Left atrial appendage occlusion device specification from 2D ICE for treating atrial fibrillation patients

Abstract

Left atrial appendage occlusion device specifications from 2D ICE for use in the treatment of atrial fibrillation patients. Systems and methods for left atrial appendage occlusion device specification from two-dimensional (2D) intracardiac echo imaging (ICE). In the imaging procedure, the 2D ICE is used to acquire an image of the patient's left atrium. A model of the left atrium is generated from the image of the left atrium. The model of the left atrium is used to validate subsequent imaging of the left atrial appendage of the patient. Anatomical measurements from validated subsequent imaging may be used to select or specify a left atrial appendage occlusion device in order, for example, to treat atrial fibrillation patients.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to medical imaging. BACKGROUND

[0002] The left atrial appendage (LAA) is a small pouch that extends off the side of the left atrium in the heart that acts as a pressure relief chamber when atrial pressure is high. The LAA plays a key role in thromboembolic risk associated with atrial fibrillation and can also have a possible triggering effect of atrial tachyarrhythmias. Atrial fibrillation (A-fib) is a type of abnormal heart rhythm in which the upper chambers of the heart beat irregularly and rapidly, increasing the risk of blood clots forming in the heart. If a blood clot in the left upper chamber (left atrium) breaks away from the heart area, it can travel to the brain and cause a stroke, which is a leading cause of death. Studies have shown that the LAA is a particularly common site of blood clot formation in people with A-fib.

[0003] Understanding the morphology and function of the LAA is critical for the treatment of A-fib. Measurement of the size of the LAA ostium has typically been done from cardiac CT or transesophageal echocardiography (TEE). Both CT and TEE have drawbacks. CT is a pre-procedure imaging procedure that uses ionizing radiation. TEE requires general anesthesia and associated risks, and management of anesthesia requires the presence of an anesthesiologist. In addition, the imaging quality of TEE is typically dependent on the operator. SUMMARY

[0004] By way of introduction, the preferred embodiments described below include methods, systems, instructions, and computer readable media for LAA closure device specification from 2D ICE.

[0005] In a first aspect, a method for left atrial appendage (LAA) assessment is provided, the method comprising: acquiring a plurality of images of a left atrium of a patient; generating a model of the left atrium from the plurality of images of the left atrium; acquiring a plurality of images of a LAA of the patient; determining, based on the model, whether the plurality of images of the LAA are sufficient to accurately estimate a specification of a LAA closure device; and when the plurality of images of the LAA are sufficient, computing and providing an anatomical measurement of the LAA, or when the plurality of images are insufficient, providing instructions for an operator to acquire additional images of the LAA.

[0006] In a second aspect, a system for left atrial appendage (LAA) device selection is provided, the system comprising: a 2D ICE imaging system configured to acquire left atrium image data of a heart of a patient and left atrial appendage image data of the heart of the patient; an imaging processor configured to generate a three-dimensional model of a left atrium of the patient from the left atrium image data, the imaging processor being further configured to register the left atrial appendage image data to the three-dimensional model and determine that the left atrial appendage is sufficiently visualized in the left atrial appendage image data; wherein the imaging processor is further configured to compute at least one anatomic measurement of the left atrial appendage and select a device for a left atrial appendage occlusion procedure based on the at least one anatomic measurement.

[0007] In a third aspect, a non-transitory computer-implemented storage medium storing machine readable instructions executable by at least one processor for left atrial appendage (LAA) evaluation is provided, the machine readable instructions comprising: acquiring a plurality of images of a left atrium of a patient; generating a model of the left atrium from the plurality of images of the left atrium; acquiring a plurality of images of a LAA of the patient; determining whether the plurality of images of the LAA are sufficient to accurately estimate specifications of a LAA occlusion device based on the model; and when the plurality of images of the LAA are sufficient, computing and providing anatomic measurements of the LAA, or when the plurality of images are not sufficient, providing instructions for an operator to acquire additional images of the LAA.

[0008] Any one or more of the aspects described above can be used alone or in combination. These and other aspects, features, and advantages will become apparent from the following detailed description of the preferred embodiments, taken in conjunction with the accompanying drawings. The invention is defined by the appended claims, and no limitation for these claims should be read into this section. Additional aspects and advantages of the present invention will be discussed below in connection with preferred embodiments and will be apparent from a reading of the following detailed description taken in conjunction with the attached drawings. Further aspects and advantages of the present invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as in the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0009] The components and the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the embodiments. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.

[0010] Figure 1 An example of the components of a heart is depicted.

[0011] Figure 2 An example of an ICE image of a patient's LA and LAA is depicted.

[0012] Figure 3A And Figure 3B An example of different orientations of an ICE catheter and different scan planes is depicted.

[0013] Figure 4An example system for LAA occlusion device specification from 2D ICE is depicted in accordance with an embodiment.

[0014] Figure 5 An example workflow for LAA occlusion device specification from 2D ICE is depicted in accordance with an embodiment.

[0015] Figure 6A and Figure 6B A 3D model of a heart and a 2D ICE scan plane are depicted in accordance with an embodiment.

[0016] Figure 7 A method for LAA occlusion device specification from 2D ICE is depicted in accordance with an embodiment.

[0017] Figure 8 An example artificial neural network is depicted Figure 9 An example convolutional neural network is depicted. DETAILED DESCRIPTION

[0018] Embodiments described herein provide systems and methods for LAA occlusion device specification from two-dimensional (2D) intra-cardiac echocardiography (ICE). In an imaging procedure, 2D ICE is used to acquire images of a patient's left atrium. A model of the left atrium is generated from the images. The model is used to validate subsequent imaging of the patient's LAA. Anatomical measurements from the validated subsequent imaging can be used to select or specify an LAA occlusion device, for example, to treat a patient with atrial fibrillation.

[0019] 2D ICE is an imaging modality that provides high-resolution real-time visualization of cardiac structures, continuous monitoring of intracardiac catheter position, and early identification of procedural complications such as pericardial effusion or thrombus formation. ICE has largely replaced transesophageal echocardiography as an imaging modality for guiding certain procedures, such as atrial septal defect closure and catheter ablation of arrhythmias, and has an emerging role in other procedures including mitral valvuloplasty, transcatheter aortic valve replacement, and left atrial appendage closure. ICE catheters can be augmented with tracking devices such as radiofrequency transmitters that make it possible to determine the position of the catheter while images are being acquired.

[0020] Figure 1 An example of different components of a patient's heart is depicted. Figure 1An example left atrium 20 and surrounding structures of interest are depicted. The structures include the left atrial appendage (LAA) 26, left inferior pulmonary vein (LIPV) 25, left superior pulmonary vein (LSPV) 28, right inferior pulmonary vein (RIPV) 22, and right superior pulmonary vein (RSPV) 24. The left ventricle or other heart chambers can be anatomical structures of interest when performing an ultrasound procedure. While the embodiments described herein focus specifically on the LAA, the method and system can be applied to other organs, tissues, or structures that can benefit from real-time segmentation and quantification.

[0021] As Figure 1 The LAA 26 is located on the side of the pulmonary artery, which brings blood from the lungs. The location of the LAA 26 can cause blood to pool there instead of flowing into the left ventricle of the heart. For patients with normal heart rhythm, the LAA 26 is rhythmically squeezed with the rest of the left atrium. This rhythmic contraction ensures that the blood in the pocket is expelled when the left atrium empties into the left ventricle, where the blood is then pumped to the body. However, AFib patients experience poor atrial contractions during episodes. These weak contractions, combined with blood pooling, can lead to clot formation, which can form in the LAA 26. Anticoagulation therapy is commonly used as the primary embolic prevention therapy for patients with atrial fibrillation, however, it can have poor long-term compliance and potential bleeding complications. Left atrial appendage closure (LAAC) is a minimally invasive cardiac intervention to prevent blood clots from forming in the LAA 26 by closing it with a device. This can significantly reduce the risk of stroke in people with A-fib who are at high risk of developing blood clots. It is used as an alternative solution to anticoagulation therapy and is recommended for people with A-fib who cannot take blood-thinning drugs.

[0022] ICE is increasingly used to guide and provide navigation support for LAAC surgical procedures. For example, ICE is the most frequently used imaging modality for, e.g., Watchman FLX implants and other LAA 26 implants. Using ICE for LAAC has advantages over TEE-guided approaches. The procedure can be performed with local anesthesia with the patient awake and responsive. Risks and post-procedural discomfort associated with general anesthesia or deeper sedation and endotracheal intubation can be avoided, and the procedure can be performed in patients with general anesthesia contraindications and in patients with gastroesophageal disease. The anesthesia team and TEE operator are not required, and procedure turnaround time is reduced.

[0023] However, there are challenges in using 2D ICE in procedures such as LAAC. One of the primary measurements used in selecting a particular LAAC device is the diameter of the LAA ostium (opening). The size of the ostium varies significantly among different patents. When performing a LAAC, measuring the size is an important task, for example, in order to determine which of several variations of a LAAC device to use for a particular patient. This can be challenging because 2D ICE provides a 2D view of the 3D anatomy of the LAA 26, and many clinical sites have limited experience working with ICE and obtaining optimal images. Although the LAA ostium can be seen in a particular ICE image frame, it can not show the full range of the LAA ostium diameter. From a single image frame or from a series of image frames, it can not be apparent that the full diameter has not been fully seen.

[0024] Figure 2 An example of a 2D ICE image of the LAA 26 is depicted. In the 2D image, both the LA 20 and the LAA 26 of the patient are included. While the location of the LAA 26 can be readily ascertained, it can not be clear whether the 2D image includes a full view of the LAA 26 or only a portion. FIG. 3 depicts an example of how different scan planes can result in different views of the LAA 26 and depicted and in particular the depicted size of the LAA ostium 205. FIG. 3 depicts two different scenarios in which an ICE catheter 210 is used to image the LAA ostium 205, represented as an ellipse. The ICE image frames are shown as wedges of ultrasound sectors representing image information. The diameter of the LAA ostium 205 as measured (LAA ostium size 215) is depicted in the ICE image frames. Figure 3A The diameter in the LAA ostium 205 is underestimated. Figure 3B The diameter in the LAA ostium 205 is underestimated. The diameter in the LAA ostium 205 is underestimated.

[0025] Embodiments provide mechanisms for certifying or verifying that the acquired images are sufficient to specify a LAAC device. In embodiments, the full diameter of the LAA ostium 205 has been seen using an estimated model of the LA anatomy including the LAA 26 and the known position of the 2D ICE frame relative to this model. Any uncertainty in the model and ICE image frame position can be used to estimate the probability that the LAA ostium 205 diameter has been measured to a certain confidence. If the LAA ostium 205 diameter has not been confidently seen, the system can suggest, based on the model, in which way to move the ICE catheter 210 to obtain a better estimate of the LAA ostium 205 diameter. For example, move the catheter forward or rotate the catheter ("clocking"). Simulating alternative views from the LA model that the ICE catheter 210 can reach can allow for simulated calculation of the LAA ostium 205 diameter. All different such views can be calculated and the best view reported to the user in the form of guidance to move the ICE catheter 210 in order to obtain the best view.

[0026] Whether the images are sufficient to estimate the size of the LAA for a LAAC device depends on the specific occlusion device and the recommendations of the device manufacturer for device deployment. The device manufacturer can specify several measurements that need to be made when choosing which occlusion device should be used for a patient. For example, the device manufacturer can require that the diameter of the LAAC opening is measured with an accuracy of 2 mm. If the images show that the diameter of the LAA is within 2 mm of the actual diameter of the LAA, then the images will be sufficient. Since there can be some uncertainty in the model recovered from the ICE images, the diameter measured from the model can only be known with 90% accuracy. In an example, given that the diameter seen on the images is within 2 mm of the maximum diameter as measured on the model, the system outputs with 90% probability that the current images are sufficient.

[0027] Figure 4An ultrasound system 100 for LAAC device specification from 2D ICE is depicted. The system includes an image processing system 100, a medical imaging device 130, and an optional server 140. The server 140 can be configured to perform any of the tasks of the image processing system 100, including processing and / or storage of data and models. The server 140 can be or include a cloud-based platform. The image processing system 100 includes a processor 110 (imaging processor 110), a memory 120, and a display 115. The medical imaging device 130 can be utilized including the image processing system 100 or the image processing system 100 is coupled to the medical imaging device 130. The image processing system 100 is configured to generate a model of the LA using acquired 2D ICE images of the LA. The image processing system 100 is configured to use the model to verify that the 2D ICE images of the LAC are sufficient to specify a LAAC device for the patient. In an example, images of the LAC acquired without proper orientation / scan plane can be rejected as these images are insufficient to accurately measure the LA ostium. Sufficiently, for example, can mean that at least 80, 90, 95% of the LAA / LA ostium is visible in at least one image so that an accurate assessment can be performed. The image processing system 100 can also be configured to train or store machine learning models for these tasks. Imaging data is acquired in real-time from the medical imaging device 130, for example as an ultrasound sequence. Additional, different, or fewer components can be provided. For example, a computer network is included for remote processing of locally captured ultrasound data by the server 140, for example. As another example, a user input device (e.g., a keyboard, buttons, sliders, dials, trackballs, mice, or other devices) is provided for user to vary or place one or more markers. In yet another example, one or more devices or components can be included for use in a medical procedure such as left atrial appendage closure (LAAC). For example, a device to occlude the opening of the left atrial appendage to prevent blood clots formed in the LAA 26 from entering the bloodstream, a device to pinch the base of the LAA 26 to close it, or a device to close the LAA 26 using a band or suture loop, among other medical devices.

[0028] The medical imaging device 130 can be an ultrasound system 130 configured to acquire a scan plane of the LA and LAA 26. Ultrasound imaging uses sound waves to image internal body structures. Different ultrasound procedures include, among others, transthoracic echocardiography (TTE), transesophageal echocardiography (TEE), and intracardiac echocardiography (ICE). TTE is a non-invasive procedure in which a transducer (or probe) is placed on the patient’s chest. Images are recorded using ultrasound data. For TEE, the probe is passed through the patient’s esophagus to be close to the patient’s heart. The probe can have an ultrasound transducer at the tip to provide imaging capabilities. ICE uses an ultrasound tipped catheter in which the catheter is passed through a vein in the groin and up into the heart. ICE can be used to produce detailed images of the size, structure, and function of the heart, as well as detailed images of the heart’s valves. The context of use as described herein is for treating atrial fibrillation under the guidance of ICE imaging, but other procedures and ultrasound techniques can be used. For this context, the left atrium can be the most common anatomical structure of interest. In other embodiments, the left ventricle or other heart chamber can be the anatomical structure of interest. Other ICE imaging locations can result in other anatomical structures of interest, such as arteries or veins. The left atrium and corresponding anatomical structures are used as examples. In embodiments, for a LAAC procedure, the catheter is introduced via femoral venous access and placed in the right atrium, right ventricular outflow tract, coronary sinus, or left atrium. The medical imaging device 130 is configured to acquire a 2D scan plane of the left atrium and LAA 26.

[0029] The processor 110 is a general purpose processor, a digital signal processor, a graphics processing unit, an application specific integrated circuit, a field programmable gate array, an artificial intelligence processor, digital circuitry, analog circuitry, a combination thereof, or other now known or later developed device for performing the functions described below, in addition to other processes, from the LAAC device specification from the 2D ICE images. The processor 110 is a single device, multiple devices, or a network. For more than one device, parallel or sequential partitioning of the processing can be used. Different devices making up the processor 110 can perform different functions. In one embodiment, the processor 110 is a control processor or other processor of the medical imaging device 130. In other embodiments, the processor 110 is part of a separate workstation or computer. The processor 110 operates according to stored instructions to perform the various actions described herein. The processor 110 is configured by software, design, firmware, and / or hardware to perform any or all of the actions of Figure 5 and Figure 7 any other calculations described herein.

[0030] Figure 5A workflow for LAA occlusion device specification from 2D ICE according to embodiments is depicted. A patient can be prepared for a procedure. ICE uses an ultrasound tipped catheter where the catheter is threaded through a vein in the groin and up into the heart. An ICE overview of the left atrium is acquired 310. The left atrium is modeled, generating a left atrium model 320. ICE imaging of the LAA 26 is performed to acquire ICE scan data of the LAA 315. The ICE imaging of the LAA 26 is checked against the model for ICE measurements. If the measurements are acceptable, a LAAC device is selected / specified based on the measurements and LAA device specifications 325 and the procedure is performed. If the measurements are not acceptable, the operator is notified and additional imaging of the LAA 26 is performed.

[0031] The processor 110 is configured to generate a model of the LA of the patient from images acquired by the medical imaging device 130. The processor 110 is configured to use the model to determine whether the acquired images of the LAA 26 are sufficient to specify a device for a LAAC procedure. The processor 110 can also be configured to use the model to verify measurements from subsequent images acquired by the medical imaging device 130 of the LAA 26. The verification can for example include determining the orientation of the scan plane and whether there is an acceptable view of the ostium.

[0032] In embodiments, the processor 110 is configured to generate a 3D model of the left atrium by combining multiple scan planes acquired by the medical imaging device 130. In an ICE imaging procedure, the sensor provides the location / orientation of the scan plane. The provided known location is used to fill a volume with data from the scan plane. The location of the catheter can be identified from for example a magnetic position sensor. The location and orientation of the catheter / transducer define the scan plane. Different scan planes have different locations / orientations. The sensed location is used to assign a scalar or other ultrasound data to different voxels in the 3D volume. Alternatively, the sensed location is used to align the plane position represented by the ultrasound data within the volume. The ultrasound data is mapped to three dimensions using the location of the scan plane.

[0033] Figure 6A An example of different scan plans 370 overlaid on a 3D environment is depicted. As the catheter / transducer is rotated to different orientations, different scan planes are captured. When data from some or all of the scan planes 370 are combined, a 3D view can be visualized. The 3D view can be sparse, e.g., missing some data between the scan planes. However, the 3D view can include enough data to generate a mesh, and / or allow the system to identify the location of each scan plane 370 relative to each other and any landmarks or features of the heart. Figure 6BAn example model generated by combining different scan planes, segmenting the data, generating a mesh model, and adding some appearance data to flesh out the model is depicted. In both 6A and 6B, scan plane 370 is visible.

[0034] In embodiments, the processor 110 is configured to perform segmentation on the acquired images to generate a LA model, e.g., using a machine trained model. Different types of models or networks can be trained and used for the segmentation task. Segmentation divides an image into regions based on a specified description, such as segmenting a body organ / tissue. In embodiments of segmenting a volume, the segmented data includes a plurality of pixels. Each pixel represents a two-dimensional display element. For example, a pixel represents an amount of 2D data. For segmentation, landmark detection, and tracking, the processor 110 can apply one or more trained machine learning networks or models that have been trained for the respective task.

[0035] The machine learning network(s) or model(s) can include a neural network defined as a plurality of sequential feature units or layers. Sequential is used to indicate the general flow of output feature values from one layer to the next. Sequential is used to indicate the general flow of output feature values from one layer to the next. Information from the next layer is fed to the next layer, and so on until the final output. The layers can be only feed forward or can be bidirectional, including some feedback to previous layers. The nodes of each layer or unit can connect to all or only a subset of the nodes of the previous and / or subsequent layers or units. Skip connections can be used, such as layers that output to the sequentially next layer as well as other layers. Rather than pre-programming features and trying to correlate the features to attributes, a deep architecture is defined to learn features at different levels of abstraction based on the input data. The features are learned to reconstruct lower level features (i.e., more abstract or compressed level features). Each node of a unit represents a feature. Different units are provided for learning different features. Various units or layers can be used, such as convolutional, pooling (e.g., max pooling), deconvolutional, fully connected, or other types of layers. Within a unit or layer, any number of nodes are provided. For example, 100 nodes are provided. Later or subsequent units can have more, fewer, or the same number of nodes.

[0036] The segmented data can be used to form a mesh model of the LA of the patient. An algorithm can be used to compute a graph mesh representation from the image segmentation. Features such as landmarks / tissues / contours / boundaries, etc. can be identified and labeled in the segmented data / 3D model. These features can be used to determine whether newly acquired ultrasound data of the LA is acquired at an acceptable angle / orientation.

[0037] Image data, machine trained networks, training data, computed metrics, and other data can be stored in memory 120. Memory 120 can be or include external storage devices, RAM, ROM, databases, and / or local storage (e.g., solid state drive or hard disk drive). The same or different non-transitory computer-readable media can be used for instructions and other data. Memory 120 can be implemented using a database management system (DBMS) and resides on a memory 120 such as a hard disk, RAM, or removable media. Alternatively, memory 120 is internal to processor 110 (e.g., cache). Instructions for implementing processes, methods, and / or techniques discussed herein are provided on a non-transitory computer-readable storage medium or memory such as a cache, buffer, RAM, removable media, hard disk drive, or other computer-readable storage media (e.g., memory 120). The instructions are executable by processor 110 or another processor. Computer-readable storage media include various types of volatile and non-volatile storage media. The functions, acts or tasks illustrated in the figures or described herein are implemented in response to one or more sets of instructions embodied in or on computer-readable storage medium. The functions, acts or tasks are independent of the

[0038] Processor 110 is configured to determine whether subsequent image data of LAA 26 is sufficient for use in designating LAAC devices. To determine whether image data of LAA 26 is sufficient, processor 110 can register the image slices of LAA 26 to the heart model and then determine its orientation. For example, a view of the LA from a particular scan plane can be acquired at an orientation that is 5, 10, 20 degrees from optimal. The offset angle makes it difficult to properly assess the LA orifice. Processor 110 is configured to determine that newly acquired image data of the LA is insufficient by matching features of the newly acquired image to the 3D model and deriving where the scan plane cuts through the heart region.

[0039] The processor 110 is also configured to check / validate various measurements from ICE imaging of the LAA 26 against the LA model. In embodiments, the processor 110 is configured to measure features of the LAA 26 and validate the measurements based at least in part on the heart model. The processor 110 can determine and use the orientation of the catheter / scan plane and an algorithm to determine whether the measurements of the patient's LA ostium are within a predefined range of estimated LA ostium measurements derived from the heart model. This algorithm is then used to determine whether the ICE images of the LAA 26 are sufficient to accurately estimate the patient's LAA occlusion device needs based on the ICE images of the LAA 26 and the model of the LA. If the ICE images of the LAA 26 are not sufficient, the user of the system is notified of this and can be told how to acquire additional appropriate images to better determine the patient's specific needs for a LAA occlusion device. If the ICE images are sufficient, the processor 110 can be further configured to select a device for the LAAC procedure based on the LAA 26 measurements.

[0040] The display 115 is configured to display or otherwise provide images and measurements of the patient to a user. The display can be configured to display the 3D model, the acquired 2D ICE images, and an overlay of the 2D ICE images of the LAA 26 and the 3D model. In examples, the scan plane of the acquired LAA 2D ICE images can be displayed to the operator so that the operator can understand why the images are not sufficient, e.g., how much the scan plane is offset from the optimal or useful scan plane. The display 115 is a CRT, LCD, projector, plasma, printer, tablet, smart phone, or other now known or later developed display device for displaying output.

[0041] In embodiments, the system is configured to measure not only the LAA diameter, but also any measurement of the size or shape of the LAA 26 used to select an LAA 26 occlusion device. Many manufacturers of LAA occlusion devices offer solutions that vary in size. If the only choice between devices is based on size, this is referred to as "sizing" the device. If anatomic measurements are used in the selection between different available devices, the system can be used to assist in sizing or selecting between different manufacturer devices.

[0042] In embodiments, the system can detect whether the LAA 26 is visible in the ICE frames. The system also registers the ICE frames to a model of the left atrial anatomy and estimates variability in the alignment of the ICE frames to the anatomical model. The system is also configured to reason, based on the uncertainty in the ICE frame alignment and the uncertainty in the anatomical model, to determine the uncertainty in a particular measurement such as the LAA ostium 205 diameter. The system mitigates the need for pre-procedural CT imaging, TEE imaging, or other imaging than ICE that can be used as guidance during the LAA occlusion procedure. The system provides a measure of the confidence of the LAA 26 measurements from the ICE for LAA occlusion device sizing. In addition, the system provides instructions on how to better position the ICE catheter 210 in order to obtain ICE images that will better support measurements of the LAA 26 useful for selecting a LAA occlusion device.

[0043] Figure 7 An example method is depicted that uses an estimated model of the anatomy to guide 2D ICE imaging of the LAA ostium 205 for optimal selection of a device for LAA occlusion. These actions are performed by the systems of FIGS. 1, 2, 3, 4, 5, 6, 7, 8, 9, other systems, workstations, computers, and / or servers. Additional, different, or fewer actions can be provided. Actions are performed in the order shown (e.g., top to bottom) or in other orders. Certain actions can be omitted or changed depending on the results of previous actions and the state of the patient. Figure 4 、 5 The system of FIGS. 1, 2, 3, 4, 5, 6, 7, 8, 9, other systems, workstations, computers, and / or servers perform the actions. Additional, different, or fewer actions can be provided. Actions are performed in the order shown (e.g., top to bottom) or in other orders. Certain actions can be omitted or changed depending on the results of previous actions and the state of the patient.

[0044] At action A110, a plurality of images of the patient's left atrium are acquired. An ultrasound scanner images using an ICE catheter 210. The transducer of the ICE catheter 210 scans a plane. The scan plane is oriented based on the position of the catheter. Different scan planes are scanned as the catheter is moved (e.g., translated or rotated). Each scan generates a data frame representing the scan plane at that time. The ultrasound data frames can be display values (e.g., RGB) or scalar values in polar or Cartesian coordinate format. The ultrasound data frames can be B-mode, color flow, or other ultrasound images. A sequence of data frames results from the ICE imaging. Each frame represents a 2D scan plane, so a collection of frames representing different 2D scan planes in and / or around the volume of the heart are acquired.

[0045] At action A120, a model of the left atrium is generated from the plurality of images of the left atrium. The model is generated by combining a plurality of slices of the left atrium and registering the slices to a common coordinate system using the known position of the catheter / transducer. In Figure 6A and Figure 6BExamples of scanning planes (slices) and how they can be combined to generate a model are provided. Known locations represented by ultrasound data (multiple images) are used to fill a volume. The sensed locations are used to assign scalar or other ultrasound data to different voxels. Alternatively, the sensed locations are used to align the plane locations represented by the ultrasound data within the volume. The locations of the scanning planes are used to map the ultrasound data to three dimensions. The mapping forms a 3D sparse ICE volume using the location information associated with each ICE image. A set of input 2D ICE images are input, each including a portion of a heart in its field of view. The sensed 3D locations are used to map all of the 2D ICE images to 3D space, forming a sparse ICE volume. The generated 3D sparse ICE volume preserves the spatial relationships between the individual ICE views.

[0046] In embodiments, a machine learning network is used to segment the model. The 3D segmentation is a labeling of voxels or locations by different anatomical structures. The labeling indicates the anatomical structure represented by each location. Alternatively, the segmentation forms a 3D mesh for each anatomical structure. Other segmentation results can be provided. The 3D segmentation provides a boundary for one or more structures in 3D. The segmentation has one or more structures of interest, such as identifying a subset or all of the locations of the anatomical structures of interest, e.g., different.

[0047] In embodiments, the processor 110 generates a 3D segmentation from the ICE volume input of the machine learning network. The 3D segmentation is a labeling of voxels or locations by different anatomical structures. The labeling indicates the anatomical structure represented by each location. Alternatively, the segmentation forms a 3D mesh for each anatomical structure. Other segmentation results can be provided. The 3D segmentation provides a boundary for one or more structures in 3D. The segmentation has one or more structures of interest, such as identifying a subset or all of the locations of the anatomical structures of interest, e.g., the leaflets of the heart valves (tricuspid, aortic, mitral), segments of each heart chamber, proximal portions of arteries and veins attached to the heart. The feature data / identified landmarks can be used to register newly acquired images to the 3D heart model.

[0048] In embodiments, the 3D segmentation uses a machine learning network. The ICE volume and / or multiple 2D slices with or without other data are input to the machine learning network and a 3D segmentation is output in response. The machine learning network can also be configured to fill in portions of the 3D volume that arise due to missing data from a limited number of 2D slices. Examples are provided in U.S. Patent No. 11,534,136 B2, which is incorporated by reference herein in its entirety.

[0049] Machine learning for image segmentation can be done by extracting a selection of features from an input image. These features can include, for example, pixel gray levels, pixel positions, image time, information about a neighborhood of a pixel, etc. A vector of image features is then fed into a learned classifier that classifies each pixel of the image into a class. The parameters of the classifier are learned automatically by giving the classifier input images with known ground truth classification results. The output of the model can then be compared to the ground truth and the parameters of the model are adjusted so that the output of the model better matches the ground truth. This procedure is repeated for a large number of input images so that the learned parameters generalize to new, unseen examples. The process of adjusting the parameters of the model is called training. Deep learning can also be used for segmentation, for example using neural networks. Deep learning based image segmentation can be done, for example, using a convolutional neural network (CNN). A convolutional neural network has a hierarchical structure where a series of convolutions are performed on an input image. The kernels of the convolutions are learned during training. The convolution results are then combined using a learned statistical model that outputs a segmented image.

[0050] The machine learning network can be an image-to-image network, such as a fully convolutional U-net trained to convert an ICE volume to a 3D segmentation. For example, trained convolutional units, weights, links, and / or other characteristics of the network are applied to data of the ICE volume and / or derived feature values to extract corresponding features through multiple layers and output a 3D segmentation. The features input are extracted from ICE images arranged in 3D. Other, more abstract features can be extracted from those extracted features using this architecture. Other features are extracted from the input depending on the number and / or arrangement of units or layers. The network includes an encoder (convolutional) network and a decoder (transposed convolutional) network forming a "U" shape with connections between the encoder and the decoder passing features at a maximum level of compression or abstraction. Skip connections can be provided. Any now known or later developed U-Net architecture can be used. Other fully convolutional networks can be used. In one embodiment, the network is a U-Net with one or more skip connections. The skip connections pass features from the encoder to the decoder at other levels of abstraction or resolution than the most abstract (i.e., other than the bottleneck). The skip connections provide more information to the decoding layers. The fully convolutional layers can be at a bottleneck of the network (i.e., between the encoder and the decoder at the most abstract level of the layers). The fully connected layers can ensure that as much information as possible is encoded. Batch normalization can be added to stabilize training.

[0051] To train any network, various optimizers can be used, such as Adadelta, SGD, RMSprop, or Adam. The weights of the network are randomly initialized, but another initialization can be used. End-to-end training is performed, but one or more features can be set. Batch normalization, dropout, and data augmentation can or can not be used. During optimization, different discriminative features are learned. Features are learned that provide an indication of the anatomical structure location or missing volume information given an input sparse ICE volume. The network minimizes an error or loss, such as mean squared error (MSE), Huber loss, LI loss, or L2 loss. In one embodiment, the machine training uses a combination of adversarial loss (e.g., using a GAN) and a reconstruction loss. The discriminator of the GAN provides an adversarial loss for segmentation and / or volume completion. The reconstruction loss is a measure of the difference of the 3D segmentation from the ground truth segmentation and the difference of the complete volume from the ground truth volume.

[0052] Figure 8 An embodiment of an artificial neural network 500 is shown in accordance with one or more embodiments. Alternative terms for "artificial neural network" are "neural network," "artificial neural net," or "neural net." The artificial neural network 500 can be used, in part, in one or more machine learning based networks, e.g., for segmentation, rendering, etc.

[0053] The artificial neural network 500 includes nodes 502-522 and edges 532, 534,... 536, where each edge 532, 534,... 536 is a directed connection from a first node 502-522 to a second node 502-522. Typically, the first node 502-522 and the second node 502-522 are different nodes 502-522, but the first node 502-522 and the second node 502-522 can also be the same. In this embodiment, the artificial neural network 500 includes three types of nodes 502-522: input nodes 502, hidden nodes 504, and output nodes 506. Figure 8 In this embodiment, the edge 532 is a directed connection from the node 502 to the node 506, and the edge 534 is a directed connection from the node 504 to the node 506. The edges 532, 534,... 536 from the first nodes 502-522 to the second nodes 502-522 are also represented as "ingoing edges" of the second nodes 502-522 and "outgoing edges" of the first nodes 502-522.

[0054] In this embodiment, the nodes 502-522 of the artificial neural network 500 can be arranged in layers 524-530, where these layers can include an inherent order introduced by the edges 532, 534,... 536 between the nodes 502-522. In particular, the edges 532, 534,... 536 can only exist between adjacent layers of nodes. In this embodiment, the layers 524-530 include an input layer 524 of input nodes 502, a first hidden layer 526 of hidden nodes 504, a second hidden layer 528 of hidden nodes 504, and an output layer 530 of output nodes 506. Figure 8In the embodiment shown in Fig. 5, there is an input layer 524 comprising only nodes 502 and 504 without incoming edges, an output layer 530 comprising only a node 522 without outgoing edges, and hidden layers 526, 528 between the input layer 524 and the output layer 530. Generally, the number of hidden layers 526, 528 can be chosen arbitrarily. The number of nodes 502 and 504 within the input layer 524 is generally related to the number of input values of the neural network 500, and the number of nodes 522 within the output layer 530 is generally related to the number of output values of the neural network 500.

[0055] In particular, a (real) number can be assigned as a value to each node 502-522 of the neural network 500. Here, x （n） i denotes the value of the i-th node 502-522 of the n-th layer 524-530. The values of the nodes 502-522 of the input layer 524 are identical to the input values of the neural network 500, and the value of the node 522 of the output layer 530 is identical to the output value of the neural network 500. Furthermore, each edge 532, 534,..., 536 can comprise a weight which is a real number, in particular, the weight is a real number within the interval [-1, 1] or within the interval [0, 1]. Here, w （m,n） i,j denotes the weight of the edge between the i-th node 502-522 of the m-th layer 524-530 and the j-th node 502-522 of the n-th layer 524-530. Furthermore, the weights w （n,n+1） i,j The abbreviation w （n） i,j .

[0056] In particular, for calculating the output values of the neural network 500, the input values are propagated through the neural network. In particular, the values of the nodes 502-522 of the (n+1)-th layer 524-530 can be calculated based on the values of the nodes 502-522 of the n-th layer 524-530 by .

[0057] Here, the function f is a transfer function (another term is "activation function"). Known transfer functions are step functions, sigmoid functions (e.g. logistic function, generalized logistic function, hyperbolic tangent, arctangent function, error function, smooth step function), or rectifier functions. The transfer function is mainly used for normalization purposes.

[0058] In particular, the values are propagated through the neural network layer by layer, where the values of the input layer 524 are given by the input of the neural network 500, where the values of the first hidden layer 526 can be computed based on the values of the input layer 524 of the neural network, where the values of the second hidden layer 528 can be computed based on the values of the first hidden layer 526, and so on.

[0059] In order to set the values w （m,n） i,j of the edges, the neural network 500 has to be trained using training data. In particular, the training data comprises training input data and training output data (denoted as t i ). For a training step, the neural network 500 is applied to the training input data to generate computed output data. In particular, the training data and the computed output data comprise a number of values, said number being equal to the number of nodes of the output layer.

[0060] In particular, a comparison between the computed output data and the training data is used to recursively adapt the weights within the neural network 500 (backpropagation algorithm). In particular, the weights are changed according to where γ is a learning rate, and the number δ （n） j can be recursively computed as based on δ （n+1） j if the (n+1)-th layer is not the output layer, and if the (n+1)-th layer is the output layer 530, where f' is the first derivative of the activation function, and y （n+1） j is the comparison training value of the j-th node of the output layer 530.

[0061] Figure 9 A convolutional neural network 600 according to one or more embodiments is shown. The machine learning networks described herein, such as for example for segmentation, model generation, rendering, etc., can be implemented using the convolutional neural network 600.

[0062] In the embodiment shown in Figure 9 , the convolutional neural network 600 comprises an input layer 602, convolutional layers 604, pooling layers 606, fully connected layers 608, and an output layer 610. Alternatively, the convolutional neural network 600 can comprise several convolutional layers 604, several pooling layers 606, and several fully connected layers 608, as well as other types of layers. The order of the layers can be chosen arbitrarily, typically the fully connected layers 608 are used as the last layers before the output layer 610.

[0063] In particular, within the convolutional neural network 600, the nodes 612-620 of one layer 602-610 can be considered to be arranged as a d-dimensional matrix or d-dimensional image. In particular, in the two-dimensional case, the values of the nodes 612-620 in the nth layer 602-610, indexed with i and j, can be denoted as x （n） [i,j] However, the arrangement of the nodes 612-620 of one layer 602-610 has no influence on the computations performed within the convolutional neural network 600 as these are given by the structure of the edges and the weights only.

[0064] In particular, the convolutional layer 604 is characterized by the structure and the weights of the incoming edges forming a convolution operation based on a certain number of kernels. In particular, the structure and the weights of the incoming edges are chosen such that the value x （n） k of a node 614 of the convolutional layer 604 is computed based on the values x （n-1） of the nodes 612 of the preceding layer 602 as a convolution where the convolution * is defined in the two-dimensional case as: .

[0065] Here, the k-th kernel K k is a d-dimensional matrix (in this embodiment a two-dimensional matrix) which is typically small compared to the number of nodes 612-618 (e.g. a 3x3 matrix or a 5x5 matrix). In particular, this means that the weights of the incoming edges are not independent but chosen such that they result in the convolution equation. In particular, for a kernel being a 3x3 matrix, there are only 9 independent weights (each entry of the kernel matrix corresponds to one independent weight) independent of the number of nodes 612-620 in the respective layer 602-610. In particular, for the convolutional layer 604, the number of nodes 614 in the convolutional layer is equal to the number of nodes 612 in the preceding layer 602 multiplied by the number of kernels.

[0066] If the nodes 612 of the preceding layer 602 are arranged as a d-dimensional matrix, the use of multiple kernels can be interpreted as adding an additional dimension (denoted as “depth” dimension) such that the nodes 614 of the convolutional layer 604 are arranged as a (d+1)-dimensional matrix. If the nodes 612 of the preceding layer 602 are already arranged as a (d+1)-dimensional matrix including the depth dimension, the use of multiple kernels can be interpreted as an extension along the depth dimension such that the nodes 614 of the convolutional layer 604 are also arranged as a (d+1)-dimensional matrix, wherein the size of the (d+1)-dimensional matrix with respect to the depth dimension is a multiple of the size in the preceding layer 602 by the number of kernels.

[0067] An advantage of using a convolutional layer 604 is that spatial local correlations of the input data can be exploited by implementing a local connectivity pattern between the nodes of adjacent layers, in particular by each node being connected to only a small region of the nodes of the previous layer.

[0068] In the embodiment shown in Figure 9 , the input layer 602 comprises 36 nodes 612 arranged as a two-dimensional 6x6 matrix. The convolutional layer 604 comprises 72 nodes 614 arranged as two two-dimensional 6x6 matrices, each of the two matrices being the result of a convolution of the values of the input layer with a kernel. Equivalently, the nodes 614 of the convolutional layer 604 can be interpreted as being arranged as a three-dimensional 6x6x2 matrix, where the last dimension is a depth dimension.

[0069] The pooling layer 606 can be characterized by the structure and weights of the incoming edges and by the activation function of its nodes 616 forming a pooling operation based on a non-linear pooling function f. For example, in the two-dimensional case, the value x of a node 616 of the pooling layer 606 is computed as （n） based on the values x of the nodes 614 of the previous layer 604. （n-1） being computed as .

[0070] In other words, by using a pooling layer 606, the number of nodes 614, 616 and parameters can be reduced by replacing a number d1-d2of adjacent nodes 614 in the previous layer 604 with a single node 616, the single node 616 being computed as a function of the values of the number of adjacent nodes in the pooling layer. In particular, the pooling function f can be a max-function, an average or an L2-norm. In particular, for the pooling layer 606, the weights of the incoming edges are fixed and not modified by training.

[0071] An advantage of using a pooling layer 606 is that the number of nodes 614, 616 and the number of parameters is reduced. This leads to a reduction of the amount of computations in the network and to a control of overfitting.

[0072] In the embodiment shown in Figure 9 , the pooling layer 606 is a max-pooling, replacing four adjacent nodes with one node, the value being the maximum of the values of the four adjacent nodes. The max-pooling is applied to each d-dimensional matrix of the previous layer; in this embodiment, the max-pooling is applied to each of the two two-dimensional matrices, reducing the number of nodes from 72 to 18.

[0073] The fully connected layer 608 can be characterized by the fact that most, in particular all, edges between the nodes 616 of the previous layer 606 and the nodes 618 of the fully connected layer 608 exist and that the weight of each edge can be individually adjusted.

[0074] In this embodiment, the nodes 616 of the preceding layer 606 of the fully connected layer 608 are shown as a two-dimensional matrix, and are additionally shown as unassociated nodes (indicated as rows of nodes, where the number of nodes is reduced for better presentation). In this embodiment, the number of nodes 618 in the fully connected layer 608 is equal to the number of nodes 616 in the preceding layer 606. Alternatively, the number of nodes 616, 618 can be different.

[0075] Further, in this embodiment, the values of the nodes 620 of the output layer 610 are determined by applying a Softmax function to the values of the nodes 618 of the preceding layer 608. By applying the Softmax function, the sum of the values of all nodes 620 of the output layer 610 is 1, and all values of all nodes 620 of the output layer are real numbers between 0 and 1.

[0076] The convolutional neural network 600 can also include ReLU (rectified linear unit) layers or activation layers with a non-linear transfer function. In particular, the number of nodes and the structure of the nodes included in a ReLU layer is equal to the number of nodes and the structure of the nodes of the preceding layer. In particular, the value of each node in a ReLU layer is computed by applying a rectification function to the value of the corresponding node of the preceding layer.

[0077] The inputs and outputs of different convolutional neural network blocks can be wired using summation (residual / dense neural networks), element-wise multiplication (attention), or other differentiable operators. Thus, the convolutional neural network architecture can be nested rather than sequential if the entire pipeline is differentiable.

[0078] In particular, the convolutional neural network 600 can be trained based on a backpropagation algorithm. To prevent overfitting, regularization methods can be used, such as dropout of nodes 612-620, random pooling, use of artificial data, weight decay based on L1 or L2 norm, or maximum norm constraint. Different loss functions can be combined to train the same neural network to reflect joint training objectives. A subset of neural network parameters can be excluded from optimization to preserve pre-trained weights with respect to another dataset.

[0079] Any machine training architecture for segmentation can be used. Similarly, for other tasks described herein, different machine training architectures can be used. For example, a U-Net is used. A convolution-to-transposed convolution network is used. One segment of a layer or unit applies a convolution to increase abstraction or compression. The most abstract feature values are then output to another segment. Another segment of the layer or unit then applies a transposed convolution to decrease abstraction or compression, resulting in an output of an indication of class membership by location. The architecture can be a fully convolutional network. Other deep networks can be used.

[0080] Referring back Figure 7 At act A130, a plurality of images of the LAA 26 of the patient are acquired. Similar to act A110, the ultrasound scanner images the heart region of the patient using the ICE catheter 210. Images of the LAA 26 can be acquired. In embodiments, it is determined whether the LAA 26 is visible in the images. If the acquired images do not include the LAA 26, a notification can be provided. For this procedure, the transducer of the ICE catheter 210 scans a plane. The scan plane is oriented based on the position of the catheter. As the catheter is moved (e.g., translated or rotated), different scan planes are scanned. Each scan generates a frame of data representing the scan plane at that time. The frame of ultrasound data can be in polar or Cartesian coordinate format, display values (e.g., RGB), or scalar values. The frame of ultrasound data can be B-mode, color flow, or other ultrasound images.

[0081] At act A140, the system determines whether the plurality of images of the LAA 26 are sufficient to accurately estimate the dimensions of the respective LAA occlusion device based on the model. Sufficient, for example, can mean that at least 80, 90, 95% of the LAA / LA ostium is visible in at least one image such that an accurate assessment can be performed. Whether the images are sufficient to estimate the size of the LAA of the LAAC device can be determined based on the recommendations of the device manufacturer for device deployment. Different devices and device manufacturers can require different measurements or values of anatomical features. For example, a device manufacturer can require that the diameter of the LAAC ostium be measured with an accuracy of 2 mm. If the images show that the diameter of the LAA is within 2 mm of the actual diameter of the LAA, then the images are sufficient. Since there can be some uncertainty in the model recovered from the ICE images, the diameter measured from the model can only be known with 90% accuracy. Thus, given that the diameter seen on the images is within 2 mm of the maximum diameter as measured on the model, the system will have an output that the current images are sufficient with 90% probability.

[0082] In embodiments, one or more images of the LAA 26 are registered with the 3D model to determine the orientation of the scan plane relative to other features of the 3D heart model, including, for example, the LA ostium. Features such as landmarks identified in the images of the LAA 26 can be registered / matched with related features in the 3D model. The orientation can be derived therefrom. In embodiments, the images of the LAA 26 are segmented by using, for example, a machine trained network configured for segmentation as described above to identify the features. In embodiments, the uncertainty in the ICE frame alignment and the uncertainty in the anatomical model are determined and used to identify the uncertainty in particular measurements such as the LAA ostium 205 diameter. The uncertainty can be provided to the operator so that they can determine whether to acquire different views. The uncertainty of the sizing measurements can be used in determining whether the device is appropriate for the LAAC procedure.

[0083] At act A150, when the multiple images of the LAA 26 are sufficient, an appropriate LAA occlusion device is specified. In embodiments, the device can be identified prior to performing the procedure. The multiple images of the LAA 26 can be used to determine whether a device is appropriate by, for example, checking the estimated anatomical feature values against device specifications. In embodiments, multiple measurements of the LAA 26 are computed and compared to predicted measurements from the LA model. These measurements can include not only measurements of the LAA ostium 205 diameter, but also any measurements used to select the size or shape of the LAA 26 for the LAA occlusion device. For example, the LAA ostium is measured from the pulmonary vein ridge to the junction of the LA and LAA 26. The landing zone (the area within the LAA 26 where the device will be positioned) is measured within the ostium at 10 mm at an angle perpendicular to the neck axis. Another important distance to measure is the maximum length of the anchoring lobe to confirm that the lobe has enough space to accommodate the selected device. Based on the measurements, different LAAC devices can be used for different patients. Many manufacturers of LAA occlusion devices offer solutions that vary in size. If the only choice between devices is based on size, this is referred to as sizing the device. However, additional anatomical measurements beyond size can be computed and used to help size or select between different manufacturer devices.

[0084] When the multiple images of the LAA 26 are not sufficient, the operator is instructed to acquire additional images of the LAA 26. Instructions can be provided on how to better position the ICE catheter 210 so as to obtain ICE images that will better support measurements of the LAA 26 useful for selecting a LAA occlusion device. For example, the instructions can include rotating or adjusting the position of the catheter so that the orientation of the transducers provides image slices that fully depict the LA ostium.

[0085] In embodiments, the 2D ICE procedure provides adequate visualization of the LAA 26 and can be used as a replacement for TEE in guiding LAA occlusion procedures. Embodiments provide an estimated model of the anatomy to guide 2D ICE imaging of the LAA ostium 205 to optimally select a device for LAA occlusion. In addition, the guidance can be used to provide 2D ICE images that rule out LAA thrombus, select device size by measuring the dimensions of the landing zone, guide transseptal puncture, verify positioning of the delivery sheath in the LAA 26, assist device delivery and confirm stability before and after release, check for peridevice leaks with color Doppler flow imaging, and monitor for complications such as cardiac tamponade.

[0086] It is to be understood that the elements and features of the claims can be combined in different ways to produce new claims falling within the scope of the application. Thus, although the dependent claims below refer to the independent claims, the subject matter of these dependent claims can be combined with the subject matter of the other dependent claims in any appropriate manner not specified in the above paragraphs. Such new combinations are to be understood as falling within the scope of the present specification.

[0087] While the application has been described above with reference to various embodiments, it can be understood that many changes and modifications can be made to the described embodiments, and it is intended to cover in the appended claims all such changes and modifications that fall within the scope of the application. In this patent application, the terms "comprises", "comprising", "containing" and / or "having" and the like can have the meaning ascribed to them under

[0088] The following is a list of non-limiting illustrative embodiments disclosed herein: Illustrative Embodiment 1. A method for left atrial appendage evaluation, the method comprising: acquiring a plurality of images of a left atrium of a patient; generating a model of the left atrium from the plurality of images of the left atrium; acquiring a plurality of images of a left atrial appendage of the patient; determining, based on the model, whether the plurality of images of the left atrial appendage are sufficient to accurately estimate specifications of a proper left atrial appendage occlusion device; and when the plurality of images of the left atrial appendage are sufficient, computing and providing anatomical measurements of the left atrial appendage, or when the plurality of images are not sufficient, providing instructions for an operator to acquire additional images of the left atrial appendage.

[0089] Illustrative Embodiment 2. The method of illustrative embodiment 1, further comprising: identifying a proper left atrial appendage occlusion device based on the anatomical measurements of the left atrial appendage.

[0090] Illustrative Embodiment 3. The method of illustrative embodiment 2, further comprising: performing a left atrial appendage occlusion procedure using the proper left atrial appendage occlusion device.

[0091] Illustrative Embodiment 4. The method of one of the preceding embodiments, wherein acquiring comprises acquiring two-dimensional images using an ICE ultrasound imaging system.

[0092] Illustrative Embodiment 5. The method of one of the preceding embodiments, wherein the model comprises a three-dimensional segmentation of the left atrium of the patient.

[0093] Illustrative Embodiment 6. The method of one of the preceding embodiments, wherein generating the model comprises: determining a location of a scan plane within the cardiac system of the patient for the plurality of images of the left atrium; forming an ICE volume from the plurality of images of the left atrium and the location; and generating a three-dimensional segmentation model from input of the ICE volume to a machine learning network.

[0094] Illustrative Embodiment 7. The method of illustrative embodiment 6, wherein determining whether the plurality of images of the left atrial appendage are sufficient comprises: registering one or more of the plurality of images to the three-dimensional segmentation model; and using the location of at least one of the plurality of images of the left atrial appendage relative to the model to check for a full diameter of a left atrial appendage orifice in the at least one image.

[0095] Illustrative Embodiment 8. The method of one of the preceding embodiments, wherein determining comprises providing a measure of uncertainty in the model and image frame location to determine an estimate of the probability that the left atrial appendage orifice diameter has been measured to a certain confidence.

[0096] Illustrative Embodiment 9. The method of one of the preceding embodiments, wherein providing further comprises using one or more additional measurements of the size or shape of the left atrial appendage in selecting a device for left atrial appendage occlusion.

[0097] Illustrative Embodiment 10. The method of one of the preceding embodiments, wherein the instructions are based on simulated alternative views from the model of the left atrium reachable by an ICE catheter that would account for accurate calculation of left atrial appendage orifice diameter.

[0098] Illustrative Embodiment 11. A system for left atrial appendage device selection, the system comprising: a 2D ICE imaging system configured to acquire left atrial image data of a left atrium of a patient and left atrial appendage image data of a left atrial appendage of the patient; and an imaging processor configured to generate a three-dimensional model of the left atrium of the patient from the left atrial image data, the imaging processor further configured to register the left atrial appendage image data to the three-dimensional model and determine that the left atrial appendage is sufficiently visualized in the left atrial appendage image data; wherein the imaging processor is further configured to compute at least one anatomic measurement of the left atrial appendage and select a device for a left atrial appendage occlusion procedure based on the at least one anatomic measurement.

[0099] Illustrative Embodiment 12. The system according to one of the preceding embodiments, wherein the left atrium is sufficiently visualized when a left atrial appendage orifice diameter has been measured up to a certain confidence.

[0100] Illustrative Embodiment 13. The system according to one of the preceding embodiments, wherein the imaging processor is further configured to determine that the left atrial appendage is not sufficiently visualized in the left atrial appendage image data, wherein the imaging processor is configured to generate an instruction to an operator to acquire additional image data of the left atrial appendage.

[0101] Illustrative Embodiment 14. The system according to one of the preceding embodiments, wherein the three-dimensional model is generated by segmenting and combining a plurality of scan planes from the left atrial image data.

[0102] Illustrative Embodiment 15. The system of Illustrative Embodiment 14, wherein generating the three-dimensional model comprises: determining a position of a scan plane within a cardiac system of the patient for the left atrial image data; forming an ICE volume from the scan plane and the position; and generating a three-dimensional segmentation model from an input of the ICE volume to a machine learning network.

[0103] Illustrative Embodiment 16. The system according to one of the preceding embodiments, further comprising: a display configured to display the left atrial image data, the left atrial appendage image data, and / or the three-dimensional model of the left atrium.

[0104] Illustrative Embodiment 17. A non-transitory computer- implemented storage medium storing machine-readable instructions executable by at least one processor for left atrial appendage assessment, the machine-readable instructions comprising: acquiring a plurality of images of a left atrium of a patient; generating a model of the left atrium from the plurality of images of the left atrium; acquiring a plurality of images of a left atrial appendage of the patient; determining, based on the model, whether the plurality of images of the left atrial appendage are sufficient to accurately estimate specifications of a left atrial appendage occlusion device; and when the plurality of images of the left atrial appendage are sufficient, computing and providing anatomical measurements of the left atrial appendage, or when the plurality of images are not sufficient, providing instructions for an operator to acquire additional images of the left atrial appendage.

[0105] Illustrative Embodiment 18. The non-transitory computer- implemented storage medium of one of the preceding embodiments, wherein the instructions to generate the model comprise: determining a location of a scan plane within the patient’s cardiac system for the plurality of images of the left atrium; forming an ICE volume from the plurality of images and location of the left atrium; and generating a three-dimensional segmentation model from input of the ICE volume to a machine learning network.

[0106] Illustrative Embodiment 19. The non-transitory computer- implemented storage medium of one of the preceding embodiments, wherein the instructions to determine comprise: registering the plurality of images of the left atrial appendage to the model of the left atrium; and determining whether a full view of a diameter of the left atrial appendage is visualized in the plurality of images of the left atrial appendage.

[0107] Illustrative Embodiment 20. The non-transitory computer- implemented storage medium of one of the preceding embodiments, wherein acquiring is performed using a 2D ICE imaging system.

Claims

1. A method for assessing the left atrial appendage, the method comprising: Acquire multiple images of the patient's left atrium; A model of the left atrium is generated from the plurality of images of the left atrium; Multiple images of the patient's left atrial appendage were acquired; Based on the model, it is determined whether the multiple images of the left atrial appendage are sufficient to accurately estimate the specifications of an appropriate left atrial appendage occlusion device; as well as When the plurality of images of the left atrial appendage are sufficient, anatomical measurements of the left atrial appendage are calculated and provided; or when the plurality of images are insufficient, instructions are given to the operator to obtain additional images of the left atrial appendage.

2. The method according to claim 1, further comprising: The appropriate left atrial appendage occlusion device is identified based on the anatomical measurements of the left atrial appendage.

3. The method according to claim 2, further comprising: The left atrial appendage occlusion procedure is performed using the appropriate left atrial appendage occlusion device.

4. The method of claim 1, wherein acquiring includes acquiring two-dimensional images using an ICE ultrasound imaging system.

5. The method of claim 1, wherein the model comprises a three-dimensional segmentation of the left atrium of the patient.

6. The method of claim 1, wherein generating the model comprises: The location of the scanning plane within the patient's cardiac system is determined based on the multiple images of the left atrium; The ICE volume is formed from the plurality of images of the left atrium and the location; as well as A 3D segmentation model is generated from the input of the ICE volume to the machine learning network.

7. The method of claim 6, wherein determining whether the plurality of images of the left atrial appendage are sufficient comprises: Register one or more of the plurality of images to the 3D segmentation model; as well as The position of at least one of the plurality of images of the left atrial appendage relative to the model is used to check the presence of the full diameter of the left atrial appendage opening in the at least one image.

8. The method of claim 1, wherein the determination comprises: Uncertainty measurements are provided in the model and image frame locations to estimate the probability that the left atrial appendage diameter has been measured to a certain confidence level.

9. The method of claim 1, further comprising: One or more additional measurements of the size or shape of the left atrial appendage used in selecting a device for left atrial appendage occlusion.

10. The method of claim 1, wherein the instructions are based on a simulated alternative view of the model of the left atrium accessible by the ICE catheter, which takes into account an accurate calculation of the diameter of the left atrial appendage.

11. A system for left atrial appendage device selection, the system comprising: A 2D ICE imaging system is configured to acquire left atrial image data of the patient's left atrium and left atrial appendage image data of the patient's left atrial appendage; and An imaging processor is configured to generate a three-dimensional model of the patient's left atrium from the left atrial image data, and the imaging processor is further configured to register the left atrial appendage image data to the three-dimensional model and determine that the left atrial appendage is adequately visualized in the left atrial appendage image data; The imaging processor is also configured to calculate at least one anatomical measurement of the left atrial appendage and to select a device for a left atrial appendage occlusion procedure based on the at least one anatomical measurement.

12. The system of claim 11, wherein the left atrium is fully visualized when the diameter of the left atrial appendage has been measured to a certain confidence level.

13. The system of claim 11, wherein the imaging processor is further configured to determine that the left atrial appendage is not adequately visualized in the left atrial appendage image data, wherein the imaging processor is configured to generate instructions to an operator to obtain additional image data of the left atrial appendage.

14. The system of claim 11, wherein the three-dimensional model is generated by segmenting and combining multiple scan planes from the left atrial image data.

15. The system of claim 14, wherein generating the three-dimensional model comprises: The location of the scanning plane within the patient's cardiac system was determined based on the left atrial image data; An ICE volume is formed from the scan plane and the location; as well as A 3D segmentation model is generated from the input of the ICE volume to the machine learning network.

16. The system of claim 11, further comprising: The display is configured to display the left atrial image data, the left atrial appendage image data, and / or the three-dimensional model of the left atrium.

17. A non-transitory computer-implemented storage medium storing machine-readable instructions executable by at least one processor for left atrial appendage assessment, the machine-readable instructions comprising: Acquire multiple images of the patient's left atrium; A model of the left atrium is generated from the plurality of images of the left atrium; Multiple images of the patient's left atrial appendage were acquired; Based on the model, it is determined whether the multiple images of the left atrial appendage are sufficient to accurately estimate the specifications of the left atrial appendage occlusion device; as well as When the plurality of images of the left atrial appendage are sufficient, anatomical measurements of the left atrial appendage are calculated and provided; or when the plurality of images are insufficient, instructions are given to the operator to obtain additional images of the left atrial appendage.

18. The non-transitory computer-implemented storage medium of claim 17, wherein the instructions for generating the model comprise: The location of the scanning plane within the patient's cardiac system is determined based on the multiple images of the left atrium; The ICE volume is formed from the plurality of images of the left atrium and the location; as well as A 3D segmentation model is generated from the input of the ICE volume to the machine learning network.

19. The non-transitory computer-implemented storage medium of claim 17, wherein the instructions for determining comprise: The plurality of images of the left atrial appendage are registered to the model of the left atrium; as well as Determine whether a full view of the diameter of the left atrial appendage is visualized in the plurality of images of the left atrial appendage.

20. The non-transitory computer-implemented storage medium of claim 17, wherein a 2D ICE imaging system is used to perform the acquisition.

Images