System and method for reconstruction and visualization of anatomical data - Patents.com

Abstract

The present invention provides systems and methods that provide for reconstruction of three-dimensional (3D) ultrasound image data into a combination of multiple views of an anatomical region of interest, the multiple views including both 3D and two-dimensional (2D) views that are spatially related to one another. Additionally, in some embodiments, the system may provide a user with the option to seamlessly switch between various 2D views. In particular, a user may be able to utilize an interactive interface to switch to a 2D view of interest without requiring reconstruction from a 3D view, where the 2D view of interest may have the same or a different imaging configuration.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 343,771, filed May 19, 2022, the contents of which are incorporated herein by reference in their entirety.

[0002] The present disclosure relates to visualization of biological tissue, and more particularly to systems and methods that provide reconstruction of three-dimensional (3D) ultrasound image data into a combination of multiple views of an anatomical region of interest associated with intravascular and / or intracardiac tissue, the multiple views including both 3D views and two-dimensional (2D) views that are spatially related to one another. [Background technology]

[0003] Medical imaging refers to a number of different techniques used to view the human body to diagnose, monitor, or treat medical conditions. Thus, medical imaging is generally recognized as one of the most effective diagnostic and interventional tools in medicine. The most common types of medical imaging modalities include, but are not limited to, X-ray imaging, magnetic resonance imaging (MRI), and ultrasound (US) imaging. Although each type of imaging modality has its particular advantages and its associated disadvantages, ultrasound is becoming a more common imaging technique, primarily due to its portability, ease of use, non-invasiveness, and reduced cost compared to other imaging modalities.

[0004] Ultrasound imaging is a medical imaging technique for imaging organs and soft tissues within the human body. Ultrasound imaging uses non-invasive high frequency sound waves to produce ultrasound images. Ultrasound images are produced based on the reflection of the waves from body structures. The strength (amplitude) of the sound signal and the time it takes for the waves to travel through the body provide the information needed to produce the image.

[0005] Ultrasound imaging can help physicians evaluate, diagnose, and treat a variety of medical conditions. When making a diagnosis based on ultrasound examination, physicians must rely on adequate image quality, obtaining appropriate views, and adequate quantification of all relevant structures and flows.

[0006] For example, ultrasound imaging can generally be in the form of two-dimensional (2D) or three-dimensional (3D) imaging. Conventional 2D ultrasound imaging is widely used because it can dynamically display 2D images of a region of interest in real time. However, due to the absence of anatomical structure and orientation information, clinicians need to mentally imagine the volume with a planar 2D image when they need a view of the 3D anatomical structure. To address the shortcomings of 2D ultrasound imaging, 3D ultrasound imaging has been developed to help diagnosticians obtain a complete understanding of spatial anatomical relationships. In particular, physicians can view any plane of the reconstructed 3D volume and a panoramic view of the region of interest, which is intended to help surgeons check whether surgical instruments are correctly placed in the region of interest.

[0007] Although the advancement of ultrasound imaging technology has provided some improvements, current ultrasound imaging systems still have drawbacks. In particular, it has been found that the various views provided by 3D ultrasound imaging systems result in representations that are unfamiliar to clinicians. Thus, 2D ultrasound imaging remains the natural go-to imaging modality for many clinicians. Furthermore, as explained above, the main drawbacks of 2D ultrasound imaging are the limited field of view provided and the required manual interaction with a probe or catheter, which is error-prone, cumbersome, and generally challenging for users. Summary of the Invention [Means for solving the problem]

[0008] The present invention recognizes the shortcomings of current ultrasound imaging systems, namely the limited field of view and required interactions associated with 2D imaging systems, and the complexity of navigating and working with 3D imaging systems. The present invention provides an imaging system that is configured to provide the benefits of both 2D and 3D imaging techniques without the associated shortcomings, thereby providing a user (clinician or equivalent) with optimal orientation and navigation capabilities when performing ultrasound-related procedures.

[0009] Aspects of the present invention may be accomplished using an imaging system configured to reconstruct 3D ultrasound image data, specifically 3D volume data, into a combination of multiple views of an anatomical region of interest, the multiple views including both 3D and 2D views that are spatially related to each other. The imaging system is configured to provide a user with seamless interaction with both the 2D and 3D imaging views, where the user may select one or more 2D images to be dynamically reconstructed from the 3D volume data. The multiple 2D views are directly related to the 3D view, since the 2D views are digitally reconstructed from the volume data itself. The imaging system provides a user with an interactive interface, where the user may specifically select a particular portion of the region of interest to be depicted in the 3D view from which the associated 2D image is to be reconstructed. Thus, the multiple views (2D and 3D views) relate to the same anatomical region of interest. The imaging system is further configured to annotate at least the 3D view to highlight those particular portions of the region of interest that are being reconstructed in the 2D view.

[0010] Furthermore, in some embodiments, the system may provide the user with the option to seamlessly switch between various 2D views. In particular, the user may be able to utilize an interactive interface to switch to a 2D view of interest without requiring reconstruction from a 3D view, where the 2D view of interest may have the same or a different imaging configuration. For example, a given workflow may involve the user selecting a view that has been digitally reconstructed from the 3D volume data (and / or digitally steered via a steerable probe device), and the user may subsequently switch to the selected 2D view of interest, which provides a different viewing mode (e.g., a flow imaging mode or the like), via the interactive interface.

[0011] Thus, the system provides the user with an intuitive understanding of the interrelationships of views and orientations within the acquired 3D data, thereby overcoming limitations that plague current imaging systems (i.e., 2D and 3D US imaging) while providing the benefits associated with each. In particular, the relationships between the 2D and 3D views mimic current clinical imaging probes (e.g., linear / curved / phase / rotational) in both appearance and interaction. In particular, the user can steer and / or navigate the 2D views digitally (i.e., by interacting with the 3D volume via an interface) without requiring manual steering of the ultrasound probe, which can be cumbersome and error-prone.

[0012] One aspect of the invention includes an ultrasound imaging system including a console configured to be operatively associated with an imaging device, the console comprising a hardware processor coupled to a non-transitory computer readable memory including instructions executable by the processor to cause the console to receive three-dimensional (3D) image data from the ultrasound imaging device and dynamically reconstruct a plurality of images from the 3D image data, the plurality of images including at least one 3D image providing a 3D view of an anatomical region of interest and one or more corresponding two-dimensional (2D) images providing a 2D view of the anatomical region of interest and spatially related to the 3D view.

[0013] Each of the multiple images may include at least one of a slice-based image and a volume-based image. For example, in one embodiment, at least one 3D image provides a 360-degree visualization of the entire circumference of the anatomical region of interest. In such an embodiment, the systems and methods described herein are included with a catheter-based system (or a system utilizing some type of probe). At least one 3D image may provide a trajectory volume visualization of the anatomical region of interest. In some embodiments, one or more 2D images provide at least one of a phased array visualization and a circumferential radial visualization of the anatomical region of interest. Thus, in some embodiments, a user may be presented with a 3D image (providing a trajectory volume visualization of the anatomical region of interest) and two or more 2D images (a first 2D image providing a phased array visualization of the anatomical region of interest and a second 2D image providing a circumferential radial visualization of the anatomical region of interest).

[0014] It should be noted that in some embodiments, some of the 2D images may be reconstructed from data obtained by a co-located secondary imaging modality in addition to or as an alternative to being reconstructed from 3D image data.For example, in some embodiments, the systems and methods of the present invention may further utilize secondary imaging modalities such as computed tomography (CT) imaging systems, transmission imaging systems, bright-field or dark-field imaging systems, fluorescence imaging systems, phase contrast imaging systems, differential interference contrast imaging systems, hyperspectral imaging systems, Raman or surface-enhanced Raman imaging systems, and magnetic resonance imaging (MRI) systems.For example, in some embodiments, one or more 2D images may be reconstructed based on pre-registered CT volume data, MRI volume data, etc.

[0015] In some embodiments, each of the multiple images includes one or more annotations that provide a visual indication of the spatial relationship of one of the multiple images to another of the multiple images. The one or more annotations may include highlighted markings or the like.

[0016] For example, in some embodiments, the plurality of images includes a 3D image and two 2D images spatially associated with the 3D image. In such embodiments, the 3D image includes two annotations separately associated with each of the two 2D images, each of which provides a visible indication of a location of an anatomical region of interest from within a 3D view associated with a 2D view of the respective 2D image. Still further, in some embodiments, the 2D images each include annotations associated with each other. For example, an annotation in a first one of the 2D images (e.g., a 2D image providing a phased array visualization) provides a visible indication of a location of an anatomical region of interest from within a 2D view of the first one of the 2D images associated with a 2D view of the second one of the 2D images (e.g., a 2D image providing a circumferential radial visualization). Thus, an annotation in a second one of the 2D images provides a visible indication of a location of an anatomical region of interest from within a 2D view of the second one of the 2D images associated with the 2D view of the first one of the 2D images.

[0017] In some embodiments, the console may be configured to augment the images with one or more annotations based at least in part on user input using the system. For example, a user may be able to select a particular portion of a region of interest from which a 2D image should be reconstructed and further select whether the spatial relationship between the 2D and 3D images should be enhanced. In some embodiments, the system may be configured to provide the user with suggested views, such as suggested 2D views that correspond to anatomical landmarks of interest. For example, in a catheter-based application, a catheter may be positioned within a default home view within an anatomical region of interest (e.g., the right atrium), and relevant structures within this home view (e.g., the tricuspid valve, the mitral annulus, the fossa ovalis, etc.) may be automatically recognized and suggested via the system as 2D views to be displayed. The suggested 2D views may also correspond to typical standard views commonly presented in current 2D imaging systems, thereby providing an even more intuitive interaction and experience for the user.

[0018] An interactive interface operatively associated with the console may further allow the user to select the type of annotation to highlight relationships between images (i.e., select color, visibility, intensity, blur, or the like). By providing users with the ability to freely select the visual cues of their choices, the system of the present invention allows for more intuitive relationships and interpretations by the user. In particular, the system allows for a more customizable approach to providing visualization of anatomical regions of interest.

[0019] In some embodiments, the console is configured to receive the full-circumference 3D image data in real time or near real time, and the console is configured to reconstruct a plurality of images in real time or near real time based, at least in part, on user input and / or a predefined protocol.

[0020] In some embodiments, the ultrasound imaging device comprises a catheter-based ultrasound imaging device comprising a catheter including a rotatable ultrasound transducer array provided thereon configured to transmit ultrasound pulses to and receive echoes of the ultrasound pulses from intravascular and / or intracardiac tissue. Thus, the image data may be in the form of reflected signal data based on received echoes of the ultrasound pulses from the intravascular and / or intracardiac tissue. Still further, in some embodiments, the ultrasound imaging device comprises, for example, a transesophageal echocardiogram (TEE) probe. Thus, the 3D image data collected via the catheter-based or probe-based ultrasound imaging device may provide 360-degree visualization of the entire circumference of an anatomical region of interest.

[0021] It should be noted that in some embodiments, non-circumferential image data may be collected. For example, the systems and methods of the present invention may utilize a four-dimensional (4D) system (with a pyramidal field of view).

[0022] In some embodiments, the console is further configured to process the reflected signal data using one of a functional imaging algorithm and an anatomical imaging algorithm, extract associated functional and anatomical parameter data of the anatomical region of interest, reconstruct a plurality of images from the extracted functional and / or anatomical parameter data, and output the reconstructed plurality of images depicting a visualization of the anatomical region of interest to an operator via a display. The functional parameter data may include at least one of, but is not limited to, tissue perfusion, tissue motion, physiological motion, tissue stiffness or elasticity, tissue strain, tissue anisotropy, tissue coherence, certain statistical tissue parameters modeled by statistical distributions, tissue texture parameters, and spectral and frequency based parameters of tissue and fluid flow (i.e., blood flow and the like).

[0023] The functional parameter data may be indicative of a characterization of tissue in the anatomical region of interest, the anatomical parameter data including at least one of a spatial relationship and a geometric relationship of the tissue in the anatomical region of interest. The tissue characterization may include at least one of a tissue type, a tissue healthiness, a tissue depth, a lesion formation in the tissue as a result of the ablation procedure, and a lesion depth in the tissue. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a schematic diagram of a medical imaging system for providing reconstruction of 3D ultrasound image data into a combination of multiple views (2D and 3D images) of an anatomical region of interest associated with intravascular and / or intracardiac tissue.

[0025] [Diagram 2] FIG. 2 is a block diagram illustrating the exchange of data between an imaging device, a console unit, and a display of a medical imaging system consistent with this disclosure.

[0026] [Diagram 3] FIG. 3 is a block diagram illustrating one embodiment of a method for reconstruction of 3D ultrasound image data into a combination of multiple views (2D and 3D images) of an anatomical region of interest consistent with this disclosure.

[0027] [Figure 4] FIG. 4 shows an exemplary display of multiple views of an anatomical region of interest in accordance with the systems and methods of the present disclosure illustrating the presentation of a combination of a 3D image and two 2D images spatially related to the 3D image.

[0028] [Diagram 5] FIG. 5 shows a display of multiple views of an anatomical region of interest illustrating 2D and 3D images augmented with annotations that provide a visual indication of the location of the anatomical region of interest from which a 2D image is reconstructed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] (Detailed Description) In summary, the present invention is directed to systems and methods that provide reconstruction of three-dimensional (3D) ultrasound image data into a combination of multiple views of an anatomical region of interest associated with intravascular and / or intracardiac tissue, the multiple views including both 3D views and two-dimensional (2D) views that are spatially related to one another.

[0030] In particular, the system of the present invention provides a user (i.e., a clinician or equivalent) with an improved means for interacting with and utilizing ultrasound imaging technology to evaluate, diagnose, and treat various medical conditions. Aspects of the present invention may be accomplished using an imaging system configured to reconstruct 3D ultrasound image data, specifically 3D volumetric data, into a combination of multiple views of an anatomical region of interest, the multiple views including both 3D and 2D views that are spatially related to one another. The imaging system is configured to provide a user with seamless interaction with both 2D and 3D imaging views, where the user may select one or more 2D images to be dynamically reconstructed from the 3D volumetric data. The multiple 2D views are directly related to the 3D views, since the 2D views are digitally reconstructed from the volumetric data itself. The imaging system provides a user with an interactive interface where the user can specifically select a particular portion of the region of interest to be depicted in the 3D view from which the associated 2D image is to be reconstructed. Thus, the multiple views (2D and 3D views) relate to the same anatomical region of interest. The imaging system is further configured to annotate at least the 3D view to highlight those particular portions of the region of interest that have been reconstructed in the 2D view.

[0031] Thus, the system of the present invention provides the user with an intuitive understanding of the interrelationships of multiple views (i.e., both 2D and 3D views) and orientations within the acquired 3D data, thereby providing the benefits associated with each while overcoming limitations that plague current imaging systems (i.e., 2D and 3D US imaging systems). In particular, the relationships between the 2D and 3D views mimic current clinical imaging probes (e.g., linear / curved / phase / rotational) in both appearance and interaction. In particular, the user can steer and / or navigate the 2D views digitally (i.e., by interacting with the 3D volume via an interface) without requiring manual steering of the ultrasound probe, which can be cumbersome and error-prone.

[0032] It should be noted that the following description focuses on the use of the present invention for ultrasonic visualization of intravascular and / or intracardiac tissue, which may be particularly useful for catheter-based interventional procedures to assess anatomical and functional data in relation to a volume of interest. However, as will be generally understood, the systems and methods of the present invention may be used for ultrasonic visualization of any type of tissue for any type of procedure in which imaging analysis is used and / or preferred. For example, in one embodiment, the systems and methods of the present invention may be particularly useful for classifying catheter ablation procedures and the lesion formation associated therewith.

[0033] 1 is a schematic diagram of an exemplary medical imaging system 10. In the illustrated embodiment, the medical imaging system 10 is an ultrasound system and includes an imaging device 12 operably coupled to a console 14 and a display 16. As generally understood, ultrasound imaging (sonography) uses high frequency sound waves to view the inside of the body. Because ultrasound images are captured in real time, they can also show the movement of the body's internal organs and fluid flow (e.g., blood flowing through blood vessels). In sonography, an ultrasound device 12, also referred to as a transducer probe, is placed directly on the skin or inside a body orifice.

[0034] The ultrasonic transducer probe 12 is responsible for sending and receiving sound waves that produce ultrasonic images through the piezoelectric effect, a phenomenon that causes quartz crystals within the probe to rapidly vibrate and send out sound waves. These waves are then bounced off objects and reflected back to the probe.

[0035] It should be noted that in the present invention, the probe 12 may use any type of transducer to transmit and receive acoustic waves. For example, the probe 12 may include one or two dimensional arrays of electronic transducer elements to transmit and receive acoustic waves. These arrays may include microelectromechanical systems (MEMS) based transducers such as capacitive micromachined ultrasonic transducers (CMUTs) and / or piezoelectric micromachined ultrasonic transducers (PMUTs).

[0036] As is generally understood, CMUT devices can provide superior bandwidth and acoustic impedance characteristics, making them preferable over conventional piezoelectric transducers. Vibration of the CMUT membrane can be triggered by applying pressure (e.g., using ultrasound) or can be induced electrically. Often, electrical connections to the CMUT device using integrated circuits (ICs), such as application specific integrated circuits (ASICs), facilitate both the transmission and reception modes of the device. In the reception mode, changes in membrane position cause changes in capacitance that can be detected electrically, while in the transmission mode, applying an electrical signal causes vibration of the membrane.

[0037] Regarding PMUT devices, unlike bulk piezoelectric transducers that use thickness mode motion of a plate of piezoelectric ceramics such as PZT or single crystal PMN-PT, PMUT devices are based on the flexural motion of a thin membrane coupled with a thin piezoelectric film such as PVDF. In comparison to bulk piezoelectric ultrasonic transducers, PMUT devices can offer advantages such as increased bandwidth, flexible geometry, natural acoustic impedance matching with water, reduced voltage requirements, mixing of different resonant frequencies, and potential for integration with supporting electronics, especially for miniaturized high frequency applications.

[0038] The transducer probe 12 is generally operably coupled to a console 14, which controls the operation of the transducer probe 12 (i.e., the transmission of sound waves from the probe). The console 14 may generally include one or more processors (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both) and storage devices, such as a main memory, a static memory, or a combination of both, in communication with each other via a bus or the like. A memory according to the present invention may include a machine-readable medium on which one or more sets of instructions (e.g., software) are stored that embody any one or more of the methodologies or functions described herein. The software may also reside, completely or at least partially, in the main memory and / or in the processor during its execution by the computer system, and the main memory and the processor may also constitute a machine-readable medium. The software may further be transmitted or received over a network via a network interface device.

[0039] For example, in an exemplary embodiment, the console 14 may include a computing device generally configured to communicate across a network, including one or more processors and memory, and input / output mechanisms (i.e., keyboards, knobs, scroll wheels, or the like) with which an operator may interact to operate the machine, including making adjustments to the transmission characteristics of the probe, storing images, and performing other tasks described herein, including selecting particular regions of interest for subsequent reconstruction into 2D and / or 3D images.

[0040] During operation, the CPU and / or GPU may control the transmission and reception of electrical current and the subsequent resulting emission and reception of acoustic waves from the probe 12. The CPU and / or GPU also analyzes the electrical pulses that the probe generates in response to returning reflected waves and then converts this data into an image (i.e., an ultrasound image) that can then be viewed on the display 16, which may be an integrated monitor. Such images may also be stored in memory and / or printed via a printer (not shown).

[0041] In the illustrated embodiment, the imaging device 12 may generally be in the form of an imaging catheter capable of providing imaging and mapping capabilities. Such a device 12 may thus be useful for ultrasound visualization of intravascular and / or intracardiac tissue, which may be particularly useful for catheter-based interventional procedures to assess anatomical and functional data in relation to a volume of interest. As will generally be appreciated, the systems and methods of the present invention may be used for ultrasound visualization of any type of tissue for any type of procedure in which imaging analysis is used and / or preferred.

[0042] As an example, in one embodiment, the imaging device 12 may be useful in performing catheter ablation to treat cardiac disorders such as atrial fibrillation (AF) or the like. For example, in some embodiments, the catheter 12 may further include additional components that provide associated capabilities. For example, a portion of the catheter may include sensors (e.g., localization and / or tracking sensors) and / or energy delivery elements (e.g., ablation elements). Thus, the systems and methods of the present invention may be particularly useful for classifying catheter ablation procedures and lesion formations associated therewith. However, it should be noted that the systems and methods of the present invention may be useful for monitoring, diagnosing, and / or treating various disorders associated with targeted regions, and are not limited to intravascular and / or intracardiac tissues and disorders associated therewith.

[0043] The imaging catheter 12 may include a fully rotatable transducer unit consisting of an ultrasound transducer array configured to transmit ultrasound pulses to and receive echoes of the ultrasound pulses therefrom within the surrounding endovascular tissue during the procedure. Such ultrasound transmissions result in a collection of image data that is received by the console 14 and subsequently reconstructed into one or more images that provide visualization and characterization of the surrounding endovascular tissue. In particular, the console 14 may utilize image data received from the imaging assembly of the imaging catheter 12 to reconstruct one or more images, including at least 2D and 3D images, of the anatomical region of interest (i.e., endovascular and / or endocardial tissue). The console 14 may process the received image data utilizing certain imaging protocols and algorithms to reconstruct the images and subsequently output the reconstructed images to the operator via a display, depicting visualization of the anatomical region of interest. In addition to providing image reconstruction based on image data received from the imaging assembly, the console 14 may further provide control of the imaging assembly, including controlling the emission of ultrasonic pulses therefrom (intensity, frequency, duration, etc.) and controlling the movement of the ultrasonic transducer unit (i.e., controlling the rotation, including the speed and duration of rotation).

[0044] Figure 2 is a block diagram illustrating the exchange of data between the imaging device 12, the console unit 14, and the display 16 of the medical imaging system 10. Figure 3 is a block diagram illustrating one embodiment of a method for reconstruction of 3D ultrasound image data into a combination of multiple views (2D and 3D images) of an anatomical region of interest.

[0045] As shown, the console unit 14 is configured to receive 3D image data from the imaging device 12, process such data, and dynamically reconstruct a plurality of images. The 3D image data may include 3D image data in general, and specifically, 3D volumetric data captured by the imaging device 12 during an ultrasound procedure. The console 14 is configured to reconstruct such data into at least one 3D image providing a 3D view of an anatomical region of interest, and one or more corresponding 2D images providing a 2D view of the anatomical region of interest and spatially related to the 3D view. The console 14 may be equipped with certain hardware and software to provide such image reconstruction and imaging assembly control, as described in International PCT Application No. PCT / IB2019 / 000963 (published as WO 2020 / 044117) to Hennersperger et al., the contents of which are incorporated herein by reference in their entirety.

[0046] In turn, the console 14 is configured to provide a plurality of images (2D and 3D images) on the display 16 that present 2D and 3D views to the user, thereby depicting visualization of the anatomical region of interest.

[0047] The imaging system 10 provides a user with an interactive interface that allows the user to specifically select a particular portion of a region of interest to be depicted in a 3D view from which an associated 2D image is to be reconstructed, such that multiple views (2D and 3D views) relate to the same anatomical region of interest.

[0048] Thus, imaging system 10 is configured to provide a user with seamless interaction with both 2D and 3D imaging views, where the user may select one or more 2D images to be dynamically reconstructed from the 3D volume data. The multiple 2D views relate directly to the 3D view, since the 2D views are digitally reconstructed from the volume data itself.

[0049] Each of the multiple images may include at least one of a slice-based image and a volume-based image. For example, in one embodiment, at least one 3D image provides a 360-degree visualization of the anatomical region of interest. At least one 3D image may provide a trajectory volume visualization of the anatomical region of interest. In some embodiments, one or more 2D images provide at least one of a phased array visualization and a circumferential radial visualization of the anatomical region of interest. Thus, in some embodiments, a user may be presented with a 3D image (providing a trajectory volume visualization of the anatomical region of interest) and two or more 2D images (at least a first 2D image providing a phased array visualization of the anatomical region of interest, and at least a second 2D image providing a circumferential radial visualization of the anatomical region of interest). Note that in some embodiments, multiple 2D views may be generated and displayed (e.g., 4 or 8 phased array slices reconstructed at an angle).

[0050] It should be noted that in some embodiments, the system may be configured to provide persistent 3D image data. Additionally or alternatively, the system may be configured to provide high-resolution 2D imaging data that is spatially aligned with one direction of the 360-degree view. For example, the system may operate in an alternative configuration that generally allows high-resolution 2D imaging (both temporal and spatial) in a single direction.

[0051] 4 shows an exemplary embodiment of a display of multiple views of an anatomical region of interest according to the disclosed system and method, illustrating the combined presentation of a 3D image and two 2D images spatially related to the 3D image. As shown, the display 16 provides the user with a multi-view presentation of an imaginary line 3D volume visualization and corresponding planar view sections. The 2D planar views reconstructed using the present approach are shown as a phased array view (top left) and a circumferential 2D view (bottom left).

[0052] The imaging system 10 is further configured to annotate at least the 3D views to highlight those particular portions of the region of interest that are being reconstructed in the 2D views. For example, FIG. 5 shows an exemplary embodiment of a display of multiple views of an anatomical region of interest illustrating 2D and 3D images augmented with annotations that provide a visual indication of the location of the anatomical region of interest from which the 2D image is reconstructed. As illustrated in FIG. 5, each of the multiple images includes one or more annotations that provide a visual indication of the spatial relationship of one of the multiple images to another of the multiple images. The one or more annotations may include highlighting markings or the like.

[0053] In the illustrated embodiment, the 3D image includes two annotations separately associated with each of the two 2D images, each of which provides a visible indication of the location of the anatomical region of interest from within a 3D view associated with a 2D view of the respective 2D image. Similarly, each of the 2D images includes annotations associated with each other. For example, an annotation in a first one of the 2D images (e.g., a 2D image providing a phased array visualization) provides a visible indication of the location of the anatomical region of interest from within a 2D view of the first one of the 2D images associated with a 2D view of the second one of the 2D images (e.g., a 2D image providing a circumferential radial visualization). Similarly, an annotation in a second one of the 2D images provides a visible indication of the location of the anatomical region of interest from within a 2D view of the second one of the 2D images associated with the 2D view of the first one of the 2D images.

[0054] In some embodiments, the console 14 may be configured to augment the images with one or more annotations based at least in part on user input with the system. For example, a user may be able to select a particular portion of a region of interest from which a 2D image should be reconstructed, and further select whether the spatial relationship between the 2D and 3D images should be enhanced.

[0055] For example, an interactive interface operatively associated with the console is configured to allow the user to further select a type of annotation (i.e., select color, visibility, intensity, blur, or the like) to highlight the relationship between the images. Note that the annotation (i.e., highlight or the like) may be permanently visible or may be associated with a specific user input, such as a change of view, a mouse over a particular image region, or other heuristic that indicates that the highlight may be useful at a given moment. By providing the user with the ability to freely select the visual cues of their options, the system of the present invention allows for more intuitive relationships and interpretations by the user. The highlighting allows the user to make meaningful mental links between the 3D and 2D views of the anatomical region of interest, thereby allowing the user to intuitively work with the data while also providing significant added value by providing additional 3D views in real time or near real time while the data is being updated.

[0056] Furthermore, it should be noted that the 2D view can be reconstructed at any position from the 3D view. Additionally or alternatively, the 2D and 3D views can show the same but differently reconstructed information. In other words, both the 3D and 2D views may show different data reconstructed from the same raw data, or derived from the raw data, or data from other imaging modalities registered thereto, or any combination of these various data sets.

[0057] In some embodiments, the console 14 is configured to receive full-circumference 3D image data in real time or near real time, and the console is configured to reconstruct multiple images in real time or near real time based, at least in part, on user input and / or a predefined protocol.

[0058] Thus, the system provides the user with an intuitive understanding of the interrelationships of views and orientations within the acquired 3D data, thereby overcoming limitations that plague current imaging systems (i.e., 2D and 3D US imaging) while providing the benefits associated with each. In particular, the relationships between the 2D and 3D views mimic current clinical imaging probes (e.g., linear / curved / phase / rotational) in both appearance and interaction. In particular, the user can steer and / or navigate the 2D views digitally (i.e., by interacting with the 3D volume via an interface) without requiring manual steering of the ultrasound probe, which can be cumbersome and error-prone.

[0059] As used in any embodiment herein, the term "module" may refer to software, firmware, and / or circuitry configured to perform any of the aforementioned operations. Software may be embodied as a software package, code, instructions, instruction sets, and / or data recorded on a non-transitory computer-readable storage medium. Firmware may be embodied as code, instructions or instruction sets, and / or data hard-coded (e.g., non-volatile) in a memory device. "Circuitry," as used in any embodiment herein, may comprise, for example, wired circuitry, programmable circuitry such as a computer processor with one or more individual instruction processing cores, state machine circuitry, and / or firmware that stores instructions executed by the programmable circuitry, either alone or in any combination. Modules may be embodied as circuitry that collectively or individually forms part of a larger system, for example, an integrated circuit (IC), a system on a chip (SoC), a desktop computer, a laptop computer, a tablet computer, a server, a smartphone, etc.

[0060] Any of the operations described herein may be implemented in a system that includes one or more storage media having stored thereon instructions, individually or in combination, that are executed by one or more processors, where the processors may include, for example, server CPUs and / or GPUs, mobile device CPUs and / or GPUs, and / or other programmable circuitry.

[0061] It is also contemplated that the operations described herein may be distributed across multiple physical devices, such as processing structures, in more than one different physical location. The storage medium may include any type of tangible medium, such as hard disks, floppy disks, optical disks, compact disk read-only memories (CD-ROMs), rewritable compact disks (CD-RWs), and magneto-optical disks, semiconductor devices such as read-only memories (ROMs), random access memories (RAMs) such as dynamic and static RAMs, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), flash memories, solid state disks (SSDs), magnetic or optical cards, or any type of medium suitable for storing electronic instructions. Other embodiments may be implemented as software modules executed by a programmable control device. The storage medium may be non-transitory.

[0062] As described herein, various embodiments may be implemented using hardware elements, software elements, or any combination thereof. Examples of hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, application specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc.

[0063] References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in one embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0064] The term "non-transitory" should be understood to exclude only propagating, ephemeral signals, per se, from the scope of the claims, and does not disclaim all standard computer-readable media other than propagating, ephemeral signals, per se. In other words, the meaning of the terms "non-transitory computer-readable medium" and "non-transitory computer-readable storage medium" should be interpreted to exclude only those types of ephemeral computer-readable medium that were found in In Re Nuijten to fall outside the scope of patentable subject matter under 35 USC § 101.

[0065] The terms and expressions employed herein are used in a descriptive sense, rather than a limiting sense, and in the use of such terms and expressions there is no intention to exclude any equivalents of the features (or portions thereof) shown and described, it being recognized that various modifications are possible within the scope of the claims, and therefore the claims are intended to cover all such equivalents.

[0066] (Incorporated by reference) References and citations to other documents, such as patents, patent applications, patent publications, journals, books, articles, web content, etc. are made throughout this disclosure. All such documents are incorporated herein by reference in their entirety for all purposes.

[0067] (Equivalent) Various modifications of the invention and many further embodiments thereof, in addition to those shown and described herein, will become apparent to those skilled in the art from the entire contents of this document, including the scientific and patent references cited herein. The subject matter herein contains important information, exemplification, and guidance that can be adapted to the practice of this invention in its various embodiments and equivalents.

Claims

1. 1. An ultrasound imaging system comprising: a console configured to be operatively associated with an imaging device, the console comprising a hardware processor coupled to a non-transitory computer readable memory, the non-transitory computer readable memory including instructions that cause the console to: Receiving three-dimensional (3D) image data from an ultrasound imaging device; dynamically reconstructing a plurality of images from the 3D image data, the plurality of images including at least one 3D image providing a 3D view of an anatomical region of interest and one or more corresponding two-dimensional (2D) images providing 2D views of the anatomical region of interest and spatially related to the 3D view; 4. An ultrasound imaging system comprising:

2. The system of claim 1 , wherein each of the plurality of images comprises at least one of a slice-based image and a volume-based image.

3. The system of claim 1 , wherein the at least one 3D image provides a full 360 degree visualization of the anatomical region of interest.

4. The system of claim 3 , wherein the at least one 3D image provides a trajectory volume visualization of the anatomical region of interest.

5. The system of claim 1 , wherein the one or more 2D images provide at least one of a phased array visualization and a circumferential radial visualization of the anatomical region of interest.

6. The system of claim 5 , wherein the one or more 2D images provide multiple phased array views of the anatomical region of interest.

7. The system of claim 1 , wherein each of the plurality of images includes one or more annotations that provide a visual indication of a spatial relationship of one of the plurality of images to another of the plurality of images.

8. The system of claim 7 , wherein the plurality of images includes a 3D image and two or more 2D images spatially related to the 3D image.

9. 10. The system of claim 8, wherein the 3D image includes annotations separately associated with each of the two or more 2D images, each annotation providing a visible indication of a location of the anatomical region of interest from within the 3D view associated with a 2D view of a respective 2D image.

10. The system of claim 8 , wherein each of the 2D images includes annotations associated with each other.

11. 11. The system of claim 10, wherein the annotation in the first one of the 2D images provides a visible indication of a location of the anatomical region of interest from within a 2D view of the first one of the 2D images associated with a 2D view of the second one of the 2D images.

12. 12. The system of claim 11, wherein the annotation in the second one of the 2D images provides a visible indication of a location of the anatomical region of interest from within a 2D view of the second one of the 2D images that is associated with the 2D view of the first one of the 2D images.

13. The system of claim 7 , wherein the one or more annotations include emphasis markings.

14. The system of claim 13 , wherein the emphasized markings include a shade or color that has a contrasting appearance relative to a surrounding portion of the individual image to which the annotation is applied.

15. The system of claim 7 , wherein the console is configured to augment the plurality of images with the one or more annotations that are based, at least in part, on user input with the system.

16. 10. The system of claim 1, wherein the console is configured to receive the full-circumference 3D image data in real-time or near real-time, and the console is configured to reconstruct the plurality of images in real-time or near real-time based, at least in part, on user input and / or a predefined protocol.

17. 2. The system of claim 1, wherein the ultrasound imaging device comprises a catheter-based ultrasound imaging device comprising a catheter including a rotatable ultrasound transducer array provided thereon, the rotatable ultrasound transducer array configured to transmit ultrasound pulses to intravascular and / or intracardiac tissue and receive echoes of the ultrasound pulses therefrom.

18. 20. The system of claim 17, wherein the image data is in the form of reflected signal data based on received echoes of the ultrasound pulses from the intravascular and / or intracardiac tissue.

19. The console further comprises: processing the reflected signal data using one of a functional imaging algorithm and an anatomical imaging algorithm to extract associated functional and anatomical parameter data for the anatomical region of interest and reconstructing the plurality of images from the extracted functional and / or anatomical parameter data; outputting said reconstructed images depicting a visualization of the anatomical region of interest to an operator via a display; The system of claim 18 configured to:

20. 20. The system of claim 19, wherein the functional parameter data includes at least one of tissue perfusion, tissue motion, tissue stiffness or elasticity, tissue strain, tissue anisotropy, tissue coherence, certain statistical tissue parameters modeled by statistical distributions, texture parameters of the tissue, spectral and frequency based parameters of the tissue, and blood flow.

21. 21. The system of claim 20, wherein the functional parameter data is indicative of a characterization of tissue in the anatomical region of interest, the anatomical parameter data including at least one of spatial and geometric relationships of tissue in the anatomical region of interest.

22. 22. The system of claim 21, wherein the tissue characterization includes at least one of tissue type, tissue health, tissue depth, lesion formation in the tissue as a result of an ablation procedure, and lesion depth in the tissue.

23. 20. The system of claim 19, wherein the console is configured to output the reconstructed images via the display in response to user input with a user interface operatively associated with the console.

24. 20. The system of claim 19, wherein the console is configured to output, via the display, the reconstructed images based, at least in part, on invoking an algorithm that determines one or more suggested views to be displayed to the user based on the anatomical region of interest.

25. 25. The system of claim 24, wherein the algorithm comprises correlation of a targeted structure with one or more surrounding structures within the anatomical region of interest.

26. The system of claim 25 , wherein the one or more proposed views are associated with the one or more surrounding structures.

Images