Model-based automatic navigation system and method for ultrasound images

Abstract

This invention discloses a system and method that uses anatomical information provided via a detailed anatomical 3D model to generate 2D view bookmarks or shortcuts defined about the 3D model. These shortcuts utilize this information provided by the 3D model to determine the optimal location for acquiring 2D images of a specific desired view or structure based on the structure of the 3D anatomical model. Because these shortcuts are based on the structure of the 3D model, they are directly applicable to obtaining 3D / 2D images generated from the 3D dataset when the model is registered to a 3D volume generated from a 3D echocardiogram dataset for any patient. Furthermore, the 3D model assists the user in navigating the 3D dataset using the 3D volume and / or 2D planar views by providing known anatomical references within the 3D model that are correlated with references in the displayed views.

Description

Background Technology

[0001] The implementation schemes of the subject matter disclosed herein relate to medical imaging, and more specifically to the display of ultrasound imaging equipment to practitioners.

[0002] An ultrasound imaging system typically includes an ultrasound probe applied to a patient's body and a workstation or device operatively coupled to the probe. The probe can be controlled by the system operator and is configured to transmit and receive ultrasound signals / image data processed into ultrasound images by the workstation or device. The workstation or device can display or show the ultrasound images via a display device. For displaying ultrasound images, the ultrasound imaging system uses the image data / dataset acquired by the probe to generate multiple images of the structures being imaged by the probe.

[0003] In one example of these types of images, the image data / dataset is a three-dimensional (3D) image dataset that can be rendered into a 3D volume to generate images of multiple planes of the imaging structure presented to the user of the ultrasound imaging system. The process for generating these planar images involves multiplanar reconstruction (MPR) and allows for the efficient display of the 3D image dataset by providing multiple planes, typically three (3) orthogonal 2D planes associated with the 3D volume. The 3D volume 1000 is sliced ​​at different locations and has different orientations to form a 2D planar view or image 1002, which is presented together with the 3D volume 1000 in an MPR display 1004, such as... Figure 1 As shown.

[0004] To improve the speed of obtaining a desired view of a patient's anatomy, some ultrasound imaging systems provide shortcuts 1006 on the system's display 1008 for relating to a specific standard view that the patient will receive, such as... Figure 1 As shown. Shortcut 1006 provides the user with acquisition parameters, including the acquisition mode of the image dataset, such as 2D, 3D and / or 4D, with or without color flow (CF), depth, width, plane angle, etc., and provides the user with display parameters, including the layout on the screen, the orientation of the 3D data, the position of the clipping plane, etc., to achieve the selected image associated with shortcut 1006. This feature allows the user to quickly select the desired view from the menu or list 1010 of shortcut 1006 presented on the system's user interface / display 1008, and enables the system to present the required probe position and angle for that view.

[0005] One drawback of these types of systems and methods is that all stored parameters for the provided shortcut 1006 are linked to the standardized probe position of the associated view, rather than to the anatomical structures of interest within the patient. For example, when the "Inter-Com" shortcut 1006 (representing the inter-commissural view) is selected on screen 1008, the system sets the acquired probe angle to 60°, which is the "standard" or average across all patients. However, this average or standard angle / setting may not be optimal for the anatomy of the current patient. Therefore, the user will have to manipulate the probe to the correct angle to obtain the desired image via the ultrasound imaging system. Furthermore, the user will have to correct the parameters associated with that shortcut 1006 on the system for a specific view to accommodate the difference between the current patient's anatomy and the standard for all patients. Additionally, this correction must be applied to the parameters associated with each individual shortcut 1006 of the system for all other views to best match the current patient's anatomy and cardiac orientation. This is a cumbersome process and significantly increases the time required to obtain the desired image dataset for individual patients.

[0006] Furthermore, while the ability to display combined 3D and 2D views provides enhanced information about the structure being imaged, it is often difficult, especially for inexperienced users, to easily identify the anatomical structure being imaged using only the 3D and 2D images presented by the imaging system, making it difficult to properly position the probe to obtain the best 2D view of the structure of interest.

[0007] Therefore, it is desirable to develop an imaging system and method to improve visualization and navigation within 3D and 2D images provided in an MPR display that can associate parameters with various shortcuts to a specific image view based on an anatomical model of the patient's anatomy, rather than based on probe position, for obtaining the image view. Summary of the Invention

[0008] In this disclosure, an ultrasound imaging system utilizes MPR to combine 3D and 2D images provided from an image dataset with an aligned 3D model representing the anatomical structures in the images for presentation. The system and method provide automatic and optional real-time alignment and co-orientation (i.e., in motion or in movie view) of the 3D anatomical model adjacent to one or more 2D MPR views of a 3D volume / image and / or a 3D echocardiogram dataset. The 3D model is registered to a 3D volume created from the 3D echocardiogram dataset such that the 3D model is aligned with each of the displayed 3D volume and 2D images to provide the user with a visual representation of the orientation of each of the 3D volume and 2D planar views relative to the 3D model and relative to each other.

[0009] Each 2D planar view is represented in the 3D model to clearly identify the orientation of the 2D planar view relative to the anatomical structure being imaged. When viewing each 2D planar view and its associated 3D model, the user can then immediately and intuitively understand / determine the orientation of the currently displayed data, which may be real-time or retrieved from an associated electronic storage location.

[0010] Additionally, the 3D model assists users in navigating 3D datasets using 3D volumetric and / or 2D planar views by providing known anatomical references that are correlated with references in the displayed views. Due to the registration between the 3D model and the 3D volume, the location of the structure of interest within the 3D model can be easily determined, and this location can be used to instruct the user that the probe must be navigated and / or positioned appropriately to obtain the desired 2D view of the structure.

[0011] Furthermore, the system and method utilize anatomical information provided by a detailed anatomical 3D model to generate view bookmarks or shortcuts defined by the 3D anatomical model. These shortcuts leverage the information provided by the anatomical model to determine the optimal location of 2D images for a specific desired MPR view or structure based on the structure of the 3D anatomical model. Because the shortcuts are based on the structure of the 3D model, they are directly applicable to 3D and 2D cross-sections or MPR images generated from the 3D dataset when the model is registered to a 3D volume generated from a 3D echocardiogram dataset for a patient. Some of these shortcuts are pre-defined by the system, i.e., for the factory default view, but users can also create additional shortcuts based on personal preferences referencing the 3D anatomical model, with each shortcut stored in the system. Moreover, because these shortcuts are based on the anatomical structure of the 3D model, they can be used with any 3D volume subsequently registered to the 3D model, whether it is a 3D volume from the same patient or a different patient.

[0012] When a user interacts with the system to obtain a desired 2D MPR view, and when the user selects different shortcuts or alternatively interacts with the 3D model to identify different 2D views for display, the 3D model can shift according to the desired orientation or view change. As a result of registration with the 3D model, the system can demonstrate changes in the 2D view from previous shortcut 2D views to the current or selected 2D view through deformation motion. In this way, the system and method provide the user with accurate information about the orientation of the 2D image using the associated 3D model for various shortcut 2D views.

[0013] According to one aspect of this disclosure, an ultrasound imaging system includes: a probe for acquiring a 3D image dataset on a structure to be imaged; a processor operatively connected to the probe and configured to generate a 3D volume and at least one 2D image from the 3D image dataset; and a display operatively connected to the processor to present the 3D volume and the at least one 2D image on the display, wherein the processor is configured to register a 3D anatomical model stored within the ultrasound imaging system to the 3D volume and display a selected 2D image contained within the 3D volume corresponding to a selected plane within the 3D anatomical model.

[0014] According to another aspect of this disclosure, a method for displaying 2D ultrasound images using an ultrasound imaging system includes the steps of: obtaining a 3D image dataset using the ultrasound imaging system; generating a 3D volume from the 3D image dataset; registering a 3D anatomical model stored in the ultrasound imaging system to the 3D volume; and displaying at least one 2D image contained within the 3D volume corresponding to a selected plane within the 3D anatomical model.

[0015] It should be understood that the above brief description is provided to introduce selected concepts further described in the detailed embodiments in a simplified form. This is not intended to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims following the detailed embodiments. Furthermore, the claimed subject matter is not limited to embodiments that address any shortcomings mentioned above or in any part of this disclosure. Attached Figure Description

[0016] The invention will be better understood by referring to the following description of non-limiting embodiments, in which:

[0017] Figure 1 This is an illustration of a prior art display for multiplanar reconstructed views used in ultrasound imaging systems.

[0018] Figure 2 This is a schematic diagram of an ultrasound imaging system according to one implementation scheme.

[0019] Figure 3 This is an illustration of an exemplary display having a multi-plane reconstructed view of an aligned 3D model according to one embodiment.

[0020] Figure 4 This is an illustration of a method for registering and aligning a 3D model with a multi-plane reconstructed view based on an implementation scheme.

[0021] Figure 5 This is an illustration of an exemplary display screen according to one implementation, which includes shortcut selection associated with various standard and user-defined 2D views.

[0022] Figure 6 This is a schematic diagram of an exemplary display showing a 2D plan view and a 3D model according to one embodiment, wherein the 3D model has representations of the current 2D view and the selected 2D view.

[0023] Figure 7 It is a diagram illustrating the deformation motion of the 2D view and 3D model when switching between selected 2D views according to an implementation scheme.

[0024] Figure 8 This is a schematic diagram of an exemplary display that includes anatomical orientation information in a 2D view and a 3D model. Detailed Implementation

[0025] The foregoing summary of the invention and the following detailed description of certain embodiments of the invention will be better understood when read in conjunction with the accompanying drawings. As used herein, elements or steps described in the singular and beginning with the words “a” or “an” should be understood to not exclude a plurality of said elements or steps unless such exclusion is expressly stated. Furthermore, references to “an embodiment” of the invention are not intended to be construed as excluding the existence of additional embodiments that also include the referenced features. Moreover, unless expressly stated to the contrary, embodiments that “comprise” or “have” elements or multiple elements having a particular property may include other such elements that do not have that property.

[0026] The following description relates to various implementation schemes for ultrasound imaging. Specifically, systems and methods for visualizing cross-cutting information on ultrasound images using a three-dimensional (3D) model are provided. Three-dimensional (3D) ultrasound datasets can be acquired using ultrasound imaging systems, such as... Figure 2 The ultrasound imaging system described herein is similar to the ultrasound imaging system disclosed in U.S. Patent Application Publication No. US2018 / 0344290 entitled "Systems And Methods For Displaying Intersections On Ultrasound Images," the entire contents of which are expressly incorporated herein by reference for all purposes. The three-dimensional ultrasound dataset can be used to create 3D volumes and / or one or more 2D planar slices via multiplanar reconstruction (MPR) for presentation on a display of an ultrasound imaging system associated with a 3D model of the anatomical structure being imaged. For example, a graphical user interface (such as...) Figure 3 and Figure 4 The graphical user interface (GUI) depicted herein may include a visualization of a 3D model for each MPR 3D / 2D image, wherein information about the orientation of the image relative to the 3D model is displayed on the GUI, thereby enabling the user to easily navigate the 3D volume.

[0027] It should be understood that although this article describes various implementation schemes for ultrasound imaging, the methods described herein can be implemented using other 3D imaging modalities, including but not limited to computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), single-photon emission computed tomography (SPECT), etc.

[0028] Figure 2 This is a schematic diagram of an ultrasound imaging system 100 according to one embodiment of the present invention. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drive a transducer element 104 within a probe 106 to transmit pulsed ultrasound signals into the body (not shown). Various probe and transducer geometries can be used. The pulsed ultrasound signal is backscattered from a body structure such as blood cells or muscle tissue to generate an echo returning to element 104. The echo is converted into an electrical signal or ultrasound data by element 104, and the electrical signal is received by a receiver 108. The electrical signal representing the received echo passes through a receive beamformer 110 that outputs ultrasound data. According to some embodiments, the probe 106 may include electronic circuitry to perform all or part of the transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be located within the probe 106. In this disclosure, the terms "scanning" or "under scanning" can also be used to refer to the process of acquiring data by transmitting and receiving ultrasound signals. In this disclosure, the term "data" can be used to refer to one or more datasets acquired using an ultrasound system.

[0029] User interface 115 can be used to control the operation of ultrasound imaging system 100, including controlling the input of patient data, changing scan or display parameters, etc. User interface 115 may include a graphical user interface configured for display on display device 118. The graphical user interface may include information to be output to the user (such as ultrasound images, patient data, etc.), and may also include menus or other elements through which the user can input into the computing system. (See below for more information.) Figures 2 to 4 In a more detailed example, the user interface may receive input from the user instructing, for example, to adjust the position of the plane to be imaged. The user interface 115 may include one or more of the following: a rotatable device, a mouse, a keyboard, a trackball, a touch-sensitive display of the display device 118, hard keys linked to specific actions, soft keys configurable to control different functions, and a graphical user interface.

[0030] The ultrasound imaging system 100 also includes a processor 116 for controlling the transmitting beamformer 101, the transmitter 102, the receiver 108, and the receiving beamformer 110. The processor 116 communicates electronically with the probe 106. For the purposes of this disclosure, the term "electronic communication" may be defined to include both wired and wireless communication. The processor 116 controls the probe 106 to acquire data. The processor 116 controls which of the elements 104 are active and the shape of the beam emitted from the probe 106. The processor 116 also communicates electronically with a display device 118 and can process the data into an image for display on the display device 118. According to one embodiment, the processor 116 may include a central processing unit (CPU). According to other embodiments, the processor 116 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 may include multiple electronic components capable of performing processing functions. For example, processor 116 may include two or more electronic components selected from a list of electronic components, including: a central processing unit, a digital signal processor, a field-programmable gate array, and a graphics board. According to another embodiment, processor 116 may also include a composite demodulator (not shown) that demodulates RF data and generates raw data. In yet another embodiment, demodulation may be performed earlier in the processing chain.

[0031] Processor 116 is adapted to perform one or more processing operations based on multiple selectable ultrasound modalities on the data. Data can be processed in real time during a scanning session as echo signals are received. For the purposes of this disclosure, the term "real time" is defined as including processes performed without any intentional delay. For example, one embodiment is capable of acquiring images at a real-time rate of 7 to 20 volumes per second. Ultrasound imaging system 100 is capable of acquiring 2D data of one or more planes at a significantly faster rate. However, it should be understood that the real-time volume rate can depend on the length of time spent acquiring each volume of data used for display. Therefore, the real-time volume rate may be slower when acquiring relatively large volumes of data. Thus, some embodiments may have a real-time volume rate significantly faster than 20 volumes per second, while other embodiments may have a real-time volume rate slower than 7 volumes per second. Data may be temporarily stored in a buffer (not shown) during a scanning session and processed in a less real-time manner in real-time or offline operations. Some embodiments of the invention may include multiple processors (not shown) to handle the processing tasks handled by processor 116 according to the exemplary embodiments described above. For example, a first processor can be used to demodulate and extract RF signals, while a second processor can be used to further process the data before displaying the image. It should be understood that other embodiments may use different processor arrangements.

[0032] The ultrasound imaging system 100 is capable of continuously acquiring data at volumetric rates, for example, 10 Hz to 30 Hz. It can refresh images generated from a display at a similar frame rate. Other embodiments are capable of acquiring and displaying data at different rates. For example, depending on the volume size and intended application, some embodiments are capable of acquiring data at volumetric rates less than 10 Hz or greater than 30 Hz. A memory 120 is included for storing the volume of processed acquired data. In one exemplary embodiment, the memory 120 has sufficient capacity to store at least several seconds of ultrasound data volume. The data volume is stored in a manner that facilitates retrieval based on its acquisition order or time. The memory 120 may include any known data storage medium.

[0033] Optionally, contrast agents can be used to implement embodiments of the present invention. When an ultrasound contrast agent, including microbubbles, is used, contrast imaging generates enhanced images of in vivo anatomical structures and blood flow. After data acquisition using a contrast agent, image analysis includes separating harmonic and linear components, enhancing harmonic components, and generating ultrasound images by utilizing the enhanced harmonic components. A suitable filter is used to perform the separation of harmonic components from the received signal. Ultrasound imaging using contrast agents is well known to those skilled in the art and will therefore not be described in detail.

[0034] In various embodiments of the invention, processor 116 may process data through other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules may generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, and combinations thereof, etc. Image beams and / or volumes are stored, and timing information indicating the time of data acquisition in the memory may be recorded. These modules may include, for example, a scan conversion module for performing scan conversion operations to convert image volumes from beam space coordinates to display space coordinates. A video processor module may be provided that reads image volumes from the memory and displays the images in real time during surgery on a patient. The video processor module may store images in memory 120, read from and display images from the memory.

[0035] See now Figure 3When the ultrasound imaging system 100 is operated to image a patient's anatomical structures 200 (such as the anatomical structures of the heart 202 obtained during echocardiography), the probe 106 acquires a 3D image dataset that is transmitted to the processor 116 for image generation. In doing so, the processor 116 creates a reconstructed 3D volume, image, or rendering 204 of the structure 200 from the 3D image dataset. Additionally, using a multiplanar reconstruction (MPR) process, the processor 116 generates one or more 2D planar views 206 through predetermined and / or selected slices of the 3D volume / image 204. The 2D planar images 206, combined with the 3D volume / image 204, are presented on a display 118, which... Figure 3 An exemplary implementation shows three (3) 2D planar views 206 associated with the 3D volume / image 204, each set in a separate window or frame 218 of the display 118 and identified as indicating a particular view represented by the 2D planar view 206 being displayed, which may be a standardized view and / or a user-selected view, or a combination thereof.

[0036] With the creation of 3D volumetric / image 204 and 2D planar image 206, processor 116 has access to a stored 3D anatomical model 208 of the structure being imaged for display in combination with one or more 2D planar views 206 and optionally 3D volumetric / image 204. In the method for generating the 3D anatomical model 208 to be displayed, such as... Figure 4As best shown in the exemplary embodiment, in block 300, processor 116 initially registers a geometric model 210, which typically represents the shape of the imaging structure 200, to 3D volume / image 204 using known image segmentation methods, such as those disclosed in J. Hansegard, F. Orderud, and SIRabben's "Real-time Active Shape Model for Segmentation of 3D Echocardiography" in *Computer Analysis of Images and Patterns* (Berlin: Springer, 2007, Vol. 4673, LNCS, pp. 157-164) and those disclosed in U.S. Patent No. 8,265,363 entitled *Method And Apparatus For Automatically Identifying Image Views In A3D Dataset*, the full text of which is expressly incorporated herein by reference for all purposes, such that the structure and orientation of the geometric model 210 correspond to those shown in the 3D volume / image 204. Subsequently, in box 302, processor 116 performs registration between the 3D anatomical model 208 of the imaging structure 200 and the 3D volume / image 204. In this step, registration is computed by applying the known geometric transformation matrix between the anatomical model 208 and the geometric model 210 to the transformation matrix caused by segmentation, so that the structure of the anatomical model 208 is correlated with the structure of the 3D volume / image 204, such that the orientation, structure, and points of the 3D anatomical model 208 are known and correspond in the 3D coordinate system to those orientations, structures, and points presented in the 3D volume / image 204. Once the 3D anatomical model 208 is registered to the 3D volume / image 204, in box 304, the 3D model 208 can be oriented and displayed in association with the 3D volume / image 204 and each 2D planar view / image 206, where the orientation of each 3D model 208 corresponds to the orientation of the specific image 204 and the specific image 206 associated with the individual 3D model 208.

[0037] Regarding the registration of the 3D anatomical model 208 to the 3D volume / image 204, in an alternative embodiment, the 3D anatomical model 208 may also be a deformable 3D anatomical model 208. In this embodiment, the 3D image dataset used to form the 3D volume / image 204 may be applied to the 3D anatomical model 208 and / or the geometric model 210 to deform or distort the structure of the 3D anatomical model 208 and / or the geometric model 210 to more specifically conform to the actual shape of the anatomical structure 200 of the patient being imaged.

[0038] After registration and optional deformation of the 3D anatomical model 208 to the 3D volumetric / image 204, the model 208 can be presented on the display 118 in combination with one or more of the 3D volumetric / image 204 and the 2D planar MPR image 206, such as Figure 3 and Figure 4 As shown.

[0039] Regarding the presentation of model 208 on monitor 118, in Figure 3 In the exemplary embodiment shown, model 208 is presented within one of a plurality of frames 218 on display 118. This model is associated with each 3D volume / image in 3D volume / image 204 and a plurality of 2D planar images 206 on display 118, optionally presented at a scale suitable for illustrating the structure of model 208 without diminishing the information provided by each image in images 204 and 206. However, in an alternative embodiment, model 208 may be displayed with any combination of images 204 and 206, such as with each of one or more 2D planar images 206 presented solely on display 118. Each 2D planar image in 206 is associated with a specific default normalized and / or selected view, or a combination thereof, through the imaging structure 200 to provide information about the condition of the imaging structure 200. When the 3D model 208 is registered with images 204 and 206 in 3D coordinate space, the rendering of the 3D model 208 for each image is oriented to be viewed / displayed by / contains the plane 212 of the 3D volume / image 204 or the 2D planar view 206. Therefore, the plane 212 of each corresponding image 204 or image 206 rendered on the display 118 is identified within the associated 3D model 208 by an icon 213 of any suitable shape and / or color (such as a shaded square 214 aligned with the plane 212 of each image 204 or image 206). To align the square 214 with the associated image plane 212, the 3D model 208 is rotated / oriented to place the square 214 within the plane of the display 118 displaying the images 204 and 206. In this way, various images 204, 206, and model 208 are presented to the user on display 118, thereby showing the plane 212 (as represented by the square 214 within model 208) for a particular image 204, image 206, and the orientation of image 204, image 206 relative to model 208. This significantly helps in understanding the position of the 2D planar image 206 on structure 200, as well as any necessary navigation to the desired area of ​​interest within imaging structure 200.

[0040] In another exemplary embodiment, the display 118 presents separate views of a 2D planar view 206 and a 3D model 208 obtained in real time by the user positioning the probe 106 on the patient. The separate view of the 3D model may also optionally be presented on the display 118 or may be used only as an internal reference for the system 100 / processor 116.

[0041] In an implementation where the real-time registered 3D model 208 is presented on the display 118 in conjunction with the 2D view 206 during acquisition, the 3D model 208 provides the user with an understanding of the current scanned anatomy shown in the 2D view 206 in relation to the 3D model 208. This can provide useful information during scanning in 3D or 4D, especially for inexperienced users, as the user can orient the probe 106 by viewing the 3D model 208 and the associated 2D image 206 to achieve optimal acquisition of the desired 2D image 206. In addition to changing the 2D view 206 by moving the probe 106, the user can also interact directly with the 3D model 208 (or 2D image 206) to change and / or determine the 2D image 206 being displayed. For example, the user can use the user interface 115 to click on the 3D model 208 on the display 118, such as zooming in on the model 208 and the 2D image 206 to rotate the 3D model 208 and the 2D image 206, and draw slices on the 3D model 208 to identify the 2D planar view 206 to be displayed.

[0042] The view appendage of the 3D model 208 includes a plane or slice 214 represented in the model 208, which corresponds to the plane of the 2D image 206 presented in conjunction with the 3D model 208, as a result of the registration between the previously described 3D volume 204 and the 3D model 208. In the 3D anatomical model 208, the location and view of interest of all features of interest are known. Therefore, during the real-time acquisition of the 3D volume / image 204 or when analyzing the recorded 3D volume / image 204, due to the registration of the 3D volume / image 204 with the 3D model 208, the location and / or orientation of these features of interest and the view of interest are also known, and optionally, the location of the probe 106 is also known based on the correspondence between the image data / imaging field from the probe 106 and the registered 3D volume 204 and the 3D model 208. Furthermore, by interacting with the 3D model 208, the user can change the orientation of the 3D model 208 to any desired view, such as... Figure 3 As shown, plane 214 is represented by an orientation corresponding to the following: the position of plane 214 is in the selected orientation of 3D model 208.

[0043] Therefore, in such Figure 5In one exemplary embodiment shown, a user can select a shortcut from a list or array 400 of shortcuts 402 stored in memory 120, and the system 100 provides the shortcut to the user in an appropriate manner to navigate to a specific view of interest / 2D plane view 206 of the 3D volume 204. Each bookmark or shortcut 402 corresponds to a standard or user-defined 2D view of the imaging structure 200 used to evaluate and / or diagnose any conditions present in the structure 200. Shortcut 402 is associated with a 2D plane 212 within the 3D model 208, having imaging parameters for the 2D plane 212, such as depth, gain, angle, and the position and orientation of the 2D plane 212 in the 3D volume 208, for example, stored in the memory 120 associated with the shortcut 402. As in the prior art, all imaging parameters for the 2D plane 212 associated with the shortcut 402 to the selected 2D view 206 are determined relative to the 3D model 208 and not relative to the position of the patient-related probe 106. Therefore, when a user selects a specific shortcut 402, the 2D image 206 displayed from the 3D volume 204 is associated with the imaging parameters of the 2D plane 212, which is related to the structure of the 3D model 208 registered to the 3D volume 204.

[0044] Regardless of whether the 3D volume 204 registered to the 3D model 208 is acquired in real time or from an electronic storage device or database (e.g., memory 120, which is operatively connected to system 100), processor 116 can easily display the orientation of the 2D view 206 associated with the selected shortcut 402 by accessing the image parameters in memory 120 for selecting shortcut 402 / 2D view 206 and showing the slice 214 on the 3D model 208 corresponding to the position of the selected 2D view 206 on the 3D model 208. If the 2D image / view 206 is acquired from the stored 3D volume 204, processor 116 can simply present the selected 2D view 206 along the slice (not shown) of the 3D volume 204 corresponding to / registered to the slice 214 of the 3D model 208.

[0045] If a 2D image / view 206 is being acquired in real time, the processor 116 can render the 2D view 206 along slices (not shown) of the 3D volume 204 corresponding to / registered to slice 214 of the 3D model 208, as is done with the stored 3D volume 204. Alternatively, as Figure 6As shown, system 100 / processor 116 may provide the user with indication of the appropriate position of probe 106 to obtain the desired 2D image / view 206 represented by slice 214. To provide this indication using real-time 3D volume 204 to 3D model 208 registration, processor 116 may use any suitable navigation instructions, such as optional indication 406 of the actual or current position / imaging plane of probe 106 on 3D model 208 and separate indication 408 of the desired or selected position / imaging plane of probe 106 to obtain the selected 2D image 206, and optional navigation icons 411, such as arrows, indicating the direction of movement required for probe 106 to move to the desired position of probe 106 for the selected 2D image 206. In this way, processor 116 can guide or navigate the user to the appropriate position of probe 106 to match the desired position of slice 214 corresponding to the desired 2D view represented by shortcut 402.

[0046] For example, a “perfect” 3-cavity view can be well defined in the 3D anatomical model 208. After registering the real-time or stored 3D volume / image 204 to the 3D model 208, the user can select a shortcut 402 associated with the 3-cavity view, and the processor 116 can render a 2D image 206 associated with the image or view parameters of the 3-cavity view on the 3D model 208. However, based on the registration applied to the 3D volume / image 204, and / or the processor 116 can render the position of the probe 106, the user needs to move the probe 106 to provide the desired 3-cavity view. Furthermore, since the reconstructed 3D volume / image 204 and 3D model 208 are continuously and automatically registered in the background by the processor 116 during the real-time acquisition of the 3D dataset of the 3D volume / image 204, the user can access any view of interest using shortcut 402 to provide navigation direction for the position of the probe 106 during the real-time acquisition process for any selected shortcut 402. Furthermore, when the user moves / rotates the probe 106 and the corresponding 2D image / view 206 displayed on the display 118 while acquiring real-time image data, the 3D model 208 will move / rotate in sync with the movement of the displayed 2D image 206.

[0047] According to another exemplary implementation, see again Figure 5In addition to the predetermined or standard 2D planar view 206 provided by system 100, the user can create new shortcuts 404 associated with views defined using 3D model 208 but for images selected by the user. For example, during an intervention, in addition to the standard 2D view 206 provided by shortcut 402, it is desired to obtain certain 2D views 206 from the 3D volume / image 204 obtained during the procedure, and the user can place probe 106 at a position corresponding to the desired 2D view 206. System 100 then enables the user to define a new (shortcut addition function 403) specific shortcut 404 based on the registration between 3D model 208 and 3D volume 204, which corresponds to the position of the 2D image / view 206 in 3D model 208. In addition to the parameters for standard shortcut 402, the shortcut 404 and the corresponding imaging parameters associated with the 3D model 208 for the 2D view 206 associated with shortcut 404 are stored in memory 120 by system 100. Because the parameters of the user-selected 2D view 206 represented by shortcut 404 are related to the 3D model 208 stored in the memory 120 of system 100, shortcut 404 can also be used to obtain the same 2D view 206 from other 3D volumes 204 of other patients that have been registered to the 3D model 208 stored in the memory 120 of system 100. Furthermore, each of the user-defined shortcuts 404 can be deleted from memory 120 and from the 3D model 208 (shortcut deletion function 409) and / or reconfigured or edited in the 3D model 208 stored by the user in the memory 120 of system 100 (shortcut editing function 407). Additionally, user-defined shortcuts 404 can be combined in a single shortcut 405 (such as for multiple desired 2D views 206 associated with a specific intervention and / or diagnostic procedure). Then, when the user loads a new 3D volume 204 from storage or real-time acquisition into system 100, once the anatomical model 208 is automatically registered with the 3D volume 204, the user can automatically acquire all 2D images 206 associated with shortcut 405 in the 3D model 208. The user does this by selecting combination shortcut 405 so that processor 116 can access the image parameters for each 2D image 206 associated with combination shortcut 405 in the 3D model 208 stored in memory 120, and subsequently display each 2D image 206 in the registered 3D volume 204 corresponding to the stored location / imaging parameters for combination shortcut 405.

[0048] View now Figure 7As an exemplary implementation, and as a further benefit of the correlation between the stored image parameters of shortcuts 402 and 404 and the anatomical structure of the 3D model 208, the exact position of the 2D view 206 of each shortcut 402 and 404 relative to each other is also known. Therefore, when a user transitions between different 2D views 206 associated with different shortcuts 402 and 404, such as when a user selects a series of shortcuts 402 and 404 to display the associated 2D image 206, the currently displayed 2D image 206 can be presented as if the displayed image is slowly moving or “morphing”. In this morphing movement shown on display 118, there is an animated transition on display 118 from the position and orientation of the current 2D image 206 to the position and orientation of the selected 2D image 206. The movement or transition of one or more intervening 2D images 206 located or set between the current 2D image 206 and the selected 2D image 206 helps the user understand the spatial and anatomical relationship between the two 2D views 206. The deformation movement of the displayed 2D image 206 may optionally be illustrated in conjunction with a diagram of the movement, for example, arrow 410 on the 3D anatomical model 208 indicating the direction of movement between the 2D views 206 on the 3D model 208, or rotation of the 3D model 208 or rotation of the plane 214 in the 3D model 208 that indicates the position of the plane of the displayed 2D image 206, or a combination thereof.

[0049] See now Figure 8The exemplary embodiment shown utilizes the readily known structures of the stored 3D anatomical model 208, through registration between the 3D model 208 and the 3D volume 204. Any structure or feature 500 of interest, such as the left atrial appendage, mitral valve center, right ventricle, etc., and any orientation 502 of interest, such as the left side of the patient, the right side of the patient, the anterior side, the posterior side, etc., can be readily defined in the 3D model 208 and shown in the 3D volume 204 and / or the 2D planar image 206. After the 3D anatomical model 208 has been registered to the patient 3D volume / image 204, the features 500 of interest and the orientation 502 of interest can be displayed in the 3D model 208 together with the 3D volume 204 and / or the 2D planar image 206, thereby providing the user with the anatomical orientation of the acquired images 204 and 206. Feature 500 and orientation 502 can be displayed in a highlighted form (such as by providing an outline 504 in images 204 and 206 indicating the location and / or orientation 502 of the selected feature 500). Alternatively, or in combination with the outline 504, color coding (not shown) can be used to perform marking in one or more of models 208 and / or images 204 and 206, for example, showing the left ventricle as red, the right ventricle as blue, etc., as readable text and / or automatic markings (not shown) located within the associated 2D planar image 206, and / or as arrows indicating orientation 502 in the space surrounding images 204 and 206, such as left side of the patient, right side of the patient, etc.

[0050] In an alternative implementation, when acquiring a 3D dataset in real time, it is made possible to render a 3D volume 204 and one or more 2D planar images 206 in motion (e.g., in a movie view). Registering a 3D model 208 to the 3D volume 204 allows the 3D model 208 to be rendered as a 4D model 208 in motion, along with the 3D volume 204 being rendered as a 4D volume 204 and the 2D planar view / image 206 being rendered as a 3D planar image. The motion of the 3D model 208 can be determined as one or more averages of the positions of different structures in the 3D volume 204 and / or the average of the motions of those structures.

[0051] This written description uses examples to disclose the invention, including the best mode, and also enables those skilled in the art to practice the invention, including making and using any device or system and performing any included methods. The scope of the invention is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A method for displaying 2D multiplanar reconstructed ultrasound images using an ultrasound imaging system, the method comprising the following steps: - Obtain a 3D image dataset using the ultrasound imaging system described above; - Generate a 3D volume from the 3D image dataset; - Registering the 3D anatomical model stored in the ultrasound imaging system to the 3D volume; and - Displays a 2D multiplanar reconstructed image containing selected planes within the 3D volume corresponding to the 3D anatomical model; and - Identify the selected plane in the 3D anatomical model. The step of identifying the selected plane includes selecting a shortcut on the ultrasound imaging system associated with the selected plane in the 3D anatomical model. The method further includes the step of defining the shortcut by storing imaging parameters for a user-defined 2D plane associated with the shortcut in the 3D anatomical model.

2. The method of claim 1, wherein the step of defining the shortcut comprises: - Position the imaging probe at a certain location to obtain the 2D multiplanar reconstructed image within the 3D volume; as well as - Storing in memory imaging parameters related to the position of the 2D multiplanar reconstructed image relative to the 3D anatomical model for the 2D multiplanar reconstructed image.

3. The method according to claim 2, wherein the imaging parameters include probe image depth, probe image gain, probe angle, position and orientation of the 2D multiplane reconstructed image within the 3D volume, and combinations thereof.

4. The method of claim 1, wherein the step of identifying the selected plane in the 3D anatomical model further comprises: - Display the 3D anatomical model; as well as - Showing the selected plane in the 3D anatomical model corresponding to the 2D multiplanar reconstructed image.

5. The method of claim 4, wherein the step of showing the selected plane in the 3D anatomical model comprises: - A first plane is shown that corresponds to the position of a first 2D multiplanar reconstructed image defined by the current position of a probe operatively connected to the ultrasound imaging system; as well as - A second plane is shown that corresponds to the location of a second 2D multiplanar reconstructed image defined by the shortcut on the ultrasound imaging system associated with the selected plane in the 3D anatomical model.

6. The method of claim 5, further comprising the step of displaying a navigation icon between the first plane and the second plane.

7. The method of claim 1, wherein the step of registering the 3D anatomical model to the 3D volume comprises: - Register the geometric model to the 3D volume; as well as - Register the geometric model to the 3D anatomical model.

8. A method for displaying 2D multiplanar reconstructed ultrasound images using an ultrasound imaging system, the method comprising the following steps: - Obtain a 3D image dataset using the ultrasound imaging system described above; - Generate a 3D volume from the 3D image dataset; - Registering the 3D anatomical model stored in the ultrasound imaging system to the 3D volume; and - Displays a 2D multiplanar reconstructed image containing selected planes within the 3D volume, corresponding to the 3D anatomical model. The step of displaying the 2D multiplanar reconstructed image containing a selected plane within the 3D volume corresponding to the 3D anatomical model includes: - Displays a first 2D multiplanar reconstructed image of the 3D volume corresponding to a first selected plane within the 3D anatomical model; - Select a second 2D multiplanar reconstructed image within the 3D volume corresponding to a second plane within the 3D anatomical model; and - The first 2D multiplane reconstructed image is deformed into the second 2D multiplane reconstructed image by means of the 3D volume.

9. The method of claim 8, further comprising providing an illustration of the movement from the first 2D multiplanar reconstructed image to the second 2D multiplanar reconstructed image on the 3D anatomical model.

10. The method of claim 9, wherein the step of providing the illustration of the movement from the first 2D multiplanar reconstructed image to the second 2D multiplanar reconstructed image includes providing an arrow on the 3D anatomical model in the direction of movement of the 2D multiplanar reconstructed image.

11. The method of claim 10, wherein the step of providing the illustration of the movement from the first 2D multiplanar reconstructed image to the second 2D multiplanar reconstructed image includes rotating the 3D anatomical model.

12. An ultrasound imaging system, the ultrasound imaging system comprising: - A probe used to obtain a 3D image dataset on the structure to be imaged; - A processor operatively connected to the probe and configured to generate 3D volumetric and one or more 2D multiplanar reconstructed images from the 3D image dataset; - A memory operatively connected to the processor and containing information about the 3D anatomical model and imaging parameters for one or more selected 2D planes within the 3D anatomical model; and - A display operatively connected to the processor to render the 3D volume and the one or more 2D multiplanar reconstructed images on the display. The processor is configured to register the 3D anatomical model to the 3D volume and display a selected 2D multiplanar reconstructed image contained within the 3D volume, corresponding to the imaging parameters used for the selected 2D plane within the 3D anatomical model. The processor is configured to receive imaging parameters of a user-defined 2D multiplanar reconstructed image within the 3D volume, and to store the imaging parameters of the selected 2D plane in the 3D anatomical model corresponding to the user-defined 2D multiplanar reconstructed image as a user-defined shortcut in the memory.

13. The system of claim 12, wherein the processor is configured to display a plurality of shortcuts, each associated with a selected 2D plane of the 3D anatomical model, receive an instruction for selection of a shortcut, and display the 2D multiplanar reconstructed image contained within the 3D volume corresponding to the imaging parameters for the selected shortcut.

14. The system of claim 12, wherein the processor is configured to identify features of interest in the 3D anatomical model and display a selected 2D multiplanar reconstructed image containing imaging parameters corresponding to the selected 2D plane within the 3D volume, using indications of the features of interest in the selected 2D multiplanar reconstructed image.

15. An ultrasound imaging system, the ultrasound imaging system comprising: - A probe used to obtain a 3D image dataset on the structure to be imaged; - A processor operatively connected to the probe and configured to generate 3D volumetric and one or more 2D multiplanar reconstructed images from the 3D image dataset; - A memory operatively connected to the processor and containing information about the 3D anatomical model and imaging parameters for one or more selected 2D planes within the 3D anatomical model; and - A display operatively connected to the processor to render the 3D volume and the one or more 2D multiplanar reconstructed images on the display. The processor is configured to register the 3D anatomical model to the 3D volume and display a selected 2D multiplanar reconstructed image contained within the 3D volume, corresponding to the imaging parameters used for the selected 2D plane within the 3D anatomical model. The processor is configured to display a first 2D multiplanar reconstruction image within the 3D volume corresponding to a first selected 2D plane within the 3D anatomical model, determine the position of a second 2D multiplanar reconstruction image within the 3D volume corresponding to a second selected 2D plane within the 3D anatomical model, and deform the first 2D multiplanar reconstruction image into the second 2D multiplanar reconstruction image through the 3D volume.

16. The system of claim 15, wherein the processor is configured to display a diagram of movement from the first 2D multiplanar reconstructed image to the second 2D multiplanar reconstructed image on the 3D anatomical model.

Images