Systems and methods for anatomically aligned multiplanar reformatted views for ultrasound imaging

Abstract

This invention provides a system and method for the automatic alignment and co-orientation of a 3D anatomical model adjacent to one or more 2D MPR views of a 3D echocardiographic image dataset. The 3D model is presented in alignment with the displayed 3D and 2D images to provide an indication of the orientation of each of the 3D volumetric and 2D planar views relative to the 3D model and relative to each other. The 3D model may include labels or information about anatomical features of interest to enable the user to easily visualize the arrangement of anatomical features in the 2D planar views relative to the 3D model. When interacting with either the 2D view or the 3D model, both the 2D view and the 3D anatomical model simultaneously change orientation, wherein the representation of the 2D planes in the 3D model corresponds to movement of the 2D image.

Description

Systems and methods for anatomically aligned multiplanar reconstructed views for ultrasound imaging Background Technology

[0001] The embodiments of the subject matter disclosed herein relate to medical imaging, and more specifically to the display of ultrasound imaging equipment to practicing physicians.

[0002] An ultrasound imaging system typically includes an ultrasound probe applied to a patient's body and a workstation or device operatively coupled to the probe. The probe is controlled by the system operator and configured to transmit and receive ultrasound signals / image data processed into ultrasound images by the workstation or device. The workstation or device can display the ultrasound images via a display device. For displaying ultrasound images, the ultrasound imaging system uses the image data / dataset acquired by the probe to generate multiple images of the structures imaged by the probe.

[0003] In one example of these types of images, the image data / dataset is a three-dimensional (3D) image dataset that can be rendered into a 3D volume to generate images of multiple planes of an imaging structure presented to a user of an ultrasound imaging system. The process for generating these planar images involves multiplanar reconstruction (MPR) and allows for efficient display of the 3D image dataset by providing multiple planes associated with the 3D volume, typically three (3) orthogonal 2D planes. As shown in Figure 1, the 3D volume 1000 is sliced ​​at different locations with different orientations to form a 2D planar view or image 1002 presented together with the 3D volume 1000 in an MPR display 1004.

[0004] However, when using MPR, whether in 3D static images or 4D echocardiographic images, in order to navigate and observe the desired region of the imaging structure of interest, with the 3D volume and planar image presented on the monitor in a motion manner, it is often difficult to understand the position and orientation of the 2D planar slice relative to the 3D volume and / or the patient's actual anatomical structures.

[0005] Therefore, it is desirable to develop an imaging system and method for improving the visualization and navigation of 3D and 2D images presented in MPR displays. Summary of the Invention

[0006] In this disclosure, 3D and 2D images provided by an ultrasound imaging system utilizing MPR are presented together with aligned 3D models of the anatomical structures represented in the images. The system and method provide automatic and optional real-time (i.e., in motion or in cinematic view) alignment and co-orientation of a 3D anatomical model adjacent to one or more 2D DPR views of a 3D echocardiographic dataset. The 3D model is presented aligned with each of the displayed 3D and 2D images to provide the user with an indication of the orientation of each of the 3D volumetric and 2D planar views relative to the 3D model and relative to each other. Each 2D planar view is represented in the 3D model to clearly identify the orientation of the 2D planar view relative to the imaged anatomical structure. When viewing each 2D planar view and the associated 3D model, the user can then immediately and intuitively understand / determine the orientation of the currently displayed data, which may be real-time or retrieved from an associated electronic storage location. Furthermore, the 3D model assists the user in navigating the 3D dataset using 3D volumetric and / or 2D planar views by providing known anatomical references associated with the displayed views.

[0007] This system and method utilize anatomical information provided via a 3D image dataset to create a detailed anatomical 3D model to assist user navigation within 3D echocardiographic data. The display includes a 3D model presented aligned with the MPR view, where the planes of the 2D MPR view are represented within the 3D model. The 3D model may additionally include labels or information about anatomical features of interest to allow the user to easily visualize the arrangement of the 2D planar view relative to the anatomical features identified in the 3D model. The 3D model may also include information related to other aspects of the image, such as probe representation, to assist the user in orienting the probe to obtain improved or optimal images of the structures of interest.

[0008] When interacting with the MPR view, as the user manipulates the MPR view, both the MPR view and the 3D anatomical model simultaneously change orientation, either through rotation between the 3D and 2D images, or through the representation of the 2D planes in the 3D model corresponding to movement of the 2D image. Additionally, when the user interacts with the 3D model, the corresponding 3D or 2D image will shift, for example, through translation and / or rotation, depending on changes in the orientation of the 3D model or the position and orientation of the 2D plane view. In this way, the system and method provide the user with accurate information about the orientation of the 2D image using the associated 3D model.

[0009] According to one aspect of this disclosure, an ultrasound imaging system includes: a probe for acquiring a 3D image dataset of a structure to be imaged; a processor operatively connected to the probe and configured to generate a 3D volume and at least one 2D image from the 3D image dataset; and a display operatively connected to the processor to present the 3D volume and at least one 2D image on the display, wherein the processor is configured to register a 3D anatomical model with the 3D volume and display the 3D anatomical model aligned with at least one 2D image.

[0010] According to another aspect of this disclosure, a method for displaying a three-dimensional (3D) anatomical model aligned with an ultrasound image provided by an ultrasound imaging system is provided, the method comprising the steps of: obtaining a 3D image dataset using an ultrasound imaging system; generating a 3D volume and at least one two-dimensional (2D) image from the 3D image dataset; registering the 3D anatomical model with the 3D volume; and displaying the 3D anatomical model aligned with at least one 2D image.

[0011] It should be understood that the above brief description is provided to introduce selected concepts further described in the detailed embodiments in a simplified form. This is not intended to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims following the detailed embodiments. Furthermore, the claimed subject matter is not limited to embodiments that address any shortcomings mentioned above or in any part of this disclosure. Attached Figure Description

[0012] The invention will be better understood by referring to the following description of non-limiting embodiments, in which:

[0013] Figure 1 is an illustration of a prior art display for a multiplanar reconstruction view of an ultrasound imaging system.

[0014] Figure 2 is a schematic diagram of an ultrasound imaging system according to one embodiment.

[0015] Figure 3 is an illustration of an exemplary display of a multi-plane reconstructed view of an aligned 3D model according to one embodiment.

[0016] Figure 4 is an illustration of a method for registering and aligning a 3D model with a multi-planar reconstructed view according to an implementation scheme.

[0017] Figure 5 is an illustration of an exemplary display of a multi-plane reconstructed view of a 3D model with alignment and markings according to one embodiment.

[0018] Figure 6 is an illustration of an exemplary display of a multi-plane reconstructed view of a static 3D model according to another embodiment. Detailed Implementation

[0019] The foregoing summary of the invention and the following detailed description of certain embodiments of the invention will be better understood when read in conjunction with the accompanying drawings. As used herein, elements or steps described in the singular and beginning with the words “a” or “an” should be understood to not exclude a plurality of said elements or steps unless such exclusion is expressly stated. Furthermore, references to “an embodiment” of the invention are not intended to be construed as excluding the existence of additional embodiments that also include the referenced features. Moreover, unless expressly stated to the contrary, embodiments that “comprise” or “have” elements or multiple elements having a particular property may include other such elements that do not have that property.

[0020] The following description relates to various implementations of ultrasound imaging. Specifically, systems and methods are provided for visualizing cross-sectional information on ultrasound images using a three-dimensional (3D) model. Ultrasound imaging systems, such as the one depicted in Figure 2, can be used to acquire three-dimensional (3D) ultrasound datasets. This ultrasound imaging system is similar to the one disclosed in U.S. Patent Application Publication US2018 / 0344290, entitled “Systems and Methods For Displaying Intersections On Ultrasound Images,” the entire contents of which are expressly incorporated herein by reference for all purposes. This patent application can be used to create 3D volumes and / or one or more 2D planar slices via multiplanar reconstruction (MPR) for presentation on a display of an ultrasound imaging system associated with a 3D model of the imaged anatomy. For example, a graphical user interface, such as those depicted in Figures 3, 4, and 6, may include a visualization of a 3D model of each MPR 3D / 2D image, displaying information about the orientation of the image relative to the 3D model, thereby enabling the user to easily navigate the 3D volume. Methods for creating 3D models from 3D ultrasound datasets (such as those depicted in Figure 5) may include generating a 3D model that has been modified to fit the specific anatomy of the patient being imaged in order to provide a more accurate representation of the 3D model.

[0021] It should be understood that although this article describes various implementation schemes in conjunction with ultrasound imaging, the methods described herein can also be implemented using other 3D imaging modalities, including but not limited to computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), single-photon emission computed tomography (SPECT), etc.

[0022] Figure 2 is a schematic diagram of an ultrasound imaging system 100 according to one embodiment of the present invention. The ultrasound imaging system 100 includes a transmit beamformer 101 and a transmitter 102 that drive a transducer element 104 within a probe 106 to transmit pulsed ultrasound signals into the body (not shown). Various geometries of probes and transducer elements can be used. The pulsed ultrasound signal is backscattered from a body structure such as blood cells or muscle tissue to generate an echo returning to element 104. The echo is converted by element 104 into an electrical signal or ultrasound data, and the electrical signal is received by a receiver 108. The electrical signal representing the received echo passes through a receive beamformer 110 that outputs ultrasound data. According to some embodiments, the probe 106 may include electronic circuitry to perform all or part of the transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 101, transmitter 102, receiver 108, and receive beamformer 110 may be located within the probe 106. In this disclosure, the terms "scanning" or "under scanning" may also be used to refer to the process of acquiring data by transmitting and receiving ultrasound signals. In this disclosure, the term "data" may be used to refer to one or more datasets acquired using an ultrasound system.

[0023] User interface 115 can be used to control the operation of ultrasound imaging system 100, including controlling the input of patient data, changing scan or display parameters, etc. User interface 115 may include a graphical user interface configured for display on display device 118. The graphical user interface may include information to be output to the user (such as ultrasound images, patient data, etc.) and may also include menus or other elements through which the user inputs content into the computing system. In the example described in more detail below in conjunction with Figures 2 to 4, the user interface may receive input from the user indicating, for example, adjustments to the position of the plane to be imaged. User interface 115 may include one or more of the following: a rotary knob, a mouse, a keyboard, a trackball, a touch-sensitive display that may also be connected to display device 118, hard keys linked to specific actions, soft keys that can be configured to control different functions, and the graphical user interface.

[0024] The ultrasound imaging system 100 also includes a processor 116 for controlling the transmitting beamformer 101, the transmitter 102, the receiver 108, and the receiving beamformer 110. The processor 116 communicates electronically with the probe 106. For the purposes of this disclosure, the term "electronic communication" may be defined to include both wired and wireless communication. The processor 116 controls the probe 106 to acquire data. The processor 116 controls which of the elements 104 are active and the shape of the beam emitted from the probe 106. The processor 116 also communicates electronically with a display device 118, and the processor 116 can process the data into an image for display on the display device 118. According to one embodiment, the processor 116 may include a central processing unit (CPU). According to other embodiments, the processor 116 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 116 may include multiple electronic components capable of performing processing functions. For example, processor 116 may include two or more electronic components selected from a list of electronic components, including: a central processing unit, a digital signal processor, a field-programmable gate array, and a graphics board. According to another embodiment, processor 116 may also include a composite demodulator (not shown) that demodulates RF data and generates raw data. In yet another embodiment, demodulation may be performed earlier in the processing chain.

[0025] Processor 116 is adapted to perform one or more processing operations based on multiple selectable ultrasound modalities on the data. Data can be processed in real time during a scanning session as echo signals are received. For the purposes of this disclosure, the term "real time" is defined as including procedures executed without any intentional delay. For example, one embodiment is capable of acquiring images at a real-time rate of 7 to 20 volumes per second. Ultrasound imaging system 100 is capable of acquiring 2D data of one or more planes at significantly faster rates. However, it should be understood that the real-time volume rate can depend on the length of time spent acquiring each volume of data used for display. Therefore, the real-time volume rate may be slower when acquiring relatively large volumes of data. Thus, some embodiments may have a real-time volume rate significantly faster than 20 volumes per second, while other embodiments may have a real-time volume rate slower than 7 volumes per second. Data may be temporarily stored in a buffer (not shown) during a scanning session and processed in a less real-time manner in real-time or offline operations. Some embodiments of the invention may include multiple processors (not shown) to handle processing tasks processed by processor 116 according to the exemplary embodiments described above. For example, a first processor can be used to demodulate and extract RF signals, while a second processor can be used to further process the data before displaying the image. It should be understood that other embodiments may use different processor arrangements.

[0026] The ultrasound imaging system 100 is capable of continuously acquiring data at volumetric rates, for example, 10 Hz to 30 Hz. Images generated from the display are refreshed at similar frame rates. Other embodiments are capable of acquiring and displaying data at different rates. For example, depending on the volume size and intended application, some embodiments are capable of acquiring data at volumetric rates less than 10 Hz or greater than 30 Hz. A memory 120 is included for storing the volume of processed acquired data. In one exemplary embodiment, the memory 120 has sufficient capacity to store at least several seconds of ultrasound data volume. The data volume is stored in a manner that facilitates retrieval based on its acquisition order or time. The memory 120 may include any known data storage medium.

[0027] Optionally, contrast agents can be used to implement embodiments of the present invention. When an ultrasound contrast agent, including microbubbles, is used, contrast imaging generates enhanced images of in vivo anatomical structures and blood flow. After data acquisition using a contrast agent, image analysis includes separating harmonic and linear components, enhancing harmonic components, and generating ultrasound images by utilizing the enhanced harmonic components. A suitable filter is used to perform the separation of harmonic components from the received signal. Ultrasound imaging using contrast agents is well known to those skilled in the art and will therefore not be described in detail.

[0028] In various embodiments of the invention, processor 116 may process data through other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules may generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastography, TVI, strain, strain rate, and combinations thereof. Image beams and / or volumes are stored, and timing information indicating the time of data acquisition in the memory may be recorded. These modules may include, for example, a scan conversion module for performing scan conversion operations to convert image volumes from beam space coordinates to display space coordinates. A video processor module may be provided that reads image volumes from the memory and displays the images in real time during surgery on a patient. The video processor module may store images in memory 120, read from and display images from the memory.

[0029] Referring now to Figure 3, when the ultrasound imaging system 100 is operated to image a patient's anatomical structures 200, such as the heart 202 obtained during echocardiography, the probe 106 acquires a 3D image dataset, which is transmitted to the processor 116 for image generation. In doing so, the processor 116 creates a reconstructed 3D volume / image or render 204 of the structure 200. Furthermore, using a multiplanar reconstruction (MPR) process, processor 116 generates one or more 2D planar views 206 of predetermined and / or selected slices from 3D volume / image 204. The 2D planar images 206 are presented together with the 3D volume / image 204 on display 118, which, in the exemplary embodiment of FIG3, shows three (3) 2D planar views 206 associated with the 3D volume / image 204. Each 2D planar view is set in a separate window or frame 218 of display 118 and is identified to indicate a specific view represented by the displayed 2D planar views 206, which may be a standardized view and / or a user-selected view or a combination thereof.

[0030] With the creation of 3D volumetric / image 204 and 2D planar image 206, processor 116 generates a 3D anatomical model 208 of the imaged structure for display together with one or more 2D planar views 206 and optional 3D volumetric / image 204. In the method for generating the 3D anatomical model 208, most clearly shown in the exemplary embodiment of FIG4, in block 300, processor 116 initially uses a known image segmentation method to register a geometric model 210, which generally represents the form of the imaging structure 200, with a 3D volume / image 204, such that the structure and orientation of the geometric model 210 correspond to the structure and orientation shown in the 3D volume / image 204. This known image segmentation method includes those disclosed in J. Hansegard, F. Orderud, and SIRabben, “Real-time active shape models for segmentation of 3d cardiac ultrasound” in Computer Analysis of Images and Patterns. Berlin, Germany: Springer, 2007, Vol. 4673, LNCS, pp. 157-164, and Orderud et al., entitled “Method And Apparatus For Automatically Identifying Image Views In A 3D Dataset,” the entire contents of which are expressly incorporated herein by reference for all purposes. Subsequently, in box 302, processor 116 performs registration between the 3D anatomical model 208 of the imaging structure 200 and the 3D volume / image 204. In this step, registration is computed by applying a known geometric transformation matrix between the anatomical model 208 and the geometric model 210 to the transformation matrix generated by segmentation, so that the structure of the anatomical model 208 is associated with the structure of the 3D volume / image 204, such that the orientation, structure, and points of the 3D anatomical model 208 are known and correspond in the 3D coordinate system to those present in the 3D volume / image 204. Once the 3D anatomical model 208 is registered with the 3D volume / image 204, in box 304, the 3D model 208 can be oriented and displayed in association with the 3D volume / image 204 and each 2D planar view / image 206, wherein the orientation of each 3D model 208 corresponds to the orientation of the specific image 204, 206 associated with the individual 3D model 208.

[0031] Regarding the registration of the 3D anatomical model 208 with the 3D volume / image 204, in an alternative embodiment, the 3D anatomical model 208 may also be a deformable 3D anatomical model 208. In this embodiment, the 3D image dataset used to form the 3D volume / image 204 may be applied to the 3D anatomical model 208 and / or the geometric model 210 to deform or distort the structure of the 3D anatomical model 208 and / or the geometric model 210 to more specifically conform to the actual form of the anatomical structure 200 of the imaged patient.

[0032] After registration and optional deformation of the 3D anatomical model 208 with the 3D volume / image 204, the model 208 can be presented on the display 118 together with one or more of the 3D volume / image 204 and the 2D planar MPR image 206, as shown in Figures 3 to 6.

[0033] Regarding the presentation of model 208 on display 118, in the exemplary embodiment shown in FIG3, model 208 is presented on display 118 in association with each of 3D volumetric / image 204 and 2D planar image 206, optionally at a scale suitable for showing the structure of model 208 without diminishing the information provided by each of the images 204, 206. However, in alternative embodiments, model 208 may be displayed with any combination of images 204, 206, such as only with each of the plurality of 2D planar images 206 presented on display 118. Each of the 2D planar images 206 is associated with a particular default, normalized, and / or selected view or combination thereof through imaging structure 200 in order to provide information about the condition of imaging structure 200. When the 3D model 208 is registered with images 204, 206 in 3D coordinate space, the rendering of the 3D model 208 for each image is oriented to a plane 212 that is viewed / displayed by the 3D volume / image 204 or contains the 2D planar view 206. Therefore, the plane 212 of each corresponding image 204, 206 rendered on the display 118 is identified within the associated 3D model 208 by an icon 213 of any suitable shape and / or color, such as a shaded square 214 aligned with the plane 212 of each image 204, 206. To align the square 214 with the associated image plane 212, the 3D model 208 is rotated / oriented to place the square 214 within the plane of the display 118 on which the images 204, 206 are rendered. In this way, various images 204, 206 and model 208 are presented to the user on display 118, showing the plane 212 of a particular image 204, 206 (represented by a square 214 within model 208) and the orientation of images 204, 206 relative to model 208. This significantly helps in understanding the position of the 2D planar image 206 on structure 200, and any necessary navigation to the desired region of interest within imaging structure 200.

[0034] In an implementation that acquires and displays in real time 3D image datasets for creating images 204, 206 and model 208 to provide a four-dimensional (4D) volume 204 including motion, a user can use information provided by model 208 to reposition / navigate probe 106 relative to structure 200 to shift one or more 2D planar images in the presented 2D planar images 206, thereby obtaining an improved or optimal acquisition or presentation of features of structure 200 to be imaged within one or more 2D planar images in 2D planar images 206.

[0035] In the exemplary embodiment shown, model 208 is positioned adjacent to and partially overlaps with associated images 204, 206, but may be located at any desired position within frame 218 containing associated images 204, 206, or at a position on display 118 separate from images 204, 206. Model 208 may be a full 3D representation of model 208 displayed adjacent to associated images 204, 206, where icons 213 / squares 214 indicate the positions of associated 204, 206 within model 208, as previously described, or it may be displayed as a slice or segment of model 208 along plane 212 of images 204, 206 within model 208, where portions of model 208 positioned between the observer and icons 213 / squares 214 are removed, allowing the observer to see the internal structure of model 208 corresponding to associated images 204, 206 for a clearer view of the structures shown in images 204, 206. Sliced ​​model 208 may also be shown with or without icons 213 / squares 214.

[0036] Furthermore, model 208 is interactive, allowing a user to define a desired position for the associated 2D planar image 206 within model 208 by shifting (e.g., moving, translating, and / or rotating) model 208 or the plane 214 represented on model 208 to different orientations of the associated 2D planar image 206. When model 208 or plane 214 is shifted (e.g., moved, translated, and / or rotated), in an exemplary embodiment, the 2D planar image 206 will be modified accordingly to conform to the image of the plane 214 shown in association with model 208; that is, one of model 208 or plane 214 will move, but the other of model 208 or plane / square 214 will remain stationary, wherein the 2D planar image 206 is modified to maintain alignment with the position of the square / plane 214 within model 208. Conversely, the user can shift (e.g., move, translate, and / or rotate) the 2D planar image 206 to cause the 2D planar image 206 presented on the display 118 to shift, and the orientation of the model 208 will change according to the changed position of the square / plane 214 within the model 208. Alternatively, when the image 206 is shifted by the user, the model 208 may remain stationary, and the position of the square 214 of the plane 212 representing the image 206 may be moved to correspond to the new orientation of the image 206 relative to the model 208.

[0037] In other exemplary embodiments, a user can interact with model 208 by drawing lines (not shown) on model 208, which represent slices of structure 200 / model 208 that are expected to be shown in or as an associated 2D planar image 206. When these lines are created by the user, model 208 is reoriented, i.e., translated and / or rotated to align plane 212 of the desired 2D planar image 206 defined by the line with square 214, wherein the 2D planar image 206 representing the desired plane 212 is displayed in association with the reoriented model 208.

[0038] Furthermore, while in some exemplary embodiments, a change to the plane 212 of the 3D volume / image 204 or the 2D planar image 206 (e.g., by rotating the model 208 and / or drawing or moving the image 206 thereon) may only change the orientation of the associated model 208, in other embodiments, the movement or change of the 3D volume / image 204 or a 2D planar image 206 or its associated model 208 will correspondingly passively change the orientation of each remaining image 204, 206 and the associated model 208 to maintain correspondence with the actively shifted image 204, 206.

[0039] In other exemplary embodiments, this aspect can also be applied to the movement of a cursor 220 presented on display 118 and used in association with user interface 115. When a user interacts with an image 204, 206 and / or a pattern 208 located in a specific window / quadrant / frame 218 of display 118, the cursor 220 in that frame 218 is presented in a first color (e.g., white) to indicate the location where the user is actively interacting with display 118 via interface 115. Other passive cursors 222 are simultaneously displayed in other frames 218 of display 118. When the user moves the active cursor 220 relative to an image 204 / 206 or pattern 208 in a relevant frame 218, as a result of registration between images 204, 206 and pattern 208, the passive cursors 222 also move accordingly over their respective images 204, 206 and / or pattern 208 to indicate to the user where any feature selected by the active cursor 220 in one frame 218 is located by one or more passive cursors 222 in other frames 218.

[0040] Additionally, if it is desired to highlight a specific anatomical feature 224 of structure 200 (e.g., left atrial appendage, mitral valve center, right ventricle, etc.) in images 204, 206, that feature 224 can be selected in model 208. Because the 3D anatomical model 208 is registered with the 3D volumetric / image 204 and the 2D planar image 206, the region 226 of images 204, 206 corresponding to the selected feature 224 of model 208 can be highlighted, for example, by providing a contour 228 in images 204, 206 indicating the location and alignment of the selected feature 224. Alternatively, or in conjunction with the contour 228, color coding (not shown) can be used to perform labeling, for example, indicating the left ventricle in red and the right ventricle in blue within the 3D model 208, and / or as readable text (not shown) located within the associated 2D planar image 206.

[0041] Referring now to the exemplary embodiment shown in Figure 5, the known specific relationship between the 2D planar image 206 and the 3D anatomical model 208, resulting from the performed registration, can also be used to indicate the position of the probe 106 and the imaging field 107 together with the anatomical model 208, thereby showing the user the direction of acquisition and further facilitating the user's navigation of the probe 106 to obtain the desired images 204, 206 for viewing. Furthermore, when the probe 106 is moved to change the displayed images 204, 206, the 3D model 208 rotates in association with the movement of the probe 106 to maintain alignment of the model 208 with the displayed images 204, 206.

[0042] Referring now to the exemplary embodiment shown in FIG6, instead of displaying the 3D model 208 to orient the square 214 to align with the currently displayed 2D planar MPR image / view 206, as described with respect to the previous embodiments, the model 208 for each image / view 204, 206 may be positioned on the display 118 in each frame 218 at a default view (e.g., from the front, from the side, etc.), wherein the position and orientation of the square 214 representing the plane 212 of the associated 2D planar view / image 206 on the 3D model 208 varies relative to the orientation of the plane 212 relative to the 3D model 208. Furthermore, depending on the specific orientation of the plane 212 of the 2D planar view / image 206, the representation of the plane 212 on the 3D model 208 may be shown as a square 214 or a line in space (not shown).

[0043] In an alternative implementation, when acquiring a 3D dataset in real time, the 3D volume 204 and one or more 2D planar images 206 are rendered in a motion-like manner. For example, in a movie view, registration of the 3D model 208 to the 3D volume 204 allows the 3D model 208 to be rendered as a 4D model 208 along with the 3D volume 204 (4D volume 204) and the 2D planar views / images 206 (3D planar images) in a motion-like manner. The motion of the 3D model 208 can be determined as one or more averages of the positions of different structures in the 3D volume 204 and / or the average of the motions of these structures.

[0044] This written description uses examples to disclose the invention, including the best mode, and also enables those skilled in the art to practice the invention, including making and using any device or system and performing any included methods. The scope of the invention is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A method for displaying and aligning a three-dimensional (3D) anatomical model with ultrasound images provided by an ultrasound imaging system, the method comprising the steps of: - Obtaining a 3D image dataset, wherein the step of obtaining the 3D image dataset includes operating the ultrasound imaging system to obtain the 3D image dataset; - Generating a 3D volume and at least one two-dimensional (2D) image from the 3D image dataset; - Registering a 3D anatomical model with the 3D volume, wherein the step of registering the 3D anatomical model with the 3D volume includes: registering a geometric model with the 3D volume; and applying a geometric transformation matrix between the geometric model and the 3D anatomical model to register the geometric model with the 3D anatomical model; and - Displaying the 3D anatomical model aligned with the at least one 2D image, wherein the step of displaying the 3D anatomical model aligned with the at least one 2D image includes: selecting a structure of interest in the 3D anatomical model; and identifying the structure in the at least one 2D image.

2. The method of claim 1, wherein the step of displaying the 3D anatomical model aligned with the at least one 2D image comprises: The 3D anatomical model shows a slice along the location of the at least one 2D image in the 3D anatomical model.

3. The method of claim 1, wherein the step of displaying the 3D anatomical model aligned with the at least one 2D image comprises: The location of the at least one 2D image in the 3D anatomical model is indicated by an icon.

4. The method according to claim 3, further comprising the following step: The icon or one of the at least 2D images is rotated in relation to the rotation of the other icon or at least 2D image.

5. The method according to claim 1, further comprising the following steps: The 3D anatomical model or the at least one 2D image is rotated in relation to the rotation of the other 3D anatomical model or the at least one 2D image.

6. The method of claim 1, wherein the step of identifying the structure in the at least one 2D image comprises: Provide the outline of the structure in the at least one 2D image.

7. The method of claim 1, wherein the step of selecting the structure of interest in the 3D anatomical model comprises: Identify the structures of interest in the 3D anatomical model.

8. The method of claim 7, wherein the step of identifying the structure of interest in the 3D anatomical model comprises: Color is applied to the structures of interest in the 3D anatomical model.

9. The method of claim 1, further comprising providing a label on the structure of interest in the at least one 2D image.

10. The method of claim 1, further comprising interacting with the 3D anatomical model to shift the at least one 2D image.

11. The method of claim 10, wherein the step of interacting with the 3D anatomical model comprises: Lines are drawn on the 3D anatomical model to shift the at least one 2D image, thereby showing a plane of the 3D anatomical model oriented along the lines.

12. The method of claim 1, wherein the step of registering the geometric model with the 3D volume comprises: An image segmentation process is performed on the 3D volume.

13. The method of claim 1, further comprising using the 3D volume to distort at least one of the geometric model or the 3D anatomical model.

14. The method of claim 1, wherein the 3D volume is displayed in real time to form a 4D volume.

15. The method of claim 14, wherein the 3D anatomical model is presented as a 4D model.

16. An ultrasound imaging system, the ultrasound imaging system comprising: -Probe; - A processor, operatively connected to the probe; and a display operatively connected to the processor to render the 3D volume and the at least one 2D image on the display, wherein the processor is configured to perform the method according to any one of claims 1 to 15.

Images