Fusion of ultrasound images with preoperative structured 3D image data for ultrasound image guided surgery

Abstract

An ultrasound imaging system (102) includes a single transducer array (110) having a long axis and configured to transmit and receive in a first mode to acquire real-time ultrasound images of a sagittal plane during a surgical procedure, or in a second mode, acquiring 3D ultrasound data comprising a plurality of sagittal planes by rotating the single transducer array about an axis parallel to the long axis of the transducer array; a resampler (202) configured to resample the 3D ultrasound data and generate a set of resampled ultrasound planes that are approximately orthogonal to the surface of tissue of interest; a divider (204) configured to segment a first profile of the tissue of interest in the set of resampled ultrasound planes; and a registration engine (206) configured to register the first profile with a second profile of the tissue of interest in the pre-operative 3D MR image data and generate a fused 3D image.

Description

Technical Field

[0001] The following content relates generally to ultrasound imaging, and more specifically to the fusion of ultrasound images with preoperative structured 3D image data for ultrasound-guided surgery. Background Technology

[0002] An ultrasound imaging system includes a transducer array that transmits an ultrasound beam into the field of view. As the beam passes through a structure in the field of view (e.g., a structure of a sub-section of an object or subject), portions of the beam are attenuated, scattered, and / or reflected by that structure, and some of these reflections (echoes) return to the transducer array. The transducer array receives the echoes and processes them to generate an ultrasound image of that portion of the object or subject. The ultrasound image is then displayed visually.

[0003] Ultrasound imaging is used in a wide range of medical applications. One example of a medical application is ultrasound-guided biopsy or treatment. Typically, a biopsy is a surgical procedure that involves removing a small sample of tissue of interest (e.g., prostate, lung, breast, etc.) for subsequent examination to check for abnormalities such as cancer cells. During a biopsy, a needle is inserted through the skin and advanced to the target tissue from which the sample is to be collected. In ultrasound-guided biopsies or treatments, ultrasound is used to help clinicians locate and / or navigate instruments to the tissue of interest.

[0004] One approach is to combine real-time ultrasound images with preoperative structured (e.g., magnetic resonance (MR)) 3D image data during surgery to provide an anatomical reference frame for tracking instrument advancement. To achieve this, firstly, a transducer array is used to acquire sagittal and transverse images of the tissue of interest (TPO). Then, the contours of the TPO segmented in the two orthogonal planes are co-registered with tissue contours obtained from the 3D MR image data segmentation. Segmentation can include various features, including lesions. Then, during surgery, the real-time ultrasound images are overlaid with the feature contours in the corresponding planes of the 3D MR images.

[0005] Unfortunately, unless the transducer array is moved to a different location that reduces the tissue overview, the two orthogonal ultrasound imaging planes typically cannot simultaneously display a specific point of interest. Therefore, the registration of the contours in the two orthogonal ultrasound imaging planes with the 3D MR image data may be inaccurate, and the current position of the instrument in the real-time ultrasound image relative to the anatomical reference frame provided by the 3D MR image data may not accurately represent the actual position of the instrument relative to the tissue of interest.

[0006] Therefore, an improved method is still needed to mitigate the above and / or other drawbacks of existing methods in fusing ultrasound images and structured 3D image data for ultrasound-guided surgery. Summary of the Invention

[0007] This application addresses the aforementioned and other issues. The present invention provides a more detailed description of concepts in the specific embodiments. It should not be used to identify the essential features of the claimed subject matter, nor should it be used to limit the scope of the claimed subject matter.

[0008] In one aspect, an ultrasound imaging system includes a single transducer array having a long axis and configured to transmit and receive in a first mode to acquire real-time ultrasound images in the sagittal plane during surgery, or in a second mode to acquire 3D volumetric ultrasound data comprising multiple sagittal planes by rotating the single transducer array about an axis parallel to the long axis of the transducer array. The ultrasound imaging system also includes a resampler configured to resample the 3D volumetric ultrasound data and generate a set of resampled ultrasound planes approximately orthogonal to the surface of the tissue of interest. The ultrasound imaging system also includes a segmenter configured to segment a first contour of the tissue of interest within the set of resampled ultrasound planes. The ultrasound imaging system further includes a registration engine configured to register the first contour with a second contour of the tissue of interest from preoperative three-dimensional (3D) magnetic resonance (MR) image data and generate a fused 3D image. The ultrasound imaging system also includes a rendering engine configured to overlay the real-time ultrasound image with the segmented feature contours from the preoperative 3D MR image data.

[0009] On the other hand, a computer-implemented method includes: acquiring real-time sagittal ultrasound images during surgery using a single transducer array having a long axis, or acquiring 3D volumetric ultrasound data while the single transducer array rotates about an axis parallel to the long axis of the single transducer array. The computer-implemented method further includes: resampling the 3D volumetric ultrasound data to generate a set of resampled ultrasound planes approximately orthogonal to the surface of the tissue of interest. The computer-implemented method further includes: segmenting a first contour of the tissue of interest within the set of resampled ultrasound planes. The computer-implemented method further includes: registering the first contour with a second contour of the tissue of interest from preoperative 3D MR image data to generate a fused 3D image. The computer-implemented method further includes: overlaying the real-time ultrasound images with the segmented feature contours from the preoperative 3D MR image data on a display monitor.

[0010] On the other hand, a computer-readable medium is encoded with computer-executable instructions that, when executed by a processor, cause the processor to: acquire real-time ultrasound images in the sagittal plane during surgery using a single transducer array having a long axis, or acquire 3D volumetric ultrasound data while the single transducer array rotates about an axis parallel to the long axis of the single transducer array; resample the 3D volumetric ultrasound data to generate a set of resampled ultrasound planes approximately orthogonal to the surface of the tissue of interest; segment a first contour of the tissue of interest in the set of sagittal ultrasound planes; register the first contour with a second contour of the tissue of interest in preoperative 3D MR image data to generate a fused 3D image; and overlay the real-time ultrasound image with the segmented feature contours from the preoperative 3D MR image data on a display monitor.

[0011] Other aspects of this application will be recognized by those skilled in the art upon reading and understanding the accompanying specification. Attached Figure Description

[0012] This application is illustrated by way of example and is not limited to the figures in the accompanying drawings, in which the same reference numerals indicate similar elements.

[0013] Figure 1 A non-limiting example of an ultrasound imaging system including a fusion module according to one aspect of the embodiments described herein is illustrated.

[0014] Figure 2 A non-limiting example of a fusion module comprising a resampler, a segmenter, and a registration engine, according to one aspect of an embodiment of the present invention, is illustrated.

[0015] Figure 3 An example coordinate system for an ultrasound probe used for resampling v3-D ultrasound volumes according to the embodiment described herein is illustrated.

[0016] Figure 4 An example method for resampling 3D ultrasound volume into an isometric 2D image according to the embodiment described herein is illustrated.

[0017] Figure 5A A portion of a 2D slice of sagittal MR image data according to one aspect of an embodiment of the present invention is shown, containing a tissue of interest contour and a suspicious region contour within the tissue of interest contour.

[0018] Figure 5B A portion of a 2D slice of 3D MR image data, showing a cross-sectional view according to one aspect of an embodiment of the present invention, is illustrated, containing the outline of the tissue of interest and the outline of a suspicious region within the outline of the tissue of interest.

[0019] Figure 6A A fused image comprising sagittal ultrasound images, including a portion of a 2D slice from 3D MR image data, is shown according to one aspect of an embodiment herein. Figure 5A The organization of interest (OI) profile and the suspicious region profile within the OI profile registered with the profile.

[0020] Figure 6B A fused image comprising cross-sectional views of ultrasound images is shown according to one aspect of an embodiment of the present invention, containing a portion of a 2D slice from 3D MR image data. Figure 5B The organization of interest (OI) profile and the suspicious region profile within the OI profile registered with the profile.

[0021] Figure 7 An example of one aspect of the implementation scheme according to this document is illustrated. Figure 2 It also includes a variant of the fusion module of the main point determiner.

[0022] Figure 8 A fused image showing the outline of an organization of interest and graphical markers representing key targets therein, according to one aspect of an embodiment of this paper, is illustrated.

[0023] Figure 9 Another fused image is shown, according to one aspect of the embodiment described herein, displaying the outline of the organization of interest and graphical markers representing the main target points therein.

[0024] Figure 10 An example of one aspect of the implementation scheme according to this document is illustrated. Figure 7 A variant of the fusion module.

[0025] Figure 11 A non-limiting example of a flowchart illustrating a computer-implemented method for generating fused images for ultrasound-guided surgery according to an embodiment of this article is provided.

[0026] Figure 12 A non-limiting example of a flowchart illustrating a computer-implemented method for generating a fused image containing key targets for ultrasound-guided surgery, according to an embodiment of this article.

[0027] Figure 13 A non-limiting example of a flowchart illustrating a computer-implemented method for beamforming real-time cross-sectional images by scanning a sagittal transducer array in a cross-sectional direction during ultrasound image-guided surgery, according to an embodiment of this article.

[0028] Figure 14 A non-limiting example of a magnetic resonance (MR) imaging system according to one aspect of the embodiments described herein is illustrated. Detailed Implementation

[0029] Embodiments of this disclosure will now be described by way of example with reference to the accompanying drawings, wherein systems, methods, and / or instructions on a computer-readable medium can efficiently provide precise fusion of 3D ultrasound data and preoperative 3D MR image data for ultrasound-guided surgery (such as biopsy, treatment, and / or other procedures). In one example, this includes: acquiring 3D ultrasound data via scanning of a sagittal transducer array; resampling the 3D ultrasound data to generate a set of ultrasound image planes approximately orthogonal to the surface of the tissue of interest; and co-registering the tissue of interest contour segmented in this set of ultrasound image planes with the tissue of interest contour segmented in the preoperative 3D MR image data to create a fused 3D image. Then, during the procedure, the real-time ultrasound image is superimposed with the feature contours segmented from the corresponding planes of the fused 3D image.

[0030] As described above, existing methods include: acquiring sagittal ultrasound images; acquiring transverse ultrasound images; and registering tissue-of-interest (TOI) contours from two orthogonal planes with segmented TOI contours from preoperative 3D MR image data. However, if the transducer is not moved to different locations that would reduce the tissue overview, the two orthogonal ultrasound image planes typically cannot simultaneously display specific TOIs, and therefore, the co-registration of contours in the two orthogonal ultrasound image planes with contours in the 3D MR image data may be inaccurate. The method described herein overcomes these and / or other drawbacks. Compared to configurations without the method described herein (where 3D ultrasound data acquired by scanning is processed to generate 3D ultrasound images, and then the ultrasound images are co-registered with preoperative 3D MR image data), the method described herein further reduces memory and / or processing resource consumption and / or processing time.

[0031] In some instances, the method described herein also identifies principal targets within the tissue of interest based on preoperative 3D MR image data, and graphically indicates these principal targets by overlaying markers onto the corresponding planes of the displayed real-time ultrasound and fused 3D images. This allows clinicians to use the graphical markers to advance instruments to the principal targets during surgery. In some instances, the method described herein not only displays real-time sagittal images but also forms real-time transverse images from the beam of 3D ultrasound data acquired through the principal targets, simultaneously displaying both sagittal and transverse real-time images.

[0032] First refer to Figure 1The illustration schematically illustrates a non-limiting example of an ultrasound system 102. The ultrasound system 102 includes an ultrasound probe 104 and a console 106. In the illustrated embodiment, the probe 104 and the console 106 are coupled to each other via a communication channel 108, which includes wired (e.g., a complementary interface and a cable between them) and / or wireless technologies (e.g., Wi-Fi, etc.). In another example, the probe 104 and the console 106 are integrated into the same housing, such as part of a handheld ultrasound system.

[0033] The probe 104 includes a transducer array 110. The transducer array 110 has a long axis and includes one or more transducer elements 112. Suitable array examples include 64, 128, 192, 256, and / or other arrays, including larger and smaller arrays. The transducer array 110 can be linear, curved, and / or other shaped, fully filled, sparse, and / or combinations thereof. In one example, the transducer array 110 is configured to rotate within the probe 104. For example, the transducer array 110 is spatially oriented along the sagittal plane of the probe 104 and is configured to pivot or scan within at least 180° of the probe 104 in the transverse direction by rotating the transducer array 110 about an axis parallel to the long axis of the transducer array.

[0034] In one example, probe 104 may further include a transducer array rotation assembly with electromechanical and control components for rotating transducer array 110 in the cross-sectional direction within probe 104 and controlling such rotation. In another example, transducer array 110 is not configured to rotate within probe 104, and the transducer array rotation assembly is omitted; probe 104 is manually rotated, for example, by a user, robot, etc., by rotating probe 104 in the cross-sectional direction about an axis parallel to the long axis of the transducer array.

[0035] One or more transducer elements 112 are configured to convert an excitation electrical signal into an ultrasonic pressure field and to convert the reflected ultrasonic pressure field into an electrical signal. For example, one or more transducer elements 112 may be selectively excited via an excitation electrical signal, such that at least a subset of the transducer elements 112 emits an ultrasonic pressure field into the scanning field of view. The one or more transducer elements 112 receive echo signals and generate analog electrical signals indicative of the echo signals. Echo signals are generated in response to the emitted ultrasonic pressure field interacting with a structure, such as tissue, blood cells, etc.

[0036] The control console 106 includes a transmitting circuit 116 configured to generate an excitation electrical signal for the transducer array 110 to transmit an ultrasonic pressure field. In one example, this includes generating a delay for each transmission of the individual elements 112 of the transducer array 110, stimulating all elements 112, or stimulating fewer than all elements 112, etc.

[0037] The console 106 also includes receiving circuitry 118 configured to receive and preprocess analog echo signals. In one example, this includes applying fixed amplification, applying analog time gain compensation, converting the analog echo signal to a digital echo signal, down-converting the signal to shift its center frequency to baseband, decimating the signal to reduce the data rate, and / or otherwise preprocessing the signal.

[0038] The console 106 also includes a switch 120 configured to switch between the transmitting circuit 116 and the receiving circuit 118, for example, by electrically connecting the transmitting circuit 116 to the transducer array 110 for transmitting operations and electrically connecting the receiving circuit 118 to the transducer array 110 for receiving operations. In an alternative embodiment, separate switches are used, such that the transmitting circuit 116 has a switch and the receiving circuit 118 has a different switch.

[0039] The console 106 also includes a beamformer 122. The beamformer 122 is configured to beamform signals from the receiving circuit 118, for example via delay summation (e.g., matched filter beamformer, etc.) and / or other beamforming, and to construct a scan plane of scan lines of radio frequency (RF) data or in-phase / quadrature (IQ) data for each receiving operation.

[0040] The console 106 also includes a scanline processor 124. The scanline processor 124 is configured to perform one or more of the following: filtering (e.g., via a finite impulse response (FIR) filter, an infinite impulse response (IIR) filter, a bandpass filter (BPF), etc.), in-phase and quadrature (I / Q) demodulation, envelope detection, dynamic range compression, compositing, dynamic range expansion, noise suppression, downconversion, decimation, adaptive gain, etc.; and output a set of scanlines as a one-frame / B-mode image.

[0041] The console 106 also includes a buffer memory 126. The buffer memory 126 is configured to store scan lines for volumetric or 3D processing. For example, in one instance, when the transducer array 110 rotates (or scans) in the transverse direction to acquire 3D volumetric ultrasound data (e.g., a set of sagittal planes angularly offset in the transverse direction), scan lines corresponding to each sagittal plane are stored in the buffer memory 126 for use in volumetric or 3D processing.

[0042] Console 106 also includes a fusion module 128. In one example, fusion module 128 is configured to co-register a tissue-of-interest (TOO) contour segmented from 3D volumetric ultrasound data in buffer memory 126 with a TOO contour segmented from preoperative 3D MR image data. As described in more detail below, in one example, this includes: processing the 3D volumetric ultrasound data to generate a set of planes with a specific orientation relative to the TOO; and co-registering the TOO contours from this set of images with the TOO contours in the preoperative 3D MR image data.

[0043] As described in more detail below, in some instances, the fusion module 128 is further configured to identify a principal target in the tissue of interest based on preoperative 3D MR image data, wherein the principal target is visualized via graphical markers along with a registered contour of interest, allowing clinicians to advance instruments to the principal target using the graphical markers. As described in more detail below, in some instances, the fusion module 128 is further configured to form a real-time cross-sectional image from a 3D volumetric ultrasound data beam passing through the principal target, wherein the real-time cross-sectional image formed by this beam is also visualized.

[0044] Console 106 also includes a rendering engine 130 configured to overlay real-time ultrasound images onto a corresponding plane of a fused image on display 132. Console 106 also includes a user interface (U / I) 134 configured to allow a user to control the operation of console 102. Console 106 also includes a controller 136 comprising a processor such as a microprocessor (μP), a central processing unit (CPU), a graphics processing unit (GPU), etc., and a computer-readable medium, including non-transitory media but excluding transient media (signals, carrier waves, etc.). The processor is configured to execute instructions in the computer-readable medium to perform the functions of one or more components of console 106 and / or the imaging system 102 described herein.

[0045] Figure 2 An example of a fusion module 128 is illustrated schematically. In this example, the fusion module 128 includes a resampler 202, a segmenter 204, and a registration engine 206.

[0046] Resampler 202 retrieves and / or receives 3D volumetric ultrasound data from buffer memory 126 as input. Similarly, the 3D volumetric ultrasound data includes signals from multiple angularly spaced sagittal planes acquired during scanning of transducer array 110. In one example, the 3D volumetric ultrasound data may include approximately four hundred (400) sagittal plane signals, not all of which are orthogonal to the interface of the tissue of interest. Resampler 202 is configured to resample the signals and generate a set of planes approximately orthogonal to the interface of the tissue of interest.

[0047] This set of planes includes at least two (2) sagittal planes, such as three (3) planes, nine (9) planes, twelve (12) planes, twenty (20) planes, etc., but fewer than the number of planes acquired for 3D volumetric ultrasound data. For example, in one instance, this set of planes may select twelve (12) planes spaced at equal angles from hundreds of 3D volumetric ultrasound data planes acquired within a scanning range (such as zero degrees (0) to one hundred and eighty degrees (180)). Resampling may include interpolation and / or other processing. This method reduces the required processing resources compared to a configuration that generates planes for all 3D volumetric ultrasound data.

[0048] Combination Figure 3 and Figure 4 Unrestricted examples were discussed. Figure 3 A portion of the example probe and its coordinate system are schematically illustrated, and Figure 4 An example resampling is illustrated schematically. From Figure 4 Initially, in this example, probe 104 includes a top region 302, an elongated shaft 304, and a handle 306, with a transducer array 110 located within the top region 304. In this example, the transducer array 110 is configured to rotate within the top region 304. A non-limiting example of such a probe 104 is the 3D X14L intracavitary transducer manufactured by the Danish company BK Medical. The user typically places the tissue of interest 308 at the center of the 3D scan. Coordinates (x, y, z) form a right-handed coordinate system.

[0049] In this coordinate system, the x-axis 310 points to the page, the y-axis 312 is defined as a line on the xy plane that divides the total scan angle into two equal parts, and the z-axis 314 points in the direction of axis 304 of the probe 104. The z-axis 312 is perpendicular to the xy plane, the yz plane is the sagittal plane of the body, and the origin is located on the rotation axis. Figure 4 This shows that after resampling, the 3D ultrasound data is such that all N slices (S0, S1, S2, ..., S...) are... (N-1)All slices contain a y-axis, are perpendicular to the xz plane, and are spaced at equal angular intervals such that the angle between slices is π / N. An example of this resampling is described in U.S. Patent 10,779,798 B2, filed September 24, 2018, and granted to Martins, entitled “Ultrasound Three-Dimensional (3-D) segmentation,” the entire contents of which are incorporated herein by reference.

[0050] In one instance, the resampler 202 is further configured to perform similar functions to the scanline processor 124, such as one or more of the following: filtering (e.g., via an FIR filter, IIR filter, BPF, etc.), I / Q demodulation, envelope detection, dynamic range compression, compositing, dynamic range expansion, noise suppression, downconversion, decimation, adaptive gain, etc.; and outputting a set of scanlines as a one-frame / B-mode image. Additionally or alternatively, the resampler 202 employs the scanline processor 124 and / or other components to perform one or more of these functions.

[0051] Segmenter 204 retrieves and / or receives a set of planes generated by resampler 202 as input. Segmenter 204 is configured to segment the contour of the tissue of interest from this set of planes. In one instance, the output of segmenter 204 includes points in a point cloud or 3D ultrasound space. Various known and / or other methods can be used to segment the contour. For example, in one instance, segmenter 204 includes a neural network trained to segment the contour from this set of planes. In this example, the segmented output can be used as training data for subsequent and / or adaptive training of segmenter 204.

[0052] The registration engine 206 retrieves and / or receives tissue-of-interest (TOI) contour segmentation from the segmenter 204, as well as segmentation from preoperative 3D MR images (e.g., T2-weighted acquisition, etc.) as input. The 3D MR images include the TOI and suspicious regions within the TOI (such as potential lesions, nodules, tumors, etc.). MR segmentation comprises point clouds or points in 3D MR space. Known and / or other techniques for segmenting MR data can be used for MR segmentation, including automated segmentation techniques that follow protocols used by clinicians (such as radiologists) to manually segment the TOI and / or suspicious regions.

[0053] The registration engine 206 is configured to perform co-registration of the segmentation of the tissue of interest in the resampled 3D volumetric ultrasound data from the segmenter 204 and the segmentation of the preoperative 3D MR image data. This includes converting the MR segmentation from MR space to ultrasound space. In one instance, this includes performing point cloud registration from MR coordinates to ultrasound space for at least the segmented tissue of interest and suspicious regions (i.e., potential lesions, nodules, tumors, etc.) in the preoperative 3D MR image data to generate fused 3D data.

[0054] Rendering engine 130 retrieves and / or receives real-time ultrasound images from scanline processor 124, as well as fused 3D data as input. Rendering engine 130 is configured to combine the real-time ultrasound images from scanline processor 124 with corresponding planes from the fused 3D data on display 132 during the procedure.

[0055] Figure 5A A portion 502 of a 2D slice of 3D MR image data in a sagittal view is shown, and includes a tissue of interest contour 504 and a suspicious region contour 506 within the tissue of interest contour 504. Figure 5B A portion 508 of a 2D slice of 3D MR image data in a cross-sectional view is shown, and includes a tissue of interest outline 510 and a suspicious region outline 512 within the tissue of interest outline 510.

[0056] Figure 6A A fused image plane 602 is shown, comprising ultrasound image planes including sagittal views, containing images from... Figure 5A The 2D slice shown contains a tissue of interest outline 504 and a suspicious region outline 506 within the tissue of interest outline registered with the outline. Figure 6B A fused image 604 is shown, comprising ultrasound images including cross-sectional views, containing images from... Figure 5B The 2D slice shown contains a portion 508 of the tissue of interest outline 510 and a suspicious region outline 512 within the tissue of interest outline registered with that outline.

[0057] Figure 7 The combination is illustrated schematically. Figure 2 A variation of the described example. In this variation, the fusion module 128 further includes a primary target determiner 702. The primary target determiner 702 retrieves and / or receives MR data as input, such as segmented T2-weighted MR images, apparent diffusion coefficient (ADC) images, diffusion-weighted (DW) images, etc. In one instance, the primary target determiner 702 is configured to determine the primary target within a suspicious region (based on MR data). Generally, tissues of interest, such as lesions and tumors, appear hypoechoic in ADC images and hyperechoic in DW images.

[0058] Now, a non-limiting example for determining the primary target is described. In ADC images, the ADC value outside the prostate is set to a high value (e.g., 2000, etc.). In DW images, the B value outside the prostate is set to zero (0). The number of voxels corresponds to a predetermined volume. In one instance, the predetermined volume is based on visual examination. Generally, this size is chosen to be larger than the average lesion volume seen through visual examination of multiple patients. For example, for 1.3 cm... 3 The average volume, a suitable volume could be 1.6 cm³. 3 1.8cm 3 2.1cm 3 The smaller size allows the ADC to independently pinpoint one or more focal lesions. For this number of voxels, the voxel with the lowest ADC value in the ADC image is identified, and the values ​​of the remaining voxels are set to high values ​​(e.g., 2000, etc.). For the same voxel, the voxel with the highest b value in the DW image is identified, and the values ​​of the remaining voxels are set to zero (0).

[0059] For each voxel, the severity value is calculated as the ratio of the b-value to the ADC value plus a constant. Suitable values ​​for the constant are 5, 7, 10, 13, higher, or lower. The number of voxels corresponding to another predetermined volume is determined. Again, the predetermined volume is based on visual examination. In one instance, the predetermined volume is approximately the average size of the lesion, such as 1.3 cm in the example above. 3 In other instances, the volume can be larger. For this number of voxels, the highest severity value is retained, and the remaining severity values ​​are set to zero (0). A connected component algorithm (e.g., processing one region at a time, two-pass method, etc.) is then applied to create a set of disconnected lesions, nodules, tumors, etc. For each region, a smoothing function (e.g., a 3D Gaussian kernel, etc.) is applied to the severity value, and the 3D coordinates of the highest severity value among the smoothed severity values ​​are selected as the primary target.

[0060] Rendering engine 130 retrieves and / or receives real-time ultrasound images, fused 3D data, and 3D coordinates of the primary target points from scanline processor 124 as input. Rendering engine 130 is configured to combine, during surgery, the real-time ultrasound images from scanline processor 124, the corresponding planes in the fused 3D data on display 132, and graphical markers on display 132 representing the primary target points within the outline of the suspicious region.

[0061] Figure 8 The fused image 802 is shown, in which the outline 804 of the suspicious region and the main target point 806 are shown. The centroid 808 of the outline 804 of the suspicious region is graphically illustrated as a reference frame. Figure 9The fused image 902 is shown, illustrating the outline 904 of the suspicious region and the main target point 906. The centroid 908 of the outline 904 of the suspicious region is illustrated as a reference frame. Figure 8 and Figure 9 In this paper, for illustrative purposes, the primary targets 806 and 906 are marked with black borders. Other markers are envisioned. For example, markers could include shapes, patterns, pointers, colors, user-controlled levels of transparency, etc.

[0062] Figure 10 The combination is illustrated schematically. Figure 7 A variation of the described example. In this example, the transducer array 110 of probe 104 is configured to acquire sagittal planes not only during surgery but also to perform one or more scans between each sagittal plane acquisition during surgery. In this example, the 3D volumetric data is again stored in buffer memory 126. However, in this example, resampler 202 also receives the 3D coordinates of the primary target from primary target determiner 702.

[0063] In this example, resampler 202 is also configured to beamform a real-time cross-sectional image based on 3D volumetric data and the 3D coordinates of the primary target point. The beamformed real-time cross-sectional image will include a plane passing through the primary target point. In this example, rendering engine 130 further receives the beamformed real-time cross-sectional image. In one instance, the real-time sagittal image and the corresponding sagittal plane in the fused image are displayed in one viewport, and simultaneously, the beamformed real-time cross-sectional image and the corresponding cross-section in the fused image are displayed in another viewport.

[0064] In one example, real-time sagittal images can be used to track the advancement of an instrument toward a primary target within a delineated region of suspicion, while real-time beamforming cross-sectional images can be used to visualize the instrument outside the delineated region of suspicion, for example, to align the instrument for advancement toward the primary target within the delineated region of suspicion. In one example, the user can control whether to display real-time beamforming cross-sectional images.

[0065] Further reference Figure 1 , Figure 2 , Figure 7 and Figure 10 The segmenter 204 can apply an automatic segmentation algorithm to segment the contours of suspicious regions in each image plane in the image plane group, and the rendering engine 130 can display an image with automatically segmented contours. Figure 11 This document illustrates a non-limiting example of a flowchart of a computer-implemented method for generating fused images for ultrasound-guided surgery. It should be understood that the order of actions in this method is not restrictive. Therefore, other orders are contemplated herein. Furthermore, one or more actions may be omitted, and / or one or more additional actions may be included.

[0066] At 1102, segmented preoperative 3D MR image data is received, as described herein and / or otherwise. For example, in one instance, 3D MR image data acquired from T2-weighted acquisition, etc., is obtained. In the 3D MR image data, the contours of the tissue of interest and the contours of the suspicious region are segmented. In one instance, segmentation includes known and / or other techniques, including automated segmentation techniques that follow protocols used by clinicians (such as radiologists) to manually segment the tissue of interest and / or the suspicious region, and generate a point cloud or points in 3D MR space.

[0067] At position 1104, 3D volumetric ultrasound data of the tissue of interest are acquired, as described herein and / or otherwise. For example, for the prostate, probe 104 is positioned below the prostate, and transducer array 110 is guided to the prostate using real-time ultrasound images. Probe 104 is then activated, causing transducer array 110 to scan (or rotate) in the transverse direction and acquire multiple sagittal planes angularly offset in the transverse direction, thereby providing 3D volumetric ultrasound data.

[0068] At point 1106, the 3D volumetric ultrasound data of the tissue of interest is processed, as described herein and / or otherwise. For example, in one instance, the 3D volumetric ultrasound data of the tissue of interest includes the prostate and more than 100 sagittal planes. Resampler 202 resamples the signal and generates, for example, a set of 3 to 12 sagittal planes spaced at equal angles, which are approximately orthogonal to the surface of the tissue of interest. As discussed herein, this method improves registration accuracy and reduces processing requirements.

[0069] At 1108, as described herein and / or otherwise, the contour of the tissue of interest is segmented in a set of sagittal planes spaced at equal angular intervals. For example, in one instance, segmenter 204 includes and / or utilizes a trained neural network and / or other methods to segment the contour of the tissue of interest in a set of sagittal planes spaced at equal angular intervals. In one instance, the segmentation generates a point cloud or points in 3D ultrasound space.

[0070] At 1110, as described herein and / or otherwise, the tissue of interest contour segmented in a set of sagittal planes at equal angular intervals is fused with the tissue of interest contour segmented in the 3D MR data. For example, in one instance, registration engine 206 performs point cloud registration, which at least transforms the MR coordinates of the tissue of interest and the region of suspicion to ultrasound space.

[0071] At 1112, as described herein and / or otherwise, the real-time sagittal image is overlaid with feature contours segmented from the preoperative 3D MR image data. For example, in one instance, rendering engine 130 retrieves and / or receives real-time ultrasound images and 3D data from scanline processor 124 as input and combines the real-time ultrasound images from scanline processor 124 with feature contours segmented from the preoperative 3D MR image data on display 132.

[0072] Figure 12 Another non-limiting example is illustrated by a flowchart of a computer-implemented method for generating fused images and key targets for ultrasound-guided surgery. It should be understood that the order of actions in this method is not restrictive. Therefore, other orders are contemplated herein. Furthermore, one or more actions may be omitted, and / or one or more additional actions may be included.

[0073] At 1202, segmented 3D MR data are received as described herein and / or otherwise. At 1204, 3D volumetric ultrasound data of the tissue of interest are acquired as described herein and / or otherwise. At 1206, the 3D volumetric ultrasound data of the tissue of interest are processed as described herein and / or otherwise. At 1208, the contour of the tissue of interest is segmented in a set of sagittal planes spaced at equal angular intervals as described herein and / or otherwise. At 1210, the tissue of interest contour segmented in a set of sagittal planes spaced at equal angular intervals is fused with the tissue of interest contour segmented in the 3D MR data as described herein and / or otherwise.

[0074] At point 1212, as described herein and / or otherwise, the primary target point for surgery is identified within the suspicious region. For example, in one instance, the primary point determiner 702 retrieves and / or receives segmented MR images, ADC images, and DW images. The primary point determiner 702 identifies the primary target point within the suspicious tissue based on the MR data. For example, in a non-limiting instance, the extraprostatic ADC value in the ADC image is set to a high value (e.g., 2000, etc.), and the extraprostatic B value in the DW image is set to zero (0). A predetermined volume (e.g., 1.6 cm) is determined. 3 The number of voxels corresponding to (etc.). For this number of voxels, the voxel with the lowest ADC value in the ADC image is identified, and the values ​​of the remaining voxels are set to high values ​​(e.g., 2000, etc.), and the voxel with the highest b value in the DW image is identified, and the values ​​of the remaining voxels are set to zero (0).

[0075] For each voxel, a severity value is calculated based on the ratio of the b-value to the ADC, plus a constant. This is determined in relation to another predetermined volume (e.g., 1.3 cm). 3The number of voxels corresponding to (etc.). For this number of voxels, the severity value is retained, and the remaining severity values ​​are set to zero (0). Then, a connected component algorithm is applied to create a set of disconnected lesion, tumor, etc. regions. For each region, a smoothing function (e.g., a 3D Gaussian kernel) is applied to the severity, and the 3D coordinates of the highest severity value among the smoothed severity values ​​are selected as the primary target.

[0076] At 1214, as described herein and / or otherwise, the real-time sagittal image is overlaid with feature contours segmented from preoperative 3D MR image data and the principal target points. For example, in one instance, rendering engine 130 retrieves and / or receives a real-time ultrasound image from scanline processor 124, the 3D coordinates of feature contours segmented from preoperative 3D MR image data and the principal target points as input, and combines the real-time ultrasound image from scanline processor 124 with the feature contours segmented from preoperative 3D MR image data and the principal target points on display 132.

[0077] Figure 13 Another non-limiting example of a flowchart illustrating a computer-implemented method for beamforming real-time cross-sectional images by scanning in the cross-sectional direction using a sagittal transducer array during ultrasound-guided surgery is provided. It should be understood that the order of actions in this method is not restrictive. Therefore, other orders are contemplated herein. Furthermore, one or more actions may be omitted, and / or one or more additional actions may be included.

[0078] At 1302, segmented 3D MR image data is received, as described herein and / or otherwise. At 1304, 3D volumetric ultrasound data of the tissue of interest is acquired, as described herein and / or otherwise. At 1306, the 3D volumetric ultrasound data of the tissue of interest is processed, as described herein and / or otherwise. At 1308, the contour of the tissue of interest is segmented in a set of sagittal planes spaced at equal angular intervals, as described herein and / or otherwise. At 1310, the tissue of interest contour segmented in a set of sagittal planes spaced at equal angular intervals is fused with the tissue of interest contour segmented in the 3D MR data, as described herein and / or otherwise.

[0079] At 1312, as described herein and / or otherwise, a principal point for surgery is identified within the suspected region. At 1314, as described herein and / or otherwise, a real-time cross-sectional image is beamformed. For example, in one instance, during surgery, a rotating transducer array 110 is used to acquire 3D volumetric data. A resampler 202 processes the 3D volumetric data and beamforms the real-time cross-sectional image based on the coordinates of the principal point. The beamformed real-time cross-sectional image will include a plane passing through the principal point.

[0080] At position 1316, as described herein and / or otherwise, the real-time sagittal image and the real-time beamforming cross-sectional image are respectively overlaid with and displayed on feature contours segmented from preoperative 3D MR image data. The real-time sagittal image can be used to track the advancement of the instrument toward the primary target within the delineated suspicious region, while the real-time beamforming cross-sectional image can be used to visualize the instrument outside the delineated suspicious region, for example, for aligning the instrument to advance toward the primary target within the delineated suspicious region.

[0081] As discussed in this paper, MR data such as 3D MR image data, ADC images, and DW images are utilized. Figure 14 An example of an imaging system 1400 configured for at least MR imaging is schematically illustrated. The imaging system 1400 includes a main magnet 1402. The main magnet 1402 is configured to provide a substantially uniform, time-constant main magnetic field (B0) within an examination region 1404. Various magnet technologies (e.g., superconducting, resistive, or permanent magnet technologies) and / or physical magnet configurations (e.g., solenoid or open magnet configurations) have been implemented depending on the required main magnetic field strength and the requirements of the specific application.

[0082] The imaging system 1400 also includes a gradient coil 1406. The gradient coil 1406 is configured to generate a time-varying gradient magnetic field. The gradient coil 1406 includes an x-gradient coil for generating a gradient field along the x-direction, a y-gradient coil for generating a gradient field along the y-direction, and a z-gradient coil for generating a gradient field along the z-direction. The function of the gradient coil 1406 is to spatially encode the MR signal to distinguish signals from different locations within the body. The gradient coil 1406 is also used in various techniques such as diffusion imaging, perfusion imaging, functional imaging, elastography, and angiography. For diffusion imaging, the gradient coil 1406 is configured to generate a diffusion-sensitive gradient that affects image contrast.

[0083] The imaging system 1400 also includes a transmitting radio frequency (RF) coil 1408. The transmitting RF coil 1408 is configured to generate an RF signal that excites and / or otherwise manipulates hydrogen and / or other magnetically resonant nuclei in the object and / or subject within the examination area 1404. The imaging system 1400 also includes a receiving RF coil 1410. The receiving RF coil 1410 is configured to receive magnetic resonance (MR) signals generated by the excited nuclei in the examination area 1404. The illustrated transmitting RF coil 1408 and receiving RF coil 1410 are volumetric coils or whole-body coils integrated into the imaging system 1400.

[0084] For example, RF coil 1408 is configured as a receiving coil, and RF coil 1410 is configured as a transmitting coil. In another example, transmitting RF coil 1408 and receiving RF coil 1410 are part of the same transmit-receive RF coil, and a switch or similar device is configured to switch between transmit and receive operations. In yet another example, transmitting RF coil 1408 and receiving RF coil 1410 are separate from and installed within imaging system 1400 for use with the imaging system to scan an object or subject. Other coils are envisioned herein. Examples include smaller volume coils configured for limbs (such as the head), surface coils, etc.

[0085] The imaging system 1400 also includes an RF source 1414. The RF source 1414 is configured to generate an RF signal having a desired frequency (e.g., the Larmor frequency of the MR active nucleus under investigation). The imaging system 1400 also includes an RF pulse programmer 1416. The RF pulse programmer 1416 is configured to set the timing and / or shape of the RF signal generated by the RF source 1414. The imaging system 1400 also includes an RF amplifier 1418. The RF amplifier 1418 is configured to amplify the shaped RF signal to the level required by the transmitting RF coil 1408 to excite nuclei in an object or subject within the examination area 1404.

[0086] The imaging system 1400 also includes a gradient pulse programmer 1420. The gradient pulse programmer 1420 is configured to set the timing, intensity, and / or shape of the time-varying magnetic field generated by the gradient coils 1406 during scanning of the object and / or subject. The imaging system 1400 also includes a gradient amplifier 1422. The gradient amplifier 1422 is configured to amplify the time-varying magnetic field to the desired level for each gradient coil 1406. The gradient amplifier 1422 includes an independent power amplifier for each gradient coil 1406, each gradient coil including an x-gradient coil, a y-gradient coil, and a z-gradient coil. In one example, the x-gradient coil and the y-gradient coil each include a Golay coil, and the z-gradient coil includes a Maxwell coil.

[0087] Controller 1432 controls RF source 1414, RF pulse programmer 1416, and gradient pulse programmer 1420. RF pulse programmer 1416 and gradient pulse programmer 1420 respectively control RF amplifier 1426 and gradient amplifier 1422 based on the imaging technique used for the scanned object or subject. Examples of different imaging techniques include diffusion imaging, perfusion imaging, functional imaging, elastography, angiography, etc.

[0088] The imaging system 1400 also includes an RF detector 1424. The RF detector 1424 is configured to receive an analog MR signal generated by the RF receiving coil 1410 during a data acquisition window having a given timing and length. The imaging system 1400 also includes an RF amplifier 1426. The RF amplifier 1426 is configured to amplify the received analog MR signal. The imaging system 1400 also includes a signal conditioner 1428. The signal conditioner 1428 is configured to condition the amplified analog MR signal, for example, by demodulating, filtering, etc. The imaging system 1400 also includes an analog-to-digital (A / D) converter 1430. The A / D converter 1430 is configured to digitize the conditioned analog MR signal, i.e., to convert the conditioned analog MR signal into a digital MR signal.

[0089] The imaging system 1400 also includes a subject / object support 1434. The subject / object support 1434 includes a stage movably coupled to a frame / base. In one example, the stage is slidably coupled to the frame / base via bearings or the like, and a drive system (not visible) including a controller, motor, lead screw, and nut (or other drive system) translates the stage along the frame / base into and out of the inspection area 1404. The stage is configured to support an object or subject in the inspection area 1404 for loading, scanning, and / or unloading the subject or object. A stage controller (not visible) controls the drive system.

[0090] The imaging system 1400 also includes a reconstructor 1436. The reconstructor 1436 is configured to reconstruct the digitized MR signal and generate independent axial (2D) and / or volumetric (3D) image data. The MR signal includes coded imaging data (k-space), which is transformed by an image reconstruction algorithm using Fourier transform and / or other algorithms. The 2D and / or 3D image data can be visually presented via a display monitor, film printer, etc. In one example, the reconstructor 1436 is configured to process the MR signal and generate T2-weighted images, ADC images, DW images, etc.

[0091] Imaging system 1400 also includes computing system 1438. Computing system 1438 serves as an operator console for imaging system 1400. Computing system 1438 communicates with reconstructor 1436. In one example, imaging system 1400 is configured to transmit reconstructed images to ultrasound imaging system 102, servers, databases, workstations, radiology information systems (RIS), hospital information systems (HIS), electronic medical record systems (EMR), picture archiving and communication systems (PACS), etc.

[0092] The above method can be implemented by computer-readable instructions encoded or embedded on a computer-readable storage medium, which, when executed by a computer processor, cause the processor to perform the described action or function. Additionally or alternatively, at least one of the computer-readable instructions may be executed by a signal, a carrier wave, or other transient medium that is not a computer-readable storage medium.

[0093] As used herein, elements or steps listed in the singular and beginning with the word "a" or "an" should be understood to not exclude multiple said elements or steps unless such exclusion is explicitly stated. Furthermore, references to "an embodiment" of the invention are not intended to be construed as excluding the existence of additional embodiments that also include the referenced features. Moreover, unless explicitly stated to the contrary, embodiments that "comprise," "include," or "have" one or more elements having a particular attribute may include additional elements that do not have that attribute. The terms "comprise" and "in" are used as concise linguistic equivalents to the corresponding terms "comprising" and "wherein". Furthermore, the terms "first," "second," and "third," etc., are used merely as notations and are not intended to impose numerical requirements or a particular order of position on their objects.

[0094] Various implementations and / or components (e.g., modules or components and controllers therein) may also be implemented as part of one or more computers or processors. The computer or processor may include computing devices, input devices, display units, and interfaces, such as for accessing the Internet. The computer or processor may include a microprocessor. The microprocessor may be connected to a communication bus. The computer or processor may also include memory. Memory may include random access memory (RAM) and read-only memory (ROM). The computer or processor may further include a storage device, which may be a hard disk drive or a removable storage drive, such as a floppy disk drive, optical disk drive, etc. The storage device may also be other similar means for loading computer programs or other instructions into the computer or processor.

[0095] As used herein, the terms "computer" or "module" can include any processor-based or microprocessor-based system, including systems using microcontrollers, reduced instruction set computers (RISCs), application-specific integrated circuits (ASICs), logic circuits, and any other circuitry or processors capable of performing the functions described herein. The examples above are merely illustrative and are therefore not intended to limit the definition and / or meaning of the term "computer" in any way. A computer or processor executes a set of instructions stored in one or more storage elements to process input data. Storage elements may also store data or other information as desired or required. Storage elements may take the form of an information source within the processor or a physical memory element.

[0096] An instruction set may include various commands that instruct a computer or processor to perform specific operations (such as methods and processes according to various embodiments of the present invention) as a processing machine. The instruction set may be in the form of a software program. Software may take various forms, such as system software or application software. Furthermore, software may take the form of a collection of separate programs or modules, a program module within a larger program, or a portion of a program module. Software may also include modular programming in the form of object-oriented programming. The processor's processing of input data may be in response to operator commands, the results of previous processing, or a request from another processor.

[0097] As used herein, the terms “software” and “firmware” are interchangeable and include any computer program stored in memory for execution by a computer, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The memory types described above are merely exemplary and therefore do not limit the types of memory that can be used to store computer programs.

[0098] It should be understood that the above description is intended to be illustrative and not restrictive. For example, the above embodiments (and / or aspects thereof) may be used in combination with each other. Furthermore, many modifications may be made to adapt particular situations or materials to the teachings of various embodiments of the invention without departing from the scope of the invention. While the dimensions and types of materials described herein are intended to define parameters of various embodiments of the invention, these embodiments are by no means restrictive but exemplary. Many other embodiments will be apparent to those skilled in the art upon review of the above description.

[0099] This written description uses examples to disclose various embodiments of the invention, including the best mode, and also enables those skilled in the art to practice various embodiments of the invention, including making and using any device or system and performing any included methods. The patent scope of the various embodiments of the invention is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that differ only slightly from the literal language of the claims.

[0100] The embodiments shown in the accompanying drawings and described above are merely exemplary embodiments and are not intended to limit the scope of the appended claims, including any equivalents included within the scope of the claims. Various modifications are possible and will be apparent to those skilled in the art. Any combination of non-mutually exclusive features described herein is intended to be within the scope of this disclosure. That is, features of the described embodiments may be combined with any suitable aspect described above, and optional features of any aspect may be combined with any other suitable aspect. Similarly, features listed in dependent claims may be combined with non-mutually exclusive features of other dependent claims, particularly where the dependent claims are subordinate to the same independent claim. In some jurisdictions that claim single-claim dependents, such single-claim dependents may have been used in practice, but this should not be construed as meaning that features in dependent claims are mutually exclusive.

Claims

1. An ultrasound imaging system (102), the ultrasound imaging system comprising: A single transducer array (110) having a long axis and configured to transmit and receive in a first mode to acquire real-time ultrasound images of the sagittal plane during surgery, or in a second mode to acquire 3D volumetric ultrasound data containing multiple sagittal planes by rotating the single transducer array about an axis parallel to the long axis of the transducer array. A resampler (202) is configured to resample the 3D volumetric ultrasound data and generate a set of resampled ultrasound planes that are approximately orthogonal to the surface of the tissue of interest. A segmenter (204) is configured to segment a first contour of the tissue of interest in a set of resampled ultrasound planes; A registration engine (206) configured to register the first contour with a second contour of the tissue of interest in preoperative three-dimensional (3D) magnetic resonance (MR) image data and generate a fused 3D image; and A rendering engine (130) is configured to overlay the real-time ultrasound image with feature contours segmented from the preoperative 3D MR image data.

2. The ultrasound imaging system according to claim 1, wherein, The set of resampling ultrasound planes includes only three to twelve planes.

3. The ultrasound imaging system according to claim 1, further comprising: A principal point determiner (702) is configured to determine the principal target of the surgery based on 3D MR image data.

4. The ultrasound imaging system according to claim 3, wherein, The rendering engine is configured to overlay markers representing the locations of the main target points of the surgery onto the fused 3D image.

5. The ultrasound imaging system according to claim 3, wherein, The resampler is configured to perform beamforming on real-time cross-sectional ultrasound images passing through the primary target point based on additional 3D volumetric ultrasound data obtained during the surgery.

6. The ultrasound imaging system according to claim 5, wherein, The rendering engine is configured to display the real-time ultrasound image of the sagittal plane in a sagittal view of the fused 3D image in a first display port, and to display the real-time cross-sectional ultrasound image of the beamforming in a cross-sectional view of the fused 3D image in a second display port.

7. The ultrasound imaging system according to claim 6, wherein, The rendering engine is configured to overlay the markers representing the locations of the primary target points of the surgery onto the beamforming real-time cross-sectional ultrasound image.

8. A computer-implemented method, the computer-implemented method comprising: Real-time ultrasound images in the sagittal plane are acquired during surgery using a single transducer array with a long axis, or 3D volumetric ultrasound data are acquired as the single transducer array rotates about an axis parallel to the long axis of the transducer array. The 3D volumetric ultrasound data is resampled to generate a set of resampled ultrasound planes that are approximately orthogonal to the surface of the tissue of interest; The first contour of the tissue of interest is segmented in the set of resampled ultrasound planes; The first contour is registered with the second contour of the tissue of interest in the preoperative 3D MR image data to generate a fused 3D image; and On a display monitor, the real-time ultrasound image is overlaid with feature contours segmented from the preoperative 3D MR image data.

9. The computer-implemented method according to claim 8, wherein, The set of sagittal ultrasound planes includes three to twelve sagittal planes.

10. The computer-implemented method according to claim 8, further comprising: The main points of the surgery were identified based on MR data.

11. The computer-implemented method according to claim 10, further comprising: The display monitor shows markers indicating the positions of the key points on the fused 3D image.

12. The computer-implemented method according to claim 8, further comprising: Beamforming is performed on a real-time cross-section passing through the key point based on additional 3D volumetric ultrasound data obtained during the procedure.

13. The computer-implemented method according to claim 12, further comprising: The real-time ultrasound image in the sagittal plane is displayed on the sagittal view of the fused 3D image in the first display port; as well as The beamforming real-time cross-sectional ultrasound image is displayed on a cross-sectional view of the fused 3D image in the second display port.

14. The computer-implemented method according to claim 12, further comprising: Acquire sagittal plane images; Acquire cross-sectional images; Segment the first contour of the tissue of interest in the sagittal image; Segment the second contour of the tissue of interest in the cross-sectional image; Display the first contour and the second contour; The system receives user input to accept or reject the use of the sagittal and transverse images for registration with the preoperative 3DMR image data.

15. A computer-readable medium encoded with computer-executable instructions, which, when executed by a processor, cause the processor to: Real-time ultrasound images in the sagittal plane are acquired during surgery using a single transducer array with a long axis, or 3D volumetric ultrasound data are acquired as the single transducer array rotates about an axis parallel to the long axis of the transducer array. The 3D volumetric ultrasound data is resampled to generate a set of resampled ultrasound planes that are approximately orthogonal to the surface of the tissue of interest; The first contour of the tissue of interest is segmented in the set of sagittal ultrasound planes; The first contour is registered with the second contour of the tissue of interest in the 3D preoperative MR image data to generate a fused 3D image; as well as On a display monitor, the real-time ultrasound image is overlaid with feature contours segmented from the preoperative 3D MR image data.

Images