Provides 3D ultrasound images

Abstract

A method and processing system for providing a three-dimensional 3D ultrasound image and an additional ultrasound acquisition. A position of the additional ultrasound acquisition relative to the 3D ultrasound image is determined or obtained. After obtaining the 3D ultrasound image and the additional ultrasound acquisition, initial display data is used to display only the 3D ultrasound image. In response to user input, second display data is generated for displaying both the 3D ultrasound image and the additional ultrasound acquisition. The display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition relative to the 3D ultrasound image.

Description

Technical Field

[0001] The present invention relates to the field of ultrasound imaging, and in particular to the field of three-dimensional ultrasound imaging systems. Background Art

[0002] Ultrasound imaging is increasingly being adopted in a variety of different applications. It is important to provide the user with enough information about the object being scanned in order to accurately assess the object's condition. This is particularly true when the object in question is a patient undergoing a medical ultrasound scan.

[0003] Some ultrasound imaging systems include an ultrasound probe and an ultrasound probe tracker adapted to track the position of the ultrasound probe. The ultrasound probe is typically adapted to be held by a clinician or other user of the ultrasound imaging system.

[0004] The ultrasound probe may include a CMUT transducer array for transmitting ultrasound and receiving echo information. The transducer array preferably includes a piezoelectric transducer formed of a material such as PZT or PVDF. The transducer array may include a two-dimensional array of transducers that can be scanned in a 2D plane or in three dimensions for 3D imaging. In another example, the transducer array may be a 1D array.

[0005] Known ultrasound probe trackers include electromagnetic or optical tracking systems and transducer-based tracking systems.

[0006] Methods for generating three-dimensional (3D) ultrasound images have been proposed. It has been demonstrated that 3D ultrasound images significantly improve the user's understanding of the imaging volume. Typically, to capture a 3D ultrasound image, an ultrasound probe captures a series of 2D ultrasound images, and an ultrasound probe tracker identifies the position (of the probe) at which each image was captured. The captured 2D ultrasound images are stacked based on their respective capture positions to form a 3D image of the imaging volume.

[0007] WO 2017 / 076758 A1 discloses an ultrasound system that provides an ultrasound image of a volume region. A region of interest within the volume region can be imaged using a beam with a higher frequency than the rest of the volume region, thereby controlling the image quality of the region of interest. Summary of the invention

[0008] The invention is defined by the claims.

[0009] According to an exemplary embodiment of the present invention, a method for providing a three-dimensional 3D ultrasound image and an additional ultrasound acquisition is provided, the method comprising: obtaining a 3D ultrasound image of a volume from an ultrasound imaging system comprising an ultrasound probe and an ultrasound probe tracker; obtaining an additional ultrasound acquisition of a portion of interest of the volume from the ultrasound imaging system; identifying a position of the additional ultrasound acquisition relative to the 3D ultrasound image; and after the steps of obtaining the 3D ultrasound image and obtaining the additional ultrasound acquisition: generating first display data for displaying the 3D ultrasound image; receiving a first user input; and in response to the first user input, generating second display data for displaying the 3D ultrasound image and the additional ultrasound acquisition, wherein the display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition.

[0010] Thus, the initial display data may be for displaying only the 3D ultrasound image (and not for displaying the additional ultrasound acquisition). In response to a first user input, the display data may be modified to display the additional ultrasound acquisition as well as the 3D ultrasound image. Thus, data for displaying the additional ultrasound acquisition is included in the display data only in response to a user input.

[0011] The 3D ultrasound image and the additional ultrasound acquisition(s) are obtained prior to generating the display data.Thus, the proposed method provides a concept for improving the manipulation of previously acquired ultrasound data.

[0012] The display of the additional ultrasound acquisition in the (second) display data depends on the relative position of the region imaged by the additional ultrasound acquisition with respect to the 3D ultrasound image.

[0013] The present invention relies on the concept of determining or otherwise identifying the relative location of ultrasound acquisitions within a 3D ultrasound image. This can be used to provide useful information to clinicians to help them understand the location of ultrasound acquisitions. Therefore, the proposed concept advantageously increases the user's awareness of the information output by the ultrasound imaging system.

[0014] In the proposed embodiment, the position of the region of interest represented by the additional ultrasound acquisition is identified within the 3D ultrasound image and used for defining the display of the additional ultrasound acquisition in the second display data.

[0015] The display data may be used or processed by an ultrasound image display, for example comprising a monitor, to display the 3D ultrasound image and the additional ultrasound acquisition. The display of the additional ultrasound acquisition is dependent on user input. The additional ultrasound acquisition may be a 2D ultrasound image, a video or a cine replay.

[0016] This reduces the memory or processing power required to provide the display data, as the user may not need to initially display the additional ultrasound acquisitions. The additional ultrasound acquisitions are only provided in response to user input, thereby reducing the number of memory accesses required to provide the relevant ultrasound acquisitions to the user. The memory reduction effect increases with the number of additional ultrasound acquisitions.

[0017] In some examples, the first display data is used for display of the 3D ultrasound image and a user-selectable marker, the display position of the user-selectable marker being based on the position of the additional ultrasound acquisition; and the step of receiving a first user input includes receiving the first user input in response to the user selecting the user-selectable marker.

[0018] Therefore, a concept is proposed of displaying an additional ultrasound acquisition only when (ie in response to) a user selecting a marker that has been provided at the location of the additional ultrasound acquisition on the 3D ultrasound image.

[0019] Providing user-selectable tags allows the user to more efficiently search and retrieve stored ultrasound acquisitions. Displaying ultrasound acquisitions only in response to selection of a corresponding tag provides the user with a simultaneous overview of the existence and availability of various ultrasound acquisitions (by way of multiple tags) and increases the user's awareness of the total amount of information available. The memory and processing power (including the number of memory accesses) required to provide the user with relevant ultrasound acquisitions is also reduced, because the ultrasound acquisitions need only be retrieved when the user selects the corresponding tag (e.g., from a database).

[0020] Furthermore, in the case of clinical analysis of 3D ultrasound images, providing markers at ultrasound acquisition locations reduces the likelihood of clinician error and reduces the risk of clinician misinterpreting, misidentifying, or mislocating ultrasound acquisitions or portions of ultrasound acquisition imaging, which may increase risk to the subject. For example, if a clinician misidentifies the location of a tumor, this may result in unnecessary or harmful treatment of the subject at an incorrect location.

[0021] Some embodiments comprise the step of determining an orientation of the additional ultrasound acquisition relative to the 3D ultrasound image, wherein the display of the additional ultrasound acquisition (in the second display data) is based on the determined orientation of the additional ultrasound acquisition.

[0022] Therefore, embodiments propose adjusting the display data such that the displayed ultrasound acquisition is appropriately oriented relative to the 3D ultrasound image. This provides additional information to the user and improves their understanding of the relevance of the displayed ultrasound acquisition. The orientation allows the user to more accurately understand the position of the ultrasound acquisition relative to the 3D ultrasound image. This results in significantly reduced user errors.

[0023] In some examples, there is a step of determining how the additional ultrasound acquisition overlays the 3D ultrasound image based on the identified position; wherein the step of generating the display data includes generating (second) display data for display of the 3D ultrasound image, wherein the additional ultrasound acquisition overlays the 3D ultrasound image at the identified position.

[0024] Hence, the display of the additional ultrasound acquisition (based on the display data) may overlay a corresponding part of the displayed 3D image.In particular, if the position and optionally the orientation of the additional ultrasound acquisition is known with respect to the 3D ultrasound image, the ultrasound acquisition may be suitably displayed to overlay the 3D ultrasound image.

[0025] Embodiments allow for a more accurate and relevant display of images. Superimposing the additional ultrasound acquisition on the corresponding portion of the 3D image reduces the amount of information displayed to the user, since the relevant portion of the 3D image will be replaced by the additional ultrasound acquisition, thereby solving the problem of how to integrate the additional ultrasound in the 3D image. For example, a slice or segment of the 3D image can be replaced with a 2D ultrasound image / video (an example of an additional ultrasound acquisition) at that location. This allows for providing additional information (e.g., a high-resolution 2D image or blood flow depicted in color) while allowing the user to more effectively identify the location and relevance of the additional ultrasound acquisition and minimizing the amount of data displayed.

[0026] Preferably, the step of identifying the location of the additional ultrasound acquisition comprises obtaining the location from the ultrasound probe tracker.

[0027] Since the 3D ultrasound images are generated from the same ultrasound probe tracker, this increases the accuracy and precision of determining the relative positions of the additional ultrasound acquisitions.

[0028] In some examples, identifying the location of the additional ultrasound acquisition includes receiving another user input indicating the location of the additional ultrasound acquisition with respect to the 3D ultrasound image.

[0029] This allows the user to define the position of the additional ultrasound acquisition within the 3D ultrasound image. For example, it can be utilized to provide further information about the 3D ultrasound image.

[0030] For example, the user may extract a portion of the 3D ultrasound image to form the additional ultrasound acquisition, the location of the extracted portion being the location of the additional ultrasound acquisition. This advantageously allows the user to provide or mark a specific portion of the 3D ultrasound image for later review.

[0031] In some embodiments, the method includes the following steps: obtaining multiple additional ultrasound acquisitions of different portions of interest of the volume from an ultrasound imaging system; and identifying the position of each additional ultrasound acquisition relative to the 3D ultrasound image; wherein the step of generating second display data includes generating second display data for displaying the 3D ultrasound image and at least one additional ultrasound acquisition of the multiple additional ultrasound acquisitions through a 3D ultrasound image display, wherein the display of the at least one additional ultrasound acquisition is based on the corresponding position of each at least one additional ultrasound acquisition.

[0032] Thus, more than one additional ultrasound acquisition is available, wherein the second display data comprises data for displaying at least one of these additional ultrasound acquisitions.This enables a selection of ultrasound acquisitions to be displayed to the user, thereby increasing their awareness of the information.

[0033] The step of generating second display data may include generating display data for display of the 3D ultrasound image and sequential display of each additional ultrasound acquisition, wherein the sequential display of each additional ultrasound acquisition is based on a respective position of each of the additional ultrasound acquisitions.

[0034] Thus, based on the relative positions of the ultrasound acquisitions with respect to the 3D ultrasound image, a plurality of additional ultrasound acquisitions may be obtained and display data may be sequentially generated, the data causing the ultrasound acquisitions to be sequentially displayed.

[0035] Displaying the additional ultrasound acquisitions sequentially can help improve the user's understanding of the ultrasound imaging process, thereby preventing information overload for the user. Sequential display also reduces the processing power required to display the additional ultrasound acquisitions because they do not need to be displayed simultaneously.

[0036] In some other embodiments, the step of generating second display data may include generating display data for displaying the 3D ultrasound image and a sequential display of each additional ultrasound acquisition, wherein the sequential display of each additional ultrasound acquisition is based on a respective time at which each additional ultrasound acquisition is captured by the ultrasound imaging system.

[0037] Therefore, a concept is proposed which sequentially displays ultrasound acquisitions based on the time at which each ultrasound acquisition was captured.

[0038] This advantageously allows the display of ultrasound acquisitions to mirror the examination process of the subject. This allows review of the manner in which the ultrasound acquisitions were obtained (e.g., by a colleague, review team, or observer). Such embodiments thus advantageously provide additional information about the subject, as the timing or pattern of acquiring different ultrasound acquisitions may reflect the subject's suspected diagnosis.

[0039] Optionally, the method may also include: receiving a second user input; identifying a location of a region of interest in the 3D ultrasound image based on the second user input; obtaining information about the region of interest; and generating display data for display of the 3D ultrasound image and the information about the region of interest, wherein the display of the information about the region of interest is based on the location of the region of interest.

[0040] Preferably, the information about the region of interest comprises annotations or measurements for the region of interest. Thus, the information may comprise textual information or any other annotations (e.g., circles or diagrams). The second user input may thereby define further information of the 3D ultrasound image. The information of the region of interest may be obtained from the user input, from the additional ultrasound acquisition or from processing (e.g., image processing, analysis or recognition) of the 3D ultrasound image.

[0041] In some examples, information about the region of interest is associated with the additional ultrasound acquisition. In this way, user input can provide information (e.g., labels or annotations) about the additional ultrasound acquisition. This enables the user to provide additional information for subsequent users of the method, for example, for review.

[0042] Thus, in some examples, a concept is proposed of receiving a third user input. And based on the third user input, modifying the additional ultrasound acquisition to provide additional information. The modification may be performed before generating display data for displaying the additional ultrasound acquisition. Alternatively, the display data may be modified to reflect the modification of the additional ultrasound acquisition.

[0043] Thus, the user can modify the additional ultrasound acquisition, for example to add labels, measurements or data to the acquisition. Advantageously, this allows the user to add and control the information provided by the additional ultrasound acquisition.

[0044] In at least one embodiment, the step of obtaining the additional ultrasound acquisitions includes: obtaining multiple additional ultrasound acquisitions of corresponding portions of interest of the volume; the step of identifying the positions of the additional ultrasound acquisitions includes: for each additional ultrasound acquisition, identifying the corresponding position of the additional ultrasound acquisition relative to the 3D ultrasound image; and the step of generating second display data for displaying the 3D ultrasound image and selecting additional ultrasound acquisitions through a 3D ultrasound image display, wherein the display of the selection of the additional ultrasound acquisition is based on the position of each selected additional ultrasound acquisition, and wherein the selection of the additional ultrasound acquisition is less than the total number of additional ultrasound acquisitions.

[0045] Therefore, a general concept is proposed in which two or more additional ultrasound acquisitions are obtained, but the display data is controlled such that not all acquired ultrasound acquisitions are displayed simultaneously. This significantly reduces the processing power required to display the additional ultrasound images while making them available for display.

[0046] An embodiment also provides a computer program product comprising a computer-readable storage medium having computer-readable program code embodied thereon, the computer-readable program code being configured to perform all the steps of any method described herein.

[0047] According to other exemplary embodiments of the present invention, a three-dimensional 3D ultrasound image processing system is provided, the system being adapted to obtain a 3D ultrasound image of a volume from an ultrasound imaging system comprising an ultrasound probe and an ultrasound probe tracker, obtain an additional ultrasound acquisition of a portion of interest of the volume from the ultrasound imaging system, identify a position of the additional ultrasound acquisition relative to the 3D ultrasound image, and generate display data for displaying the 3D ultrasound image and the additional ultrasound acquisition by a 3D ultrasound image display, wherein the display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition.

[0048] A three-dimensional 3D ultrasound image display system may be provided, comprising: a 3D ultrasound image processing system as described herein; and a 3D ultrasound image display adapted to receive display data; and display the 3D ultrasound image and the additional ultrasound acquisition, wherein the display of the additional ultrasound acquisition is based on a position of the additional ultrasound acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Examples of the present invention will now be described in detail with reference to the accompanying drawings, in which:

[0050] Figure 1 An ultrasonic diagnostic imaging system is shown;

[0051] Figure 2 An ultrasound imaging system according to an embodiment is shown;

[0052] Figure 3 and Figure 4 illustrates display of a three-dimensional ultrasound image according to an embodiment;

[0053] Figure 5 illustrates display of a three-dimensional ultrasound image according to another embodiment;

[0054] Figure 6 A three-dimensional ultrasound image display system according to an embodiment is illustrated; and

[0055] Figure 7A method of providing a three-dimensional 3D ultrasound image according to one embodiment is illustrated. DETAILED DESCRIPTION

[0056] The present invention provides a method and processing system for providing a three-dimensional 3D ultrasound image and an additional ultrasound acquisition. The position of the additional ultrasound acquisition with respect to the three-dimensional ultrasound image is determined or obtained. After obtaining the 3D ultrasound image and the additional ultrasound acquisition, initial display data is used to display only the 3D ultrasound image. In response to user input, second display data is generated for displaying both the 3D ultrasound image and the additional ultrasound acquisition. The display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition with respect to the three-dimensional ultrasound image.

[0057] Embodiments are based, at least in part, on the recognition that the understanding of a three-dimensional ultrasound image can be enhanced by enabling the provision of additional ultrasound acquisitions. By basing the provision or display of the additional ultrasound acquisitions on the location of the acquisitions relative to the three-dimensional ultrasound image, the correlation between the 3D ultrasound image and the acquisitions can be more easily understood. The display is based on the location of the acquisitions, thereby providing a viewer of the display with more information than a non-location-based display of the additional ultrasound acquisitions.

[0058] Exemplary embodiments may be employed, for example, in a medical ultrasound imaging system, wherein the additional ultrasound acquisitions represent images of different organs, bones, blood vessels, tumors, etc. Preferably, the additional ultrasound acquisitions are functional images or videos, such as Doppler cine loops.

[0059] The term "ultrasound acquisition" describes any data that can be obtained by an ultrasound probe / system following an ultrasound imaging procedure.An ultrasound acquisition may be an image, video or cine replay of the region / area of ​​interest imaged by the ultrasound probe.

[0060] An ultrasound imaging system will be described hereinafter to help understand the present invention. Figure 1An ultrasonic diagnostic imaging system 101 with an array transducer probe 102 is shown in block diagram form. The array transducer probe 102 includes a transducer unit. Traditionally, piezoelectric materials have been used for ultrasonic transducers. Examples are lead zirconate titanate (PZT) and polyvinylidene fluoride (PVDF), of which PZT is particularly popular as a material of choice. The piezoelectric effect is a reversible process, which means that a mechanically deformed piezoelectric crystal generates internal charge and mechanical strain when it encounters an applied electric field. Introducing alternating current (AC) into a piezoelectric material creates ultrasonic pressure waves at a frequency related to the AC frequency. Single crystal piezoelectric materials can be used to achieve high voltage electrical and electromechanical coupling constants for high performance transducers. Recent developments have led to medical ultrasonic transducers that can be mass-produced by semiconductor processes. Desirably, these processes should be the same as the processes used to produce the application-specific integrated circuits (ASICs) required for ultrasonic probes, such as complementary metal oxide semiconductor (CMOS) processes, particularly for 3D ultrasound. These developments have produced micromechanical ultrasonic transducers (MUTs), preferably in the form of capacitive MUTs (CMUTs). A CMUT transducer is a tiny, diaphragm-like device with electrodes that convert the acoustic vibrations of a received ultrasound signal into a modulated capacitance. A CMUT transducer is able to operate over a wide bandwidth, enabling high-resolution and high-sensitivity imaging, and produces a large pressure output, allowing for a large depth of field for receiving acoustic signals at ultrasound frequencies.

[0061] Figure 1 The transducer array 106 of the CMUT cells 108 described above is shown for transmitting ultrasound and receiving echo information. The transducer array 101 of the system 110 may be a one-dimensional or two-dimensional array of transducer elements that can be scanned in a 2D plane or in three dimensions for 3D imaging.

[0062] The transducer array 106 is coupled to a microbeamformer 112, which controls the transmission and reception of signals by the CMUT array elements. Beamforming is a signal processing method that allows for directional transmission or reception of signals such as ultrasound. Signals at specific angles undergo constructive or destructive interference in the transducer array 106, allowing the desired signal to be selected while ignoring other signals. Receive beamforming can also utilize time delays to receive signals due to differences in echo depth.

[0063] Microbeamformers can at least partially perform beamforming by applying delay and sum beamforming to signals received by adjacent or small groups of transducer elements, for example as described in U.S. Pat. Nos. 5,997,479 (Savord et al.), 6,013,032 (Savord), and 6,623,432 (Powers et al.). Microbeamforming is often performed internally to the probe to reduce the number of signals sent to the main beamformer for processing.

[0064] The microbeamformer 112 is coupled to a transmit / receive (T / R) switch 116 that switches between transmit and receive modes via a probe cable (e.g., a coaxial cable). The T / R switch protects the main beamformer 120 from high energy transmit signals when the microbeamformer is not present or used. The transducer array 106 is directly controlled by the main system beamformer 120. Under the control of the microbeamformer 112, the transmission of ultrasound beams from the transducer array 106 is coupled to the microbeamformer via the T / R switch 116 and the main system beamformer 120 by a transducer controller 118, which receives input from a user operating a user interface or control panel 138. One of the functions controlled by the transducer controller 118 is the direction in which the beam is steered and focused. The beam can be directed straight ahead (perpendicular to the transducer array 106) from the transducer array 106, or steered at different angles to obtain a wider field of view by delaying the excitation pulses sent from the array transducer elements.

[0065] The transducer controller 118 may be coupled to control a voltage source 145 of the transducer array. For example, the voltage source 145 may define DC and AC bias voltages applied to the CMUT cells of the CMUT array 106, for example to generate ultrasound RF pulses in a transmit mode.

[0066] The partially beamformed signals generated by the microbeamformer 112 are forwarded to the main beamformer 120, where the partially beamformed signals from individual patches of transducer elements are combined into a fully beamformed signal and digitized. For example, the main beamformer 120 may have 128 channels, each of which receives partially beamformed signals from tens or hundreds of patches of the CMUT transducer unit 108. In this way, the signals received by thousands of transducer elements of the transducer array 106 can effectively contribute to a single beamformed signal.

[0067] The beamformed signals are coupled to the signal processor 122. The signal processor 122 may process the received echo signals in various ways, such as: bandpass filtering, which passes frequencies within a given range and attenuates frequencies outside the range; decimation, which is the process of reducing the sampling rate of the signal by some typical order of magnitude or integer value; I and Q component separation, which demodulates the wave and its samples to be 80 degrees out of phase; and harmonic signal separation, which is used to separate linear and nonlinear signals in order to identify nonlinear (higher harmonics of the fundamental frequency) echo signals returning from tissue and microbubbles.

[0068] The signal processor 122 may optionally perform other signal enhancements such as: ripple reduction, which removes coherent waves of the same frequency; signal synthesis, which combines multiple ultrasound signals from a given target into one signal by combining the data received from each angle; or noise removal, which removes random or systematic noise from the signal.

[0069] The bandpass filter in the signal processor 122 may be a tracking filter in which the passband slides from a higher frequency band to a lower frequency band as echo signals are received from increasing depths, thereby rejecting noise at higher frequencies from greater attempts where there is no anatomical information in these frequencies. The processed signals are coupled to a bright mode (B-mode) processor 126 and optionally to a Doppler processor 128. The B-mode processor 126 employs detection of the amplitudes of the received ultrasound signals to image structures in the body, such as tissue of organs and blood vessels, and normalizes these amplitudes to grayscale for subsequent display as an image.

[0070] B-mode images of body structures can be formed in a harmonic image mode. Harmonic image mode exploits the nonlinear propagation of ultrasound through the body and its tissues. Harmonic imaging transmits at one frequency and receives at harmonics of that frequency. Typically, the second harmonic is used because higher harmonics have very low amplitudes and are difficult to detect. B-mode images can be formed in a fundamental image mode or a combination of a fundamental image mode and a harmonic image mode. The inclusion of harmonic imaging can provide greater contrast and spatial resolution. Prior work has been performed on the fusion of image modes, and algorithms on this combination are further explained in U.S. Patent No. 6,283,919 (Roundhill et al.) and U.S. Patent No. 6,458,083 (Jago et al.).

[0071] The Doppler processor 128 may, if any, process temporally separate signals from tissue motion and blood flow for detecting motion of material, such as flow of blood cells in the image field. The Doppler processor 40 typically includes a wall filter having parameters that can be set to and / or reject echoes returned from selected types of material in the body. For example, the wall filter may be set to a passband characteristic that passes signals with relatively low amplitude from higher velocity material while rejecting relatively strong signals from lower or zero velocity material.

[0072] The passband characteristic will pass signals from flowing blood while rejecting signals from nearby fixed or slowly moving targets (such as the walls of the heart). The opposite characteristic will pass signals from the moving tissue of the heart while rejecting blood flow signals, which is called tissue Doppler imaging, detecting and depicting the motion of tissue. The Doppler processor receives and processes a series of time-discrete echo signals from different points in the image field; the sequence of echoes from a particular point, called a ensemble. A ensemble of echoes received in rapid succession over a relatively short interval can be used to estimate the Doppler shift of flowing blood, which has a correspondence of Doppler frequency to velocity, which is indicative of blood flow velocity. A ensemble of echoes received over a longer period of time is used to estimate the velocity of slower flowing blood or slower moving tissue. The structure and motion signals generated by the B-mode (and Doppler) processor(s) are coupled to the scan converter 132 and the multiplanar reformatter 144. The scan converter 132 arranges the echo signals in the desired image format according to the spatial relationship in which the echo signals are received. For example, a scan converter may arrange the echo signals into a two-dimensional sector-shaped format, or a pyramidal three-dimensional (3D) image.

[0073] The scan converter may superimpose a B-mode structural image with colors corresponding to the motion of points in the image field with their Doppler estimated velocities to generate a color Doppler image that depicts the motion of tissue and blood flow in the image field. A multiplanar reformatter 144 converts echoes received from points in a common plane in a volumetric region of the body into an ultrasound image of that plane, for example, as described in U.S. Pat. No. 6,443,896 (Detmer). The minimum number of data points required to describe a plane is three, and then the plane measurement may be repeated after measuring these three points by moving a fixed amount in a direction orthogonal to the plane, thereby establishing a volumetric region without acquiring data from the entire volume itself. A volume renderer 142 converts the echo signals of a 3D data set into a 3D image as projected as seen from a given reference point, as described in U.S. Pat. No. 6,530,885 (Entrekin et al.).

[0074] The 2D or 3D images are coupled from the scan converter 132, the multiplanar reformatter 144, and the volume renderer 146 to the image processor 130 for further enhancement, buffering, and temporary storage for display on the image display 140. In addition to being used for imaging, the blood flow values ​​generated by the Doppler processor 128 and the tissue structure information generated by the B-mode processor 126 are coupled to the quantification processor 134. For example, the quantification processor generates measures of different flow conditions (e.g., volumetric rate of blood flow) and structural measurements (e.g., size and gestational age of an organ). The quantification processor 46 can receive output from the user control panel 138, such as points in the anatomical structure of the image to be measured.

[0075] The output data from the quantification processor is coupled to the image processor 136 to generate measurement images and values, wherein the images are on the display 140. The graphics processor 136 may also generate graphic overlays for display with the ultrasound image. These graphic overlays may include standard identifying information, such as the patient name, date and time of the image, imaging parameters, etc. For these purposes, the graphics processor receives input from the user interface 138, such as the patient name, date of birth, etc.

[0076] The user interface 138 is also coupled to the transmit controller 118 to control the generation of ultrasound signals from the transducer array 106, and therefore the images generated by the transducer array 106 and the ultrasound imaging system 101. The user interface 138 may also be coupled to a multi-planar reformatter 144 to select and control the planes of a plurality of multi-planar reformatted (MPR) images, which may be used to perform quantitative metrics in the image field of the MPR images.

[0077] As will be appreciated by those skilled in the art, the above-described embodiments of the ultrasound diagnostic imaging system are intended to give non-limiting examples of such ultrasound diagnostic imaging systems. The skilled person will immediately appreciate that several variations in the architecture of the ultrasound diagnostic imaging system are feasible without departing from the teachings of the present invention. For example, as also indicated in the above-described embodiments, the microwave beamformer 112 and / or the Doppler processor 128 may also be omitted, the ultrasound probe 102 may not have 3D imaging capabilities, etc. Other variations will be apparent to those skilled in the art.

[0078] It should be understood that the present invention is not limited to the described ultrasound diagnostic imaging system, but may relate to any ultrasound imaging system. Such an ultrasound system may be adapted to operate in a variety of ultrasound modes, such as: A-mode (amplitude modulation), B-mode (brightness modulation), M-mode (motion mode), Doppler imaging, harmonic imaging, contrast visualization, etc.

[0079] Figure 2 An object 1 is shown undergoing a three-dimensional 3D ultrasound imaging process, such as previously described, performed by an ultrasound imaging system 2. The object 1 comprises a patient's leg.

[0080] The ultrasound imaging system 2 comprises an ultrasound probe 5 and an ultrasound probe tracker 6. The ultrasound probe 5 comprises a handheld portable probe that can be manually moved by a user over a region of interest. The ultrasound probe 5 is adapted to capture two-dimensional images. The ultrasound probe tracker 6 comprises any known ultrasound probe tracker, such as that present in an electromagnetic tracking probe.

[0081] The three-dimensional 3D ultrasound image processing system 7 is adapted to generate display data for displaying a 3D ultrasound image via a 3D ultrasound image display 8. In particular, the 3D ultrasound image processing system may obtain signals from an ultrasound imaging system, store the signals and / or process the signals to display a 3D ultrasound image.

[0082] To perform a three-dimensional ultrasound imaging process, the user performs a sweep of a volume of interest (e.g., a subject's thigh). In one example, the user brings the ultrasound probe 5 into contact with the subject, initiates an image capture process, and moves the ultrasound probe along a direction of interest 9 while the ultrasound probe captures a series of 2D ultrasound images. As each image is captured, the ultrasound probe tracker 6 monitors and records the position of the ultrasound probe. When the volume of interest has passed under the ultrasound probe (e.g., the user has moved the ultrasound probe over the entire top of the subject's thigh), the user ends the image capture process.

[0083] The start and end of the image capture process may be performed by user input such as a user pressing a button.

[0084] The ultrasound image processing system 7 may, for example, be capable of reconstructing a 3D volume or ultrasound image from a series of captured 2D ultrasound images. The reconstruction takes into account the relative spatial position of each image captured during the sweep. For example, the reconstruction process may include stacking the captured images based on their relative positions and extrapolating between intermediate volumes (e.g., between each captured image) to generate a 3D ultrasound image.

[0085] Under normal imaging conditions, for example, when the subject is stationary throughout the imaging process, the above-described method provides a reconstructed 3D volume or three-dimensional ultrasound image of acceptable quality in all dimensions.

[0086] Other methods for generating a 3D volume or reconstructing a 3D ultrasound image based on a series of two-dimensional ultrasound images are known in the art. For example, an ultrasound probe can be adapted to capture a three-dimensional ultrasound image by simultaneously capturing multiple 2D ultrasound images, which are output to an ultrasound image processing system. In other examples, an array of appropriately arranged ultrasound probes is used. In some examples, the ultrasound imaging system performs reconstruction of the 3D ultrasound image.

[0087] The 3D ultrasound image display 8 is adapted to obtain display data from the ultrasound image processing system 7 to display a 3D ultrasound image. The three-dimensional ultrasound image may be displayed to a user via, for example, a monitor or a screen of the 3D ultrasound image display 8.

[0088] The 3D ultrasound image display 8, the ultrasound image processing system 7 and the ultrasound imaging system 2 can communicate with each other directly or indirectly using any known system. For example, the ultrasound image processing system or the ultrasound imaging system can store its output in a database, such as on a server, a memory stick, a cloud storage system or other storage device, which can be accessed by the 3D ultrasound image display or the ultrasound image processing system, respectively. In other examples, there is direct communication via a wired or wireless communication channel.

[0089] Thus, the ultrasound image processing system 7 is able to generate display data for displaying a 3D ultrasound image. The 3D ultrasound image display 8 is adapted to receive the display data and to display at least a 3D ultrasound image based on the display data.

[0090] The ultrasound image processing system 7 may regenerate or modify the display data to change the display of the 3D ultrasound image display 8. For example, a user may wish to manipulate the orientation or view of the 3D ultrasound image provided by the display. Thus, the display data may be modified in response to user input to reorient, rerotate, or reposition the 3D ultrasound image displayed by the 3D ultrasound image display. Such concepts are known in the art.

[0091] In the proposed invention, the ultrasound image processing system 7 is adapted to generate second display data for displaying the 3D ultrasound image and at least one said additional ultrasound acquisition of the portion of interest. In particular, the initial or first display data will display the 3D ultrasound image (instead of the (one or more) additional ultrasound acquisitions). In response to a user input, second display data for displaying both the 3D ultrasound image and the at least one additional ultrasound acquisition are generated.

[0092] The additional ultrasound acquisition may be an additional ultrasound acquisition captured / acquired by the ultrasound imaging system 2. Capturing the additional ultrasound acquisition may require holding the ultrasound probe at a selected position of the object for a minimum length of time. This may not be possible when performing data capture of a 3D ultrasound image (e.g., due to memory or time limitations when capturing the 3D ultrasound image). Furthermore, it may not be possible to capture the 3D ultrasound image and the additional ultrasound image simultaneously.

[0093] For example, the additional ultrasound acquisition may require capturing a series of 2D images of a region of interest at a specific location on the object. The series of 2D ultrasound images may be processed according to known concepts to generate a functional ultrasound video of the region of interest. One such functional ultrasound video or image is the color Doppler image described previously.

[0094] In other examples, the additional ultrasound acquisition is a high quality or contrast enhanced ultrasound image. High quality or contrast enhanced images generally require that the ultrasound probe be held in a particular position for a minimum length of time.

[0095] In yet other examples, the additional ultrasound acquisition is an XML file that defines changes or modifications to the 3D ultrasound image. For example, the XML file may define the location of blood flow within the 3D image, which is captured by an ultrasound process (e.g., Doppler capture).

[0096] Thus, the additional ultrasound acquisition may define features or information of the 3D ultrasound image (eg, blood flow, highlights, colors, tumor presence, etc.).

[0097] Thus, the additional ultrasound acquisitions may provide more or more specific information than the 3D ultrasound image alone. In most acquisition settings, if the acquisition provides images / video, the additional acquisitions are of higher quality than those acquired during the 3D ultrasound imaging because increased image acquisition time may be used and the ultrasound probe is not moved.

[0098] In still other embodiments, the user may define the additional ultrasound acquisition. In such embodiments, the additional ultrasound acquisition may be a segment or extraction of a 3D ultrasound image. The location of the additional ultrasound acquisition may be defined by a location extracted from the 3D ultrasound image. Optionally, the user may provide annotations (e.g., text, graphic shapes, or numerical indicators) on the extracted additional ultrasound acquisition. Such embodiments may be useful for marking locations where further study or ultrasound imaging is desired, or for providing additional information (e.g., in the form of annotations) for later use. It will be clear that the additional ultrasound acquisition is still derived from the information provided by the ultrasound imaging system 2.

[0099] The present invention proposes a concept wherein the display of the additional ultrasound acquisition depends on the position of the additional ultrasound acquisition relative to the 3D ultrasound image. Hence, the second display data generated by the image processing system 7 depends on the position of the additional ultrasound acquisition.

[0100] The ultrasound probe tracker 6 may record the position of the additional ultrasound acquisition by recording the position where the additional ultrasound acquisition was captured. The recorded position may be used to establish the relative position of the additional ultrasound acquisition with respect to the 3D ultrasound image.

[0101] Since the same ultrasound probe tracker 6 is used in the generation of the 3D ultrasound image, the recorded position is correctly calibrated for the 3D ultrasound image. Therefore, the ultrasound probe tracker 6 can easily define the relative position of the additional ultrasound acquisition with respect to the 3D ultrasound image. Such an embodiment provides maximum accuracy in determining the relative position of the additional ultrasound acquisition.

[0102] In another example, the user may manually define the location of the additional ultrasound acquisition, for example using a user input device.As an example only, the display may show a 3D ultrasound image and the user may select the location by selecting an appropriate location using a user input device.

[0103] In still other examples, the location of the additional ultrasound acquisition may be identified by an image recognition process or by an image comparison process, which may compare the captured additional ultrasound acquisition with the 3D ultrasound image to identify the location of the additional ultrasound acquisition within the 3D ultrasound image. In particular, a portion of interest within the additional ultrasound acquisition may be identified (e.g., a specific organ), and the location of the portion within the 3D ultrasound image may be identified as a relative location.

[0104] The relative position of the additional ultrasound acquisition may, for example, define a display position, a display timing (i.e., when to display the additional ultrasound image) or whether to display the additional ultrasound acquisition. In this way, the relative position of the additional ultrasound acquisition (relative to the 3D ultrasound image) may define whether the display data includes (displays) the data of the additional ultrasound acquisition and / or the position of the additional ultrasound acquisition.

[0105] After obtaining the 3D ultrasound image and obtaining the additional ultrasound acquisition, first display data and second display data are generated.

[0106] Will refer to Figure 3 to Figure 4 A particularly advantageous embodiment is described which uses user selectable markers to identify locations for ultrasound acquisitions.

[0107] In such an embodiment, the initially generated display data (first display data) defines user selectable markers for the 3D ultrasound image. Each user selectable marker (within the 3D ultrasound image) is positioned at the location of a corresponding additional ultrasound acquisition and is associated with the additional ultrasound acquisition.

[0108] In response to the user selecting the user-selectable marker, subsequent display data (second display data) is generated for displaying the 3D ultrasound image and the additional ultrasound acquisition associated with the marker. In this way, the additional ultrasound acquisition is not displayed (e.g., displayed via the ultrasound image) until the marker associated with the additional ultrasound acquisition is selected.

[0109] In other words, the initial display data (first display data) includes data for displaying the 3D ultrasound image and information about the location of the additional ultrasound acquisition. In response to receiving the user input, the initial display data is modified (i.e., second display data is created) to also include data for displaying the additional ultrasound acquisition associated with the user input, e.g., selection of an appropriate label.

[0110] Providing user-selectable tags allows the user to more efficiently search and retrieve stored ultrasound acquisitions. In particular, displaying ultrasound acquisitions only in response to selection of a corresponding tag provides the user with a simultaneous overview of the existence and availability of various ultrasound acquisitions and increases the user's awareness of the total amount of information available. The memory and processing power (including the number of memory accesses) required to provide the user with relevant ultrasound acquisitions is also reduced, because the ultrasound acquisitions need only be retrieved when the user selects the corresponding tag (e.g., from a database).

[0111] Furthermore, in the case of clinical analysis of 3D ultrasound images, providing a marker at the location of the portion of interest reduces the likelihood of clinician error and reduces the risk of clinician misinterpreting, misidentifying or mislocating the portion of interest, which may increase the risk to the subject. For example, if the clinician misidentifies the location of a tumor, this may result in unnecessary or harmful treatment of the subject at an incorrect location.

[0112] Figure 3 A three-dimensional (3D) ultrasonic image display 21 of a three-dimensional ultrasonic image is conceptually shown. The display 21 is generated by display data (first display data) and can be displayed by the 3D ultrasonic image display 8.

[0113] For clarity, three views of the 3D volume are illustrated: a top view 20A; an elevation view 20B (viewed from line yl - y2 ); and a side view 20C (viewed from line xl - x2 ).

[0114] As described above, the display data includes data for displaying a 3D ultrasound image, and the display data is processed by the 3D ultrasound image display 8 and displayed on the 3D ultrasound image display 21 .

[0115] The display data also defines the location of the additional ultrasound acquisition. The 3D ultrasound image display 21 thereby identifies the location of the additional ultrasound acquisition ("location of interest"). The location of interest is defined relative to the imaged 3D volume or 3D ultrasound image.

[0116] The defined positions of the additional ultrasound acquisitions define the positions of the user selectable markers 22 for display in the displayed 3D ultrasound image.Therefore, the display 21 of the 3D ultrasound image also displays the user selectable markers at the identified positions of the additional ultrasound acquisitions.

[0117] Thus, initial or first display data is provided for displaying a 3D ultrasound image and for displaying one or more user selectable markers, each marker being respectively associated with an additional ultrasound acquisition.

[0118] The initial or first display data (and thereby the display 21) is modified in response to the user selecting the user-selectable marker. In particular, the modified display data includes data for display of the 3D ultrasound image and the additional ultrasound acquisition (associated with the selected user-selectable marker 22). Thus, the display 21 displays the appropriate additional ultrasound acquisition in response to the user selecting the user-selectable marker 22.

[0119] Thus, a user selectable marker 22 is provided at the location of the additional ultrasound acquisition and is mapped or associated with that additional ultrasound acquisition. In response to the user selecting the user selectable marker, the 3D ultrasound image display displays the additional ultrasound acquisition to the user. The user can select the user selectable marker using any known user input device (e.g., a touch screen interface, a computer mouse, a keyboard, etc.).

[0120] Figure 4 Conceptually, the display of the three-dimensional ultrasound image 21 is shown after a user has selected the user selectable marker 22. As previously described, the display 21 is provided by (modified) display data (second display data) and may be displayed by the 3D ultrasound image display 8.

[0121] Figure 4 The same three views of the imaged 3D volume 21 are illustrated: a top view 30A; an elevation view 30B and a side view 30C. As previously described, upon selection of a user-selectable marker, the initial or first display data is modified for display of the 3D ultrasound image and the additional ultrasound acquisition. Thus, a second display data is provided, which is provided for displaying the 3D ultrasound image 21 and the additional ultrasound acquisition 23.

[0122] exist Figure 4 In the present invention, the additional ultrasound acquisition includes a two-dimensional 2D ultrasound image or video of the portion of interest.

[0123] By generating display data for displaying the 3D ultrasound image 21 and the additional ultrasound acquisitions 23 only in response to a user selecting a user-selectable marker 22, the total amount of data transferred or transmitted for display is reduced (because the additional ultrasound acquisitions are only transmitted when the user-selectable marker is selected). The greater the size of the additional ultrasound acquisitions or the greater the number of user-selectable markers, the greater the effect of reducing the total amount of data.

[0124] Moreover, providing user-selectable markers at the locations of the portions of interest allows the user to perform the task of diagnosing the problem more efficiently because the user is more directly aware of the location of the additional information. Thus, the arrangement and location of the user-selectable markers provides the user with previously unavailable cognitive content (indicating the location of additional ultrasound acquisitions).

[0125] As shown, the display data can be adapted so that the location of the additional ultrasound acquisition 23 displayed also depends on the location of the user-selectable marker 22. This helps to improve the user's understanding of the available information and reduce screen clutter. For example, information can be displayed next to or at the location indicated by the user-selectable marker.

[0126] Thus, in response to the user selecting the user selectable marker, the user selectable marker marking the additional ultrasound acquisition may be replaced by the additional ultrasound acquisition itself. In particular, if the additional ultrasound acquisition is a 2D image or video of the portion of interest of the object, the 2D image / video may be overlaid (e.g. Figure 4 The displayed additional ultrasound acquisition may thus replace a portion of the 3D ultrasound image.

[0127] In other examples, the displayed additional ultrasound acquisition is positioned to one side of the 3D ultrasound image, for example, in line with the user-selectable marker or a pointer pointing to the user-selectable marker. The display position of the additional ultrasound acquisition may thus depend on the position of the additional ultrasound acquisition relative to the 3D ultrasound image.

[0128] In an embodiment, the ultrasound image processing system is adapted to obtain the depth of the portion of interest in the additional ultrasound acquisition.As mentioned before, the additional ultrasound acquisition may represent an image / video containing the portion of interest (eg, an organ, bone, tumor, blood vessel, etc.).

[0129] The depth of the portion of interest may be obtained from a user input device or determined from information about the portion of interest. In one example, the length of a button press (of a user input device) may indicate the depth of the portion of interest. For example, when the additional ultrasound acquisition is captured using an ultrasound probe, the length of the button press (to capture the position) may indicate the relative depth of the portion of interest. The longer the press time, the further away the portion of interest is in the ultrasound probe.

[0130] The (first) display data may be configured such that the displayed user-selectable marker 22 includes an indication of the depth of the portion of interest relative to the current view of the 3D ultrasound image. Thus, the user-selectable marker may indicate the location of the portion of interest (associated with the additional ultrasound acquisition) within the entire 3D ultrasound image.

[0131] Thus, the 3D ultrasound image display may indicate the depth of the portion of interest (to which the additional ultrasound acquisition relates) relative to the current view of the 3D ultrasound image display. The indication may be textual or graphical (eg, different color or size).

[0132] In particular, where the acquisition is an image or video, the depth may indicate the distance from the edge of the ultrasound image or the position within the acquisition.

[0133] For example, a 2D ultrasound image may be an image of a specific organ (portion of interest) where the organ is located only in a portion of the overall 2D ultrasound image. An image recognition process or user input may identify the location of the organ within the 2D ultrasound image to establish the location of the organ relative to the edge of the ultrasound image. If the orientation and location of the 2D ultrasound image relative to the imaged 3D volume is known, the 3D location (i.e., depth) of the organ may be easily identified.

[0134] If the position and / or orientation of a 2D image / video relative to the 3D imaging volume is known, the depth of the portion of interest (within that image / video) with respect to the 3D ultrasound image can be easily identified. Providing information about depth increases the user's awareness of information beyond what was previously available to them.

[0135] although Figure 3 and Figure 4 Only a single additional ultrasound acquisition and its user-selectable label are illustrated, but it is apparent that the ultrasound image processing system may be adapted to obtain a plurality of additional ultrasound acquisitions and to provide display data for displaying the 3D ultrasound image and the user-selectable label for each additional ultrasound acquisition.

[0136] Figure 5 Another three-dimensional (3D) ultrasound image display 41 of a three-dimensional ultrasound image according to another embodiment is shown. The display 41 is provided based on display data generated by the ultrasound image processing system 7 and may be displayed by the 3D ultrasound image display 8.

[0137] The initial (first) display data contains data for the display 41 of a 3D ultrasound image and is provided by an ultrasound image processing system.

[0138] The ultrasound image processing system is adapted to obtain a plurality of additional ultrasound acquisitions and to identify the position of each ultrasound acquisition relative to the 3D ultrasound image. Here, the plurality comprises a first 42, a second 43 and a third 44 additional ultrasound acquisition.

[0139] For the purpose of conceptual understanding, Figure 5 Also shown are exemplary locations of first 42 , second 43 and third 44 additional ultrasound acquisitions relative to the 3D ultrasound image.

[0140] The initial display data (for display of the 3D ultrasound image only) may be sequentially modified in response to the first user input such that the display 41 continuously displays the 3D ultrasound image and sequentially displays a plurality of additional ultrasound acquisitions.

[0141] In one embodiment, the display data is modified sequentially based on the position of each additional ultrasound acquisition.For example, the display data may be modified sequentially such that the display 41 sequentially displays the additional ultrasound acquisitions 42, 43, 44 as they are positioned with respect to the direction.

[0142] In one such example, the initial display data may first be modified to generate second display data in which the additional ultrasound acquisitions 42 associated with the location closest to the predetermined location 45 on the 3D ultrasound image are displayed. The second display data may then be sequentially modified so that the additional ultrasound acquisitions 43, 44 are sequentially displayed at increasingly distant locations. Thus, the further the location of the additional ultrasound acquisition is from the predetermined location, the later in the order in which the display data is provided for the additional ultrasound acquisition. For example, this may allow the additional ultrasound acquisitions to be sequentially displayed from the top of the display (e.g., the top of the user's leg) to the bottom of the display (e.g., the bottom of the user's leg).

[0143] In another embodiment, the displayed data is sequentially modified based on the time at which each additional ultrasound acquisition is obtained or captured.

[0144] For example, the display data may be sequentially modified so that the earliest (i.e., earliest captured) additional ultrasound acquisition is displayed first, followed by the additional ultrasound acquisitions captured in the order of capture. Thus, the initial display data may be first modified so that the additional ultrasound acquisition captured is displayed first, then modified so that the second captured additional ultrasound acquisition is displayed, and so on. In this manner, the later the additional ultrasound acquisition is captured, the later the additional ultrasound sequence appears in the display data.

[0145] The user input triggers the sequential modification of the initial display data to generate the second display data. For example, the user can define the position of the predetermined position 45 (or the desired direction) on the displayed 3D ultrasound image, which will trigger the sequential modification of the initial display data based on the distance from the predetermined position 45.

[0146] In another example, a user may define a start time for displaying additional ultrasound acquisitions, and modify the display data such that a sequence of additional acquisitions captured after the start time are included sequentially (from oldest to newest) in the display data.

[0147] The display data is modified so as to display the relevant additional ultrasound acquisition(s) at the location of the additional ultrasound acquisition on the displayed 3D ultrasound image. This provides additional information to the user and avoids confusion or misinterpretation.

[0148] Thus, the display data may be sequentially modified such that the plurality of additional ultrasound acquisitions are sequentially displayed in the order of acquisition or in an order determined during post-processing of the additional ultrasound acquisitions. Thus, post-processing of the additional ultrasound acquisitions may be performed to determine the sequence order (e.g., along a particular direction).

[0149] Therefore, embodiments provide for sequentially modifying display data to provide a sequential display of additional ultrasound acquisitions. The order of the sequential display may be based on, for example, an order of capture, an order of position, an order representative of a typical or known medical examination procedure, etc. The sequential display of ultrasound acquisitions advantageously reduces the amount of storage required to display the images, and also provides additional information to a user or observer (i.e., the display order provides new information, such as the capture order, beyond what was previously available).

[0150] Figure 6 The ultrasonic image display system 50 according to an embodiment of the present invention is illustrated. The ultrasonic image display system 50 includes a three-dimensional ultrasonic image processing system 7 and a 3D ultrasonic image display 8. The ultrasonic image display system 50 also includes a user input device 51.

[0151] The 3D image processing system 7 is adapted to obtain a volumetric 3D ultrasound image from an ultrasound imaging system 2 comprising an ultrasound probe (not shown) and an ultrasound probe tracker (not shown).

[0152] The 3D image processing system 7 is further adapted to obtain an additional ultrasound acquisition of the portion of interest of the volume and to identify the position of the additional ultrasound acquisition within the 3D ultrasound image.

[0153] The 3D image processing system 7 directly or indirectly obtains signals (e.g., 2D ultrasound images) from the additional ultrasound acquisition of the ultrasound imaging system 2. The 3D image processing system 7 can identify the location of the additional ultrasound acquisition by obtaining the location from the ultrasound imaging system, for example, an ultrasound probe marking the portion of interest or the location of the additional ultrasound acquisition, or a user input device 43.

[0154] The 3D image processing system 7 is adapted to provide or generate display data for displaying 3D ultrasound images and additional ultrasound acquisitions using the methods described previously.

[0155] The 3D ultrasound image display 8 is adapted to display a 3D ultrasound image. In particular, the 3D ultrasound image display is adapted to receive display data generated by the 3D image processing system 7. For example, in the case where the display data is the second display data, the 3D ultrasound image display 8 will display the 3D ultrasound image and the additional ultrasound acquisition, wherein the display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition.

[0156] In case the first display data is used to display the 3D ultrasound image and the user selectable marks, the user input device 51 allows the user to select the user selectable marks displayed by the 3D ultrasound image display 8. The user input device may include, for example, a mouse, a keyboard and / or a touch sensitive interface (e.g., coupled to the display 8).

[0157] In some embodiments, the 3D image processing system 7 is adapted to generate the 3D ultrasound image itself. For example, the ultrasound imaging system may output a series of 2D ultrasound images to which the 3D image processing system 7 applies the previously described 3D reconstruction process. In other embodiments, the ultrasound imaging system 2 provides a reconstructed or captured 3D ultrasound image.

[0158] In any embodiments described herein, a relative orientation of the additional ultrasound acquisition relative to the 3D ultrasound image may be obtained. The display data may be adapted such that the orientation of the displayed additional ultrasound acquisition (or associated user-selectable marker) is based on its relative orientation. In particular, the orientation of the displayed additional ultrasound acquisition may correspond to the orientation of the additional ultrasound acquisition (when captured) relative to the 3D ultrasound image.

[0159] Optionally, as the 3D ultrasound image is rotated (eg, in a display), the additional ultrasound acquisitions are rotated to maintain their relative orientation.

[0160] The ultrasound imaging system may capture one or more orientation reference points of a 3D ultrasound image during a 3D ultrasound imaging process. For example, a first orientation reference point may be captured at the beginning of the ultrasound imaging process.

[0161] The relative orientation of the displayed information can then be captured by the ultrasound imaging system. For example, when capturing the position of the portion of interest, the ultrasound probe tracker can determine the relative orientation of the ultrasound probe relative to one or more orientation reference points. The ultrasound probe tracker may include a probe orientation determiner for capturing the orientation reference points. The determiner may include, for example, an accelerometer, a gyroscope, or other orientation determination device.

[0162] In this way, both the translational position (ie, location) and the rotational position (ie, orientation) of the portion of interest can be used to position additional ultrasound acquisitions or user-selectable markers.

[0163] In some embodiments described herein, information about the portion of interest may be provided by a user. The information may be associated with the 3D ultrasound image or an additional ultrasound acquisition. The information about the portion of interest may include a marker, graphic, text, or digital information of the portion of interest. For example, the information may include a measure of the portion of interest, an annotation of the portion of interest, graphic information associated with the portion of interest, a label of the portion of interest, a highlight of the portion of interest, a graphic overlay of the portion of interest, and the like.

[0164] In some preferred embodiments, the additional ultrasound acquisition can be modified by the user to include annotations, measurements or other text / numeric information about the portion of interest. Thus, multiple pieces of information about the portion of interest can be combined into a single additional ultrasound acquisition, thereby reducing the number of memory accesses to provide information about the portion of interest and improving the efficiency of the system. The text / numeric information can be entered via user input 51, which can include, for example, a combination of a mouse and keyboard or a touch screen.

[0165] The text information may be added to the additional ultrasound acquisition during the capture process of the additional ultrasound acquisition. Thus, the clinician / user may mark or indicate other important features during the capture of the additional ultrasound acquisition.

[0166] Figure 7 A method 60 of providing a three-dimensional 3D ultrasound image with additional ultrasound acquisition is illustrated.

[0167] The method 60 comprises a step 61 of obtaining a 3D ultrasound image of a volume from an ultrasound imaging system comprising an ultrasound probe and an ultrasound probe tracker.

[0168] The method 60 further comprises the step 62 of obtaining an additional ultrasound acquisition of the portion of interest of the volume from the ultrasound imaging system.

[0169] The method 60 further comprises a step 63 of identifying a position of the additional ultrasound acquisition relative to the 3D ultrasound image.

[0170] The method 60 comprises a step 64 of generating first display data for displaying a 3D ultrasound image.

[0171] The method 60 also includes a step 65 of receiving a first user input.

[0172] The method further comprises the following step 66: in response to the first user input, generating second display data for displaying the 3D ultrasound image and the additional ultrasound acquisition, wherein the display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition. Thus, step 65 can be a determination step for determining whether the first user input has been received.

[0173] After at least steps 61 and 62 of obtaining a 3D ultrasound image and additional ultrasound acquisition, steps 64 to 66 are performed.

[0174] The method may be adapted to perform any number of the steps performed by the three-dimensional ultrasound image processing system previously described.

[0175] In all of the described embodiments, in order to avoid adding memory or processor burden to the 3D ultrasound image display, there may be a maximum number of additional ultrasound acquisitions for display. Therefore, the display data may be adapted to provide for display of only a limited number of additional ultrasound acquisitions. For example, if more than a predetermined number (e.g., 5) of additional ultrasound acquisitions are desired, the display data associated with the longest display time may be removed from the display (e.g., the longest additional ultrasound acquisition has already been displayed).

[0176] In all described embodiments, the method or image processing system may be adapted such that, in response to a specific user input, the display data removes the display data associated with a specific additional ultrasound acquisition. By way of example only, including another additional ultrasound acquisition in the display data may remove an earlier additional ultrasound acquisition from the display data. In another example, there may be a user selectable button for removing data used to display the additional ultrasound acquisition from the display data.

[0177] In at least one embodiment, a 3D ultrasound image is displayed so that it is aligned with the main direction of the patient, thereby mimicking a traditional CT or MR scan. The ultrasound probe can be adapted to record the orientation of the patient in space (e.g., to identify an axial view of the patient). For example, this can be done by aligning the probe with the object or the patient. For example, if the object is a patient in a hospital bed, the probe can be aligned with the main direction of the bed (i.e., arranged to be located along the direction from the head of the bed to the foot of the bed). A reasonable assumption is that the patient will be well aligned with the bed, so that the probe can be aligned with the axial direction of the patient. This provides a simple and intuitive mechanism for determining the direction of the patient using an ultrasound probe.

[0178] As described in the previous paragraphs, the additional ultrasound acquisition can be a 2D ultrasound image / video, such as a Doppler or contrast-enhanced ultrasound image. Preferably, the 2D ultrasound image / video has a higher resolution, contrast, quality, or information value than the 3D ultrasound image, or provides information that is not available in the 3D ultrasound image alone (e.g., blood flow information). In this way, providing a 2D image can significantly increase the clinician or user's understanding of the imaging volume.

[0179] The proposed method and processing system are particularly suitable for providing additional ultrasound acquisitions in response to a user selecting a user-selectable marker on a display of a 3D ultrasound image, as the additional ultrasound acquisitions may require significant processing power and / or storage requirements to retrieve and display. By only displaying the additional ultrasound acquisitions when the corresponding marker is selected, the efficiency of the system can be significantly improved without affecting the usability of the image to the clinician or user.

[0180] As described above, embodiments utilize a three-dimensional ultrasound image processing system. The processing system can be implemented in a variety of ways, used in conjunction with software and / or hardware to perform various desired functions. For example, the processing system can include a controller that employs one or more microprocessors that can be programmed using software (e.g., microcode) to perform the desired functions. However, the processing system can alternatively be implemented without employing a processor, and can also be implemented as a combination of dedicated hardware for performing some functions and a processor (e.g., one or more programmed microprocessors and associated circuits) for performing other functions.

[0181] Examples of processing system components that may be used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0182] In various implementations, the processing system may be associated with one or more storage media, such as volatile and non-volatile computer memory, such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when run on one or more processors and / or controllers, perform the desired functions. The various storage media may be fixed within the processor or controller, or may be transportable so that one or more programs stored thereon may be loaded into the processing system.

[0183] Those skilled in the art will be able to understand and implement other variations of the disclosed embodiments in practicing the claimed invention by studying the drawings, the disclosure and the claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. Although specific measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. A method (60) for providing a three-dimensional 3D ultrasound image (21) and an additional ultrasound acquisition (23), the method comprising: obtaining (61) a 3D ultrasound image of the volume from an ultrasound imaging system (2) including an ultrasound probe and an ultrasound probe tracker; obtaining (62) from the ultrasound imaging system an additional ultrasound acquisition of a portion of interest of the volume, wherein, when the additional ultrasound acquisition is captured using the ultrasound probe, a duration of a button press to capture the identified location of the additional ultrasound acquisition indicates a relative depth of the portion of interest; identifying (63) a position of the additional ultrasound acquisition relative to the 3D ultrasound image; and After obtaining the 3D ultrasound image and obtaining the additional ultrasound acquisition step: generating first display data for displaying the 3D ultrasound image and not for displaying the additional ultrasound acquisition; receiving a first user input; and In response to the first user input, second display data for displaying the 3D ultrasound image (21) and the additional ultrasound acquisition (23) is generated, wherein the display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition.

2. The method according to claim 1, wherein: The first display data is used for display of the 3D ultrasound image (21) and a user selectable marker (22), the display position of the user selectable marker being based on the position of the additional ultrasound acquisition; and The step of receiving a first user input includes receiving a first user input in response to the user selecting the user-selectable indicia.

3. The method according to claim 1, further comprising: An orientation of the additional ultrasound acquisition with respect to the 3D ultrasound image is determined, wherein the display of the additional ultrasound acquisition is based on the determined orientation of the additional ultrasound acquisition.

4. The method according to any one of claims 1 to 3, further comprising: determining how the additional ultrasound acquisitions overlay the 3D ultrasound image based on the identified locations; and The step of generating the display data comprises generating display data for display of the 3D ultrasound image, wherein the additional ultrasound acquisition overlays the 3D ultrasound image at the identified location.

5. The method according to any one of claims 1 to 3, wherein: The step of identifying the location of the additional ultrasound acquisition comprises obtaining the location from the ultrasound probe tracker.

6. The method according to any one of claims 1 to 3, comprising the steps of: obtaining a plurality of additional ultrasound acquisitions (42, 43, 44) of different portions of interest of the volume from the ultrasound imaging system; and identifying a position of each additional ultrasound acquisition relative to the 3D ultrasound image (41); The step of generating the second display data comprises generating (64) second display data for displaying the 3D ultrasound image and at least one additional ultrasound acquisition of the plurality of additional ultrasound acquisitions via a 3D ultrasound image display (8), wherein the display of the at least one additional ultrasound acquisition is based on a corresponding position of each at least one additional ultrasound acquisition.

7. The method according to claim 6, wherein: The step of generating the second display data comprises: Second display data is generated for displaying the 3D ultrasound image (41) and a sequential display of each additional ultrasound acquisition (42, 43, 44), wherein the sequential display of each additional ultrasound acquisition is based on a respective position of each of the additional ultrasound acquisitions.

8. The method according to claim 6, wherein: The step of generating the second display data comprises: Second display data is generated for displaying the 3D ultrasound image (41) and a sequential display of each additional ultrasound acquisition (42, 43, 44), wherein the sequential display of each additional ultrasound acquisition is based on a respective time at which each additional ultrasound acquisition was captured by the ultrasound imaging system.

9. The method according to any one of claims 1 to 3, further comprising: receiving a second user input; identifying a location of a region of interest in the 3D ultrasound image based on the second user input; obtaining information about the region of interest; and Display data for displaying the 3D ultrasound image and information about the region of interest is generated, wherein the display of the information about the region of interest is based on the position of the region of interest.

10. The method according to any one of claims 1 to 3, further comprising: receiving a third user input; and The additional ultrasound acquisition is modified based on the third user input.

11. The method according to any one of claims 1 to 3, wherein: The step of obtaining the additional ultrasound acquisitions comprises: obtaining a plurality of additional ultrasound acquisitions of respective portions of interest of the volume; The step of identifying the positions of the additional ultrasound acquisitions comprises: for each additional ultrasound acquisition, identifying the respective position of the additional ultrasound acquisition with respect to the 3D ultrasound image; and the step of generating the second display data comprises the step of generating display data for displaying the 3D ultrasound image and the selection of the additional ultrasound acquisitions by a 3D ultrasound image display, wherein the display of the selection of the additional ultrasound acquisitions is based on the position of each selected additional ultrasound acquisition, wherein the selection of the additional ultrasound acquisitions comprises less than a total number of additional ultrasound acquisitions.

12. A computer program product comprising a computer-readable storage medium having computer-readable program code embodied therein, the computer-readable program code being configured to perform the method according to any one of claims 1 to 11.

13. A three-dimensional 3D ultrasound image processing system (7), adapted to: obtain a volumetric 3D ultrasound image (21) from an ultrasound imaging system (2) comprising an ultrasound probe (5) and an ultrasound probe tracker (6); An additional ultrasound acquisition of the portion of interest of the volume is obtained from the ultrasound imaging system (23), wherein: when capturing the additional ultrasound acquisition using the ultrasound probe, the duration of a button press to capture the identified location of the additional ultrasound acquisition indicates a relative depth of the portion of interest; identifying a position of the additional ultrasound acquisition relative to the 3D ultrasound image; and After obtaining the 3D ultrasound image and obtaining the additional ultrasound acquisition, generating (64) first display data for displaying the 3D ultrasound image and not for displaying the additional ultrasound acquisition; receiving (65) a first user input; and In response to the first user input, second display data for displaying the 3D ultrasound image (21) and the additional ultrasound acquisition (23) is generated (66), wherein the display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition.

14. A three-dimensional 3D ultrasound image display system (50), comprising: The 3D ultrasound image processing system (7) according to claim 13; as well as A 3D ultrasound image display (8) is adapted to: receive the display data; and display the 3D ultrasound image and the additional ultrasound acquisition, wherein the display of the additional ultrasound acquisition is based on the position of the additional ultrasound acquisition.

Images