Method and system for controlling a virtual light source for a volume rendered image

Abstract

The present invention is entitled as a method and system for controlling a virtual light source for a volume rendered image. The present invention discloses a system and method for controlling a virtual light source, the system and method comprising: generating a volume rendered image including a rendered object; controlling shading and lighting of the volume rendered image based on a position of the virtual light source relative to the rendered object; and receiving a control input from a user interface. The system and method comprises, in response to the control input, automatically moving the virtual light source to an updated position along a height profile established relative to a surface of a rendered object in the volume rendered image. The system and method comprises calculating updated shading and updated lighting based on the updated virtual light source position.

Description

Technical Field

[0001] The present disclosure generally relates to methods and systems for controlling the position of a virtual light source when displaying a volume-rendered image. Background Art

[0002] Volume rendering images are very useful for representing 3D medical imaging datasets. Volume rendering images are typically 2D representations of 3D medical imaging datasets. There are currently many different techniques for generating volume rendering images. One such technique, ray casting, includes projecting multiple rays through a 3D medical imaging dataset. Each sample in the 3D medical imaging dataset is mapped to a color and transparency. Data is accumulated along each ray. According to a common technique, the data accumulated along each ray is displayed as pixels in a volume rendering image. In order to obtain an additional sense of depth and perspective, the volume rendering image is often colored and illuminated based on the position of a virtual light source relative to the rendered object shown in the volume rendering image. Coloring and lighting can be used to convey the relative positioning of structures or surfaces shown in the volume rendering image. Coloring and lighting help the observer to more easily visualize the three-dimensional shape of the rendered object represented by the volume rendering image.

[0003] In order to fully understand the three-dimensional shape of the rendered object in the volume rendered image, it may be desirable for the user to adjust the position of the virtual light source used to calculate the shading and illumination of the volume rendered image. Viewing the volume rendered image when the volume rendered image is illuminated and shaded with the virtual light source in different positions will help the user to more fully understand the three-dimensional shape and geometry of the rendered object in the volume rendered image.

[0004] Conventional solutions are configured to adjust the angular position of the virtual light source relative to the rendered object. However, this often results in different distances between the virtual light source and the rendered object, depending on the shape of the rendered object represented in the volume rendering. Conventional volume rendering techniques typically model the intensity of light from the light source to decrease with distance. This results in a lower level of illumination when the virtual light source is at a greater distance from the rendered object.

[0005] According to other conventional solutions, users need to control the position of one or more virtual light sources in three-dimensional space. Due to the challenge of controlling the three-dimensional position of the virtual light source relative to the drawn object, it may be difficult for the user to quickly and accurately position the virtual light source in the desired position.

[0006] Therefore, for these and other reasons, there is a need for an improved system and method for controlling the position of a virtual light source. Summary of the invention

[0007] The above-mentioned shortcomings, disadvantages and problems are addressed herein, which will be understood by reading and understanding the following specification.

[0008] In one embodiment, a volume rendering method includes accessing a 3D medical imaging dataset and generating a volume rendered image including a rendered object from the 3D medical imaging dataset. The method includes controlling shading and lighting of the volume rendered image based on a position of a virtual light source relative to the rendered object. Controlling the shading and lighting includes performing the following steps: a) receiving a control input from a user interface; b) in response to the control input, automatically moving the virtual light source to an updated position along a height profile established relative to a surface of the rendered object in the volume rendered image; c) calculating updated shading and updated lighting based on the updated virtual light source position; d) displaying the volume rendered image with the updated shading and updated lighting; and e) repeating steps a), b), c), and d) as the virtual light source is moved in response to the control input.

[0009] In one embodiment, a system for interacting with a 3D medical imaging dataset includes a display device, a user interface, and a processor communicatively connected to the display device and the user interface. The processor is configured to access the 3D medical imaging dataset and generate a volume rendered image including a medical object from the 3D medical imaging dataset. The processor is configured to receive a control input from the user interface to adjust a position of a virtual light source relative to the rendered object, wherein the position of the virtual light source is used to calculate illumination and shading of the volume rendered image. The processor is configured to automatically move the virtual light source to an updated position along a height profile established relative to a surface of the rendered object in the volume rendered image in response to the control input. The processor is configured to display the volume rendered image with illumination and shading determined based on the real-time position of the virtual light source during the process of automatically moving the virtual light source to the updated position.

[0010] In one embodiment, an ultrasound imaging system includes an ultrasound probe, a display device, a memory, a user interface, and a processor communicatively connected to the ultrasound probe, the display device, the memory, and the user interface. The processor is configured to control the ultrasound probe to acquire a 3D ultrasound data set and generate a volume rendering image including a rendering object from the 3D ultrasound data set. The processor is configured to receive a control input from the user interface to adjust the position of a virtual light source relative to the rendering object, wherein the position of the virtual light source is used to calculate the illumination and shading of the volume rendering image. The processor is configured to automatically move the virtual light source to an updated position along a height profile established relative to the surface of the rendering object in the volume rendering image in response to the control input. The processor is configured to display the volume rendering image with illumination and shading determined based on the real-time position of the virtual light source during the process of automatically moving the virtual light source to the updated position.

[0011] Various other features, objects and advantages of the present invention will be apparent to those skilled in the art from the accompanying drawings and detailed description thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A is a schematic diagram of a system, a medical imaging device, and a memory according to one embodiment;

[0013] Figure 1B is a schematic diagram of an ultrasound imaging system according to one embodiment;

[0014] Figure 2 is a schematic diagram of a geometric shape that may be used to generate a volume rendered image according to one embodiment;

[0015] Figure 3 is a flow chart of a method 300 according to one embodiment;

[0016] Figure 4 is a representation of a first time screen shot according to one embodiment;

[0017] Figure 5 is a representation of a screenshot at a second time according to one embodiment;

[0018] Figure 6 is a representation of a screenshot at a third time according to one embodiment;

[0019] Figure 7 is a representation of a screenshot and overlay showing an exemplary path of a light source indicator according to one embodiment;

[0020] Figure 8 is a representation of a screenshot according to one embodiment; and

[0021] Fig. 9 is a representation of a screenshot according to one embodiment. DETAILED DESCRIPTION

[0022] In the following detailed description, reference is made to the accompanying drawings which form a part thereof, and in which specific embodiments that can be practiced are shown by way of illustration. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments, and it should be understood that other embodiments may be utilized, and logical, mechanical, electrical and other changes may be made without departing from the scope of the embodiments. Therefore, the following specific embodiments should not be considered to limit the scope of the present invention.

[0023] Figure 1A1 is a schematic diagram of a system 100, a medical imaging device 101, and a memory 102. According to one embodiment, the system 100 may be a workstation. According to one embodiment, the system 100 is suitable for interacting with a 3D medical imaging data set. The system 100 includes a processor 104, a user interface 106, and a display device 108. The memory 102 may include any known medium for storing digital data, including but not limited to a hard drive, a flash memory, a random access memory (RAM), a read-only memory (ROM), a compact disk (CD), and a compact disk read-only memory (CD-ROM). The memory 102 may be part of a database, a component of a PACS / RIS system, or an independent component. The medical imaging device 101 may include any type of medical imaging device capable of acquiring a 3D medical imaging data set. For example, according to various embodiments, the medical imaging device 101 may be an ultrasound imaging system, a computed tomography (CT) imaging system, a magnetic resonance imaging (MRI) system, a nuclear medical imaging system (SPECT), or a positron emission tomography (PET) imaging system. The processor 104 is communicatively connected to the memory 102. This may be connected via a wired or wireless connection. The processor 104 may include one or more separate processing components. For example, the processor 104 may include a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU), or any other electronic component capable of processing data according to specific logic instructions. Having a processor including a GPU may be advantageous for computationally intensive operations, such as volume rendering of large 3D medical imaging data sets. According to some embodiments, the memory 102 may be co-located with the processor 104. However, according to other embodiments, the memory 102 may be remotely located relative to the processor 104 and accessed through technologies including wireless networks, the Internet, or an intranet.

[0024] The user interface 106 is communicatively connected to the processor 104. The user interface 106 can be communicatively connected in a wired or wireless manner. According to an exemplary embodiment, the user input 106 may include a trackball and one or more buttons. However, according to other embodiments, the user interface 106 may include one or more of a mouse, a touchpad, a touch screen, a rotary control, or various hard keys or soft keys with defined or definable functions. The display device 108 is also communicatively connected to the processor 104. The display device 108 may include a monitor or display screen, such as a monitor, an LCD screen, an LED screen, a projector, or any other device suitable for displaying a volume rendered image. Other embodiments may include multiple display devices.

[0025] Figure 1B1 is a schematic diagram of an ultrasound imaging system 120 according to one embodiment. The ultrasound imaging system 120 includes a transmit beamformer 121 and a transmitter 122, which drive elements 124 within an ultrasound probe 126 to transmit pulsed ultrasound signals into a body (not shown). The ultrasound probe 126 can be a linear probe, a curved linear probe, a 2D array, a mechanical 3D / 4D probe, or any other type of ultrasound probe capable of acquiring ultrasound data. Still referring to FIG. 1 , the pulsed ultrasound signal is backscattered from structures in the body, such as blood cells or muscle tissue, to produce echoes that return to the elements 124. The echoes are converted into electrical signals by the elements 124, and the electrical signals are received by the receiver 128. The electrical signals representing the received echoes pass through a receive beamformer 130 that outputs ultrasound data. According to some embodiments, the ultrasound probe 126 may include electronic circuitry to perform all or part of transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 121, the transmitter 122, the receiver 128, and the receive beamformer 130 may be located within the ultrasound probe 126. In the present disclosure, the term "scanning" or "scanning" may also be used to refer to the acquisition of data by the process of transmitting and receiving ultrasound signals. The user interface 135 may be used to control the operation of the ultrasound imaging system 120, including for controlling the input of patient data, for setting acquisition presets, or for changing display parameters. The user interface 135 may include components such as a keyboard, a mouse, a trackball, a touch pad, a touch screen, a multi-touch screen, etc.

[0026] The ultrasound imaging system 120 also includes a processor 136 for controlling the transmit beamformer 121, the transmitter 122, the receiver 128, and the receive beamformer 130. The processor 136 is communicatively connected to the transmit beamformer 121, the transmitter 122, the receiver 128, and the receive beamformer 130 via wired or wireless technology. The processor 136 is in electronic communication with the ultrasound probe 126. The processor 136 can control the ultrasound probe 126 to acquire data. The processor 136 controls which elements of the elements 124 are active and the shape of the beam emitted from the ultrasound probe 126. The processor 136 is also in electronic communication with the display device 138, and the processor 136 can process the data into an image or a numerical value for display on the display device 138. The display device 138 may include a monitor, an LED display, a cathode ray tube, a projector display, or any other type of device configured to display an image. In addition, the display device 138 may include one or more separate devices. For example, the display device 138 may include two or more monitors, LED displays, cathode ray tubes, projector displays, etc. The display device 138 may also be a touch screen. For embodiments in which the display device 138 is a touch screen, the touch screen may act as an input device and it may be configured to receive touch or touch gesture input from a user. For the purposes of this disclosure, the term "electronic communication" may be defined as including wired connections and wireless connections. According to one embodiment, the processor 136 may include a central processing unit (CPU). According to other embodiments, the processor 136 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 136 may include multiple electronic components capable of performing processing functions. For example, the processor 136 may include two or more electronic components selected from a list of electronic components, including: a central processing unit, a digital signal processor, an FPGA, and a graphics board. According to another embodiment, the processor 136 may also include a composite demodulator (not shown) that demodulates RF data and generates raw data. In another embodiment, demodulation can be performed earlier in the processing chain. The processor 136 may be adapted to perform one or more processing operations based on a plurality of selectable ultrasound modalities on the data. As the echo signals are received, the data may be processed in real time during the scanning session. For the purposes of this disclosure, the term "real time" is defined as including processes that are performed without any intentional delay. For the purposes of this disclosure, the term "real time" will be additionally defined as including actions that occur within 2 seconds. For example, if data is collected, real-time display of the data will occur within 2 seconds of collection. Those skilled in the art will appreciate that most real-time programs / processes will be executed in significantly less than 2 seconds.The data may be temporarily stored in a buffer (not shown) during a scanning session and processed in a less-than-real-time manner in either real-time or offline operations.

[0027] Some embodiments of the present invention may include multiple processors (not shown) for handling processing tasks. For example, a first processor may be used to demodulate and extract RF signals, while a second processor may be used to further process the data before displaying an image. It should be understood that other embodiments may use different processor arrangements.

[0028] The ultrasound imaging system 120 can continuously acquire data at a given frame rate or volume rate. The image generated from the data can be refreshed at a similar frame rate or volume rate. A memory 140 is included for storing frames of processed acquired data. In an exemplary embodiment, the memory 140 has sufficient capacity to store at least a few seconds of ultrasound data frames. The data frames are stored in a manner that facilitates retrieval based on their acquisition order or time. The memory 140 may include any known data storage medium.

[0029] Optionally, contrast agents can be used to implement embodiments of the present invention. When using ultrasound contrast agents including microbubbles, contrast imaging generates enhanced images of anatomical structures and blood flow in the body. After the contrast agent is used to collect data, image analysis includes separation of harmonic components and linear components, enhancement of harmonic components, and generation of ultrasound images by utilizing the enhanced harmonic components. Separation of harmonic components from the received signal is performed using a suitable filter. Using contrast agents for ultrasound imaging is well known to those skilled in the art and will not be described in detail.

[0030] In various embodiments of the present invention, the processor 136 can process data through other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, etc.) to form 2D or 3D data. For example, one or more modules can generate B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, elastic imaging, TVI, strain, strain rate, and combinations thereof. Image beams and / or frames are stored, and timing information indicating the time when data is collected in the memory can be recorded. These modules may include, for example, a scan conversion module for performing a scan conversion operation to convert image frames from beam space coordinates to display space coordinates. A video processor module may be provided that reads image frames from a memory and displays the image frames in real time while surgery is performed on a patient. The video processor module may store image frames in an image memory, read and display images from the image memory.

[0031] Figure 2 is a schematic diagram of geometric shapes that may be used to generate a volume rendered image, according to one embodiment. Figure 2 A 3D medical imaging dataset 150 and a view plane 154 are included.

[0032] refer to Figure 1A and Figure 2 In addition, the processor 104 can generate the volume-rendered image according to a variety of different techniques. According to an exemplary embodiment, the processor 104 can generate the volume-rendered image from the view plane 154 through a ray casting technique. The processor 104 can cast a plurality of parallel rays from the view plane 154 to or through the 3D medical imaging dataset 150. Figure 2A first ray 156, a second ray 158, a third ray 160, and a fourth ray 162 are shown that define a viewing plane 154. It should be understood that additional rays may be cast to assign values ​​to all pixels 163 within the viewing plane 154. The 3D medical imaging data set 150 may include voxel data, where a voxel or volume element is assigned a value or intensity. In addition, an opacity may also be assigned to each voxel. The value or intensity may be mapped to a color according to some embodiments. The processor 104 may use a "front to back" or "back to front" technique for volume composition to assign a value to each pixel in the viewing plane 154 that is intersected by a ray. For example, starting from the front, i.e., from the direction in which the image is viewed, the intensities of all voxels along the corresponding ray may be summed. Then, optionally, the intensity may be multiplied by the opacity corresponding to the opacity of the voxel along the ray to generate an opacity weighted value. These opacity weighted values ​​are then accumulated along each ray in a front to back or back to front direction. The process of accumulating values ​​is repeated for each pixel 163 in the viewing plane 154 to generate a volume rendering image. According to one embodiment, the pixel values ​​from the viewing plane 154 can be displayed as a volume rendering image. The volume rendering algorithm can be additionally configured to use an opacity function that provides a gradual transition of opacity from zero (completely transparent) to 1.0 (completely opaque). When assigning values ​​to each of the pixels 163 in the viewing plane 154, the volume rendering algorithm can take into account the opacity of the voxels along each ray. For example, voxels with an opacity close to 1.0 will block most of the contribution of voxels further along the ray, while voxels with an opacity closer to zero will allow most of the contribution of voxels further along the ray. In addition, when visualizing the surface, a thresholding operation can be performed in which the opacity of the voxels is redistributed based on the value. According to an exemplary thresholding operation, the opacity of voxels with values ​​above the threshold can be set to 1.0, while the opacity of voxels with values ​​below the threshold can be set to zero. Other types of thresholding schemes can also be used. The opacity function may be used to assign an opacity other than zero and 1.0 to voxels whose values ​​are close to a threshold in a transition zone. The transition zone may be used to reduce artifacts that may occur when using a simple binary thresholding algorithm. For example, a linear function that maps opacity to a value may be used to assign opacity to voxels having a value in the transition zone. Other types of functions that progress from zero to 1.0 may also be used. Volume rendering techniques other than those described above may also be used to generate a volume rendered image from a 3D medical imaging dataset.

[0033] The volume rendered image may be colored and illuminated to present a user with a better perception of the depth of the rendered object represented in the volume rendered image. According to various embodiments, this may be performed in a number of different ways. For example, the surface of the rendered object may be defined based on the volume rendering of the 3D medical imaging dataset. According to an exemplary embodiment, a gradient may be calculated at each pixel. The processor 104 (shown in FIG. 1 ) may calculate the amount of light at a location corresponding to each pixel and apply a standard coloring method based on the gradient and the particular light direction. The viewing direction may correspond to Figure 2 The processor 104 may also use multiple virtual light sources as input when generating a volume rendered image. For example, as rays are cast, the processor 104 may calculate the amount of light reflected, scattered, or transmitted from each voxel in a particular viewing direction along each ray. This may involve summing the contributions from multiple virtual light sources. The processor 104 may calculate the contributions from all voxels in the volume. The processor 104 may then calculate a composite value from all voxels or an interpolated value from neighboring voxels in order to calculate the final value of a pixel displayed on the image. Although the above examples describe an embodiment of integrating voxel values ​​along rays, the volume rendered image may also be calculated according to other techniques, such as using the highest value along each ray, using the lowest value along each ray, using the average value along each ray, or using any other volume rendering technique.

[0034] Figure 3 is a flow chart of a method 300 according to one embodiment. According to an exemplary embodiment, the method 300 may utilize Figure 1A The system 100 or Figure 1B The method 300 is performed by the ultrasound imaging system 100 shown in FIG. The technical effect of the method 300 is to adjust the virtual light source to an updated position along the height profile established relative to the surface of the rendered object in the volume rendered image. Figure 3 To describe, in this exemplary embodiment, method 300 is performed by Figure 1A The system 100 shown performs.

[0035] See also Figure 1A and Figure 3 , the medical imaging device 101 acquires a three-dimensional (3D) medical imaging dataset and stores the medical imaging dataset in the memory 102. At step 302, the processor 104 accesses the 3D medical imaging dataset from a memory such as the memory 102.

[0036] The 3D medical imaging dataset may include voxel data, where each voxel is assigned a value and an opacity. The value and opacity may correspond to the intensity of the voxel. At step 304, the processor 104 generates a volume rendered image from the 3D medical imaging dataset. According to one embodiment, the processor 104 may generate a volume rendered image based on a previous Figure 2 At step 306, the processor 104 determines the shading and lighting of the volume rendered image. Figure 2 As described, the shading of the volume rendered image may include calculating how light from one or more virtual light sources will interact with the structure of the rendered object represented in the volume rendered image. The method 300 will be described according to an exemplary embodiment, in which the shading and illumination of the volume rendered image are calculated using a single virtual light source, but it should be understood that in other embodiments, the processor 104 may be configured to calculate the shading and illumination based on two or more virtual light sources. The algorithm controlling the shading may calculate how the light will be reflected, refracted, and diffused based on the intensity, opacity, and gradient in the 3D medical imaging dataset. The intensity, opacity, and gradient in the 3D medical imaging dataset may correspond to tissues, organs, and structures in the relevant volume from which the 3D medical imaging dataset was acquired. At step 306, the processor 104 calculates the shading and illumination of the volume rendered image based on the current or real-time position of the virtual light source or the current positions of multiple virtual light sources. According to one example, the processor 104 may calculate the shading and illumination at step 306 based on the assumption that each virtual light source acts as a point source. According to other examples, the processor 104 may calculate the shading and illumination at step 306 based on the assumption that each virtual light source is not a point source. For example, the processor 104 may calculate shading and lighting of the volume rendered image based on the assumption that each virtual light source is a sphere or other geometric shape.

[0037] At step 308, the processor 104 is configured to display the volume rendered image including the shading and lighting calculated at step 308 on the display device 108. Thus, the position of the virtual light source will directly affect the appearance of the volume rendered image displayed on the display device 108.

[0038] Step 310 is a decision step in method 300. If the processor 104 does not receive a control input from the user interface at step 310, the method 300 returns to step 308 and displays the volume rendered image with the previously calculated lighting and shading. However, if at step 310, the processor 104 receives a control input from the user interface 106 to adjust the position of one or more of the virtual light sources, the method 300 proceeds to step 312.

[0039] At step 312, the processor 106 automatically moves the virtual light source along the height profile in response to the control input. The height profile is calculated by the processor 106 at a substantially fixed distance from a surface, such as an outer surface 407 of the rendered object 403 represented in the volume rendered image 402. According to one example, the height profile may be at a constant height relative to the outer surface 407 of the rendered object in the volume rendered image. According to another example, the height profile may be at a constant height relative to a smooth surface of the volume rendered image 407. The processor 106 may, for example, apply a smoothing function to the outer surface of the rendered object represented in the volume rendered image and then calculate the height profile at a constant height relative to the smooth surface. The smoothing function may include, for example, an average function, a curve fitting function, or a moving average function.

[0040] The height contour is determined by the processor 106 as a fixed distance (height) from the outer surface of the drawn object in the volume rendered image or a substantially fixed distance (height) from the outer surface of the drawn object in the volume rendered image. The height of the height contour can be manually adjusted by the user. According to one example, the user can use a rotary control to adjust the height of the height contour from the surface of the drawn object in the volume rendered image. According to another example, the user can use a slider or a virtual slider to adjust the height of the height contour from the surface of one or more drawn objects in the volume rendered image. The height contour can be positioned at a positive height above the surface of the drawn object, the height contour can be positioned at a negative height below the surface of the drawn object, or the height contour can be positioned exactly at the surface of the drawn object. By convention, in the present disclosure, a height of zero will be defined as being exactly at the surface of the drawn object, a positive height will be defined as being from the surface in a direction away from the center of the drawn object, and a negative height will be defined as being from the surface in a direction toward the center of the drawn object. In response to input entered through the user interface 106, the processor 104 can adjust the height of the virtual contour accordingly.

[0041] After performing step 312, the method 300 returns to step 306, where the processor 106 calculates shading and lighting of the volume rendered image based on the updated position of the virtual light source. Or, for example, using multiple light sources, the processor 106 calculates shading and lighting of the volume rendered image based on the current or real-time positions of the multiple virtual light sources, one or more of which may be at the updated position.

[0042] Figure 44 is a representation of a screen shot at a first time according to one embodiment. The screen shot includes a volume rendered image 402, a first reference plane 404, and a second reference plane 406. The volume rendered image 402 includes a rendered object 403. The rendered object 403 may include one or more anatomical structures and / or objects represented in the 3D medical imaging data set. The rendered object 403 includes an outer surface 407. The outer surface 407 defines the volume of the rendered object 403 represented in the volume rendered image. The first reference plane 404 represents a plan view through the volume rendered image 402 along line 410. The second reference plane represents a plan view through the volume rendered image along line 412. Figure 4 Also included is a light source indicator 408, a first reference plane light source indicator 414, and a second reference plane light source indicator 416. The volume rendered image 402 may be generated from a 3D medical imaging dataset, as described in step 304 of the method 300. The volume rendered image 402 includes shading and lighting based on the current or real-time position of a virtual light source, as described in step 306 of the method 300. The position of the virtual light source is represented by the light source indicator 408, which is shown in the volume rendered image 402 in a geometric relationship that is used to calculate the lighting and shading in the volume rendered image 402. Figure 4 Is a representation of the screenshots at the first time.

[0043] Figure 5 is a representation of a screenshot at a second time according to one embodiment. Figure 4 In contrast, the light source indicator 408 is Figure 5 In different positions, indicating Figure 4 Compared to the position of the virtual light source Figure 5 is different.

[0044] Figure 6 is a representation of a screenshot at a third time according to one embodiment. Figure 4 and Figure 5 In contrast, the light source indicator 408 is Figure 6 are in different positions, which indicates that Figure 4 and Figure 5 In contrast, the position of the virtual light source is Figure 6 is different.

[0045] Figure 7 is a screenshot and overlay representation showing an exemplary path 420 of a light source indicator 408, according to one embodiment. The light source indicator 408 is located at Figure 4 At point A on path 420 in volume rendered image 402; light source indicator 408 is located at Figure 5 At point B on the path 420 in the volume rendered image 402 shown; and the light source indicator 408 at Figure 6 The light source indicator 408 provides a method for calculating the volume rendered image 402 at a point C on the path 420. Figure 4 The first time shown, Figure 5 The second time and Figure 6 4 is a representation of the relative positioning between the illumination and the colored virtual light source at the third time shown. The light source indicator 408 travels from point A to point B along the path 420 between the first time and the second time; and the light source indicator 408 travels from point B to point C along the path 420 between the second time and the third time. It should be understood that the path 420 represents an exemplary path, and the operator can control the light source indicator to follow other paths by changing one or more control inputs input through the user interface 106.

[0046] According to one embodiment, the control input optionally received at step 310 may be a control received from a mouse, a control input received from a trackball, or a control input received from a touch screen or touch pad. One or more control inputs may optionally track Figure 7 For example, the operator may input the path 420 on a touch screen or touch pad. Figure 7 Similarly, for embodiments using a mouse or trackpad, a control input may be received where the mouse or trackball defines a curve having the same shape as path 420 or a shape very similar to path 420.

[0047] According to one embodiment, a user may define a height of a height profile that is used to define the height of a virtual light source relative to a surface shown in a volume rendered image. The user may adjust the height of the virtual light source, for example, using a rotation, a slider, a manual increment / decrement controller, or any other type or user interface control. The user may control the height of the virtual light source, for example, using the user interface 106, and the processor 104 may optionally show the height of the virtual light source above the surface of the volume rendered image 402. The processor 104 may optionally display the height of the virtual light source in millimeters (mm), centimeters (cm), or inches on the display device 108.

[0048] According to one embodiment, at step 312, the processor 104 automatically moves the virtual light source along a height profile, which is a constant height from the surface of the volume rendered image 402 or a constant height from the smooth surface 407 of the volume rendered image 402. The height of the height profile can be determined, for example, in a direction perpendicular to the outer surface of the volume rendered image 402. At step 310, the processor 104 receives a control input to adjust the position of one or more virtual light sources. As previously described, this can be, for example, an input made through a mouse, trackball, touch screen, or touch pad. Upon receiving the control input at step 310, the processor 104 automatically moves the virtual light source along the height profile. Before receiving a separate height control adjustment input through the user interface 106, when the method 300 repeats steps 306, 308, 310, and 312 and receives additional control inputs, the processor 104 automatically maintains the virtual light source along the height profile. This allows the user to easily position the virtual light source at any position relative to the rendered object shown in the volume rendered image 402, while maintaining the virtual light source at a fixed height from the surface of the rendered object 403 (such as the outer surface 407). Advantageously, the techniques described in method 300 allow the user to easily control the position of the virtual light source in three dimensions using only two-dimensional control inputs. The processor 104 automatically maintains the virtual light source at a substantially constant height defined by a height profile from the surface of the drawn object 403. This makes it easier and faster for the clinician to reposition the virtual light source to different locations while maintaining the virtual light source at a substantially constant distance from the surface of the drawn object 403. Maintaining the virtual light source at a substantially constant distance helps ensure that the intensity levels of the virtual illumination light sources at different locations are relatively similar. In addition, maintaining the virtual light source at a substantially constant distance helps ensure that the illumination, shading, and any shadow casting of the structure of the drawn object will change smoothly with the adjustment of the position of the virtual light source.

[0049] Figure 7 The path 420 shown represents the path of the light source indicator 408 according to one embodiment. The path 420 is a graphical way of representing the position of the virtual light source indicator used to determine the lighting and coloring of the volume rendered image 402. Figure 4 At the first time shown, the light indicator 408 is at position A. Figure 5 At the second time shown, the light indicator 408 is at position B. Figure 5At the third time shown, the light source indicator 408 is at position C. Since the light source indicator 408 represents the position of the virtual light source relative to the drawn object 403 in the volume rendered image 402, the processor 104 automatically moves the virtual light source from the first time to the second time and to the third time along the height profile. The path 420 showing the path of the light source indicator represents the path of the virtual light source from the first time to the second time to the third time. The processor 104 controls the virtual light source to be at a constant height relative to the outer surface of the drawn object shown in the volume rendered image 402.

[0050] According to one embodiment, the processor 104 may also be used to control a reference plane light source indicator, such as the first reference plane light source indicator 414 or the second reference plane light source indicator 416. The processor 104 may be configured to control one or more display characteristics of each reference plane light source indicator to indicate the distance of the virtual light source from the plane represented in the reference plane. For example, the processor 104 may use the intensity, color, size, or a combination of intensity, color, and size of the reference plane light source indicator to indicate the distance of the virtual light source from the plane represented in a particular reference plane. Figure 4 , Figure 5 and Figure 6 Reference will be used to help describe how a plane light source indicator functions according to an embodiment.

[0051] As previously described, first reference plane 404 represents a first plane along line 410, and second reference plane 406 represents a second plane along line 412. Figure 4 , Figure 5 and Figure 6 In the embodiment shown, the first plane and the second plane may be perpendicular to each other. Figure 4 As shown, light source indicator 408 is positioned almost exactly where line 410 and line 412 intersect. First reference plane light source indicator 414 indicates the position of the virtual light source relative to first reference plane 404. Second reference plane light source indicator 416 indicates the position of the virtual light source relative to second reference plane 406.

[0052] In the second time, Figure 5As shown, light source indicator 408 is not positioned on line 410 or line 412. This means that the virtual light source is not positioned in the first plane or the second plane. Therefore, processor 104 adjusts the display of first reference plane light source indicator 414 and second reference plane light source indicator 416. For example, processor 104 can control the intensity of either reference plane light source indicator to indicate the position of the virtual light source relative to the reference plane. For example, the virtual light source is relatively far away from the first plane indicated by line 410. First reference plane light source indicator 414 is shown as a ring with a dark center portion. According to one embodiment, this indicates that the virtual light source is relatively far away from the first plane. The virtual light source is relatively closer to the second plane indicated by second line 412. In this way, in Figure 5 The second reference plane light source indicator 416 is brighter than the first reference plane light source indicator 414 . Figure 4 The first reference plane light source indicator 414 and the second reference plane light source indicator 416 are both shown to have a ratio Figure 5 The first reference plane light source indicator 414 and the second reference plane light source indicator 416 are shown to be brighter in intensity. This is because Figure 4 At the first time shown, the first reference plane light source indicator 414 is located in the first plane, and the second reference plane light source indicator 416 is located in the second plane. Figure 6 , indicating a third time, the virtual light source is approximately equidistant from the first plane and the second plane, as shown by the position of the light source indicator 408 relative to the first line 410 and the second line 412. Therefore, the processor 104 displays the first reference plane light source indicator 414 and the second reference plane light source indicator 416 at the same intensity. The intensity of both the first reference plane light source indicator 414 and the second reference plane light source indicator 416 is relatively low to indicate that the virtual light source is positioned relatively far from both the first plane and the second plane.

[0053] In relation to Figure 4 , Figure 5 and Figure 6 In the example described, the processor 104 adjusts the intensity of each reference plane light source indicator to indicate the distance of the virtual light source from the plane represented in the reference plane. According to one embodiment, the processor 104 may be configured to control the reference plane light source indicator to a maximum intensity when the virtual light source is in the reference plane, and to control the reference plane light source indicator to a less than maximum intensity when the virtual light source is not in the reference plane. Figure 4 In the example, the reference plane light source indicator can be adjusted to indicate the distance of the virtual light source from the plane represented in the reference plane.

[0054] Figure 8 is a representation of a screenshot according to one embodiment. Figure 8An example is shown where a virtual light source is positioned below the outer surface of a rendered object in a volume rendered image 402. The volume rendered image 402 is backlit because the virtual light source is positioned below the outer surface of the rendered object. Figure 8 The height profile of the example shown is below the surface (or in the direction opposite to the surface). In response to receiving user input for adjusting the position of the virtual light source, the processor 104 will cause the virtual light source to follow the height profile at a fixed negative distance from the outer surface of the drawn object 403.

[0055] Fig. 9 is a representation of a screenshot according to one embodiment. Fig. 9 An example of an embodiment with two virtual light sources is shown: a first virtual light source and a second virtual light source. The first virtual light source is represented by a first light source indicator 409 and the second virtual light source is represented by a second light source indicator 411 .

[0056] According to one embodiment, the user may choose to independently control the position of either virtual light source, or the user may move both virtual light sources based on a single user input. According to an exemplary embodiment, the user may select one of the virtual light sources, such as by clicking or selecting the first light source indicator 409 or the second light source indicator 411. The user may then adjust the position of the selected virtual light source according to the techniques described with respect to method 300.

[0057] exist Fig. 9 In the example shown, the first virtual light source is positioned below the outer surface of the drawing object as indicated by the first light source indicator 409, and the second virtual light source is positioned above the outer surface of the drawing object 403 as indicated by the second light source indicator 411. Fig. 9 In the illustrated embodiment, the second light source indicator 411 is shown as having a greater intensity than the first light source indicator 409 to indicate that the second virtual light source is above the outer surface and the first virtual light source is below the outer surface of the drawn object. Fig. 9 In the illustrated embodiment, the size of the second light source indicator 411 is larger than the size of the first light source indicator 409. The processor 104 can adjust the size of each light source indicator based on the height of the light source indicator relative to the surface of the drawing object 403. Fig. 9In the illustrated embodiment, the processor 104 may be configured to display the light source indicator as larger in size when the virtual light source is above the surface of the drawing object, and to display the light source indicator as smaller in size when the virtual light source is below the surface of the drawing object. The processor 104 may be configured to smoothly change the size of the light source indicator (such as the first light source indicator 409 or the second light source indicator 411) when the user adjusts the height of the associated virtual light source. In this way, the size of the light source indicator can convey information about the height of the virtual light source relative to the surface in the drawing object 104.

[0058] This written description uses examples to disclose the invention, including the best mode, and also to enable those skilled in the art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A volume rendering method, the volume rendering method comprising: Access to 3D medical imaging datasets; generating a volume rendered image including a rendered object from the 3D medical imaging dataset; as well as Controlling shading and lighting of the volume rendered image based on a position of a virtual light source relative to the rendered object, wherein controlling the shading and the lighting comprises performing the following steps: a) receiving control input from a user interface; b) in response to the control input, automatically moving the virtual light source to an updated position along a height profile established relative to a surface of the rendered object in the volume rendered image, wherein the height profile is established to allow positioning of the virtual light source at any position relative to the rendered object while maintaining the virtual light source at a constant height relative to the surface of the rendered object or a smooth surface of the rendered object; c) calculating updated shading and updated lighting based on the updated virtual light source position; d) displaying said volume rendered image with said updated shading and said updated lighting; as well as e) when the virtual light source is moved in response to the control input, repeating steps a), b), c) and d). 2 . The method of claim 1 , wherein the height profile is at a constant height relative to the surface of the drawn object.

3. The method of claim 1, wherein the height profile is at a constant height relative to the smooth surface of the drawn object, and wherein the smooth surface is determined by applying a smoothing function to the surface of the drawn object. 4 . The method of claim 1 , further comprising adjusting a height of the height profile relative to the surface of the drawn object in response to a second control input on the user interface. The method of claim 1 , wherein the height profile is above the surface of the drawn object. The method of claim 1 , wherein the height profile is below the surface of the drawn object. 7 . The method of claim 1 , wherein said controlling said shading and said lighting of said volume rendered image is further based on a second virtual light source. 8 . The method of claim 7 , wherein the second virtual light source is positioned below the surface of the drawing object, and the virtual light source is positioned above the surface of the drawing object. 9 . The method of claim 1 , further comprising displaying a light source indicator on the volume rendered image to indicate the position of the virtual light source relative to the rendered object.

10. The method of claim 9, further comprising displaying a reference plane having a reference plane light source indicator simultaneously with the volume rendered image, wherein the reference plane light source indicator indicates the position of the virtual light source relative to the reference plane.

11. The method of claim 10, further comprising adjusting an intensity of the reference plane light source indicator to indicate the position of the virtual light source relative to the reference plane.

12. A method according to claim 11, wherein when the reference plane light source indicator is in the reference plane, the reference plane light source indicator is displayed at a maximum intensity, and wherein when the reference plane light source indicator is not in the reference plane, the reference plane light source indicator is displayed at an intensity less than the maximum intensity. 13 . The method of claim 12 , wherein the intensity of the reference plane light source indicator is adjusted when the virtual light source is moved to indicate the distance of the virtual light source from the reference plane.

14. A system for interacting with a 3D medical imaging dataset, the system comprising: Display devices; user interface; and a processor communicatively connected to the display device and the user interface, wherein the processor is configured to: Access to 3D medical imaging datasets; generating a volume rendered image including a rendered object from the 3D medical imaging dataset; receiving a control input from the user interface to adjust a position of a virtual light source relative to the rendered object, wherein the position of the virtual light source is used to calculate lighting and shading of the volume rendered image; in response to the control input, automatically moving the virtual light source to an updated position along a height profile established relative to a surface of the rendered object in the volume rendered image, wherein the height profile is established to allow positioning of the virtual light source at any position relative to the rendered object while maintaining the virtual light source at a constant height relative to the surface of the rendered object or a smooth surface of the rendered object; as well as During the process of automatically moving the virtual light source to the updated position, the volume rendered image is displayed with the lighting and the shading determined based on the real-time position of the virtual light source.

15. The system of claim 14, wherein the height profile is at a constant height relative to the smooth surface of the drawn object.

16. The system of claim 14, wherein the processor is configured to receive a second control input to add a second virtual light source, and wherein the processor is configured to display the volume rendered image with the illumination and the shading determined based on the real-time positions of the virtual light source and the second virtual light source.

17. The system of claim 14, wherein the processor is configured to display a reference plane having a reference plane light source indicator simultaneously with the volume-rendered image, wherein the reference plane light source indicator indicates the position of the virtual light source relative to the reference plane.

18. An ultrasonic imaging system, comprising: Ultrasound probe; Display devices; Memory; user interface; and a processor communicatively connected to the ultrasound probe, the display device, the memory, and the user interface, wherein the processor is configured to: controlling the ultrasound probe to acquire a 3D ultrasound dataset; generating a volume rendered image including a rendered object from the 3D ultrasound dataset; receiving a control input from the user interface to adjust a position of a virtual light source relative to the rendered object, wherein the position of the virtual light source is used to calculate lighting and shading of the volume rendered image; in response to the control input, automatically moving the virtual light source to an updated position along a height profile established relative to a surface of the rendered object in the volume rendered image, wherein the height profile is established to allow positioning of the virtual light source at any position relative to the rendered object while maintaining the virtual light source at a constant height relative to the surface of the rendered object or a smooth surface of the rendered object; as well as During the process of automatically moving the virtual light source to the updated position, the volume rendered image is displayed with the lighting and the shading determined based on the real-time position of the virtual light source.

19. The ultrasound imaging system of claim 18, wherein the height profile is at a constant height relative to the surface of the volume rendered image.

20. The ultrasound imaging system of claim 18, wherein the height profile is at a constant height relative to the smooth surface of the volume rendered image, and wherein the smooth surface is determined by applying a smoothing function to the surface of the volume rendered image.

Images