rendering three-dimensional overlays on two-dimensional images

Abstract

In some examples, one or more three-dimensional (3D) objects can be rendered relative to a two-dimensional (2D) imaging slice. The 3D objects can be rendered such that the 3D objects cast a shading shadow on the 2D imaging slice. In some examples, the 3D objects can be rendered in color, where different colors indicate a distance of a portion of the 3D object from the 2D imaging slice.

Description

Technical Field

[0001] This application relates to the rendering of three-dimensional overlays in medical imaging. More specifically, this application relates to the volumetric lighting rendering of three-dimensional overlays relative to a two-dimensional image plane. Background Technology

[0002] Through training and experience, radiologists, surgeons, and other clinicians often gain greater confidence when processing two-dimensional (2D) images, even when three-dimensional (3D) imaging sources are available. Taking 3D ultrasound as an example, multiple reformulated slices are typically extracted from 3D datasets of different orientations (e.g., volumes) and displayed as 2D images to enable more confident diagnosis, interventional planning, or real-time navigation. With the increasing availability of segmentation tools, it is also possible to create 3D models and display them as virtual representations of specific structures, anatomy, or interventional devices. It is often desirable to combine these 3D targets with the primary 2D imaging source for visualization. Summary of the Invention

[0003] As disclosed herein, a three-dimensional (3D) target is superimposed on a two-dimensional (2D) frame by using volumetric lighting and assigning translucent material to the 3D target to cast shading shadows onto a grayscale image texture. Optionally, the 3D target can be shading based on its distance from the view and / or the 2D plane.

[0004] In some examples, the technique may include rendering a 3D scene comprising: a 2D image from a 2D imaging source or from a 3D volume sliced ​​in arbitrary orientation, having a known position and orientation in 3D space; one or more targets, 3D graphics, and / or surface meshes from the volume. At least one light source may project volumetric shadows from the targets(s) at an angle. The 3D scene may be rendered using photorealistic material definitions, including light absorption properties.

[0005] According to at least one example disclosed herein, an apparatus may include a processor configured to: assign material properties to voxels of an image slice obtained from a first volume; assign material properties to voxels of a three-dimensional (3D) target obtained from a second volume; and render a 3D scene including the image slice and the 3D target, wherein, when rendering the 3D scene, the material properties of the voxels assigned to the 3D target are configured to cause the 3D target to: be rendered as semi-transparent in the 3D scene, and change the color of the light from at least one virtual light source as light from at least one virtual light source propagates through the 3D target. The apparatus may include a display configured to display the 3D scene.

[0006] According to at least one example disclosed herein, a method may include: assigning material properties to voxels of an image slice obtained from a first volume; assigning material properties to voxels of a three-dimensional (3D) target obtained from a second volume; and rendering a 3D scene including the image slice and the 3D target, wherein the material properties of the voxels assigned to the 3D target are configured to cause the 3D target to change the color of light as light from at least one virtual light source propagates through the 3D target.

[0007] According to at least one example disclosed herein, a non-transient computer-readable medium includes instructions that, when executed, cause an imaging system to perform the following operations: assigning material properties to voxels of an image slice obtained from a first volume; assigning material properties to voxels of a three-dimensional (3D) target obtained from a second volume; and rendering a 3D scene including the image slice and the 3D target, wherein the material properties of the voxels assigned to the 3D target are configured to cause the 3D target to change the color of light as light from at least one virtual light source propagates through the 3D target. Attached Figure Description

[0008] Figure 1 An example image of a three-dimensional object superimposed on a two-dimensional image is shown.

[0009] Figure 2A and Figure 2B An example image of a three-dimensional object superimposed on a two-dimensional image, according to an example of the present disclosure, is shown.

[0010] Figure 3 This is a block diagram of an ultrasound imaging system arranged according to an example of the present disclosure.

[0011] Figure 4 This is a block diagram illustrating an example processor based on the present disclosure.

[0012] Figure 5 A graphical overview of a 3D scene is provided as an example drawn based on the contents of this disclosure.

[0013] Figure 6 A graphical overview of lighting passthrough and composite passthrough for drawing a 3D scene is provided, based on examples of this disclosure.

[0014] Figure 7 A graphical depiction of lighting through an example according to this disclosure is provided.

[0015] Figure 8 A graphical depiction of a composite passage based on examples of this disclosure is provided.

[0016] Figure 9Graphical depictions of enhanced visualizations of 3D scenes are provided as examples based on this disclosure.

[0017] Figure 10 This is a flowchart illustrating an example method based on this disclosure.

[0018] Figure 11A and Figure 11B A 3D scene of a 3D model of a mitral valve clip in a 2D plane of a cardiac view acquired by ultrasound imaging, as exemplified by this disclosure, is shown.

[0019] Figure 12A and Figure 12B A 3D scene of a mitral valve clip in a 2D plane of a cardiac view acquired by ultrasound imaging, as an example according to this disclosure, is shown.

[0020] Figure 13 Examples of 3D scenes of cardiac tissue acquired by ultrasound imaging and 2D plane views of the heart are shown, according to the present disclosure.

[0021] Figure 14 A 3D scene of a model of a 3D target acquired by ultrasound imaging, and a 2D plane of a fetal skull, are shown as examples of this disclosure.

[0022] Figure 15A and Figure 15B Examples of 3D scenes of blood vessels and 2D planes of joints acquired by computed tomography, according to this disclosure, are shown.

[0023] Figure 16A and Figure 16B Examples of 3D scenes and various internal organs in a 2D plane acquired by computed tomography according to this disclosure are shown. Detailed Implementation

[0024] The following description of some exemplary examples is merely illustrative in nature and is in no way intended to limit the scope of this disclosure or its application or use. In the following detailed description of examples of the systems and methods, reference is made to the accompanying drawings, which form part of the detailed description below and illustrate specific examples of the apparatuses, systems, and methods that can be practiced. These examples are described in sufficient detail herein to enable those skilled in the art to practice the currently disclosed apparatuses, systems, and methods, and it should be understood that other examples and structural and logical changes can be utilized without departing from the spirit and scope of this disclosure. Furthermore, for clarity, detailed descriptions of certain features will not be discussed in detail where they are obvious to those skilled in the art, so as not to obscure the description of this disclosure. Therefore, the following detailed description should not be considered limiting, and the scope of the apparatuses, systems, and methods is defined only by the claims.

[0025] Despite advancements in three-dimensional (3D) imaging and rendering technologies, users of medical imaging systems (e.g., radiologists, surgeons, and sonographers) are often able to make decisions with greater confidence based on two-dimensional (2D) images. Therefore, even when scanning volumes within an object, 2D images (e.g., frames) are typically extracted from the volume, for example by a multi-plane reformer, to be displayed to the user, rather than rendering the entire scan volume or selecting 3D targets (e.g., organs, implants) within the scan volume. However, in some applications, users want or need to view 3D targets associated with 2D planes. For example, a surgeon might want to view a 3D rendering of a mitral valve clip associated with a 2D image. However, even when extracting 3D targets from the same dataset as the 2D images, it is not easy for users to reconcile and visualize the 2D images and 3D targets within a 3D scene.

[0026] Many visualization toolkits (e.g., VTK, OpenGL) support blending surface-drawn meshes with 2D textures mapped onto a 3D plane. This type of 3D “overlay” is difficult to interpret and often requires manipulation to a tilted viewpoint to understand spatial relationships and distances. Depth perception is particularly limited when the view plane is parallel to the image (which is the preferred angle for displaying the image without geometric distortion).

[0027] exist Figure 1 This issue is explained in the text. Figure 1An example image of a superposition of a three-dimensional target on a two-dimensional slice is shown. In this image, a virtual transposition puncture needle 102 has been extracted from a 3D ultrasound acquisition. The needle 102 is represented as a 3D mesh model, while the needle 102 is plotted against a 2D planar image 104 generated from a slice from the same acquisition. The plotting is performed using surface and texture shaders from standard 3D visualization software (VTK in this example). In the left view 100A, the view plane is parallel to the image 104, and the distance between the needle 102 and the plane of the image 104 is difficult to perceive visually. As shown in the right view 100B, rotating the view to a tilt angle provides a slightly better perspective, but introduces geometric distortion into the image 104 and still fails to adequately express the distance between the tip of the needle 102 and the image 104.

[0028] In some applications, virtual / mixed reality headsets and stereoscopic / holographic displays can aid in depth perception by fusing 2D and 3D information, but these require specialized hardware and are not widely available. Therefore, there is a desire for improved rendering of 3D scenes that incorporate both 2D and 3D information (e.g., 2D images and 3D objects).

[0029] Photorealistic rendering applications use physically based ray tracing to generate 2D views of 3D scenes including multiple 3D objects. Compared to surface shading used in most 3D visualization packages, physically based rendering can improve depth perception through more realistic lighting effects. These advanced algorithms can improve the visualization of the spatial position of 3D virtual objects relative to a 2D image, especially the ability to cast shadows. According to examples from this disclosure, a 2D image can be blended with one or more 3D objects by using volumetric lighting and assigning translucent material to one or more 3D objects, such that these one or more 3D objects cast tinted shadows onto the grayscale (e.g., grayscale) image texture of the 2D image. Optionally, the 3D objects can be tinted at least partially based on the distance between the 3D objects and the 2D image.

[0030] Figure 2A and Figure 2B An example image of a superimposed three-dimensional object on a two-dimensional image, according to an example of this disclosure, is shown. Figure 2A As shown, when the view is parallel to the image, the shadow 204A of the 3D target 202A on the grayscale 2D image 200A provides a depth cue. Physically based calculations of light absorption and / or scattering can create soft shadows of recognizable colors (e.g., blue, orange, green, red). Projecting tinted shadows, rather than grayscale shadows, onto the 2D image 200A prevents the shadow 204A from being confused with darker areas of the 2D image 200A and / or makes it easier for the viewer to interpret. Figure 2BAs shown, in addition to the shadow 204B of the 3D target 202B provided on the grayscale 2D image 200B, the 3D target 202B also features gradient shading to provide additional depth cues. The 3D target 202B is one color near the 2D image 200B and gradually changes to a different color at a position 208B further away from the 2D image 200B. Although Figure 2B The example shown provides a gradient between two different colors as a function of the distance of a portion of the 3D target 202B from the 2D image 200B, but gradients with multiple (e.g., three, four) colors can also be used.

[0031] Figure 3 A block diagram of an ultrasound imaging system 300 constructed according to an example of this disclosure is shown. The ultrasound imaging system 300 according to this disclosure may include a transducer array 314, which may be included in an ultrasound probe 312 (e.g., an external probe or an internal probe (e.g., an intravascular ultrasound (IVUS) catheter probe)). In other examples, the transducer array 314 may be in the form of a flexible array configured to conformally apply to the surface of an object to be imaged (e.g., a patient). The transducer array 314 is configured to emit ultrasound signals (e.g., beams, waves) and receive echoes (e.g., received ultrasound signals) in response to the emitted ultrasound signals. Various transducer arrays can be used, such as linear arrays, curved arrays, or phased arrays. For example, the transducer array 314 can include a two-dimensional array of transducer elements (as shown), which is capable of scanning in both the height and azimuth dimensions for 2D and / or 3D imaging. It is generally known that: axial orientation is the orientation normal to the array plane (in the case of a curved array, axial orientation is fan-out orientation), azimuth orientation is usually defined by the longitudinal dimension of the array, and height orientation is transverse to azimuth orientation.

[0032] In some examples, transducer array 314 may be coupled to microwave beamformer 316, which may be located within ultrasonic probe 312, and the transducer elements in array 314 may be controlled to transmit and receive signals. In some examples, microwave beamformer 316 may be controlled to transmit and receive signals via active elements in array 314 (e.g., an active subset of the elements of the array that defines the active aperture at any given time).

[0033] In some examples, the microwave beamformer 316 can be coupled to a transmit / receive (T / R) switch 318, for example, via a probe cable or wirelessly. The T / R switch 318 switches between transmitting and receiving and protects the main beamformer 322 from high-energy transmitted signals. In some examples, such as in a portable ultrasound system, the T / R switch 318 and other components of the system can be included in the ultrasound probe 312, rather than in an ultrasound system base that may house image processing electronics. The ultrasound system base typically includes software and hardware components, including circuitry for signal processing and image data generation, and executable instructions for providing a user interface.

[0034] The process of emitting ultrasonic signals from transducer array 314 under the control of microwave beamformer 316 is guided by transmit controller 320, which can be coupled to T / R switch 318 and main beamformer 322. Transmit controller 320 can control the orientation of the beam. The beam can be steered forward from directly in front of (orthogonal to) transducer array 314, or at different angles to obtain a wider field of view. Transmit controller 320 can also be coupled to user interface 324 and receive input from user operation of user input devices (e.g., user controls). User interface 324 may include one or more input devices (e.g., control panel 352), which may include one or more mechanical controls (e.g., buttons, sliders, etc.), touch-sensitive controls (e.g., touchpads, touchscreens, etc.), and / or other known input devices.

[0035] In some examples, the partially beamformed signal generated by microwave beamformer 316 can be coupled to main beamformer 322, in which partially beamformed signals from individual patches of transducer elements can be combined into a fully beamformed signal. In some examples, microwave beamformer 316 is omitted. In these examples, transducer array 314 is under the control of main beamformer 322, and main beamformer 322 performs all beamforming of the signal. In embodiments with and without microwave beamformer 316, the beamformed signal from main beamformer 322 is coupled to processing circuitry 350, which may include one or more processors (e.g., signal processor 326, mode-B processor 328, Doppler processor 360, and one or more image generation and processing units 368) configured to generate ultrasound images based on the beamformed signal (i.e., beamformed RF data).

[0036] Signal processor 326 can be configured to process the received beamformed RF data in various ways, such as bandpass filtering, decimation, I-component and Q-component separation, and harmonic signal separation. Signal processor 326 can also perform additional signal enhancement, such as speckle suppression, signal recombination, and electronic noise cancellation. The processed signals (also referred to as I-component and Q-component, or IQ signals) can be coupled to additional downstream signal processing circuitry for image generation. The IQ signals can be coupled to multiple signal paths within the system, each of which can be associated with a specific arrangement of signal processing components suitable for generating different types of image data (e.g., B-mode image data, Doppler image data). For example, the system may include a B-mode signal path 358 that couples signals from signal processor 326 to B-mode processor 328 for generating B-mode image data.

[0037] The B-mode processor 328 is capable of employing amplitude detection for imaging structures within the body. The B-mode processor 328 can generate signals for tissue images and / or phase-contrast images. The signals generated by the B-mode processor 328 can be coupled to a scan converter 330 and / or a multi-plane reformer 332. The scan converter 330 can be configured to arrange the echo signals into a desired image format based on the spatial relationships at the time the echo signals are received. For example, the scan converter 330 can arrange the echo signals into a two-dimensional (2D) fan-shaped format or a three-dimensional (3D) format in the shape of a pyramid or other shapes.

[0038] In some examples, the system may include a Doppler signal path 362 that couples the output from signal processor 326 to Doppler processor 360. Doppler processor 360 may be configured to estimate the Doppler frequency shift and generate Doppler image data. The Doppler image data may include color data that is then overlaid with a B-mode (i.e., grayscale) image for display. Doppler processor 360 may be configured, for example, to use a wall filter to filter out unwanted signals (i.e., noise or clutter associated with non-moving tissue). Doppler processor 360 may also be configured to estimate velocity and power according to known techniques. For example, the Doppler processor may include a Doppler estimator (e.g., an autocorrelator) where the velocity (Doppler frequency) estimate is based on the independent variable of a lag-one autocorrelation function (e.g., R1), and the Doppler power estimate is based on the magnitude of a lag-zero autocorrelation function (e.g., R0). The velocity estimation results can be referred to as color Doppler data, and the power estimation results can be referred to as power Doppler data. Motion can also be estimated using known phase-domain (e.g., parameterized frequency estimators, such as MUSIC, ESPRIT, etc.) or time-domain (e.g., cross-correlation) signal processing techniques. Other estimators related to the temporal or spatial distribution of velocity (e.g., acceleration estimators or temporal / spatial velocity derivatives) can be used to replace or supplement the velocity estimator. In some examples, the velocity estimation results (e.g., color Doppler data) and power estimation results (e.g., power Doppler data) can undergo further thresholding to further reduce noise, and undergo segmentation and post-processing (e.g., padding and smoothing). The velocity estimation results and / or power estimation results can then be mapped to a desired range of display colors and / or intensities based on one or more color maps and / or intensity maps. The image data (also known as Doppler image data) can then be coupled to a scan converter 330, in which the Doppler image data can be converted into the desired image format to form a color Doppler image or a power Doppler image.

[0039] The multiplane reformer 332 is capable of converting echoes received from points in a common plane (e.g., a slice) within a volumetric region of the body into an ultrasound image (e.g., a B-mode image) of that plane, for example, as described in U.S. Patent US 6,443,896 (Detmer). In some examples, a user interface 324 may be coupled to the multiplane reformer 332 for selecting and controlling the display of multiple multiplane reformatted (MPR) images. In other words, a user can select a desired plane within a volume from which a 2D image can be generated. In some examples, in addition to selecting the location and / or orientation of the plane within the volume, the user can also select the thickness of the plane. In some examples, the planar data from the multiplane reformer 332 may be provided to a volume plotter 334. The volume plotter 334 may generate (also referred to as plotting) an image of a 3D dataset as observed from a given reference point (also referred to as projection, plotting, or a 3D scene), for example, as described in U.S. Patent US 6,530,885 (Entrekin et al.). Although 3D datasets can include voxels and images of 3D datasets are generated based on voxels, in some examples, images drawn from 3D datasets can be 2D images that include pixels that can be displayed on conventional 2D displays (e.g., LCD displays, plasma screens).

[0040] According to examples of this disclosure, a volume renderer 334 can render a 3D scene including one or more 3D objects associated with a 2D image. The position and orientation of the planes from which the 2D image is generated relative to the volume (e.g., 3D space) can be known; these are collectively referred to as position information. In some examples, a multi-plane reformer 332 can provide the position information of the planes, the 2D image, and / or the planes to the volume renderer 334. The position and orientation of one or more 3D objects in the volume can also be known. In some examples, 3D objects can be extracted from the same image acquisition from which the 2D image is generated. In some examples, 3D objects from separate acquisitions and / or volume datasets are registered to the volume from which the 2D image is generated.

[0041] The volume renderer 334 can assign material properties to planar and 3D targets (e.g., color, absorbance, opacity, reflectivity). In some examples, material properties can be consistent across planar and / or 3D targets. In some examples, material properties can vary across voxels of planar and / or 3D targets. Optionally, some material properties of 3D targets can be assigned at least in part based on the position of the voxels relative to the viewer's position in the planar or 3D scene. For example, the color of a voxel assigned to a 3D target can vary depending on the voxel's distance from the plane.

[0042] The volumetric renderer 334 can simulate the lighting conditions of a 3D scene using a virtual light source. In some examples, the user can adjust and / or select the position of the light source relative to the 3D target and the plane via a user interface 324 (e.g., entering distance values ​​on the keyboard, dragging a light icon closer to or further away from the previously rendered 3D scene or an icon representing a 2D image and / or 3D target on the monitor). How light propagates from the light source through the 3D scene can be based at least in part on the material properties assigned to the voxels. For example, the intensity and color of the shadows cast by the 3D target can be based on the material properties assigned to the 3D target. The volumetric renderer 334 can render the 2D image of the 3D target and the 3D scene based at least in part on the material properties and the lighting provided by the virtual light source.

[0043] Outputs from scan converter 330 (e.g., B-mode image, Doppler image), outputs from multi-plane reformer 332, and / or outputs from volume renderer 334 (e.g., volume, 3D scene) can be coupled to image processor 336 for further enhancement, caching, and temporary storage before being displayed on image display 338. In some examples, scan converter 330 and / or image processor 336 can overlay a Doppler image onto a B-mode image of a tissue structure for display.

[0044] The graphics processor 340 can generate graphic overlays for display alongside images. These overlays may contain, for example, standard identification information (e.g., patient name), the date and time of the image, imaging parameters, etc. For these purposes, the graphics processor 340 can be configured to receive input from the user interface 324 (e.g., typed patient name or other annotations).

[0045] System 300 may include local memory 342. Local memory 342 may be implemented as any suitable non-transient computer-readable medium (e.g., flash drive, disk drive). Local memory 342 may store data generated by system 300 (including images, 3D models, executable instructions, input provided by the user via user interface 324), or any other information required for the operation of system 300.

[0046] As previously mentioned, system 300 includes a user interface 324. User interface 324 may include a display 338 and a control panel 352. Display 338 may include a display device implemented using various known display technologies (e.g., LCD, LED, OLED, or plasma display technologies). In some examples, display 338 may include multiple displays. Control panel 352 may be configured to receive user input (e.g., desired image, plane, desired 3D target, etc.). Control panel 352 may include one or more hardware controls (e.g., buttons, knobs, dial pads, encoders, mice, trackballs, etc.). In some examples, control panel 352 may additionally or alternatively include software controls (e.g., GUI controls or simply GUI controls) provided on a touch-sensitive display. In some examples, display 338 may be a touch-sensitive display that includes one or more software controls of control panel 352.

[0047] In some examples, Figure 3 The various components shown can be combined. For example, image processor 336 and graphics processor 340 can be implemented as a single processor. In another example, Doppler processor 360 and mode-B processor 328 can be implemented as a single processor. In some examples, Figure 3 The various components shown can be implemented as separate components. For example, signal processor 326 can be implemented as a separate signal processor for each imaging mode (e.g., B-mode, Doppler). In some examples, Figure 3 One or more of the various processors shown may be implemented by a general-purpose processor and / or a microprocessor configured to perform a specific task. In some examples, one or more of the various processors may be implemented as an application-specific integrated circuit (ASIC). In some examples, one or more of the various processors (e.g., image processor 336) may be implemented using one or more graphics processing units (GPUs).

[0048] Figure 4 This is a block diagram illustrating an example processor 400 according to the present disclosure. Processor 400 can be used to implement one or more processors described herein, for example, Figure 3 The image processor 336 is shown. The processor 400 can be any suitable processor type, including but not limited to microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable arrays (FPGAs) (wherein the FPGA has been programmed to form a processor), graphics processing units (GPUs), application-specific integrated circuits (ASICs) (wherein the ASIC has been designed to form a processor), or combinations thereof.

[0049] Processor 400 may include one or more cores 402. Core 402 may include one or more arithmetic logic units (ALUs) 404. In some examples, in addition to or in place of ALU 404, core 402 may include a floating-point logic unit (FPLU) 406 and / or a digital signal processing unit (DSPU) 408.

[0050] Processor 400 may include one or more registers 412 communicatively coupled to core 402. Registers 412 may be implemented using dedicated logic gates (e.g., bistable flip-flops) and / or any memory technology. In some examples, registers 412 may be implemented using static memory. Registers may provide data, instructions, and addresses to core 402.

[0051] In some examples, processor 400 may include one or more levels of cache memory 410 communicatively coupled to core 402. Cache memory 410 may provide computer-readable instructions to core 402 for execution. Cache memory 410 may provide data for processing by core 402. In some examples, computer-readable instructions may have already been provided to cache memory 410 via local memory (e.g., local memory attached to external bus 416). Cache memory 410 may be implemented using any suitable cache memory type, such as metal-oxide-semiconductor (MOS) memory, e.g., static random access memory (SRAM), dynamic random access memory (DRAM), and / or any other suitable memory technology.

[0052] Processor 400 may include controller 414, which can access other processors and / or components included in the system (e.g., Figure 3 The control panel 352 and scan converter 330 shown are used for inputs to the processor 400 and / or from the processor 400 to other processors and / or components included in the system (e.g., Figure 3 The outputs of the display 338 and volume plotter 334 are shown. The controller 414 can control the data paths in the ALU 404, FPLU 406, and / or DSPU 408. The controller 414 can be implemented as one or more state machines, data paths, and / or dedicated control logic units. The gates of the controller 414 can be implemented as stand-alone gates, FPGAs, ASICs, or any other suitable technology.

[0053] Register 412 and cache memory 410 can communicate with controller 414 and core 402 via internal connections 420A, 420B, 420C and 420D. These internal connections can be implemented as buses, multiplexers, crossbar switches and / or any other suitable connection technology.

[0054] Inputs and outputs to processor 400 may be provided via bus 416, which may include one or more wires. Bus 416 may be communicatively coupled to one or more components of processor 400, such as controller 414, cache memory 410, and / or register 412. Bus 416 may be coupled to one or more components of the system, such as the previously mentioned display 338 and control panel 352.

[0055] Bus 416 may be coupled to one or more external memories. The external memory may include read-only memory (ROM) 432. ROM 432 may be a mask ROM, electronically programmable read-only memory (EPROM), or any other suitable technology. The external memory may include random access memory (RAM) 433. RAM 433 may be static RAM, battery-backed static RAM, dynamic RAM (DRAM), or any other suitable technology. The external memory may include electrically erasable programmable read-only memory (EEPROM) 435. The external memory may include flash memory 434. The external memory may include a magnetic storage device such as a disk 436. In some examples, the external memory may be included in the system (e.g., Figure 3 In the ultrasound imaging system 300 shown, for example, local memory 342.

[0056] Figure 5 A graphical overview of a 3D scene is provided as an example of rendering according to this disclosure. This can be achieved through an imaging system (e.g., such as...) Figure 3 The imaging system 300 shown acquires volume 500. However, volume can also be acquired using other imaging modalities (e.g., computed tomography or magnetic resonance imaging). A slice 506 from which a 2D image is generated can be selected by positioning two parallel clipping planes 502, 504 within volume 500. In some examples, the position and / or orientation of these two clipping planes 502, 504 can be selected by a user via a user interface (e.g., user interface 324). In some examples, the distance between the two clipping planes 502, 504 can be adjustable. That is, the thickness of slice 506 can be adjustable. In some examples, the selection and extraction of slice 506 from volume 500 can be performed at least partially using a multi-plane reformer (e.g., multi-plane reformer 332).

[0057] One or more 3D targets 508 can be extracted from another volume 510. In some examples, volume 510 and volume 500 may come from the same imaging acquisition. In other examples, volume 510 comes from a different imaging acquisition or is a simulated volume registered to volume 500. For example, anatomical landmarks common to volumes 500 and 510 can be used to register the two volumes to each other. In some examples, 3D targets 508 may be anatomical structures (e.g., blood vessels, liver) and / or invasive devices (e.g., catheters, mitral valve clips, stents) segmented from volume 510 using known image processing techniques. In some examples, 3D targets may be models of anatomical structures and / or invasive devices. In some examples, anatomical structures or devices may be segmented from volume 510 and models may be superimposed on and / or replace anatomical structures or devices in volume 510. In some examples, 3D targets 508 can be generated by volumetric rasterization of 3D surfaces, for example, by transforming a triangular mesh input into a signed distance function defined on a voxel mesh using known algorithms. Volumetric rasterization can be generated for anatomical structures or models. In some examples, extraction / segmentation of 3D target 508 and / or registration of volumes 500, 510 can be performed at least in part by a volume renderer (e.g., volume renderer 334).

[0058] The volume renderer can assign material properties to voxels in volume 500. In some examples, the material properties can be defined as: absorbing light during propagation without affecting the color of the light, and reflecting light according to the grayscale intensity of the voxels in slice 506. The material properties of slice 506 can be assigned to control the transparency of slice 506 (e.g., opaque, translucent). In some examples, material properties can be assigned to all voxels in volume 500 that are not included in slice 506, making these voxels transparent. The volume renderer can assign material properties to voxels in volume 510. In some examples, the material properties of voxels included in 3D target 508 can be defined as allowing light to penetrate deeply into the interior and pass through 3D target 508. The material properties of voxels included in 3D target 508 can be further defined as modifying the hue (e.g., color) of the light during propagation. In some examples, 3D target 508 may appear opaque or nearly opaque to the viewer, but 3D target 508 may have sufficient transparency to allow for shading shadows cast by 3D target 508. Additional material properties, such as reflection and scattering, can be assigned to voxels. For example, the reflection property can make the 3D target 508 appear "shiny" or "matte". In some examples, material properties can be assigned to voxels in volume 510 outside of the 3D target 508, making these voxels transparent.

[0059] The volume renderer can simultaneously render volumes 500 and 510 using the same light source to create a 3D scene 512 comprising a 2D image 514 generated from slice 506 and 3D target 508. The shadow 516 cast by 3D target 508 on the 2D image 514 is also visible in the 3D scene 512. Reference will now be made to... Figure 6-9 Let's discuss the drawing details of volumes 500 and 510 in more detail.

[0060] Figure 6 A graphical overview of lighting passes and composite passes for rendering a 3D scene, based on examples of this disclosure, is provided. Lighting passes and composite passes can be performed by a volume renderer (e.g., volume renderer 334). In some examples, lighting passes can be performed prior to composite passes. During a lighting pass, a virtual light source 600 is simulated relative to volumes 500 and 510. In some examples (e.g., Figure 6 In the example shown, light source 600 may be a point light source that radiates light with equal intensity in all directions. In other examples, light source 600 may be a directional light source (e.g., a spotlight, a beam of light). In some examples, multiple light sources may be simulated. The user can preset or select the number of light sources, the type of light sources, the position of light source 600 relative to volumes 500 and 510, and / or the intensity of light source 600 via a user interface. In some examples, the user may also select other properties of the light source, such as the size of light source 600 and / or the color of light source 600 (e.g., wavelength range). Examples of suitable light sources and their user control can be found in U.S. patent applications US16 / 406951 and US16 / 347739, the contents of which are incorporated herein by reference for any purpose.

[0061] As illumination continues, light from light source 600 propagates through volumes 500 and 510 in three-dimensional space. Figure 6 In the diagram, only a single ray 602 is shown; however, it should be understood that many rays propagate through volumes 500 and 510 to simulate light source 600. The amount of light reaching each voxel in volumes 500 and 510 is calculated based on the properties of light source 600 and the material properties assigned to each voxel. The calculated amount of light for each voxel is stored for composite propagation.

[0062] During the composite passage, ray 610 propagates from the view plane 608 of the virtual observer 606 through volumes 500 and 510. In some examples, rays 610 may be parallel to each other and orthogonal to the view plane 608. In some examples (not shown), rays 610 may propagate from a single point on the view plane 608 (e.g., at the location of the virtual observer 606) through volumes 500 and 510 (similar to rays projected from a point light source). Rays 610 propagating from a single point can generate a perspective view. The user can preset or select the distance between the view plane 608 and volumes 500 and 510 and / or the orientation of the view plane 608 relative to volumes 500 and 510 via a user interface. Based at least in part on the voxels through which rays 610 propagate through volumes 500 and 510, the volume renderer can calculate the final values ​​of voxels and / or pixels of the 3D scene displayed to the user on a display (e.g., display 338).

[0063] Figure 7 A graphical depiction of lighting passage according to an example of this disclosure is provided. A volume 510 including a 3D target 508 and clipping planes 502 and 504 defining a slice 506 of volume 500 are shown. See reference... Figure 6 The light ray 602 discussed can propagate from the light source 600 through space, and the amount of light reaching each location can be calculated and stored according to a physical model. In some examples, the physical model may include an exponential light attenuation equation:

[0064]

[0065] Where L(X) is the amount of light reaching each location X. In some examples, each location X can be a voxel in a volume of 500 or 510. A i The absorption coefficient corresponding to each volume i (e.g., volume 500 or 510) or a portion of the volume of each voxel (e.g., slice 506 and 3D target 508), and F(V i ) is an estimate of the material density (a material property), which is a function of the volume i voxel value.

[0066] In some examples (e.g., the example shown in Equation 1), the absorption coefficients A can be defined separately for red, green, and blue, where A = [R, G, B]. For example, slice 506 could have A. 1 = [1,1,1], and 3D target 508 can have A 2 =[1,1,0.1]. For A 2 The blue component is set very low, allowing the blue light to penetrate deeply into the 3D target 508 and propagate beyond it, thus generating blue shadows (from the angle of light 602) on the slice 506 outside the 3D target 508.2 All components are equal, so slice 506 does not affect the color of light during propagation.

[0067] exist Figure 7 This is illustrated by points along ray 602. At point 700, ray 602 has not yet struck the 3D target 508. As ray 602 propagates through the 3D target 508, the red and green components of ray 602 attenuate at a higher rate than the blue component. Therefore, the light appears bluer, as indicated by the shading of points 702 and 704. However, the blue component of ray 602 is also attenuated to a certain extent, as indicated by the shading of point 706, which is darker than point 704, and the shading of point 704, which is darker than point 702. The blue light of ray 602 then strikes slice 506, creating a blue shadow (in...). Figure 7 (Not shown in the image). If slice 506 is not set to be completely opaque, light 602 will continue to propagate through slice 506. As shown at points 708 and 710, due to A 1 With the parameters being equal, the components of light 602 are attenuated by slice 506 in the same way, so the hue of light 602 does not change as it propagates through slice 506.

[0068] Figure 8 A graphical depiction of a composite passage according to an example of this disclosure is provided. Volume 500 is shown, comprising a slice 506 defined by clipping planes 502 and 504, a virtual observer 606, an observation plane 608, and a parallel ray 610. The distance between the clipping planes controls the thickness of the volume portion being displayed, while the density F(V) 1 The value of controls the degree of translucency or image plane. In some examples, the volume renderer may implement a ray-forward numerical scheme that uses the front-to-back RGB accumulation of trilinear interpolated volume samples 604 along the direction of ray 610. Based on the light calculated for each voxel in the path of the light and the forward propagation of ray 610, the volume renderer may calculate the final values ​​of voxels and / or pixels for the 3D scene displayed to the user on a display (e.g., display 338).

[0069] In some examples, in both light transmission and composite transmission, volume 500 can be ignored anywhere except for voxels in slice 506 (e.g., the area between clipping planes 502 and 504) (as indicated by the empty "frame" of volume 500 outside clipping planes 502 and 504). In some examples, volume 510 can be ignored anywhere except for voxels in 3D target 508. In some examples, ignored voxels of volumes 500 and 510 can be removed from the volume dataset. In some examples, material properties that prevent these voxels from being drawn and / or do not affect light propagation (e.g., transparency) can be assigned to ignored voxels.

[0070] Figure 9 A graphical depiction of an enhanced visualization of a 3D scene, as exemplified by this disclosure, is provided. Optionally, to further enhance the visualization of the relative positions of 3D target 508 and slice 506, distance-based artificial coloring can be applied to the 3D target. In some examples, during composite passage, the RGB value of each voxel of 3D target 508 can be multiplied by a color map value indexed by the distance between the position of the voxel of 3D target 508 and the position of slice 506. For example, the color map value may make voxels of 3D target 508 closer to slice 506 appear bluer and voxels of 3D target 508 farther from slice 506 appear more orange. Blue and orange are provided only as examples, and other colors may also be used. Furthermore, in some examples, more than two colors may be used to generate the color map values.

[0071] In other examples, color mapping can be applied after compositing, and the color mapping can be applied to the final pixel values ​​of the 3D scene. For example... Figure 9 As shown, color mapping can be based, at least in part, on distance estimation along ray 610 (e.g., "first impact"). In this method, the distance is based on the distance from the observation plane 608 of the virtual observer 606 to a point in the 3D target 508. This contrasts with previous methods, where shading was based on the distance between the 3D target 508 and the slice 506. Figure 9In the example shown, the nearest point of the 3D target 508 on plane 902 can be the point with the smallest distance (dminimum) from plane 608, while the farthest point of the 3D target on plane 904 can be the point with the largest distance (dmax) from plane 608. Plane 900 indicates the midpoint of the 3D target 508. In some examples, color map values ​​can make pixels generated from voxels in the 3D target 508 that are farther from the viewing plane 608 appear bluer, as indicated by arrow 908. Color map values ​​can make pixels generated from voxels in the 3D target 508 that are closer to the viewing plane 608 appear orange, as indicated by arrow 906. Blue and orange are provided only as examples, and other colors can also be used. Furthermore, in some examples, more than two colors can be used to generate color map values.

[0072] In yet another example, the attenuation A of each voxel of the 3D target 508 before the composite passage and illumination passage. i The value can be multiplied by a color map value indexed by the distance between the position of the voxel of the 3D target 508 and the position of the slice 506. In these examples, the shadow cast by the 3D target 508 can change color based on the distance of the portion of the 3D target 508 responsible for casting the shadow from the slice 506. In some examples, the different colors of the shadow can complement the different colors of the 3D target 508 itself.

[0073] Figure 10 This is a flowchart illustrating an example method according to this disclosure. In some examples, a volume plotter (e.g., volume plotter 334) may perform method 1000 at least partially. In some examples, a multiplane formatter (e.g., multiplane formatter 332) may perform a portion of method 1000.

[0074] At box 1002, the action "Assigning material properties to voxels of an image slice" can be performed. In some examples, the image slice may be obtained from a first volume. In some examples, the first volume may be acquired via an ultrasound imaging system, magnetic resonance imaging system, and / or computed tomography imaging system. At box 1004, the action "Assigning material properties to voxels of a 3D target" can be performed. In some examples, the 3D target may be obtained from a second volume. In some examples, the first and second volumes may come from the same image acquisition (e.g., the same dataset). In some examples, the material properties of the voxels of the 3D target and / or image slice are wavelength-dependent. In some examples, the first and second volumes may be acquired from different image acquisitions, imaging modalities, and / or datasets. In some examples, the 3D target may be segmented from the second volume. In some examples, the segmented 3D target may be a model overlaid with the 3D target or replaced by a model of the 3D target.

[0075] At box 1006, the action "drawing a 3D scene including image slices and 3D targets" can be performed. In some examples, drawing may be performed using at least one virtual light source. In some examples, drawing includes propagating light rays from the virtual light source through a first volume and a second volume. In some examples, drawing includes projecting light rays orthogonal to the viewing plane of a virtual observer. In some examples, user input indicating the position and orientation of the viewing plane relative to the first and second volumes is received. Optionally, material properties may be assigned to voxels of the 3D targets such that voxels closer to the viewing plane in the 3D targets are drawn with a different color than voxels farther away from the viewing plane. In some examples, image slices are drawn in grayscale and 3D targets are drawn in color. In some examples, voxels in the first volume outside the image slices and voxels in the second volume outside the 3D targets are ignored during drawing.

[0076] In some examples, the material properties of the voxels assigned to the 3D target are configured to render the 3D target as semi-transparent and to change the color of the light as it propagates through the 3D target from a virtual light source. In some examples, the material properties of image slices are configured to render the image slices as semi-transparent.

[0077] Figures 11-16 show example 3D scenes drawn based on different 3D datasets, according to examples of this disclosure.

[0078] Figure 11A and Figure 11B A 3D scene of a 3D model of a mitral valve clip in a 2D plane of a cardiac view acquired by ultrasound imaging, as exemplified by this disclosure, is shown. Figure 11A An exterior plan view 1100A of the mitral valve clip 1102 relative to a 2D ultrasound slice 1104 is shown. The material properties of the mitral valve clip 1102 are configured such that the mitral valve clip 1102 is opaque, while the slice 1104 is translucent (e.g., at least some light propagates through the slice 1104), thereby allowing the other half of the mitral valve clip 1102 to be seen through the slice 1104. However, the slice 1104 has sufficient opacity to make the blue shadow 1106 projected by the mitral valve clip 1102 visible. In view 1100A, the position of the light source is shown as a sphere 1108.

[0079] Figure 11A The planar view of the mitral valve clip 1102 relative to the 2D ultrasound slice 1110 is shown. Figure 11BAlthough the transparency of slice 1110 is not very obvious, the shadow 1106 cast by the mitral flap clip 1102 is still visible. In some examples, a multiplane reformer (e.g., multiplane reformer 332) can extract both slices 1104 and 1110 from the volume dataset.

[0080] Figure 12A and Figure 12B A 3D scene of a mitral valve clip in a 2D plane of a cardiac view acquired by ultrasound imaging, as an example according to this disclosure, is shown. Figure 12A and Figure 12B It is based on and Figure 11A and Figure 11B The same dataset was used for rendering. However, instead of a 3D model of the mitral valve clip, the "raw" mitral valve clip 1202, segmented from the acquired image volume, was rendered as a 3D target. Furthermore, different material properties were applied. For example, the mitral valve clip 1202 was rendered with tissue-like visual characteristics, and a faint reddish shading 1206 was projected onto the ultrasound slice 1204. In another example, as can be seen in the out-of-plane view 1200A, slice 1204 is opaque, so only a portion of the mitral valve clip 1202 is visible. Additionally, the simulated light source is located differently compared to the case in view 1100A. In the in-plane view 1200B, the light source is positioned almost "frontally," so the shading 1206 is barely visible on slice 1210.

[0081] Figure 13 An example of a 3D scene of cardiac tissue acquired by ultrasound imaging and a 2D plane of the heart view are shown, according to this disclosure. The 3D target and the 2D slices do not need to be in any particular spatial relationship relative to each other, and the viewing plane does not need to be parallel to the slices. In the 3D scene 1300, a portion of the cardiac tissue 1302 has been segmented from the scanned ultrasound volume, and this portion of the cardiac tissue 1302 is drawn using a tilted slice 1304, which is tilted relative to a virtual observer. Figure 12A and Figure 12B Similarly, having material properties, the voxel corresponding to heart tissue 1302 is drawn using the material properties of voxels that give a tissue-like appearance, which projects a faint reddish shading 1306 onto slice 1304. Similar to slices 1204 and 1210, slice 1304 is drawn using material properties that make slice 1304 opaque.

[0082] Figure 14 A 3D scene of a model of a 3D target acquired by ultrasound imaging, and a 2D plane of a fetal skull, are shown as examples according to this disclosure. In some examples, the 3D target may be a model included in a volume separate from the volume from which the 2D slices are obtained. Figure 14 As shown, multiple models of 3D targets 1402, 1404, 1406, and 1408 can be drawn relative to a 2D slice 1400 of the fetal skull. Each 3D target 1402, 1404, 1406, and 1408 can be drawn with different material properties (e.g., color and transparency). For example, object 1402 is drawn as semi-transparent, while 3D targets 1404, 1406, and 1408 are more opaque. Furthermore, 3D targets 1402, 1404, 1406, and 1408 are projected with different shading shadows 1410, 1412, 1414, and 1416, respectively.

[0083] Although the principles of this disclosure have been described with reference to ultrasound imaging systems, the examples of this disclosure can be extended to other imaging modalities.

[0084] Figures 15A-15B Examples of 3D scenes of blood vessels and 2D planes of joints acquired by computed tomography, according to this disclosure, are shown. Figure 15A The image shows a blood vessel 1502 drawn as a 3D target relative to an image plane 1500 including joint 1504. The blood vessel 1502 has been drawn as semi-transparent. However, in... Figure 15A No shadows were drawn in the middle. Figure 15B In contrast, the blood vessel 1502 is drawn according to the example of this disclosure, and the blood vessel 1502 projects a blue shadow 1506 on the image plane 1500, which can improve the viewer's ability to discern the position of the blood vessel 1502 relative to the joint 1504.

[0085] Figures 16A-16B Examples of 3D scenes and various internal organs in a 2D plane acquired by computed tomography, according to this disclosure, are shown. Figure 16A and Figure 16B As shown, it is possible to segment various anatomical features from a 3D dataset using known methods, and selectively plot these features in relation to a desired 2D plane based on the 3D dataset. In some examples, the user can select which 2D slice to plot and which anatomical features (e.g., organs) to plot relative to that 2D slice.

[0086] As described herein, a 2D image (e.g., an imaging plane) generated from slices can be simultaneously rendered in a 3D scene having one or more 3D targets by using volumetric lighting and assigning translucent material to the one or more 3D targets, such that the one or more 3D targets cast shading shadows on the 2D image. Optionally, the 3D targets can be shading at least in part based on their distance from the 2D image and / or the viewing plane. The methods for rendering 2D images and 3D targets according to examples of this disclosure can improve a viewer's ability to discern spatial relationships between 2D images and 3D targets in some applications.

[0087] In various examples of implementing apparatus, components, systems, and / or methods using programmable devices (e.g., computer-based systems or programmable logic units), it should be understood that the aforementioned systems and methods can be implemented using various known or later-developed programming languages ​​(e.g., "C", "C++", "FORTRAN", "Pascal", "VHDL", etc.). Therefore, various storage media (e.g., computer disks, optical disks, electronic storage devices, etc., capable of containing information that can instruct devices such as computers) can be prepared to implement the aforementioned systems and / or methods. Once a suitable device accesses the information and programs contained on the storage medium, the storage medium can provide the information and programs to the device, thereby enabling the device to perform the functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate materials (e.g., source files, object files, executable files, etc.) is provided to a computer, the computer can receive this information, appropriately configure itself, and perform the functions of the various systems and methods outlined in the diagrams and flowcharts above, thereby implementing various functions. That is, the computer can receive portions of various information from the disk relating to different elements of the aforementioned systems and / or methods, implement the respective systems and / or methods, and coordinate the functions of the aforementioned systems and / or methods.

[0088] In light of this disclosure, it should be noted that the various methods and apparatuses described herein can be implemented in hardware, software, and / or firmware. Furthermore, the various methods and parameters are included by way of example only and are not intended to be limiting. In view of this disclosure, those skilled in the art can implement the teachings to influence these techniques while determining their own techniques and desired instrumentation, while remaining within the scope of this invention. The functionality of one or more processors in the processors described herein can be incorporated into a smaller number or a single processing unit (e.g., a CPU) and can be implemented using application-specific integrated circuits (ASICs) or general-purpose processing circuitry programmed to perform the functions described herein in response to executable instructions.

[0089] While this system has been specifically described with reference to ultrasound imaging systems, it is conceivable that it can be extended to other medical imaging systems to systematically acquire one or more images. Therefore, this system can be used to acquire and / or record image information relating to, but not limited to, the kidneys, testes, breasts, ovaries, uterus, thyroid gland, liver, lungs, musculoskeletal system, spleen, heart, arteries and vascular system, as well as other imaging applications related to ultrasound-guided interventions. Additionally, this system may include one or more procedures that can be used with conventional imaging systems, such that said one or more procedures can provide the features and advantages of this system. Upon studying this disclosure, those skilled in the art will readily perceive certain additional advantages and features of this disclosure, or will experience certain additional advantages and features of this disclosure upon employing the novel systems and methods of this disclosure. Another advantage of this system and method is the ability to easily upgrade conventional medical imaging systems to incorporate the features and advantages of this system, device, and method.

[0090] Of course, it should be understood that any of the examples, embodiments, or processes described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated in an apparatus or apparatus portion of the system, apparatus, and method, and / or performed in an apparatus or apparatus portion of the system, apparatus, and method.

[0091] Finally, the foregoing discussion is intended to illustrate the apparatus, system, and method only and should not be construed as limiting the claims to any particular example or group of examples. Therefore, while the apparatus, system, and method have been described in particular and in detail with reference to exemplary examples, it should be understood that many modifications and alternative examples can be devised by those skilled in the art without departing from the broader and contemplated spirit and scope of the system and method set forth in the claims. Consequently, the specification and drawings should be considered illustrative and not intended to limit the scope of the claims.

Claims

1. An image rendering apparatus, comprising: The processor is configured as follows: Material properties are assigned to voxels of an image slice obtained from a first volume, wherein a 2D image is obtained from the image slice; Assign material properties to voxels of the three-dimensional 3D target obtained from the second volume; and A 3D scene is drawn comprising the 2D image obtained from the image slices and the 3D target, wherein, when drawing the 3D scene, the material properties of the voxels assigned to the 3D target are configured such that the 3D target is drawn as semi-transparent in the 3D scene, and changes the color of the light from at least one virtual light source as it propagates through the 3D target, wherein the direction of the light from at least one virtual light source propagating through the 3D target is neither parallel to nor perpendicular to the plane of the 2D image; and A display, configured to display the 3D scene, The 3D target projects colored shadows onto the 2D image obtained from the image slice.

2. The image rendering apparatus according to claim 1, wherein, The processor communicates with a user interface configured to receive user input that defines the position and thickness of the image slice obtained from the first volume.

3. The image rendering apparatus according to claim 1, wherein, The position of the virtual light source relative to the first volume and the second volume is adjustable.

4. The image rendering apparatus according to claim 1, wherein, The first volume and the second volume are from the same image acquisition.

5. The image rendering apparatus according to claim 1, wherein, The processor is configured to: assign a first material property to one or more voxels of the 3D target that are closer to the image slice, and assign a second material property, different from the first material property, to one or more voxels of the 3D target that are farther away from the image slice.

6. The image rendering apparatus according to claim 1, wherein, The 3D target includes at least one of a model, an invasive device, or an anatomical feature.

7. An image drawing method, comprising: Material properties are assigned to voxels of an image slice obtained from a first volume, wherein a 2D image is obtained from the image slice; Assign material properties to voxels of the three-dimensional 3D target obtained from the second volume; and A 3D scene is drawn comprising the 2D image obtained from the image slices and the 3D target, wherein the material properties of the voxels assigned to the 3D target are configured to cause the 3D target to change the color of the light as light from at least one virtual light source propagates through the 3D target, wherein the direction of the light from the at least one virtual light source propagating through the 3D target is neither parallel to nor perpendicular to the plane of the 2D image. The 3D target projects colored shadows onto the 2D image obtained from the image slice.

8. The image rendering method according to claim 7, further comprising segmenting the 3D target from the second volume.

9. The image rendering method according to claim 8 further includes superimposing or replacing the 3D target with a model of the 3D target.

10. The image rendering method according to claim 7, wherein, The material properties of the voxels in the image slice determine the transparency of the image slice.

11. The image rendering method according to claim 7, wherein, The drawing process includes projecting light rays that are orthogonal to the virtual observer's observation plane.

12. The image rendering method according to claim 11, further comprising: A first material property is assigned to one or more voxels of the 3D target that are closer to the viewing plane, and a second material property, different from the first material property, is assigned to one or more voxels of the 3D target that are farther away from the viewing plane.

13. The image rendering method according to claim 11, wherein, The position and orientation of the observation plane relative to the first volume and the second volume are adjustable.

14. The image rendering method according to claim 7, wherein, The material properties of the voxels of the 3D target are wavelength-dependent.

15. The image drawing method according to claim 7, wherein, The image slices are drawn in grayscale, while the 3D target is drawn in color.

16. The image rendering method according to claim 7, wherein, The drawing process involves propagating light from the virtual light source through the first volume and the second volume.

17. The image rendering method according to claim 16, wherein, During rendering, voxels of the first volume outside the image slice and voxels of the second volume outside the 3D target are ignored.

18. The image rendering method of claim 7, further comprising assigning material properties to voxels of a second three-dimensional 3D target obtained from the second volume or the third volume. in, The 3D scene also includes the second 3D target, and wherein the material properties of the voxels assigned to the second 3D target are configured to cause the second 3D target to change the color of the light as light from the at least one virtual light source propagates through the second 3D target.

19. A non-transient computer-readable medium comprising instructions that, when executed, cause an imaging system to perform the following operations: Material properties are assigned to voxels of the image slice obtained from the first volume, where... A 2D image is obtained from the image slices; Assign material properties to voxels of a 3D target obtained from a second volume; and A 3D scene is drawn comprising the 2D image obtained from the image slices and the 3D target, wherein the material properties of the voxels assigned to the 3D target are configured to cause the 3D target to change the color of the light as light from at least one virtual light source propagates through the 3D target, wherein the direction of the light from the at least one virtual light source propagating through the 3D target is neither parallel to nor perpendicular to the plane of the 2D image. The 3D target projects colored shadows onto the 2D image obtained from the image slice.

20. The non-transient computer-readable medium of claim 19, further comprising instructions that, when executed, cause the imaging system to perform the following operations: A first material property is assigned to one or more voxels of the 3D target that are closer to the view plane of the 3D scene, and a second material property, different from the first material property, is assigned to one or more voxels of the 3D target that are farther away from the view plane.

21. The non-transient computer-readable medium of claim 19, further comprising instructions that, when executed, cause the imaging system to perform the following operations: A first material property is assigned to one or more voxels of the 3D target that are closer to the image slice, and a second material property, different from the first material property, is assigned to one or more voxels of the 3D target that are farther away from the image slice.

Images