Medical image processing device and method

Abstract

To create a volume rendering image that represents a layer included in volume data with high accuracy.SOLUTION: A medical image processing apparatus according to an embodiment comprises an acquisition unit and a determination unit. The acquisition unit acquires a volumetric data set and location information on a layer to be rendered included in the volumetric data set in a structure having the layer. In a ray casting operation in rendering processing on the volumetric data set, the determination unit determines a sampling point position on a light beam projected on the volumetric data set based on the location information.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION The embodiments disclosed herein and in the drawings relate to medical imaging devices and methods. [Background technology]

[0002] For example, volumetric medical imaging techniques that can generate three-dimensional medical imaging data using any of a variety of imaging modalities, such as CT, PET, MRI, ultrasound, and X-ray, are widely used for imaging or diagnostic purposes.

[0003] Volumetric medical image data may include a three-dimensional array of voxels, each representing a specific location in three-dimensional space and each having one or more data values. For example, in the case of CT data, each voxel may have an associated intensity value representing the attenuation of applied X-ray radiation imparted to the location represented by that voxel. Intensity values ​​may be referred to as image values, gray values, gray levels, voxel values, or CT values. Intensity values ​​may be measured in Hounsfield units (HU).

[0004] Various rendering techniques are known for use in processing volumetric image data to display a desired representation of a patient or other subject. For example, the volumetric image data may be rendered to display a two-dimensional projection of the three-dimensional volumetric data. The two-dimensional projection may include desired texture or shading effects and may give the viewer the impression of a surface in three dimensions.

[0005] Commonly used volume rendering techniques use transfer functions to map each of a series of sampled voxel values ​​to an opacity value and a color value (usually represented by a combination of red, green, and blue color values). In ray casting and similar techniques, for each pixel in the 2D projection, an imaginary ray is cast from the viewer's viewpoint to a corresponding point on a 2D imaginary image plane and through the volume represented by the volumetric image dataset.

[0006] Many volume-rendered views rely on displaying anatomical structures as thin, transparent layers. For example, the surface of an anatomical structure, such as the surface of an organ, may be displayed as a thin, transparent layer. By representing the surface of an anatomical structure as a thin, transparent layer, a user viewing the image may be able to see the boundary of the anatomical structure while still being able to see the interior of the anatomical volume.

[0007] The rendered lamina may also be referred to as a hull or shell.

[0008] A view in which the anatomical structures are displayed as a thin, highly transparent layer is sometimes called, for example, a glass view.

[0009] Anatomical structures that are often rendered as thin layers include the lungs, heart, head, and chest. For example, a glass view of the lungs might be used with global illumination. Breast MR imaging might display a mammography image with thin skin boundaries present.

[0010] Two-level rendering may be used, in which an image rendered with a first rendering modality, e.g., Shaded Volume Rendering, is fused with an image of the same anatomy rendered with a second rendering modality, e.g., Maximum Intensity Projection. Multi-object opacity views, in which multi-object opacity is a subset of two-level rendering, may be used.

[0011] Thin, transparent layers may be used for rendering the fetus, specifically during ultrasound imaging in the first trimester or early second trimester. A fetal glass view may be used to identify fetal organs.

[0012] Thin layers of high transparency would traditionally be achieved by inserting a transfer function, sometimes called a spike transfer function. An example of a spike transfer function is shown in Figure 2 and described below. The spike transfer function targets the sub-volume boundaries between tissue intensities.

[0013] Figure 1 shows a simplified example of a volume with spatial variation in intensity values. The horizontal and vertical axes in Figure 1 represent spatial axes. A circular region of high intensity is located in the center of the volume in Figure 1, and the intensity decreases with radius.

[0014] FIG. 2 shows an example of a spike transfer function, spike transfer function 2, for opacity only. The horizontal axis 4 of FIG. 2 shows intensity values ​​on a scale of 0 to 1. The vertical axis 6 of FIG. 2 shows opacity values ​​on a scale of 0 to 1. Spike transfer function 2 comprises a narrow spike of opacity at a particular intensity value, approximately 0.38 in the illustrated example. Within the narrow spike, the opacity value is high. Outside the narrow spike, the opacity value is zero.

[0015] Figure 3 shows an example of the results of applying the spike transfer function 2 of Figure 2 to the volume of Figure 1. The spike transfer function results in a circular boundary that outlines the circular shape of the volume of Figure 1.

[0016] We find that the use of spike transfer functions typically works well in direct volume rendering when the object being outlined is smooth and its gradient is consistent, but there may be situations where the use of spike functions is prone to artifacts.

[0017] The sharp boundaries produced by spike transfer functions may be difficult for rendering algorithms. Shading artifacts may occur. Pre-integration may reduce shading artifacts, but some shading artifacts will remain.

[0018] If a layer is very thin, a rendering method that samples at a regular sampling interval will sometimes hit the layer and sometimes cross it. The rendering of the layer may appear inconsistent to the viewer; for example, the layer may appear to have varying thicknesses.

[0019] Pre-integration is a technique that involves using pre-calculated colors for each sample spacing. In two-dimensional (2D) pre-integration, the opacity and color values ​​for each possible set of intensity values ​​and sample spacings are pre-calculated using a transfer function and stored in a lookup table. In one-dimensional (1D) pre-integration, a 1D lookup table is used to approximate the functionality of 2D pre-integration techniques while reducing memory and processing load. An example of such a 1D technique is described in H. Kye et al., Interactive Classification for Pre-Integrated Volume Rendering of High-Precision Volume Data, Graphical Models, 70 (2008), 125-132.

[0020] Figure 4 shows an example of an image containing a thin layer, sometimes called a thin hull. The image is rendered using conventional volume rendering without pre-integration. Conventional volume rendering struggles to solve the thin layer consistently. Solving a thin layer consistently means that the layer appears smooth in the rendered image, without discontinuities or exaggerated irregularities.

[0021] Figure 5 shows an example of the same image as Figure 4, but rendered using preintegrated volume rendering. Preintegrated volume rendering helps produce an improved image, although some artifacts remain.

[0022] As shown in Figures 4 and 5, the use of spike transfer functions may result in artifacts. Also, it may only be possible to visualize certain types of boundaries using spike transfer functions. For example, it may be possible to visualize air-tissue or bone-tissue boundaries using spike transfer functions, but it may be more difficult to visualize boundaries between different types of tissue. When rendering using spike transfer functions, very strong intensity variations may be necessary to obtain a smooth surface. Even with strong intensity variations, shading may still be problematic.

[0023] Especially in ultrasound images, the gradients across the surface may not have sufficient consistency, which may lead to difficulties in rendering using spike transfer functions.

[0024] When using a spike transfer function, the thickness of the rendered layer depends on the rate of change of intensity across the layer. If the opacity is constant across samples, the layer may extend across all of the samples. Intensity values ​​across samples may be said to be in spikes. Where the layer is thicker, the rendered image will appear more opaque. Where the layer is thinner, it will appear more transparent. Thickness variations may lack visual appeal and make the resulting rendered image difficult to interpret. [Prior art documents] [Patent documents]

[0025] [Patent Document 1] U.S. Patent No. 10,008,026 Summary of the Invention [Problem to be solved by the invention]

[0026] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to generate a volume rendering image that depicts layers included in volume data with high accuracy. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0027] According to an embodiment, a medical image processing apparatus includes an acquisition unit and a determination unit. The acquisition unit acquires a volumetric dataset and location information of a layer in a structure having the layer to be rendered, the layer being included in the volumetric dataset. The determination unit determines, based on the location information, sampling point positions on a ray cast onto the volumetric dataset in a ray casting operation in a rendering process for the volumetric dataset. The location information includes a geometry function that defines the geometry of the layer. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 1 shows an example of a simplified representation of a volume with spatial variations in intensity. [Figure 2] FIG. 2 is a diagram showing an example of a plot of a spike transfer function. [Figure 3] FIG. 3 shows an example image illustrating the results of applying the spike transfer function of FIG. 2 to the volume of FIG. [Figure 4] FIG. 4 shows an example of an image rendered using conventional volume rendering without pre-integration operations. [Figure 5] FIG. 5 is a diagram showing an example of an image rendered using normal volume rendering with pre-integration calculation. [Figure 6] FIG. 6 is a schematic diagram of an example of an apparatus according to an embodiment. [Figure 7]FIG. 7 is a flowchart illustrating an example of a general method according to an embodiment. [Figure 8] FIG. 8 is an example of a schematic diagram of multiple rays intersecting the hull at different ray angles. [Figure 9] Figure 9 is an example of a schematic diagram of light incidence on the hull from different light angles. [Figure 10] FIG. 10 shows an example plot representing Hull's complete integral space. [Figure 11] FIG. 11 is a schematic diagram of a ray intersecting a hull, showing an example of sample locations. [Figure 12] FIG. 12 is an example of a detailed view of the spacing between the sample points and the hull of FIG. [Figure 13] FIG. 13 illustrates an example of an ultrasound image rendered using known rendering methods. [Figure 14] FIG. 14 illustrates an example of an ultrasound image rendered using a method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0029] A medical image processing apparatus 10 according to an embodiment is shown schematically in Figure 6. The medical image processing apparatus 10 may also be referred to as a medical imaging apparatus.

[0030] The medical imaging system 10 comprises a computing device 12, which in this example is a personal computer (PC) or workstation, connected to a scanner 14 via a data store 20.

[0031] The medical imaging system 10 further includes one or more display screens 16 and one or more input devices 18, such as a computer keyboard, mouse, or trackball.

[0032] In this embodiment, scanner 14 is a computed tomography (CT) scanner configured to obtain volumetric CT scans. Scanner 14 is configured to generate image data representative of at least one anatomical region of a patient or other subject. The image data includes a plurality of voxels, each having a corresponding data value. The data values, in the case of CT scanning, are intensity values, such as values ​​in Hounsfield units.

[0033] In other embodiments, scanner 14 may be configured to obtain two-dimensional, three-dimensional, or four-dimensional image data using any imaging modality, such as a magnetic resonance (MR) scanner, a computed tomography (CT) scanner, a cone-beam CT scanner, a positron emission tomography (PET) scanner, an X-ray scanner, or an ultrasound scanner.

[0034] In this embodiment, the image data set obtained by the scanner 14 is stored in the data store 20 and then provided to the computing device 12. In an alternative embodiment, the image data set is provided from a remote data store (not shown). The data store 20 or the remote data store may comprise any suitable form of memory storage. In some embodiments, the medical imaging device 10 is not connected to the scanner. That is, the medical imaging device 10 may be a workstation or the like that is separate from the scanner.

[0035] Computing device 12 includes a processing unit 22 for processing data. The processing unit includes a central processing unit (CPU) and a graphical processing unit (GPU). Processing unit 22 provides processing resources for automatically or semi-automatically processing medical image datasets. In other embodiments, the data to be processed may include any image data that is not medical image data.

[0036] The processing unit 22 comprises a geometry circuit 24 configured to determine and apply a geometry function representing the thin layer, a rendering circuit 26 configured to render an image from the volumetric image data, and a memory 28 .

[0037] In this embodiment, the circuits 24, 26 and memory 28 are each implemented in a CPU and / or GPU by a computer program having computer-readable instructions executable to perform the method of the embodiment. In other embodiments, the various circuits may be implemented as one or more Application Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs).

[0038] Computing device 12 has a hard drive and other components of a PC, including RAM, ROM, a data bus, an operating system including various device drivers, and hardware devices including a graphics card, although such components are not shown in FIG. 6 for clarity.

[0039] The apparatus of FIG. 6 is configured to perform the method for processing medical image data shown in FIG.

[0040] 7 is a flowchart illustrating an example of a generalized rendering method according to an embodiment. In the discussion of rendering below, including ray casting methods, operations (e.g., surface traversal or surface lighting) may be referred to as occurring in physical space. However, it will be apparent that the operations described below are virtual (simulated) operations that occur as arithmetic operations on a volumetric image dataset. While the stages of FIG. 7 are described in a particular order, in other embodiments, the stages of FIG. 7 may occur in any suitable order.

[0041] At stage 30 of FIG. 7, the rendering circuitry 26 receives a volumetric image dataset obtained from a CT scan performed by the scanner 14. The volumetric image dataset includes individual data values ​​for a plurality of voxels of the volumetric image dataset. The data values ​​represent intensity values. In further embodiments, the volumetric image dataset may include any suitable medical image data in any suitable modality. In further embodiments, the volumetric image dataset may include any suitable non-medical image data. Note that the volumetric image dataset may also be referred to simply as a volumetric dataset.

[0042] At stage 32, rendering circuitry 26 receives a transfer function that associates data values ​​with color and opacity values. In this embodiment, the color values ​​are RGB values. The transfer function assigns respective red, green, blue, and alpha (opacity) values ​​to each data value within the domain of the data values. In other embodiments, the color values ​​may be described in any suitable color space. The color values ​​may also be grayscale values. The transfer function is an example of a precomputed function.

[0043] In stage 34, geometry circuitry 24 obtains location information. The location information includes a geometry function that describes a thin hull, which may also be described as a layer or shell. In this embodiment, the thin hull is closed. In other embodiments, the thin hull may be open. The thin hull may contain one or more holes. In other embodiments, the thin hull may be convex or non-convex. The volumetric image data set, transfer function, and location information acquisition functions in rendering circuitry 26 are examples of an acquisition unit.

[0044] Geometry functions are continuous functions R 3 →R. The geometry function may be described as a user-defined function. The process of obtaining location information may involve manual user input. The user may select the type of geometry function to be used. The user may manually modify the scene to precisely define where the layers are.

[0045] The structures to be rendered in the volumetric image dataset include at least one of an anatomical structure, an organ, a tumor, and a tissue region. Additionally, a layer (hull) represents at least one of a surface of the structure to be rendered, an external structure, a navigation aid, an implant, and an acquisition device. For example, the hull may be considered to form a surface boundary of the anatomical structure to be rendered. For example, the anatomical structure may be a blood vessel, a lung, a heart, or a head. In some embodiments where the volumetric image dataset represents ultrasound data, the anatomical structure may be a fetus or a portion of a fetus. For example, but not limited to, the hull may represent a boundary of an anatomical organ that has thickness and exists between the object to be depicted and the viewpoint.

[0046] The hull is rendered to be mostly transparent, so that the user is aware of the hull's presence but still has a glass-over-glass view that allows them to see features inside or behind the hull.

[0047] In this embodiment, geometry circuitry 24 processes the volumetric image data set to obtain a geometry function that describes the hull. In other embodiments, geometry circuitry 24 may receive the geometry function from other circuitry in medical imaging system 10 or from additional devices.

[0048] In this embodiment, processing the volumetric image dataset to obtain the geometry function includes a combination of volume thresholding of the volumetric image dataset and segmentation of the volumetric image dataset. Volume thresholding includes applying a threshold to data values ​​in the volumetric image dataset to distinguish between voxels having data values ​​below the threshold and voxels having data values ​​above the threshold. Thresholding may be used, for example, to distinguish between voxels representing a first type of tissue and voxels representing a second type of tissue.

[0049] Segmentation involves identifying voxels that represent anatomical structures and may be performed using any suitable segmentation method. Segmentation methods may include, for example, anatomical or pathological segmentation using an analysis package, such as a deep learning-based analysis package. Segmentation may include manual editing, semi-automatic edge tracing tools, such as live wire techniques, or morphological modifications, such as erosion, dilation, opening, or closing.

[0050] In other embodiments, any suitable method may be used to determine the geometry function representing the hull. The method may include thresholding a volumetric image dataset, sometimes referred to as a primary volume. The method may include thresholding a second volumetric image dataset, sometimes referred to as a secondary volume or fusion volume. The second volume may represent the same anatomy as the primary volume. The second volume may be acquired at a different time than the primary volume. The second volume may be acquired with a different modality than that used for the primary volume, for example, using a different type of scanner. In methods where thresholding of the second volume is used to obtain the geometry function, there may be an additional step of registration between the primary volume and the second volume.

[0051] The method for determining the geometry function may comprise any suitable form of segmentation of any suitable object within the volumetric image dataset.

[0052] In some embodiments, the geometry function includes or is derived from a distance field, such as a signed distance field. An external geometry may be provided as a distance field. The external geometry may include geometry not obtained from a volumetric image dataset. The external geometry may include, for example, highlighting of a region. In some embodiments, the external geometry may include or be derived from a model of an implant. The external geometry may be used to simulate experimental placement of an implant before the implant is physically placed. The external geometry may represent at least one navigation aid. For example, the navigation aid may include at least one arrow or other symbol. The navigation aid may include geometry representing a path, such as a long tube. The navigation aid may represent an acquisition device, such as an ultrasound probe.

[0053] A signed distance field (in 3 dimensions) is a continuous function R with values ​​representing the distance to the boundary of an object. 3 →R. The signed distance field may be written as f(x,y,z). The value of f(x,y,z) at each point inside the object is the negative minimum distance from that point to the boundary of the object. The value of f(x,y,z) at each point outside the object is the minimum distance from that point to the boundary. The signed distance field contains a representation of the surface of the object. The surface is where the signed distance field f(x,y,z) is zero.

[0054] In some embodiments, landmark clustering regions may be used to obtain a geometry function for the hull. Obtaining the geometry function may include determining distances to multiple points, such as multiple landmarks. In other embodiments, other derived volumetric regions may be used. In some embodiments, a landmark algorithm may be used to find points of the anatomy. For example, a landmark algorithm may be used to find points of the anatomy within the liver. Geometry may be attached to the points of the anatomy and rendered using the methods described below. The geometry attached to the points of the anatomy may include a simple geometry, such as a bounding sphere, or the smallest smooth object that encompasses all of the points. In other embodiments, the geometry attached to the points of the anatomy may include a model, such as a liver model, attached to match the points. A central difference method may be used to obtain the geometry function.

[0055] A geometry function may represent an isosurface, which is a surface of constant data value, for example constant intensity.

[0056] In some embodiments, surfaces across multiple phases may be presented as a set of separate hulls. Isosurfaces may be tracked across multiple phases. For example, a set of cardiac images may comprise images acquired at different phases of cardiac motion. Moving anatomy from multiple scans may be projected into a single scene, possibly with data from one of the scans. Combining layers into a single scene may provide a way to visualize shape differences across a sequence without actually performing animation.

[0057] In further embodiments, any form of location information may be used to represent the geometry of the hull.

[0058] In this embodiment, the geometry function is determined by the geometry circuitry 24. In other embodiments, the geometry function may be determined by the rendering circuitry 26 or by any suitable circuitry of the medical imaging system 10. The geometry function may be obtained by an additional device and passed to the circuitry of the medical imaging system 10. The geometry function may be obtained in a pre-processing stage before the method of Figure 7 begins.

[0059] In this embodiment, the geometry function is determined by processing a volumetric image dataset, although in other embodiments the geometry function may be determined based on any suitable dataset or datasets.

[0060] In this embodiment, the geometry function describes the hull as an infinitely thin layer. As described below with reference to stage 38, the hull is treated as infinitely thin when considering sampling locations along a ray. However, as described below with reference to stage 36, a thickness is assigned to the hull when calculating the hull's contribution to the color value accumulated by the ray. In other embodiments, a geometry function is calculated that includes the hull's thickness, but that thickness is ignored when considering sampling locations.

[0061] In stage 36, the rendering circuitry 26 pre-calculates the hull's contribution to the accumulated color and opacity along the ray. The pre-calculation of the contribution occurs prior to the ray casting that occurs in stage 38. The pre-calculation involves pre-integrating the entire parameter space of the hull. The pre-integrated values ​​may be calculated discretely (numerically) or analytically.

[0062] For pre-integration operations, rendering circuitry 26 receives or assigns a predefined set of properties to the hull. The set of properties includes thickness and one or more material properties, such as attenuation. In this embodiment, rendering circuitry 26 assigns a constant thickness to the hull. Rendering circuitry 26 defines the hull as having constant material properties, such as constant attenuation. In other embodiments, any suitable set of properties may be used. The set of properties may include a thin hull profile and its color and opacity. The color and opacity may be specified as a color-opacity table. In some embodiments, the profile may be predictable such that no transfer function input is required.

[0063] In the pre-integration operation, rendering circuitry 26 determines the contribution of the hull to the light rays passing through it. The contribution includes color and opacity. The contribution of the hull depends on the angle of the light rays intersecting the hull. In this embodiment, the contribution of the hull also depends on the angle at which the hull is illuminated by the light source.

[0064] Figure 8 shows an example of the traversal of a hull 50 having a constant thickness t. The hull may also be referred to as a film or layer. Hull 50 is traversed by multiple rays 51, 52, 53, 54, 55, 56, 57, 58, and 59, also referred to as view rays. Each of rays 51-59 intersects hull 50 at a different angle. Each of rays 51-59 has a different ray direction.

[0065] Rays that intersect hull 50 at a steeper angle will travel further through the thickness of the material than rays that intersect hull 50 at a closer to normal angle. For example, in Figure 8, rays 51 and 59 travel further through the thickness of the material than ray 55.

[0066] Figure 9 shows an example of multiple light directions 61, 62, 63, 64, 65, 66, 67, and 68 combined into a single light ray 60. Light directions are sometimes referred to as illumination angles. Different illumination angles produce different lighting effects. For example, a steep angle between the light ray and the hull and a steep angle between the light direction and the hull causes the surface to cast a shadow on itself and appear dark. The angle determines how much light is attenuated as it passes through the hull. The multiple lights shown in Figure 9 include scattered rays.

[0067] The rendering circuitry 26 pre-integrates the color and opacity contributions resulting from rays intersecting the hull for each possible combination of ray angle and illumination angle. The hull has a constant thickness, measured along the normal, because the hull's material is constant. Therefore, the hull's color and opacity contributions to a ray are larger when the hull is intersected at a large angle from the normal than when the hull is intersected at an angle closer to the normal.

[0068] Figure 10 represents the complete integral space of the hull and shows an example of the color and opacity contribution of the hull at each angle. The image in Figure 10 is reproduced in grayscale rather than full color, so the color contribution is shown in gray.

[0069] The integration space is shown using a checkerboard background, which is obscured in parts of the space with large contributing opacities.

[0070] The x-axis (horizontal axis) of Figure 10 represents the cosine of the angle between the hull and the ray. This angle, sometimes referred to as the ray_angle, is defined as the angle between the ray and the normal to the hull. The ray_angle angle is sometimes described as the angle relative to the camera, since the ray can be considered to be cast from the camera position.

[0071] The horizontal midpoint in Figure 10 is where cosine(ray_angle) = 0 and occurs when ray_angle is 90 degrees. Angles close to 90 degrees are sometimes described as being steep relative to the hull or grazing the hull.

[0072] Negative values ​​of x represent rays traversing the hull from inside the hull toward the outside of the hull. Positive values ​​of x represent rays traversing the hull from outside the hull toward the inside of the hull. In this embodiment, the contributions of negative values ​​of cosine(ray_angle) and positive values ​​of cosine(ray_angle) are symmetric. The contribution from rays traversing the hull from inside to outside is the same as the contribution from rays traversing the hull from outside to inside. In other embodiments, the contributions of positive and negative values ​​may be different. The contribution from rays traversing the hull from inside to outside may be different from the contribution from rays traversing the hull from outside to inside. Steps traversing the hull from outside to inside may be integrated differently than steps traversing the hull from inside to outside.

[0073] The y-axis (vertical axis) in Figure 10 represents the cosine of the angle between the hull and the light direction. This angle, sometimes denoted as light_angle, is defined as the angle between the light direction and the hull normal.

[0074] The vertical midpoint in FIG. 10 occurs when cosine(light_angle)=0 and light_angle is 90 degrees. A negative value of y indicates that the light source is located inside the hull and light traverses the hull from inside to outside the hull. A positive value of y indicates that the light source is located outside the hull and light traverses the hull from outside to inside the hull. In this embodiment, the contribution of negative values ​​of cosine(light_angle) is symmetrical with the contribution of positive values ​​of cosine(light_angle). In other embodiments, the contributions of positive and negative values ​​may be different.

[0075] Near (0,0), both the ray and the light graze the surface of the hull. Near (1,0), the view ray goes straight into the hull, but the light grazes the surface. When cosine(ray_angle) is near zero, both angles are very steep, and the ray passes through a lot of material, resulting in high opacity.

[0076] The pre-integrated contributions at all values ​​of cosine(ray_angle) and cosine(light_angle) are stored in memory 28. In this embodiment, the pre-integrated contributions are stored as a look-up table representing all possible interactions with the hull. In other embodiments, the pre-integrated contributions may be stored as a look-up table or a list, or in any suitable format.

[0077] The pre-integrated contribution is sometimes referred to as the pre-integrated Hull integral.

[0078] In some embodiments, the pre-integrated contribution includes a thin film effect. A thin film effect occurs when the thickness of a hull is similar to the wavelength of the light used to illuminate the hull. When the thickness of a hull is similar to the wavelength, the color seen in the rendering is determined by the angle at which the hull is illuminated. The color contribution varies with angle and is wavelength dependent. For example, the color across the hull may vary, similar to the color variation seen in a soap bubble.

[0079] In some embodiments, the pre-integrated contributions may be approximated using any suitable approximation. In other embodiments, for example when the hull is very thin, it may be preferable to use the exact integral.

[0080] In this embodiment, the pre-integration operation of stage 36 is performed by rendering circuitry 26. In other embodiments, the pre-integration operation may be performed by any suitable circuitry or by additional devices in medical imaging system 10. In some embodiments, the pre-integration operation may be obtained in a pre-processing stage before the method of FIG. 7 begins.

[0081] At stage 38, the rendering circuitry 26 performs a rendering process, also referred to as a rendering operation or rendering algorithm, to render an image from the volumetric image dataset. The rendering circuitry 26 determines a virtual camera position and viewing direction from which the volume of the volumetric image dataset is viewed, thereby determining a virtual image plane. The rendering circuitry 26 casts multiple virtual rays into the volume of the volumetric image dataset. Each ray corresponds to a distinct pixel in the resulting 2D image.

[0082] Stage 38 includes a number of individual stages 40-46 that are performed on each of a number of beams.

[0083] Assuming one of the rays is cast into the volume, at stage 40 the rendering circuitry 26 begins stepping through the volume accumulating color integrals along the ray according to conventional ray-casting methods.

[0084] The rendering circuitry 26 samples data values, such as intensity values, at a plurality of sample points along the path of the ray through the volumetric data set. In this embodiment, the sample points are equally spaced along the ray. In other embodiments, any suitable sample spacing may be used, and may be variable or adaptive. The rendering circuitry 26 determines an opacity and color value for each sample point using the transfer function obtained in stage 32. The rendering circuitry 26 constructs the opacity and color values ​​along the ray and accumulates the contributions from each sample point. The accumulation function in the rendering circuitry 26 that accumulates the opacity and color values ​​is an example of an accumulator.

[0085] In stage 42, the rendering circuitry 26 uses the geometry function obtained in stage 34 to detect whether the ray intersected the hull.

[0086] FIG. 11 illustrates an example of a portion of a ray 70 intersecting a hull 72, sometimes referred to as a layer or shell. Although the hull 72 shown in FIG. 11 has a thickness, the hull is treated by the rendering circuitry 26 as being infinitely thin in stage 42. The portion of the ray shown in FIG. 11 includes three sample points 74, 76, and 78. The three sample points 74, 76, and 78 shown in FIG. 11 are examples of initial positions for the sample points. These initial positions are automatically set by the rendering circuitry 26, for example, at equal intervals along the ray 70.

[0087] Rendering circuitry 26 samples a data value at a first sample point 74 and determines a contribution for that sample point 74. The contribution includes color and opacity. Color includes red, green, and blue values. Rendering circuitry 26 combines the contribution for sample point 74 with the contributions of previous sample points of the ray, not shown in FIG. 11, to obtain the accumulated color and opacity.

[0088] Rendering circuitry 26 advances to a second sample point 76 on ray 70. The step between first sample point 74 and second sample point 76 is indicated by arrow 80.

[0089] In stepping from the first sample point 74 to the second sample point 76, the rendering circuitry 26 determines that it has stepped over the hull 72 located between the first sample point 74 and the second sample point 76. The determination that the hull 72 is located between the first sample point 74 and the second sample point 76 is based on a geometry function.

[0090] The rendering circuitry 26 estimates the location of the intersection where the hull is intersected by the ray. In estimating the intersection, the hull is treated as an infinitely thin surface. The hull is defined as having a thickness much thinner than a voxel, much thinner than the renderer steps along the ray that represents a pixel. In rendering in stage 38, the ray cannot step to a location that is inside the boundary formed by the hull. Instead, the hull will always be straddled by the ray. The estimator function in the rendering circuitry 26 that estimates the location of the intersection is an example of an estimator.

[0091] The estimated position of the intersection point may also be referred to as the estimated substep position of the hull. The method for estimating the position of the intersection point may depend on how the geometry function is defined. Methods for estimating the position may include, for example, thresholding and gradient descent for signed distance fields. For functions where gradient descent is not possible, the rendering circuitry 26 may employ a binary search to a particular precision.

[0092] The rendering circuitry 26 steps back along the ray to the estimated sub-step position of the hull, which is a position between the position of the first sample point 74 and the position of the second sample point 76.

[0093] The rendering circuitry 26 calculates the surface normal of the hull 72 at the intersection point. The geometry function is differentiable so that the normal to the hull at each point on the hull can be determined. For example, the surface normal may be calculated by computing the gradient using any suitable gradient estimation method. In other embodiments, the surface orientation may be defined as a separate function rather than by the geometry function itself.

[0094] 12 shows an example of stepping back from sample point 76 to the intersection of ray 70 and hull 72. The stepping back is indicated by arrow 82. In other words, rendering circuitry 26 moves second sample point 76 in a direction opposite to the direction of travel of ray 70, and determines the position of second sample point 76 on the intersection of ray 70 and hull 72. The determination function in rendering circuitry 26 that determines the position of second sample point 76 is an example of a determination unit.

[0095] At stages 44 and 46, the rendering circuitry 26 calculates cosine(ray_angle), which is the cosine of the angle between the normal and the ray. The rendering circuitry 26 calculates cosine(light_angle), which is the cosine of the angle between the normal and the lighting direction. Note that, in general, determining the cosine of an angle between vectors may be easier than determining the angle itself. Thus, storing and calculating cosine values ​​may be more efficient than storing and calculating angle values, which may improve performance.

[0096] The rendering circuitry retrieves hull integral pre-integration values, including color and opacity values, stored in memory 28. The rendering circuitry 26 applies the color and opacity to the main integral accumulated along the ray using the sub-step position.

[0097] Once the contribution of hull 72 has been added to the accumulated color and opacity values, the rendering process returns to stage 38 to continue sampling at sample points along the ray, including third sample point 78 .

[0098] In some situations, a single ray of light may pass through hull 72 more than once. For example, hull 72 may form a closed shell. Light ray 70 may pass through a first surface of hull 72 from the outside to the inside, and then pass through a second surface of hull 72 from the inside to the outside. Hull 72 may have one or more corrugations, causing the ray of light to pass through hull 72 two, three, four, or more times.

[0099] If the ray passes through hull 72 more than once, stages 42-46 are repeated each time the ray intersects hull 72. Each time the ray intersects hull 72, a respective contribution is added to the ray. The ray casting operation of this embodiment uses a pre-computed function to estimate, for at least some of the multiple rays, the location of a second intersection point where the ray intersects hull 72 a second time and determine the contribution of the second intersection point to the ray's aggregate color.

[0100] In some embodiments, there is more than one hull, for example, a first hull may represent the boundary of a first anatomical structure and a second hull may represent the boundary of a second anatomical structure.

[0101] Multiple geometry functions may be obtained, each representing one of the multiple hulls, and the contribution from each of the multiple hulls may be pre-computed and stored in a respective look-up table.

[0102] If the hulls do not coincide and there is no instance where two or more hulls occur in a given spacing of adjacent sampling points, rendering may proceed as described above with reference to Figure 7. Rendering circuitry 26 determines when one of the hulls is crossed. If a hull is crossed, rendering circuitry 26 determines which of the hulls has been crossed and searches the appropriate lookup table to provide the contribution of that hull.

[0103] If two or more intersections of hull 72 occur between adjacent first and second sampling points, rendering may order the intersections to determine the order in which to accumulate the contributions of the two hulls. If the two hulls coincide, rendering circuitry 26 may designate one hull as the inner hull and the other as the outer hull, depending on predetermined rules. By designating one hull as inner and the other as outer, rendering circuitry 26 may determine the order in which to add the hull contributions.

[0104] Stages 40-46 are repeated for each ray to obtain a color value for each pixel in the rendered image. Color and opacity values ​​are accumulated along each ray. The accumulated color and opacity values ​​for a ray include contributions from one or more hulls intersected by the ray. In the ray-casting process, hulls 72 are treated as infinitely thin surfaces for which the fixed color and opacity values ​​calculated in stage 36 are used.

[0105] At stage 48, rendering circuitry 26 displays the rendered image on display screen 16 or one or more other display screens. In some other embodiments, the rendered image may not be displayed, but instead may be provided to further circuitry for further operations on data representing the rendered image.

[0106] The method of FIG. 7 may provide a fast path to generating high-quality glass-like hulls or shells. The method of FIG. 7 may be faster than using existing glass-like presets. Each shell has a precise opacity and does not have varying thicknesses. The method may be controlled by segmentation. The geometry function may be considered very flexible, specifically, more flexible than the intensity-to-color mapping defined by volume rendering presets. Better and faster rendering performance may be obtained.

[0107] Rendering a hull is done by a simple lookup of pre-computed contributions using a geometry function. Thin hulls may be rendered to a constant state in a single step. The rendering step does not exactly sample the hull, but instead assumes that the hull is always between two sample steps. Volumetric integration is applied in exactly one step.

[0108] By assigning a thickness to the hull when precomputing its contribution, you can achieve a glass-view effect where the outline of the structure becomes visible in the rendered image. When precomputing the contribution, the thickness seen by a ray depends on the angle between the ray and the hull. You get a rendered image in which the hull appears to have a consistent thickness.

[0109] Some existing methods incorporate thin hulls into volume rendering by targeting partial volume boundaries in the transfer function. Such existing methods may be considered less than ideal. Only selective boundaries will be possible with reasonable quality. Artifacts may be present.

[0110] The method of FIG. 7 works by pre-computing the hull, including shading. Efficient and substantially artifact-free visualization of the hull may be obtained. A user-defined function may be used to describe the hull's geometry. The geometry function may be applied independently of the volume being rendered. This may allow for the freedom to represent complex geometries. In some situations, a perfectly or nearly perfectly constant smooth hull may be obtained.

[0111] It has been found that using pre-integration of the hull integral may give richer shading. A subtle transition between high and low opacity areas of the hull may be obtained. If the geometry function defines a good surface, the surface may appear to have perfect resolution in the rendered image.

[0112] The surface may be quite distinct. In embodiments where the hull curves on itself or has a complex shape, it may be particularly important that the hull appear to have a consistent thickness.

[0113] In some cases, multiple surfaces may be displayed. The presence of many complex surfaces may make the image difficult to interpret. In such cases, it would be particularly helpful if the surfaces had a clear appearance.

[0114] FIG. 13 shows an example of an ultrasound image rendered using an existing rendering method in which hull pre-integration is not used. The hull is represented by a sample set pre-integration using the spike transfer function described above with reference to FIGS. 1-3. Surfaces appear unclear due to apparent variations in the thickness of the surface. Some areas of the surface appear thicker and more opaque. Some areas appear thinner and more transparent. It may be difficult to distinguish between multiple surfaces.

[0115] Figure 14 is an example of the same image rendered using the method of Figure 7. You can see that the surfaces in Figure 14 look simpler and clearer than those in Figure 13.

[0116] The method of Figure 7 may also be adapted for global illumination rendering. The above embodiment is described with reference to the view ray used in single pass rendering. The light direction is considered as part of that single rendering pass. The thickness seen by the ray depends on its angle relative to the hull.

[0117] In a global illumination embodiment, a first pass is made in which light rays are cast into the volume to obtain a photon map containing irradiance values. A second pass is made in which view rays are cast into the volume to accumulate irradiance values. In a global illumination embodiment, the thickness seen by a light ray depends on the angle between the photon ray and the hull. The relationship between the photon ray and the hull may be thought of as similar to the relationship between the light ray and the hull in the previous embodiment. When determining the photon map, only the cosine of the angle between the photon ray and the hull normal is used, making the table of pre-accumulation values ​​one-dimensional.

[0118] The first pass needs to take the hull into account, since it casts shadows, and the second pass takes the hull's lighting integration into account, since its finite thickness means that the correspondence between the hull and a single voxel's proximity may be poor.

[0119] In some embodiments, multiple lookup tables may be obtained using different optical models. Global illumination rendering may utilize several lookup tables. The tables may represent different geometric functions.

[0120] 12, the rendering circuitry 26 samples points on the hull 72 by moving the sample points onto the intersections of the ray 70 and the hull 72, but the method for changing the sample points is not limited to this. For example, the rendering circuitry 26 may add new sample points onto the intersections of the ray 70 and the hull 72 in addition to the sample points set as the initial positions.

[0121] While the above embodiments have been described with respect to medical imaging data, in other embodiments, any data may be rendered using the above methods. For example, the data may include oil and gas data. The data may include three-dimensional microscopy data. The data may include, for example, climate data, geological survey data, demographic data, or gaming data.

[0122] Although particular circuits are described herein, in alternative embodiments, the functionality of one or more of these circuits may be provided by a single processing resource or other component, or the functionality provided by a single circuit may be provided by a combination of two or more processing resources or other components. A reference to a single circuit encompasses multiple components that provide the functionality of that circuit, whether or not such components are separate from one another. A reference to multiple circuits encompasses a single component that provides the functionality of those circuits.

[0123] In a first aspect, a medical imaging device is provided that includes a processing circuit configured to acquire a volumetric dataset, acquire location information for layers that are part of a structure represented in the volumetric dataset, and execute a ray casting algorithm to generate a rendered image from the volumetric dataset, the ray casting algorithm including: for each of a plurality of rays cast into the volume of the volumetric dataset, determining data values ​​for a plurality of sample locations along the ray, estimating a location of an intersection point where the ray intersects the layer based on the location information, and using a pre-computed function to determine a contribution of the intersection point to an aggregate color of the ray.

[0124] The contribution may depend on the angle between the layer and the light ray. The contribution may depend on the angle between the layer and the direction of light from a light source.

[0125] The pre-computed functions may be pre-computed integrals that integrate color and opacity over the layer depending on certain properties of the layer.

[0126] The location information may include a geometry function that defines the geometry of the layer.

[0127] The obtaining of the location information may include thresholding data values ​​in the volumetric data set. The obtaining of the location information may include thresholding a further data set representing the structure.

[0128] The obtaining of the location information may comprise segmenting the volumetric dataset. The obtaining of the location information may comprise segmenting a further dataset representing the structure.

[0129] The obtaining of the location information may include applying a distance field to external geometry, performing a landmark clustering process, and tracking iso-surfaces across multiple phases.

[0130] The location information may include a numerical function, the location information may include an analytical function, or the location information may include a geometric function.

[0131] The structure may include an anatomical structure. The structure may include an organ. The structure may include a tumor. The structure may include a tissue region.

[0132] The layer may represent a surface of the structure. The layer may represent an external structure. The layer may represent a navigation aid. The layer may represent an implant. The layer may represent an acquisition device.

[0133] The ray casting algorithm may further include estimating, for at least some of the plurality of rays, a location of a second intersection point where the ray intersects the layer a second time, and using the pre-computed function to determine a contribution of the second intersection point to an aggregate color of the ray.

[0134] The ray casting algorithm may further include estimating, for at least some of the plurality of rays, a location of a further intersection point where the ray intersects a further layer that is part of a further structure represented in the volumetric dataset, and may further include using the pre-computed function or a further pre-computed function to determine a contribution of the further intersection point to an aggregate color of the ray.

[0135] The ray casting algorithm may further comprise determining an order of the intersection point and the further intersection point relative to the direction of the ray.

[0136] The value of the pre-calculated function at the intersection point may depend on whether the ray intersects the layer passing from a first side of the layer to a second side of the layer, or whether the ray intersects the layer passing from the second side of the layer to the first side of the layer.

[0137] The contribution may be wavelength dependent, thereby giving a thin film effect.

[0138] The ray casting algorithm may be a global illumination algorithm. The rays may be photon rays that perform photon mapping.

[0139] The volumetric data set may include at least one of CT data, MR data, PET data, cone-beam CT data, X-ray data, and ultrasound data.

[0140] In a further aspect that may be provided independently, a method is provided that includes acquiring a volumetric dataset, acquiring location information for layers that are part of a structure represented in the volumetric dataset, and executing a ray casting algorithm to generate a rendered image from the volumetric dataset, the ray casting algorithm including: for each of a plurality of rays cast into the volume of the volumetric dataset, determining data values ​​for a plurality of sample locations along the ray, estimating a location of an intersection point where the ray intersects the layer based on the location information, and using a pre-computed function to determine a contribution of the intersection point to an aggregate color of the ray.

[0141] In a further aspect of the invention, which may be provided independently, there is provided a medical imaging method comprising a ray casting algorithm, a thin-hull geometry defined by a user-defined R3 → R function, and a pre-calculated integral of the hull profile with respect to the angle of the surface relative to the ray / light, the ray casting algorithm detecting transitions across the hull geometry, backing out steps into the hull, and applying the pre-calculated color integral based on the angle of the hull relative to the ray and light.

[0142] The hull geometry function may be given by one or a combination of the following 1 to 6: 1. Main volume / threshold 2. Second or Additional Volume / Threshold 3. Segmentation Objects 4. External geometry given as a distance field 5. Isosurfaces traced across multiple facies 6. Parsing Functions

[0143] The pre-calculated color integral may also capture wavelength-dependent thin film effects.

[0144] Steps across the hull from the outside may be integrated differently than steps across the hull from the inside.

[0145] In a further aspect, which may be provided independently, there is provided a medical imaging apparatus comprising a processing circuit for obtaining location information of layers that are part of a structure within volume data, the structure being the subject of a volume rendering process; setting positions of sample points corresponding to rays used in a ray casting algorithm based on the location information; and calculating ray casting operations for the volume data using the positions of the sample points to generate a rendered image.

[0146] Features in one embodiment may be provided as features in any other embodiment, as appropriate. For example, method features may be provided as apparatus features, and vice versa. Any one or more features in one embodiment may be provided in combination with any suitable one or more features of any other embodiment.

[0147] According to at least one of the embodiments described above, it is possible to generate a volume rendering image that depicts layers included in volume data with high accuracy.

[0148] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0149] 10 Medical image processing device 12 Computing Devices 14 Scanner 16 display screens 18 Input Devices 20 Data storage unit 22 Processing equipment 24 Geometry Circuit 26 Rendering Circuit 28 memory 50,72 Hal 51,52,53,54,55,56,57,58,59,60,70 rays 61,62,63,64,65,66,67,68 Light direction 74, 76, 78 sample points

Claims

1. an acquisition unit for acquiring a volumetric dataset and location information of a layer to be rendered in a structure having the layer included in the volumetric dataset; a determination unit that determines sampling point positions on rays cast onto the volumetric dataset based on the location information in a ray casting calculation in a rendering process for the volumetric dataset; Equipped with the location information includes a geometry function that defines a geometry of the layer; Medical imaging equipment.

2. an estimation unit that estimates a position of an intersection point where the ray intersects the layer based on the location information, the determination unit moves the sampling point position from an initial position onto the estimated intersection point. The medical image processing device according to claim 1 .

3. an estimation unit that estimates a position of an intersection point where the ray intersects the layer based on the location information, the determination unit newly adds the sampling point position to the estimated intersection point. The medical image processing device according to claim 1 .

4. an accumulation unit that accumulates contributions to a color value for each of the plurality of sampling points on the ray based on a pre-calculated function; 4. The medical image processing device according to claim 2 or 3.

5. The contribution depends on the angle between the layer and the light ray. The medical image processing device according to claim 4 .

6. The contribution depends on the angle between the layer and the direction of light from the light source. the light includes scattered radiation; The medical image processing device according to claim 4 .

7. the pre-computed functions are pre-computed integrals that integrate color and opacity over the layer depending on predetermined properties of the layer; The medical image processing device according to claim 4 .

8. obtaining the location information comprises thresholding data values ​​in the volumetric data set or in a further data set representing the structure; The medical image processing device according to claim 1 .

9. - obtaining the location information comprises segmentation of the volumetric dataset or of a further dataset representing the structure; The medical image processing device according to claim 1 .

10. The acquisition of the location information includes: a) applying a distance field to the external geometry; b) performing a landmark clustering process; c) tracking iso-surfaces across multiple layers; At least one of a) to c) above is included, The medical image processing device according to claim 1 .

11. the location information includes at least one of a numerical function, an analytical function, and a geometric function; The medical image processing device according to claim 1 .

12. the structure includes at least one of an anatomical structure, an organ, a tumor, and a tissue region, and the layer represents at least one of a surface of the structure, an external structure, a navigation aid, an implant, and an acquisition device; The medical image processing device according to claim 1 .

13. the ray casting operation further includes, for at least some of a plurality of rays cast at the volumetric data set, estimating a location of a second intersection point where the ray intersects the layer a second time and using a pre-computed function to determine a contribution of the second intersection point to an aggregate color of the ray. The medical image processing device according to claim 1 .

14. the ray casting operation further includes using a pre-computed function or a further pre-computed function to estimate, for at least some of the rays cast at the volumetric dataset, a location of a further intersection point where the ray intersects a further layer that is part of a further structure represented in the volumetric dataset, and to determine a contribution of the further intersection point to an aggregate color of the ray. The medical image processing device according to any one of claims 1 to 13.

15. the ray casting operation further includes determining an order of the intersection point and the further intersection point relative to the direction of the ray. The medical image processing device according to claim 14.

16. the value of the pre-calculated function at the intersection point depends on whether the ray intersects the layer passing from a first side of the layer to a second side of the layer, or whether the ray intersects the layer passing from the second side of the layer to the first side of the layer. The medical image processing device according to claim 4 .

17. The contribution is wavelength dependent, thereby giving a thin film effect. The medical image processing device according to claim 4 .

18. The ray casting operation is a global illumination algorithm, and the rays are photon rays that perform photon mapping. The medical image processing device according to any one of claims 1 to 17.

19. the volumetric data set includes at least one of CT data, MR data, PET data, cone-beam CT data, X-ray data, and ultrasound data; The medical image processing device according to any one of claims 1 to 18.

20. a first acquisition step of acquiring a volumetric data set; a second acquisition step of acquiring location information of a layer to be rendered in a structure having the layer included in the volumetric dataset; a determining step of determining, in a ray casting calculation in a rendering process for the volumetric dataset, sampling point positions on a ray cast onto the volumetric dataset based on the location information; Including, the location information includes a geometry function that defines a geometry of the layer; method.

Images