Methods and devices for rendering volumetric datasets
By combining ray tracing or path tracing processes from multiple volume datasets, visual parameter values are generated, overcoming the shortcomings of existing rendering methods and achieving more accurate and information-rich visualization of volume datasets.
Patent Information
- Application Number
- CN202610662482.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2026-05-14
- Publication Date
- 2026-11-17
AI Technical Summary
Existing rendering methods struggle to effectively combine volumetric datasets with different imaging characteristics across multiple imaging processes, resulting in insufficient or inaccurate visual representations.
By combining multiple volume datasets, ray tracing or path tracing processes are performed to generate visual parameter values, including sampling, interpolation, boundary recognition, and lighting models, to produce more accurate volume rendering results.
It improves the rendering accuracy and information richness of volumetric datasets, reveals the relationships between different data types, and generates more realistic visualizations.
Smart Images

Figure CN123118879A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of computer graphics and image processing, and more particularly to methods and systems for rendering volumetric datasets. More specifically, this disclosure relates to techniques for rendering volumetric data obtained from multiple imaging processes with different imaging characteristics. Background Technology
[0002] Volumetric data can be used to visualize the volume of objects of interest, such as body parts in medical scans. Scans of a single volume, comprising a collection of multiple volumetric data with different properties, such as multispectral CT and multiparametric MRI, have recently enabled improvements in the detail of body parts that can be imaged.
[0003] The goal is to improve the methods used to render collections of such volumetric data in order to provide a better / more useful visual representation of such data. Summary of the Invention
[0004] According to a first aspect of the invention, a computer-implemented method is provided for rendering a volume dataset representing a volume. The volume dataset includes a set of multiple volume data obtained through corresponding multiple imaging processes performed on the volume, wherein each imaging process has a set of its own distinct imaging characteristics. The method includes performing a volume rendering process using the set of multiple volume data to render an image of the volume. The volume rendering process includes generating one or more visual parameter values for a given location in the image to determine an image display value for that location. The given visual parameter values are generated based on a combination of the sets of multiple volume data.
[0005] Multiple imaging processes can be distinct pulse sequences derived from multiparameter magnetic resonance imaging (MRI) studies. In some implementations, a first set of imaging characteristics may include a first set of wavelengths of electromagnetic radiation, and a second set of imaging characteristics may include a different second set of wavelengths of electromagnetic radiation. In some implementations, particularly in photon-counting CT imaging, multiple imaging processes can be distinct spectral ranges.
[0006] Generating one or more visual parameter values may include performing a ray tracing process. The ray tracing process may include performing a sampling process at multiple sampling points along the ray for each ray traced during the ray tracing process. The sampling process may include: generating a set of corresponding sample values for the multiple sampling points based on at least one set of volumetric data, and using the corresponding sample values to generate one or more visual parameter values. The sampling distance may be determined based on one or more of a set of imaging characteristics from multiple imaging processes.
[0007] For a first ray tracked during the ray tracing process, a first sampling process can be performed at a first plurality of sampling points separated by a first sampling distance along the first ray. This first sampling process may include generating a first set of corresponding sample values based on a first set of volume data. Similarly, for a second ray tracked during the ray tracing process, a second sampling process can be performed at a second plurality of sampling points separated by a second sampling distance along the second ray, generating a second set of corresponding sample values based on a second set of volume data. The first sampling distance may be determined based on a set of imaging characteristics of the first set of volume data, and / or the second sampling distance may be determined based on a set of imaging characteristics of the second set of volume data.
[0008] The method may include: using a first set of volume data to identify the approximate location of the boundary between two media in the volume, and using a second set of volume data to identify the refined location of the boundary.
[0009] When generating a given visual parameter value, the method may include: for a given sampling location in the volume data set from which the given visual parameter value is calculated at least in part, determining whether a given set of the volume data should be used when determining the sample value for the given sampling location. This determination may be based on segmentation data indicating the segment associated with the sampling location. In some implementations, determining whether to use the given set of the volume data may include: determining that the given set of volume data does not contribute to the sample value when the given sampling location and the surface of the segment associated with the given sampling location satisfy a proximity condition.
[0010] The method may further include: a set-based interpolation method for determining image characteristics of one or more sets of volume data, using the interpolation method to interpolate volume values at sampling locations from a first set of volume data, and using the volume values when generating one or more visual parameter values.
[0011] The method may further include determining an additional interpolation method based on a set of image characteristics of a second set of volume data. The additional interpolation method can be used to interpolate additional volume values at the sampling locations from the set of volume data, and these additional volume values can be used when generating one or more visual parameter values.
[0012] The method may include: using a combination of sets of multiple volumetric data to determine the input to a lighting model for sampling locations, and using the lighting model to perform a volumetric rendering process to generate one or more visual parameter values. The input to the lighting model may include various parameters such as opacity, color, geometric properties, and / or surface optical properties.
[0013] When the input to the lighting model includes geometric properties, these properties may include surface normals, surface tangents, or surface curvature. When the input to the lighting model includes surface optical properties, these properties may include specular reflectivity, roughness, anisotropy, or subsurface translucency.
[0014] The volumetric rendering process may include a path tracing process. In an implementation involving path tracing, the path tracing process may include determining a first cumulative opacity value along a path using a first set of volume data, and comparing the first cumulative opacity value with a first threshold corresponding to the set of volume data. A scattering event may be triggered at least in part based on the determination that the first cumulative opacity value exceeds the first threshold. Similarly, the path tracing process may include determining a second cumulative opacity value along a path using a second set of volume data, and comparing the second cumulative opacity value with a different second threshold corresponding to the set of volume data. A scattering event may be triggered at least in part based on the determination that the second cumulative opacity value exceeds the second threshold.
[0015] In some implementations, the first cumulative opacity value and the second cumulative opacity value can be for the same path, and the scattering event can be triggered only if both the first cumulative opacity value exceeds a first threshold and the second cumulative opacity value exceeds a second threshold. In some implementations, the first threshold can be equal to the second threshold. Alternatively, the first threshold can be a first random value generated from a first distribution, and the second threshold can be a second random value generated from a different second distribution.
[0016] During path tracing, the method may include determining a set of contributing volumetric data from a set of multiple volumetric data for sampling locations on path segments of the path being traced during the path tracing process. This determination may be based on at least one of the following: the ray type of the path segment (which may be primary ray, secondary ray, or shadow ray); if the ray type of the path segment is secondary ray, whether the path segment was generated by a volumetric scattering event or an interface scattering event; the distance between the sampling location and the location of the scattering event that generated the path segment; the number of path segments traversed by the path up to the path segment; input received via a user interface; a body region represented by the sampling location; at least one of the imaging processes; an imaging device used in at least one of the imaging processes; or a process for interpreting the image.
[0017] The path tracing process may include triggering a scattering event at the scattering location. Triggering the scattering event may include determining the direction of a new ray to be used in the path tracing process based on a set of volume data at the scattering location, wherein the new ray originates from the scattering location. Determining the direction of the new ray may include: selecting a phase function or a surface scattering distribution function from a set of corresponding phase functions corresponding to the set of volume data, and based on the volume values of the set of volume data at the scattering location; and using the phase function or surface scattering distribution function to determine a new direction. Alternatively or additionally, determining the direction of the new ray may include determining the parameters of the phase function based on the volume values of the set of volume data at the scattering location, and using the phase function or surface scattering distribution function to determine a new direction.
[0018] The path tracing process may also include a light source function that determines which of the set of volume data was used during the accumulation of sample values along the path leaving the volume.
[0019] When determining the light source function for a path leaving the volume, the method may base the determination on which of the most frequently used volume data sets during the accumulation of opacity values. Alternatively or additionally, determining the light source function for the path may include determining the function based on whether the initial scattering event along the path leaving the volume is a volume scattering event or a surface scattering event.
[0020] In some implementations, multiple imaging processes can be performed on a volume simultaneously.
[0021] According to a second aspect of the invention, a machine-readable instruction set is provided that, when executed by a processor, causes the processor to perform the method according to the first aspect.
[0022] According to a third aspect of the invention, an apparatus is provided comprising a processor and a storage device, the storage device comprising a machine-readable instruction set. Attached Figure Description
[0023] Figure 1 This is a flowchart illustrating a method for generating visual parameter values from a set of multiple volume data according to an embodiment;
[0024] Figure 2 This is a perspective view illustrating the ray tracing process used in volumetric rendering according to an embodiment;
[0025] Figure 3 This is a perspective view illustrating the path tracing process used in physically based volumetric rendering according to an embodiment;
[0026] Figure 4 This shows the implementation Figure 1 A block diagram of the device for the volumetric rendering method is shown in the figure; and
[0027] Figure 5 This is a graph illustrating the multidimensional transfer function according to the implementation method. Detailed Implementation
[0028] Figure 1 A method for generating visual parameter values from a set of multiple volumetric data is illustrated. The method begins at step 102, obtaining a set of multiple volumetric data representing volumes, which have been obtained through multiple corresponding imaging processes, each of which has its own set of distinct imaging characteristics. Each set of volumetric data may include multiple volume values representing discrete samples of a scalar field. For example, the set of volumetric data can be received by loading from memory, a sensor, and / or other sources. Volumetric dataset 206 represents a volume that may include a patient or a portion of a patient, such as a human or animal patient.
[0029] Typically, any suitable imaging procedure can be used to generate a set of data for each volume. For example, the imaging procedure may include using computed tomography (CT) or magnetic resonance imaging (MRI). The scan data may be provided as multiple two-dimensional (2D) scans, or it may be formatted from the scan. In some examples, the set of data for each volume may include a stack of 2D image slices. 2D image slices may include compressed slice data in, for example, JPEG2000 format.
[0030] In some examples, multiple imaging procedures are performed on the volume simultaneously. For example, the multiple imaging procedures can be distinct pulse sequences from a multiparameter magnetic resonance imaging (MRI) study. Pulse sequences can include any of the following: spin echo sequences (e.g., T1-weighted, T2-weighted, or proton density-weighted sequences), gradient echo sequences, inversion recovery sequences, diffusion-weighted sequences, perfusion-weighted sequences, functional MRI sequences, or magnetic resonance angiography sequences. That is, the set of imaging characteristics can represent the pulse sequences of an MRI study. Alternatively, a first set of imaging characteristics can include a first set of wavelengths of electromagnetic radiation, and a second set of imaging characteristics can include a different second set of wavelengths of electromagnetic radiation. For example, volume dataset 206 can represent a dual-energy computed tomography (CT) study or a photon-counting computed tomography (PCCT) study.
[0031] In some examples, volume dataset 206 is a large volume dataset generated by a high-resolution scanning mode. Volume dataset 206 may, for example, have a data size of several gigabytes, such as hundreds of gigabytes or several terabytes.
[0032] Volume dataset 206 may include data formatted as multiple volume values associated with corresponding voxels. Each set of volume data may include a volume value associated with each voxel. That is, each voxel may be associated with a corresponding volume value in each set of volume data. For example, voxels may be arranged in a uniform or non-uniform grid, or in some other type of geometry (e.g., polar coordinates). Voxels in each set of volume data may spatially overlap; that is, at least one voxel in one set of volume data has a corresponding voxel representing the same portion of the volume in each of the other sets of volume data. Volume values may be isotropic or anisotropic. Each volume value may typically represent a scalar value obtained by sampling a scalar field, although in some examples, volume dataset 206 may include data associated with a non-scalar field. The type of volume value represented for each voxel may depend on how volume dataset 206 was obtained. For example, in the case where a CT scanner is used to generate volume dataset 206, the dataset may include Hounsfield values. The volume value for each voxel may be represented with a given precision, such as 8 bits or 16 bits.
[0033] At step 104, the method includes performing a volumetric rendering process using a set of multiple volumetric data to render an image of that volume. The volumetric rendering process may include a ray tracing process. The ray tracing process can be a non-physical process, such as a ray casting process, as referenced below. Figure 2 The described or physical-based process (e.g., path tracing process, as referred to below) Figure 3 The process described, or another process such as Metropolis optical transport, radiance, or photon mapping.
[0034] The volumetric rendering process includes generating one or more visual parameter values for a given location in the image, such as a pixel, to determine the image display value for that location. The given visual parameter value is generated based on a combination of multiple sets of volumetric data.
[0035] For example, visual parameter values can be opacity or color values calculated for a light source or path. A given visual parameter value can, for example, be an opacity and / or color value accumulated across multiple sampling locations 2^14. During the rendering process, a combination of multiple sets of volumetric data can be used at various points to generate visual parameter values, as shown below. Figure 2 and Figure 3Described by way of example. For example, a given visual parameter value could be an RGBA value representing color and opacity. In other examples, multiple visual parameter values could represent color and opacity separately, such as RGB values and alpha values. In other examples, visual parameter values could correspond to different visual parameters that can be used to determine the image display value. For example, the image display value could be the pixel intensity, brightness, or color value at a given location in the image. The image display value can be obtained by synthesizing multiple visual parameter values for a given location, such as a pixel, in the image plane. For example, the visual parameter value obtained for a given image location could be stored in a buffer, such as an HDR buffer, and synthesized to generate the image display value. The visual parameter values stored in the buffer can be updated as more light is simulated through that volume.
[0036] By generating visual parameter values through the combination of multiple sets of volumetric data, rendering can produce more accurate and information-rich visualizations of volumetric datasets. The method disclosed in this paper utilizes complementary information contained in different sets of volumetric data to create renderings that reveal relationships between different data types that may not be apparent when viewing a pixel-by-pixel blended image composed of different renderings of different sets of volumetric data.
[0037] Figure 2 An example ray tracing process used in volumetric rendering is illustrated schematically. A ray tracing process is performed on volume dataset 206. The collection of volume data in volume dataset 206 is registered with each other relative to the volumes. In this example, the volume represented by volume dataset 206 has a cubic geometry. This volume may include a patient or a portion of a patient, such as a human or animal patient.
[0038] During ray tracing, a viewpoint 212 is defined relative to a volume dataset 206. A view plane 202 is also defined and positioned in front of the viewpoint 212. The view plane comprises, for example, multiple pixels arranged in a grid. Figure 2 (not shown in the image), and allows the construction of visualizations of a volume dataset of 206.
[0039] In this example, the ray tracing process includes determining, for example, multiple rays 208, 210 passing through the volume dataset 206 via ray projection. Each ray 208, 210 begins at viewpoint 212 and crosses the view plane 202 at a predefined location. The ray direction (i.e., the predefined location) is randomly generated. In some examples, the direction of ray 208 used to sample a first set of volume data is randomly generated, and the direction of ray 210 is also randomly generated, independent of the direction of ray 210 used to sample a second set of volume data. However, in other examples, the same ray can be used to sample both sets of volume data.
[0040] Performing the ray tracing process includes: for the ray 208 traced during the ray tracing process, performing a sampling process at multiple sampling points along the ray 208. The sampling process includes generating a set of corresponding sample values for the multiple sampling points 214 based on at least one set of volume data, and using the corresponding sample values to generate one or more visual parameter values.
[0041] In some examples, the sampling distance is determined based on one or more of the imaging characteristics of multiple imaging processes. For example, the sampling distance can be determined based on the finest spatial frequency component measured by one of the imaging processes (in an example where only a set of volumetric data is used to calculate the sample value, this could be the imaging process used for that set of volumetric data). For example, the sampling distance can be determined as 1 / (2f) using the Nyquist theorem. max (Optionally further divided by the zoom of the imaging device used), where f max It is the finest (highest) spatial frequency component in the volumetric data obtained through this imaging process. max This can be a typical representative value of the imaging process. However, in other examples, the finest spatial frequency component can be determined based on the volumetric data itself, for example, through a Fourier transform of the volumetric data. The finest spatial frequency component can be identified as, for example, the highest spatial frequency component whose amplitude exceeds a threshold.
[0042] In some examples, the sampling distance is determined based on one or more volume values in the set of volume data, such as near the sampling location (e.g., within a predefined number of voxels). For example, the sampling distance can be determined based on local values of contrast in one or more of the set of volume data. For example, the contrast can be calculated as the magnitude of the gradient of the volume data at the sampling location.
[0043] By determining the sampling distance based on the imaging characteristics of the imaging process, this method can optimize the rendering process to take into account the different resolutions or properties of various imaging processes.
[0044] In some examples, when performing the ray tracing process, a first set of volume data is sampled for a first ray 208, and a second set of volume data is sampled for a second ray 210. In some such examples, the method includes applying different corresponding sampling distances to sample the first set of volume data and the second set of volume data. For example, the method may include determining a first sampling distance based on a set of imaging characteristics of the first set of volume data; and / or determining a second sampling distance based on a set of imaging characteristics of the second set of volume data. By using different sampling distances for different rays 208, 210 based on the specific imaging characteristics of each volume dataset, the method achieves more efficient and / or more accurate rendering that adapts to the unique properties of each data representation. For example, ray 208 used to sample a set of volume data with high contrast can use a short sampling distance to capture the fine structure of the volume data in detail, while ray 210 used to sample a set of volume data with low contrast can use a long sampling distance to improve computational efficiency. Although the sampling described above is performed for different rays, in some examples, a ray can be used to sample a first set of volume data at a first sampling distance and also to sample a second set of volume data at a second sampling distance.
[0045] During sampling of the first set of volumetric data, the boundary between two media within the volume can be identified, for example, based on determining that two adjacent sampling locations belong to different segments (segmentation will be further described below). To refine the estimation of the location of the boundary along the light ray, the sampling process can be performed using a second set of volumetric data, which can have higher contrast than the first set of volumetric data. (See reference...) Figure 3 Furthermore, the refined surface location can also be used as the location of scattering events during path tracing.
[0046] In the example, at each sampling location 214, for example, for each set of volume data, it is determined whether to use the given set of volume data when determining sample values for the sampling location. This determination is performed based on segmentation data that indicates the segment associated with the sampling location (e.g., the segment associated with the voxel closest to the sampling location). For example, segmentation data corresponding to a volume may indicate multiple segments within the volume. The segmentation data can be obtained through a segmentation process performed as part of the rendering method, or through a segmentation process performed prior to this method. A segment may represent a different part of a patient depicted in the set of volume data (e.g., heart, lungs), or other objects depicted in the set of volume data (e.g., a patient or a portion of a patient placed on a table when the set of volume data is captured). For each voxel, the segmentation data may include a value indicating the segment to which the voxel is determined to belong. For each voxel, the segmentation data may include a value indicating the segment to which the voxel is determined to belong (for the set of volume data). Determining which volumetric dataset to use for targeted visualization based on segmentation data allows for the rendering of different anatomical structures using the set of volumetric data most suitable for a given sampling location 214. For example, if the segmentation data of a given set of volumetric data indicates that a given sampling location 214 represents a portion of a stage on which a body part is placed during the imaging process, it might be desirable to exclude that set of volumetric data when calculating the sample values for that given sampling location 214. Alternatively, if a particular voxel is determined to represent a body part, such as a patient's heart, then sets of volumetric data with poor contrast for that body part can be excluded when calculating the sample values.
[0047] This method may include preprocessing segmented data to remove high-frequency components, i.e., segments below a certain size (e.g., width or volume), to generate preprocessed segmented data. The determination of whether to use a given set of volume data when determining sample values is based on this preprocessed segmented data. Preprocessing can be used to eliminate such elements, for example, in cases where it is desired not to display small metallic objects in an image.
[0048] The sampling process involves interpolating the volume value of sampling location 214 from at least one of the sets of volume data. Any suitable interpolation method can be used, such as nearest neighbor interpolation, trilinear interpolation, or cubic interpolation. Sampling points typically do not perfectly coincide with the center of any given voxel; therefore, it is expected that interpolation will be performed from multiple voxels surrounding sampling location 214.
[0049] Determining whether to use a given set of volumetric data when determining sample values may include: in response to determining that the given set of volumetric data does not contribute to the sample values, the surface of the given sampling location 214 and the segment to which the given sampling location 214 belongs satisfies a proximity condition. By excluding certain volumetric data from contributing to the sample values based on their proximity to the segment surface, this method can prevent artifacts from appearing near segment interfaces. For example, it may be determined that a set of volumetric data depicts a segment, such as a metal part, that is not depicted in other sets of volumetric data. For a set of volumetric data depicting this segment, it would be undesirable to interpolate volume values partly from within and partly from outside the segment when generating sample values, because such sample values would not effectively represent any of the segments. Therefore, if it is determined that the surfaces of a given sampling location 214 and the segment to which the given sampling location 214 belongs satisfy the proximity condition of the given set of volume data (e.g., they are separated by fewer than a predefined number of voxels, such as 1 voxel, or if the interpolation method used would result in the use of volume values from multiple segments when generating sample values), it can be determined that the given set of volume data does not contribute to the sample values.
[0050] While in some examples (as described above), the entire set of volumetric data can be ignored when calculating the sample value at a given sampling location 214, in other examples, segmentation data based on a given voxel can be used to determine whether that voxel contributes to the sample value. That is, while one volume value in the set of volumetric data may not contribute to a given sample value, it can be determined that another volume value in the same set of volumetric data does contribute to that sample value. For example, in the example mentioned above where the set of volumetric data depicts a metal part, when determining the sample values at sampling locations (e.g., sampling locations near the boundary of the metal part), the segmentation data indicates that the volume values of the metal part can be ignored, while the segmentation indicates that the volume values of the metal part cannot be used.
[0051] Returning the interpolation, in some examples, this method involves determining the interpolation method based on a set of one or more image features from a collection of volumetric data. The f mentioned above can be used... max To determine the interpolation method. Determining the interpolation method based on image characteristics allows the rendering process to optimize the sampling quality of different types of volumetric data, improving overall visualization accuracy and / or computational efficiency. For example, if the set of volumetric data is associated with high contrast (e.g., if f...), the interpolation method can be optimized. max If the volume value is above a threshold, then a first interpolation method (e.g., a method with higher accuracy, such as cubic interpolation) can be used to interpolate the volume value in order to effectively display high contrast; however, if the set of volume data is associated with low contrast (e.g., if f...), then...max If the volume is below a threshold, a computationally efficient interpolation method (e.g., nearest neighbor) can be used to interpolate the volume value. Therefore, the interpolation method can be determined based on the contrast value (e.g., calculated as the difference between the maximum and minimum volume values in the data set or the segment to which sampling position 214 belongs), and different interpolation methods can be used for different sets of volume data.
[0052] Alternatively or additionally, the method includes determining the kernel size of the interpolation method (the number of voxels contributing to sample values in a given set of volumetric data, or the maximum distance from the sampling location where a voxel contributes to the sample value). This determination can be based on f max To execute. For example, the maximum distance from the sampling location where the voxel contributes can be determined as 1 / (4f). max (Optionally divided by the zoom of the camera device).
[0053] Once the volume values at sampling position 214 have been interpolated, a transfer function can be used to determine the sample opacity and / or sample color to be input to the lighting model based on those volume values. In some examples, the transfer function can take multiple volume values from a set of corresponding volume data as input. (See brief reference.) Figure 5 The diagram illustrates a graph demonstrating such a transfer function. As can be seen, the sample opacity in this example is a two-dimensional function of the volume values of a first set of volume data (e.g., interpolated) and a second set of volume data (e.g., interpolated). Sample color can also vary as a two-dimensional function of these variables. Although... Figure 5 The example illustrates a two-dimensional function, but the transfer function can take more than two inputs (i.e., volume values from three or more sets). Furthermore, while sample opacity and sample color are used as inputs to the lighting model in this example, properties derived from the volume values (e.g., specular reflectivity, roughness, anisotropy, or subsurface translucency) can be additionally or alternatively used as inputs to the lighting model.
[0054] Using a combination of multiple sets of volumetric data as input to the transfer function allows information from more than one imaging process to be used when generating visual parameter values. For example, if an object, such as a metal piece, exists in one set of volumetric data but not in another, the transfer function can prioritize input from the set of volumetric data that shows the metal piece. In this way, by comparing it to an average of an image showing, for example, the metal piece and an image not showing the metal piece, the resulting final image can systematically incorporate additional details by utilizing the characteristics of each imaging process.
[0055] Additionally, other inputs to the illumination model can be determined based on volume data. For example, geometric properties such as surface normals (used in, for example, Phong illumination models), surface tangents (and corresponding double tangents), or surface curvature (used in curvature-dependent reflection models) can be determined. As an example, the surface normal can be determined as the gradient of the volume data at sampling location 214, for example, from the same set of volume data used to interpolate the volume values at sampling location 214. Other inputs to the illumination model include surface optical properties such as specular reflectivity, roughness, anisotropy, or subsurface translucency. These surface optical properties can be determined based on the volume values themselves or based on the segment to which sampling location 214 belongs. Geometric properties and surface optical properties can be determined based on a combination of multiple sets of volume data. For example, any of the geometric properties mentioned above can be determined as the average (e.g., mean) of the geometric properties across the set of volume data for each set of volume data (or determined as the set of volume data contributing to the sample value). That is, for example, the surface normal can be determined as the average of the surface normals determined for each set of volume data.
[0056] The lighting model is then used to model the volume illuminated by light source 204. Light source 204 can be a point source, a directional source, or may include a lightmap. The simulated light source can also be any other type of light source—for example, a model of any luminous object—or a combination of multiple different light sources. In some examples, a portion of the volume dataset 206 itself is luminous. In some examples, the light source may include a high-resolution lightmap. For example, when volume dataset 206 is a cube, the lightmap may have six sides corresponding to the outer surfaces of volume dataset 206.
[0057] Then, taking into account the simulated light at sampling position 214, a shading calculation is applied at sampling position 214 to generate the opacity and / or color value for that sampling position 214. The shading calculation can use an appropriate lighting model that takes into account the inputs mentioned above, such as the Phong lighting model or a curvature-dependent lighting model.
[0058] The opacity and / or color values generated for each sampling position 214 of a given ray 208 are then accumulated along ray 208 to determine the visual parameter values of that ray 208. The visual parameter values generated for the ray corresponding to a given position (e.g., pixel position) in the image can then be summed to determine the image display value (e.g., pixel intensity value) at that position. For example, the visual parameter values can be stored in a buffer such as an HDR buffer and synthesized to generate one or more pixel display values for a given position.
[0059] The methods described above can be applied not only to non-physical volume rendering processes, such as non-physical ray tracing, such as ray casting, but also to physically based volume rendering processes, such as path tracing.
[0060] Figure 3 The diagram schematically illustrates a path tracing process that can be used in volumetric rendering. Similar to ray tracing, the path tracing process determines one or more visual parameter values for a given location 310 in the image to determine the image display value for that location. The given visual parameter value is generated based on a combination of multiple sets of volumetric data. (Refer to the above...) Figure 2 As described, a combination of multiple sets of volumetric data can be used at the same points during path tracing (e.g., using different sampling distances for different sets of volumetric data; determining which sets of volumetric data contribute to the sample values based on segmentation data; using different interpolation methods for different sets of volumetric data; calculating average geometric or surface optical properties). During path tracing, the combination of multiple sets of volumetric data can be used in other additional ways, further utilizing the information content of the volumetric dataset 206.
[0061] exist Figure 3 In the path tracing process shown, path 306 is traced through a volume containing a set of multiple volume data. Unlike non-physical ray tracing, path tracing more closely follows the physical behavior of light by simulating various scattering events that can occur when light interacts with different materials. Path 306 may include multiple path segments generated by these scattering events.
[0062] Scattering event 302 represents a point where the path direction changes due to the interaction of simulated light rays with a volume. When a scattering event occurs, the path can continue in the new direction based on the properties of the volume at that location. The path tracing process determines when these scattering events are triggered based on the cumulative opacity value along the path.
[0063] For the first set of volume data, the path tracing process determines a first cumulative opacity value for a path segment along path 306. This cumulative opacity value can be considered as representing how much light has been attenuated in that volume as it travels along the path segment. When this first cumulative opacity value exceeds a first threshold corresponding to the first set of volume data, a scattering event 302 can be triggered.
[0064] Similarly, for the second set of volumetric data, the path tracing process determines a second cumulative opacity value along path 306. When this second cumulative opacity value exceeds a different second threshold corresponding to the second set of volumetric data, another scattering event can be triggered. Using independent cumulative opacity values and / or thresholds for different sets of volumetric data allows for finer control over light scattering events, thus enabling more accurate visualization of the unique properties of each set of volumetric data. For example, if the first imaging process is known to be associated with a short scattering length (e.g., in a specific segment), a lower threshold can be used when triggering a scattering event.
[0065] While the thresholds are equal in some examples, in others, the first threshold is a first random value generated from a first distribution, and the second threshold is a second random value generated from a different second distribution. Using random thresholds produces a more accurate model of the volumetric scattering process. By using random thresholds generated from different distributions on different sets of volume data, the properties of light transport in different imaging processes can be modeled more accurately.
[0066] Volumetric absorption can be modeled, for example, by using a maximum threshold of the number of scattering events or probabilistically by using an extinction probability density function based on the number of scattering events. In one example, the Russian Roulette algorithm can manage the extinction of path 306 by pseudo-randomly determining whether to continue or terminate path 306 at each scattering location.
[0067] In some examples, a scattering event can be triggered only when both a first cumulative opacity value exceeds a first threshold and a second cumulative opacity value exceeds a second threshold. For computational efficiency, it is preferable to identify the path of the guiding light source rather than the extinction event; for this reason, when tracing a given path segment, it is preferable to identify a set of volume data in which the scattering event is unlikely to occur. Therefore, when determining when to trigger a scattering event, it is expected that for a set of multiple or all volume data, the event will only be triggered if a threshold is exceeded.
[0068] Interface scattering event 304 represents a special type of scattering that occurs at the boundary between different materials or structures within a volume. These interface scattering events are handled differently from volume scattering events that occur within a uniform region of the volume. Interface scattering event 304 can be triggered, for example, by determining that a segment of the sampling location (for any one of the volume data sets) differs from a segment of a previously sampled location (e.g., for the same volume data set).
[0069] When a scattering event is triggered, the direction of the new ray originating from the scattering location is determined. For volume scattering event 302, the direction of the new ray is determined, for example, using a phase function (probability density function) representing the scattering probability of a given scattering angle. For surface scattering event 304, the direction of the new ray is determined, for example, using a surface scattering distribution function (e.g., a bidirectional reflection distribution function, BRDF).
[0070] The direction of the new ray can be determined based on the volume values of a set of volume data at the scattering location. For example, for volume scattering event 302, a phase function can be selected from multiple phase functions corresponding to the set of volume data. For surface scattering event 304, a surface scattering distribution function can be selected from multiple phase functions corresponding to multiple sets of volume data. Continuing with the muscle-skeleton interface example, a surface scattering distribution function (e.g., BRDF) known to show bright reflections from the bone in an imaging process can be selected as the surface scattering distribution function (e.g., BRDF) for determining the direction of the new ray. Additionally or alternatively, the parameters of the phase function / surface scattering distribution function (such as the geometric or surface optical properties described above) can be determined based on the set of volume data. The parameters can be determined simply based on which set(s) of volume data will contribute to the sample value, or in a manner similar to that described above for the reference ray tracing process, based on the volume data at the sampling location. For example, the surface normal of the surface scattering distribution function (e.g., BRDF) can be determined by averaging the surface normals calculated from the set of volume data that contribute.
[0071] and Figure 2 Similar to the non-physical ray tracing process shown, path 306 includes multiple sampling locations, which may differ for different sets of volume data. In some examples, the path tracing process includes determining a set of contributing volume data from a set of multiple volume data for a given sampling location on the path segment. Then, one or more sets of contributing volume data are used to determine the volume values to be input into the lighting model.
[0072] The set of one or more contributing volume data can be determined based on the type of light rays in the path segment. The type of light ray can be, for example, a primary ray, a secondary ray, or a shadow ray 308. A primary ray originates at viewpoint 212. A secondary ray is any ray along the simulated path other than a primary ray. A shadow ray 308 is a ray selected to extend between the light source (e.g., a second light source 310) and the sampling location.
[0073] For example, one or more sets of contributing volume data can be determined based on the precision of the set of volume data (e.g., contrast, or alternatively, the number of bits used to represent each volume value). For instance, if the path segment is a primary ray, the set of one or more contributing volume data can be selected as the set of volume data associated with high precision (e.g., high contrast). If the path segment is a secondary ray or shadow ray 308, the set of volume data associated with low precision (e.g., low number of bits used to represent each volume value) can be selected. This achieves a good balance between rendering accuracy and computational efficiency, since primary rays are significant contributors to the final opacity / color value of a given path.
[0074] Alternatively, the set of one or more contributing volumetric data can be determined based on whether the path segment was generated by volumetric scattering event 302 or interface scattering event 304. For example, if the scattering event is an interface scattering event occurring at, for example, a muscle-bone interface, then the set of volumetric data known to show a bright reflection from the bone can be determined as the set of contributing volumetric data.
[0075] Alternatively, or alternatively, a set of one or more contributing volume data points can be determined based on the distance between the sampling location and the location of the scattering event that created the path segment, or the number of path segments traversed by the path up to the path segment. For example, for a given set of volume data points, this distance can be compared to the volume scattering path length (i.e., a value representing how far light typically travels within a volume), and if the distance is shorter than the path length, it can be determined that the set of volume data points contributes to the volume value, and if the distance is longer than the path length, it can be determined that the set of volume data points does not contribute to the volume value. This allows for the correct selection of the set of volume data points during shading.
[0076] Alternatively or additionally, the set of one or more contributing volumetric data can be determined based on a user-defined configuration of the program implementing the volumetric rendering process, or a configuration preset within the program. This configuration can consider any of a range of factors to determine the set of one or more contributing volumetric data. For example, the configuration can use a body region (e.g., heart, lung) represented by a sampling location. Alternatively or additionally, the configuration can consider at least one imaging process used to generate the set of volumetric data. For example, the configuration can select a set of one or more volumetric data obtained through a specific imaging process or using a specific imaging device, for example, based on a body region represented by a sampling location. Alternatively or additionally, the configuration can consider a process used to interpret the image (e.g., a clinical reading process). For example, if the process used to interpret the image is a process using those body regions (e.g., for diagnosis), the set of one or more contributing volumetric data can be selected as a set of volumetric data known to have good contrast for a particular body region / boundary between body regions.
[0077] Path 306 can eventually leave the volume. In addition to the shading at the sampling location, further shading steps can be performed, where the light source function used for leaving the path determines the color and / or opacity values. According to certain examples of this disclosure, the light source function is determined based on the set of volumetric data used during the accumulation of sample values along the path. Different imaging processes can be associated with different light source functions; for example, multispectral CT can be modeled using ambient light sources with uniform intensity across all wavelengths, and point light sources with varying intensities for different wavelengths. Determining the light source function based on the set of volumetric data used during path traversal allows for more accurate and / or more suitable modeling of light transmission in a way that conveys visual information to the user. The light source function can be determined as the light source function used for the imaging process, whose associated set of volumetric data is most commonly used during the accumulation of opacity values. Alternatively, a light source function used for the imaging process can be used, whose associated set of volumetric data is used during the accumulation of primary or exit rays. Additionally or alternatively, the light source function can be determined based on whether the initial scattering event along the path is a volumetric scattering event or a surface scattering event. For example, when it is desirable to convey information about the first interaction between light and volume, it may be useful to use a light source that is used during the accumulation of light rays from the set of volume data associated with it, while when it is desirable to convey information about a portion of the volume facing viewpoint 212 (e.g., a surface), it may be useful to use a primary ray.
[0078] Therefore, this method achieves enhanced visualization of volume dataset 206 by generating visual parameter values based on the combination of multiple sets of volume data. When different imaging processes capture the same volume, each process provides unique information based on its specific imaging characteristics. By combining these different datasets during the volume rendering process, rather than using only a single reconstruction result, the visualization retains and integrates information that would otherwise be lost, resulting in a more comprehensive and information-rich rendering.
[0079] By directly incorporating multidimensional, multispectral data during the rendering process, this method can provide an improved ability to display volumetric data to users, for example, to aid in medical diagnosis or research. Instead of simply blending the fully rendered image at the pixel level, this method integrates low-level volumetric data from different imaging processes during rendering computation.
[0080] For example, rendering techniques can allow radiologists to gain new insights by directly incorporating the multidimensionality of multispectral data into the rendering process. Instead of simply mixing the individual pixel values of a fully rendered image, this approach works with underlying volumetric data to create a more integrated representation. This allows medical professionals to better visualize complex relationships between different tissue types or structures that might only become apparent when considering multiple imaging characteristics simultaneously.
[0081] Figure 4 It shows the method for execution Figure 1 An example device for the method shown. Device 408 includes a processor 402 and a storage device 404. Storage device 404 may include a non-transitory computer-readable medium storing instructions that, when executed by processor 402, cause processor 402 to perform reference... Figure 1 The method described. Storage device 404 may also store a volume dataset 206 that includes a collection of multiple volume data.
[0082] Processor 402 can be any suitable processing device, such as a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or any other suitable processing device. In some examples, processor 402 may include multiple processing cores or multiple discrete processing devices that work together to perform the processing operations described herein.
[0083] Storage device 404 may include any suitable storage medium, such as random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive, solid-state drive, or any other suitable storage medium. Storage device 404 may be directly coupled to processor 402, or it may be coupled via a data bus or network connection.
[0084] Device 408 can generate image 406 based on volume dataset 206. Image 406 can be displayed on a display device (not shown) coupled to device 408. (Refer to reference...) Figure 1 The volumetric rendering process described (and optionally using references) Figure 2 The described ray tracing process and / or reference Figure 3 The described path tracing process is used to render image 406, where visual parameter values are generated based on a combination of sets of multiple volume data in volume dataset 206.
[0085] Processor 402 can execute instructions to perform operations, including selecting different sets of volume data from volume dataset 206 for different operations in the rendering algorithm, as shown in reference. Figure 1 As described. Then the processor 402 can generate visual parameter values based on the selected set of volume data, and use these values to determine the image display values of image 406.
[0086] The above embodiments should be understood as illustrative examples of the invention. Other embodiments are contemplated. For example, although multiple imaging processes are performed simultaneously in the above examples, the multiple imaging processes can be performed at different times, wherein the sets of volumetric data are closely registered with each other and suitable for combination.
[0087] It should be understood that any feature described with respect to any embodiment may be used alone or in combination with other described features, and may also be used in combination with one or more features of any other embodiment, or in any combination of any other embodiment. Furthermore, equivalents and modifications not described above may be employed without departing from the scope of the invention as defined by the appended claims.
Claims
1. A computer-implemented method for rendering a volume dataset (206) representing volumes, the volume dataset comprising a set of multiple volume data obtained by performing corresponding multiple imaging processes on the volumes (102), wherein, Each of the multiple imaging processes has its own set of different imaging characteristics, and the method includes: A volume rendering process (104) is performed using the set of the plurality of volume data to render an image of the volume (406), the volume rendering process including: For a given location in an image, generate one or more visual parameter values to determine the image display value for that location; The given visual parameter value among the one or more visual parameter values is generated based on a combination of the sets of the multiple volume data.
2. The method according to claim 1, wherein, The first set of the imaging characteristics includes a first set of wavelengths of electromagnetic radiation, and the second set of the imaging characteristics includes a second set of different wavelengths of electromagnetic radiation.
3. The method according to any one of the preceding claims, wherein, Generating the one or more visual parameter values includes performing a ray tracing process, wherein performing the ray tracing process includes, for the ray (208, 210) traced during the ray tracing process: A sampling process is performed at multiple sampling points (214) along the light ray, the sampling process comprising: generating a set of corresponding sample values for the multiple sampling points based on at least one set of the set of volume data; and The corresponding sample values are used to generate the visual parameter values among the one or more visual parameter values.
4. The method according to claim 3, wherein, Performing the ray tracing process includes: For a first ray (208) tracked during the ray tracing process, a first sampling process is performed at a first plurality of sampling points separated by a first sampling distance along the first ray, the first sampling process comprising: generating a first set of corresponding sample values for the first plurality of sampling points based on a first set of the set of volume data; and For the second ray (210) tracked during the ray tracing process, a second sampling process is performed at a second plurality of sampling points separated by a second sampling distance along the second ray. The second sampling process includes: generating a second set of corresponding sample values for the second plurality of sampling points based on a second set of the set of volume data. The method includes: The first sampling distance is determined based on the set of imaging characteristics of the first set of the volume data set; and / or The second sampling distance is determined based on the set of imaging characteristics of the second set of the set of volume data.
5. The method according to any one of the preceding claims, comprising: Using a set of the volume data or the first set, identify the approximate location of the boundary between the two media in the volume; as well as The refined location of the boundary is identified using a set of the volume data or the second set.
6. The method according to any one of the preceding claims, wherein, Generating a given visual parameter value includes: for a given sampling location in the volume dataset from which the given visual parameter value is calculated in at least part, determining whether a given set of the set of volume data should be used when determining the sample value for the given sampling location, wherein the determination is based on segmentation data indicating the segment associated with the sampling location.
7. The method according to claim 6, wherein, Determining whether to use a given set of volume data when determining sample values includes: in response to determining that a given sampling location and the surface of a segment associated with the given sampling location satisfy a proximity condition, determining that the given set of volume data does not contribute to the sample values.
8. The method according to any one of the preceding claims, comprising: An interpolation method is determined based on a set of image characteristics from one or more sets of the volume data set. The volume value at the sampling location is interpolated from a set of the volume data or the first set using an interpolation method; as well as The volume value is used when generating the one or more visual parameter values.
9. The method according to claim 8, wherein, The interpolation method is used for a set in the set of volume data or the first set, and the method includes: Based on a set of image characteristics from the set of volume data or the second set, an additional interpolation method is determined, and a sampling location or an additional volume value at the sampling location is interpolated from the set of volume data; and The additional volume value is used when generating the one or more visual parameter values.
10. The method according to any one of the preceding claims, wherein, The volume rendering process includes a path tracing process.
11. The method according to claim 10, wherein, The path tracing process includes: A first cumulative opacity value along the path is determined using a set of the volume data or the first set, and the first cumulative opacity value is compared with a first threshold corresponding to the set of volume data; The scattering event (302) is triggered at least in part based on the determination that the first cumulative opacity value exceeds the first threshold. A second cumulative opacity value is determined along the path using a set from the set of volume data or the second set, and the second cumulative opacity value is compared with a different second threshold corresponding to the set of volume data; and The scattering event is triggered at least in part based on the determination that the second cumulative opacity value exceeds the second threshold.
12. The method according to claim 11, wherein, The first cumulative opacity value and the second cumulative opacity value are for the same path, and the scattering event is triggered only when both the first cumulative opacity value exceeds the first threshold and the second cumulative opacity value exceeds the second threshold.
13. The method according to any one of claims 10 to 12, wherein, The path tracing process includes: for sampling locations on path segments of the path traced during the path tracing process, determining a set of contributing volume data from the set of multiple volume data based on at least one of the following: The light type of the path segment, wherein the light type is primary light, secondary light or shadow light (308). If the ray type of the path segment is a secondary ray, then the path segment is generated by either a volume scattering event (302) or an interface scattering event (304); The distance between the sampling location and the location of the scattering event that generated the path segment; The path extends up to the number of path segments traversed by the path segment; Input received via the user interface; The body region represented by the sampling location; At least one of the imaging processes; At least one imaging device used in the imaging process; or The process used to interpret images.
14. The method according to any one of claims 10 to 13, wherein, The path tracing process includes triggering a scattering event at the scattering location, and triggering the scattering event includes: determining the direction of a new ray to be used in the path tracing process based on the volume value of the set of volume data at the scattering location, the new ray originating from the scattering location.
15. An apparatus (408) including a processor (402) and a storage device (404), the storage device (404) including a machine-readable instruction set that, when executed by the processor, causes the processor (402) to perform the method according to any one of the preceding claims.
