Spacetime self-guided shadow denoising in ray tracing applications

Abstract

In an example, a filter used to denoise shadows for a pixel or pixels can be adapted based on at least a variance in time-accumulated ray-tracing samples. The variance can be used to define a range of filter values for a spatio-temporal filter and to exclude time ray-tracing samples outside of the range. Data used to compute a first moment of a distribution can be used to compute a second moment of the distribution used to compute the variance. For a binary signal, such as visibility, the first moment (e.g., an accumulated average) can be equivalent to the second moment (e.g., a mean square). In a further aspect, spatial filtering of a pixel or pixels can be skipped based on comparing a mean of a variance of the pixel or pixels to one or more thresholds and based on an accumulated number of values for the pixel.

Description

BACKGROUND

[0001] Ray tracing is a method for rendering images by tracing the path of light in a virtual environment and simulating the interaction of light with virtual objects. Ray tracing techniques can be used to simulate various optical effects, such as shadows, reflections and refractions, scattering phenomena, and dispersion phenomena (e.g., chromatic aberration). When rendering soft shadows using ray tracing, a traditional shadow tracing method can cast an arbitrary number of shadow rays from a certain location in the virtual environment to sample the lighting conditions of a pixel with respect to a light source. The ray tracing samples can be combined and applied to the pixel. In a half-shadow (a shadowed area where light is partially occluded), some shadow rays can be visible to the light source, while other shadow rays can be occluded. A large number of shadow rays can be needed in order for the combined lighting conditions to converge to an accurate result. To save computational resources and reduce rendering time, shadow rays can be sparsely sampled, resulting in noisy rendering data. Denoising techniques can be used to filter the noisy rendering data to reduce noise and produce a final rendering that is closer to a fully sampled rendering of the scene.

[0002] Temporal-spatial filters can be used for denoising to combine spatial information from a current frame with temporal accumulated information to increase the effective sample count by leveraging information from previous frames. To avoid over-blurring, ray hit distances can be used to guide a traditional shadow denoising filter, where the filter radius is adjusted to adapt based on the distance to the occluder of the light source. This approach requires bandwidth for storing hit distances and denoising the hit distance data to account for its randomness. Variance of color luminance has been used to guide a temporal-spatial filter that denoises color luminance in global illumination. While global illumination can provide shadows for the rendering of an environment as part of accounting for direct and indirect lighting, traditional techniques are typically much more expensive than shadow tracing. SUMMARY

[0003] Embodiments of the present disclosure relate to temporal-spatial self-guided shadow denoising. The present disclosure provides methods for adapting a denoising filter (e.g., a shadow denoising filter) based at least on variance in temporally accumulated ray tracing samples.

[0004] In contrast to conventional methods for ray-traced shadow denoising, the disclosed methods can adapt a filter for pixel(s) shadow denoising based on at least variance in time-accumulated ray-traced samples. In various examples, a range of filter values for a spatio-temporal filter can be defined based on the variance and used to exclude a set of time ray-traced samples outside the range. In further aspects, data used to compute a first moment of a distribution used to compute the variance can be used to compute a second moment of the distribution. For binary signals, such as visibility, a first moment (e.g., accumulated average) can be equivalent to a second moment (e.g., mean square). Thus, computational resources can be saved by accumulating data used to derive only one of the moments. In embodiments where the moments are not equivalent, a first moment can still be sufficient to define an approximation of a second moment. In further aspects, spatial and / or temporal filtering of pixel(s) can be skipped based on at least one or more of a mean value and / or a first moment of a distribution used to compute a variance of the pixel(s), the one or more of the mean value and / or the first moment being less than or equal to a first threshold value, or greater than or equal to a second threshold value, and an accumulated value or a number of samples of the pixel being greater than or equal to a third threshold value. BRIEF DESCRIPTION OF DRAWINGS

[0005] Systems and methods for adaptive ray tracing for shadow rendering are described in detail below in connection with the appended drawings, of which:

[0006] Figure 1 is a dataflow diagram illustrating an example process for generating an output image using an image rendering system, in accordance with some embodiments of the present disclosure;

[0007] Figure 2 is a diagram illustrating an example of capturing ray-traced samples of a virtual environment, in accordance with some embodiments of the present disclosure;

[0008] Figure 3 is a diagram illustrating an example of a distribution function that can be used to determine a range of filter values for a filter, in accordance with some embodiments of the present disclosure;

[0009] Figure 4 is a diagram illustrating an example of a frame of a virtual environment that can be generated using shadow denoising, in accordance with some embodiments of the present disclosure;

[0010] Figure 5 is a flow diagram illustrating an example of a method for defining a range of filter values based on at least a mean value of visibility values, in accordance with some embodiments of the present disclosure;

[0011] Figure 6is a flowchart illustrating an example of a method for defining a second moment of a distribution using a first moment of the distribution for computing variance in filtered ray-tracing samples, in accordance with some embodiments of the present disclosure;

[0012] Figure 7 is a flowchart illustrating an example of a method for applying data corresponding to an accumulated first moment of a distribution to a second moment of the distribution in filtered ray-tracing samples corresponding to a binary signal, in accordance with some embodiments of the present disclosure;

[0013] Figure 8 is a block diagram of an example computing environment suitable for implementing some embodiments of the present disclosure; and

[0014] Figure 9 is a block diagram of an example data center suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0015] The present disclosure relates to spatio-temporal self-guided shadow denoising. The present disclosure provides methods for adapting a denoising filter, such as a shadow denoising filter, based at least on variance in temporally accumulated ray-tracing samples.

[0016] In contrast to conventional methods for ray-tracing shadow denoising, the disclosed methods adapt a filter for shadow denoising of a pixel based at least on variance in temporally accumulated ray-tracing samples. Using the disclosed methods, shadow denoising can be performed without the need to use ray hit distance to guide the filter, avoiding bandwidth or other more expensive methods for storing and denoising hit distance data.

[0017] In various examples, a range of filter values for a spatio-temporal filter can be defined based on variance. When applying the spatio-temporal filter, a set of temporal ray-tracing samples can be excluded from filtering based on the set being outside the range. The range can be at a maximum value when the mean is between the maximum and minimum values (e.g., between 0 and 1), the maximum value of the mean (e.g., 0.5) corresponding to an area that can be in a penumbra of a shadow. Further, the range can be at a minimum value (e.g., 0) when the mean is at or near the maximum or minimum values, corresponding to an area that can be fully illuminated by a light source or can be fully in a shadow.

[0018] In further aspects, the present disclosure provides methods of computing variance based on accumulated samples of a first moment and a second moment, the variance used to guide filtering of temporal ray-tracing samples (e.g., visibility). The first moment can include an average of values of the temporal ray-tracing samples, and the second moment can include an average of squares of the values. For binary signals, such as visibility, the average can be accumulated over multiple frames, the accumulated average equal to a mean square. Thus, memory bandwidth can be saved by accumulating data for only one of the moments used to derive the other, which can then be defined using that data. For non-binary signals, while not equivalent, accumulating the average is still sufficient to define an approximation of the second moment.

[0019] According to further aspects of the present disclosure, spatial and / or temporal filtering of the pixel(s) can be skipped based at least on one or more of the average of the distribution and / or the first moment being less than or equal to a first threshold, or greater than or equal to a second threshold. For example, using a binary signal of visibility, spatial and temporal filtering can be skipped where the average is 0 and / or the average is 1 and at least two values and / or frames have been accumulated. An average of 0 can indicate that the pixel is completely occluded, while an average of 1 can indicate that the pixel is completely illuminated. Additional factors for determining whether to skip spatial and / or temporal filtering of the pixel can be based on the accumulated number of values (and / or frames) of the pixel being greater than or equal to a third threshold, such as 8 frames. Any of the various types of ray-tracing samples can be adjusted for different embodiments and / or different types of ray-tracing samples.

[0020] Reference Figure 1 , Figure 1 is a data flow diagram illustrating an example process 140 for generating an output image using an image rendering system 100 according to some embodiments of the present disclosure. It should be understood that this and other arrangements described herein are set forth only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements can be wholly omitted depending on the context. Further, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination, arrangement, or location. The various functions described herein as being performed by entities can be performed by hardware, firmware, and / or software. For instance, various functions can be performed by a processor executing instructions stored in memory. In at least one embodiment, the image rendering system 100 can be implemented at least in part in a cloud computing environment. Figure 9implemented in a data center 900. As various examples, the image rendering system 100 can be included in or be one or more of: a system for performing simulation operations, a system for performing simulation operations to test or validate autonomous machine applications, a system for performing deep learning operations, a system implemented using edge devices, a system incorporating one or more virtual machines (VMs), a system implemented at least in part in a data center, or a system implemented at least in part using cloud computing resources.

[0021] The image rendering system 100 can be configured to render images of a virtual environment, such as the virtual environment 200 of Figure 2 To render images of a virtual environment, the image rendering system 100 can employ a ray tracer 102, a moment history manager 104, a sample history manager 106, a temporal moment accumulator 108, a temporal sample accumulator 110, a variance determiner 112, an image filter 114, an image combiner 116, and a 3D renderer 118.

[0022] The ray tracer 102 can be configured to trace rays through the virtual environment using any of a variety of potential ray tracing techniques in order to generate ray tracing samples of one or more aspects of the virtual environment (e.g., lighting conditions) relative to locations in the virtual environment. The temporal moment accumulator 108 can be configured to accumulate distributed spatial moments that are used to compute variance in the ray tracing samples from the ray tracer 102, data representing temporal accumulations of moments from the moment history manager 104 (e.g., a single value representing an average of these values) to update the data representing temporal accumulations of moments. The moment history manager 104 can be configured to manage data representing temporal accumulations of moments (e.g., for each pixel), such as in a moment history buffer. The variance determiner 112 can use the data representing temporal accumulations of moments for particular pixels to compute variance for one or more particular pixels, and can provide the variance to the image filter 114 for spatio-temporal filtering of data corresponding to spatial ray tracing samples from the ray tracer 102.

[0023] The time sample accumulator 110 can be configured to accumulate the ray tracing samples from the ray tracer 102 with data representing time-accumulated values of ray tracing samples from the sample history manager 106 (e.g., a single value representing can be an average value) to provide data representing time-accumulated values of ray tracing samples to the image filter 114 for spatio-temporal filtering of data corresponding to spatial ray tracing samples from the ray tracer 102. The sample history manager 106 can be configured to manage data representing time-accumulated values of ray tracing samples (e.g., for each pixel), e.g., in a sample history buffer. In the illustrated example, the sample history manager 106 can update the sample history buffer using the spatio-temporally filtered ray tracing samples.

[0024] In some embodiments, the image combiner 116 can combine data corresponding to the spatio-temporally filtered ray tracing samples (e.g., filtered shadowing / visibility data) with data representing a 3D rendering of the virtual environment (e.g., shadowing data without ray tracing) to generate the output image 120. The 3D renderer 118 can be configured to generate the 3D rendering using any suitable method, which can or can not include ray tracing. In embodiments, the 3D rendering can include pixel color information for a frame of the virtual environment.

[0025] As described herein, the ray tracer 102 can be configured to trace rays through the virtual environment using any of a variety of potential ray tracing techniques in order to generate ray tracing samples relating to one or more aspects of the virtual environment with respect to locations in the virtual environment. Reference is now made to Figure 2 , Figure 2 is a diagram illustrating an example of capturing ray tracing samples for a virtual environment 200, in accordance with some embodiments of the present disclosure. The image rendering system 100 can be configured to use the ray tracer 102 to render an image using any number of ray tracing passes in order to sample conditions of the virtual environment. Figure 2 The examples are described with respect to visibility samples, and more specifically, visibility samples for one or more pixels of a light source in the virtual environment. However, the disclosed methods can be implemented with other types of ray tracing samples, which can include those that form a binary signal (e.g., with values of 0 or 1). In some embodiments, the temporal ray tracing samples can represent, indicate, or otherwise correspond to environmental occlusions with respect to one or more pixels of the virtual environment. The ray tracing techniques can be adapted to accommodate the effect(s) being simulated when sampling different aspects of the virtual environment.

[0026] With respect to one or more ray tracing passes 214, various examples of rays that can be traced by the ray tracer 102 through the virtual environment 200 are illustrated in Figure 2The rays are used to jointly sample one or more aspects of the virtual environment with respect to the locations in the virtual environment 200. An example of nine locations is shown (possibly more, e.g., one per pixel), with locations 230, 232, and 234 individually labeled. In at least one embodiment, each ray is associated with (e.g., cast from) one of the locations and used to generate a ray-traced sample for that location. For example, ray 240 is associated with location 232, ray 242 is associated with location 230, and ray 244 is associated with location 234.

[0027] In some embodiments, each location at which a ray is cast by the ray tracer 102 corresponds to a respective pixel. For example, the locations (e.g., locations 230, 232, and 234) can be determined by transforming a virtual screen of pixels (e.g., from a z-buffer) to world space. The virtual screen can represent a view of a camera in the virtual environment 200, and in some embodiments, these locations can be referred to as pixels or world-space pixels. In other examples, the locations can not have such a one-to-one correspondence with pixels. Further, in other examples, the locations can be determined as respective eye rays (e.g., cast from a camera through a virtual screen including the pixel) intersecting the virtual environment 200 at respective points and / or areas.

[0028] In various embodiments, the accuracy of the samples at the locations can be limited, as each ray can only provide partial information for the location. Thus, sampling the virtual environment 200 using a limited number of rays can tend to cause noise in the image, particularly for certain locations in the virtual environment 200. To illustrate the foregoing example, the rays used in the illustrated example are shadow rays used to sample one or more aspects of lighting conditions at the location of the light source 220 in the virtual environment 200. The image rendering system 100 can use this information, e.g., to render shadows in the image based on the lighting conditions at the locations. In some embodiments, the rays are cast from the locations to sample random or pseudo-random locations at the light source 220. The ray tracer 102 can use any suitable method for ray tracing, e.g., stochastic ray tracing. Examples of stochastic ray tracing techniques that can be used include those that employ Monte Carlo or quasi-Monte Carlo sampling strategies. In the illustrated example, the ray tracer 102 casts one ray for sampling at each location and / or pixel in the ray tracing pass(es) 214. In other embodiments, a different number of rays can be cast for each location or pixel, no rays can be cast for certain locations or pixels, and / or a different amount of rays can be cast for different locations or pixels.

[0029] Although only light source 220 is shown, lighting conditions at locations can similarly be sampled with respect to other light sources and / or objects in virtual environment 200, which can be combined with the set of ray-tracing samples derived with respect to light source 220. In some examples, lighting conditions for different light sources can be determined and filtered separately (e.g., using the filtering techniques described with respect to Figure 1 image combiner 116 (e.g., as another input to image combiner 116). Moreover, in the present example, no additional rays are cast from a location in virtual environment 200 when a ray interacts with that location (e.g., at light source 220 or occluder 222). However, for other ray-tracing effects or techniques, one or more additional rays can be cast therefrom.

[0030] As shown, some rays (e.g., ray 244) can interact with light source 220, causing the ray-tracing samples to indicate that light from light source 220 can illuminate the corresponding locations. In some embodiments, rays falling into this category can be assigned a visibility value of 1 to indicate that they are visible with respect to light source 220. Other rays, such as ray 240 and ray 242, can interact with objects, causing the ray-tracing samples to indicate that light from light source 220 is at least partially blocked and / or prevented from reaching these locations. One example of such an object is occluder 222, which can block light rays from reaching light source 220. In some embodiments, rays falling into this category can be assigned a visibility value of 0 to indicate that they are not visible with respect to light source 220. Since the visibility value can assume one of two potential values, it can correspond to a binary signal.

[0031] Location 230 is an example of a location that can be within the penumbra of a shadow that can be cast by occluder 222, and the lighting conditions can be more accurately computed by combining ray-tracing samples derived from multiple rays. For example, the ray-tracing samples for location 230 generated using only ray 242 can indicate that location 230 is completely blocked from receiving light from light source 220. However, the ray-tracing samples for location 230 generated using another ray can indicate that location 230 is at least partially illuminated by light source 220.

[0032] Accordingly, limiting the number of rays used to generate location samples can result in noise, which can result in visual artifacts in the data rendered by image rendering system 100. Image filter 114 can be used to implement a denoising technique to reduce the noise. In various examples, the denoising technique can include image filter 114 filtering the lighting condition data corresponding to the ray-tracing samples from ray-tracer 102 in space and / or time. For example, image filter 114 can apply one or more spatial filter pass and / or temporal filter pass to the lighting condition data from ray-tracer 102.

[0033] Temporal filtering of lighting condition data can utilize ray-traced samples, which can be similar to reference samples. Figure 2 The described samples are generated, but used for previous states and / or previous output frames of the virtual environment 200. Therefore, temporal filtering can increase the effective sample count of ray-traced samples used to determine the filtered lighting condition data of a pixel and / or increase the temporal stability of the filtered lighting condition data of a pixel. However, since temporal ray-traced samples may correspond to different states of the virtual environment 200, some samples may be irrelevant to or related to the current state of the virtual environment 200 (e.g., objects or cameras may move, light sources may change), posing a risk of visual artifacts when they are used for filtering. The disclosed embodiments can use variations in values ​​corresponding to the temporal ray-traced samples to guide temporal filtering in order to reduce or eliminate these potential artifacts.

[0034] According to the disclosed embodiments, the variance (e.g., per pixel) of the values ​​corresponding to the temporal ray-traced samples can be determined by variance determiner 112 using the calculation of the variance of a plurality of moments comprising a distribution. Variance is a measure of how far a set of values ​​is dispersed (e.g., from a mean or other reference value). The moments of the distribution used to calculate the variance are quantitative measures of the shape of a function (e.g., a variance function). The combined moments define the variance of the values. The variance Var in a set of values ​​i can be determined at least based on equation (1):

[0035] Var(i) = μ1 i –μ2 i 2 (1)

[0036] Where μ1 is the first moment and μ2 is the second moment. The first moment μ1 corresponds to the average value of the values, and the second moment μ2 corresponds to the average value of the squares of the values.

[0037] In some cases, the first moment can be used to calculate the second moment. For example, for binary signals, such as visibility, the average might equal the mean square. Specifically, as mentioned above, the second moment can be the average of the squares of the values. A binary signal may have values ​​of 0 or 1, and the value remains unchanged when the square of the value is taken. To illustrate the above, assume a series of visibility values: 1, 0, 1, 1, 0 (e.g., pixels over 5 frames). The average of the values ​​(e.g., the first moment) is (1 + 0 + 1 + 1 + 0) / 5 = "0.6". The average of the squares of the values ​​(e.g., the second moment) is (1 + 0 + 1 + 1 + 0) / 5 = "0.6". 2 +0 2 +1 2 +1 2 +0 2 ) / 5 = "0.6". Therefore, when calculating variance, the first moment can be used as the second moment. Therefore, the variance can be calculated using equation (2):

[0038] Var(i) = μ1 i - μ1 i 2 (2)

[0039] In embodiments where the values correspond to non-binary signals, similar approaches can be used. For example, while not equivalent, the first moments can be close enough to the second moments to be used as the second moments and / or the second moments can be otherwise derived or approximated from the first moments. In some examples, the first moments can be adjusted using factors or formulas to derive the second moments. As a further example, when calculating the first moments, the values can be rounded, binned, or otherwise modified, and the first moments can be used to derive the second moments. For example, if the values are between 0 and 1 (or normalized to fall within that range), the values can be rounded to the nearest value (0 or 1) for calculating the first moments.

[0040] As described herein, the temporal moment accumulator 108 can calculate and accumulate (using the moment history manager 104) the first moments (e.g., the mean values) over multiple frames. Since the first moments can be used to derive the second moments, bandwidth can be saved by accumulating data for only one of the moments used to derive the other, which can then be defined using that data.

[0041] For temporal filtering, the temporal moment accumulator 108 can accumulate moments of the variance in the ray-traced lighting condition data for the pixels from any number of previous states and / or frames of the virtual environment. The moment history manager 104 can store the accumulated moments for temporal filtering in a moment history buffer, which can be updated for each rendered state and / or frame using the update 162. For example, in an iteration of the process 140, the temporal moment accumulator 108 can provide data capturing the temporal accumulated values of the moments to the variance determiner 112, which can compute the variance values for the pixels based at least on that data. In some examples, the temporal moment accumulator 108 can merge the spatial values of the moments of the distribution of the pixels with the temporal accumulated values of the corresponding moments, which can be captured by the data provided to the variance determiner 112. In any examples, the update 162 can replace the currently stored temporal accumulated values with the updated temporal accumulated values for use in subsequent iterations (e.g., states or frames).

[0042] Similarly for temporal filtering, the temporal sample accumulator 110 can accumulate samples of ray-traced lighting condition data for any number of previous states and / or frames of pixels from the virtual environment. The sample history manager 106 can store the accumulated samples for temporal filtering in a sample history buffer, which can be updated for each rendered state and / or frame using updates 164. For example, in an iteration of the process 140, the temporal sample accumulator 110 can provide data capturing the temporal accumulated values of the ray-traced samples to the image filter 114 for one or more filter passes. In some examples, the temporal moment accumulator 108 can merge the spatial values of the samples with the corresponding temporal accumulated samples, which can be captured by the data provided to the image filter 114. In any examples, the updates 164 can replace the currently stored temporal accumulated values with updated temporal accumulated values for subsequent iterations (e.g., states or frames).

[0043] The process 140 illustrates and describes certain methods of updating 162 and 164 with respect to temporal accumulation. However, in various embodiments, any of the updates of temporal accumulation can be implemented using pre-accumulation, post-accumulation, and / or recursive temporal accumulation (e.g., iteratively accumulating frame noise rendering data with temporal accumulated data, then blurring the accumulated data to generate an output that is fed back to update the temporal accumulated data for a subsequent iteration).

[0044] In determining the samples and / or moments of temporal accumulation, the number of accumulated samples and / or moments can be capped, for example, with an upper limit number (e.g., anywhere from eight to sixteen samples and / or moments). As an example, for each pixel, the temporal sample accumulator 110 can accumulate up to eight ray-traced samples from eight previous frames, thereby increasing the potential effective sample count by a factor of eight. Thus, in the case where each rendered frame corresponds to a single sample per pixel, the temporal sample accumulator 110 can increase the effective sample count to up to eight samples per pixel. The upper limit number(s) can remain constant or can vary over time, for example, based on the state of the virtual environment 200. Further, the upper limit number can be different for samples and moments.

[0045] In various examples, the buffers (e.g., the moment history buffer and / or the sample history buffer) can include a rolling window of values that can be updated for each rendered frame (e.g., up to a maximum number of values). In various examples, the data values for a pixel (e.g., used to compute the variance(s) for the pixel) can be an aggregation of data values for the pixel(s) of the current state of the virtual environment 200 and data values of one or more previous states (e.g., frames) of the virtual environment 200. In determining the aggregated data values for a pixel, the temporal moment accumulator 108 and / or the temporal sample accumulator 108 can weight each data value. For example, each data value can be weighted equally or different data values can be assigned different weights. In some examples, the historical data values for each pixel are accumulated in the buffer and combined with the current spatial data value for the pixel to determine the aggregated data value for the pixel used for filtering. In various examples, an exponential moving average can be used to compute the aggregated data value(s) for the pixel. The exponential moving average can be computed using a linear combination of the spatial data value and the temporal accumulated value for each iteration, resulting in a directed aggregation.

[0046] Along with the time-integrated lighting condition data (e.g., visibility data) and the time-integrated moment data, the image rendering system 100 can maintain (e.g., in a G-buffer) depth, object or world space normals, mesh identifiers (IDs), screen space motion vectors generated from the rasterization process, and / or other information (e.g., spatial information). The image rendering system 100 can also maintain the depth, normals, mesh IDs, and / or other information (e.g., spatial information from that frame) for previous frames. In various examples, the temporal moment accumulator 108 and / or the temporal sample accumulator 110 can use any of this information to reproject samples and / or moments from one frame time to another (e.g., map pixels across frames). In the event of a reprojecting failure, spatial moment and / or sample data can be used.

[0047] The image filter 114 can filter the data using any of a variety of possible filtering techniques. In some examples, the image filter 114 performs filtering using a cross (or joint) bilateral filter. The cross bilateral filter can replace each pixel with a weighted average of nearby pixels, using Gaussian distribution weights that take into account distance, variance, and / or other differences between pixels to guide the image. An edge stop function can be used to identify common surfaces using G-buffer properties to improve the robustness of the cross bilateral filter to input noise.

[0048] Any of the various filters and / or filter passes described herein can be applied using a filter kernel. Filters can also have one or more filter directions. A filter kernel of a filter can refer to a matrix (e.g., a rectangular array) defining one or more convolutions to process image data (and / or illumination condition data or rendering data) of an image (e.g., data values of pixels) to change one or more characteristics of the image, such as shading and / or color of image pixels. In some examples, a filter kernel can be applied as a separable filter. In applying a filter as a separable filter, the matrix can be represented using multiple sub-matrices or filters that can be applied separately to the image data in multiple passes. When determining or calculating a filter kernel of a separable filter, the present disclosure contemplates that the sub-matrices can be calculated directly or can be derived from another matrix.

[0049] Each element of the matrix of the filter kernel can correspond to a respective pixel position. One of the pixel positions of the matrix can represent an initial pixel position that corresponds to a pixel to which the filter is applied and can be located at the center of the matrix (e.g., for determining a position of the filter). For example, when applying a filter to a pixel corresponding to a position 232 of Figure 2 , the pixel can define the initial pixel position. In applying some filters, data values (e.g., visibility values) of other pixels can be used to determine the data value(s) of the pixels within the filter kernel footprint at image positions determined relative to the pixel. A filter direction can define an alignment of the matrix relative to the image and / or pixels along which the filter is applied. Thus, when applying the filter to the pixel, other pixels of the other pixel positions of the filter kernel matrix can be determined using the filter direction(s) and the filter kernel relative to the initial pixel position.

[0050] Each element of the matrix of the filter kernel can include a filter weight for a pixel position. The matrix can be applied to an image using a convolution, where a data value of each pixel of the image corresponding to a pixel position of the matrix can be added or otherwise combined with data values of pixels corresponding to local neighbors in the matrix, as weighted by the filter values (also referred to as filter weights). For one or more filters described herein, the filter values can be configured to blur pixels, for example, by fitting a distribution(s) to the filter kernel (e.g., to the width and height).

[0051] The data values of the applied filter(s) can correspond to illumination condition data (e.g., visibility data) of the pixels. Thus, applying the matrix of the filter kernel to the pixels can cause the illumination condition data to be at least partially shared between the pixels corresponding to the pixel positions of the filter kernel. The sharing of the illumination condition data can mitigate noise due to the sparse sampling illumination in ray tracing.

[0052] In at least one embodiment, filtering of the pixel(s) can be guided based at least in part on a variance determined by variance determiner 112 for the pixel(s). For example, one or more parameters of the filter(s) and / or filter pass applied by image filter 114 can be determined based at least on the variance. In at least one embodiment, the one or more parameters define a range of filter values for the filter(s) and / or filter pass, and image filter 114 can merge samples of temporal ray tracing with spatial samples and / or other rendering data based on the range of filter values. In various examples, the range can define a set of filter values for the pixel(s) and can be based on the variance for the pixel(s). For example, when applying a filter and / or filter pass to a pixel, image filter 114 can exclude a set of filter values from filtering based at least on the set of filter values being outside of the range.

[0053] Reference is now made to Figure 3 , Figure 3 is a diagram illustrating an example of a distribution function 300 that can be used to determine a range of filter values for a filter in accordance with some embodiments of the present disclosure. Distribution function 300 can be used to define one or more parameters of the filter(s), and in the illustrated example corresponds to a Bernoulli distribution. For example, distribution function 300 can correspond to equation (2), where i corresponds to a binary random or pseudo-random (e.g., guessed) signal. In various examples, distribution function 300 can differ from that illustrated, and / or can be derived from and / or otherwise correspond to the variance and / or equation (2). Further, in some examples, distribution function 300 can correspond to a distribution that can be defined using any number of moments.

[0054] In some examples, when applying a filter to an initial pixel location using a convolution, as described herein, image filter 114 can evaluate data values corresponding to locally adjacent pixels in the matrix, and avoid combining data values that fall outside of a range. Additionally or alternatively, data values within the range can be combined using the convolution. In some examples, one or more data values can have a filter weight that is based at least on the range. The filter weight of a data value of an adjacent pixel can be reduced (or lowered) based at least on a distance of the data value from a data value of the initial pixel location. In any example, the range can be centered on the data value of the initial pixel location. In some examples, filter weights of data values within the range can not be modified based on falling within the range, but can be modified for other reasons.

[0055] In at least one embodiment, one or more parameters of the filter and / or filter pass can be based at least in part on applying results or solutions corresponding to one or more variables of the first moment and / or second moment (e.g., variance variable) of the distribution function 300. For example, referring to Figure 3 The solution of equation (2) can define one or more ranges or bounds of the range of filter values. For example, the range can be defined by the data value plus and minus the solution of the distribution function 300. To illustrate the foregoing, when the mean (first moment) is 0.5, the solution at point 310 of the distribution function 300 can be 0.25. The image filter 114 can then define the range as 0.5+ or -0.25, i.e., from 0.25 to 0.75, which means that the filter(s) can accept most data values. At point 312, only data values of 1 can be accepted by the filter(s). As shown at point 314, the range can decrease toward the maximum and minimum of the first moment (e.g., as the variance decreases from the maximum).

[0056] Figure 3 Examples of the foregoing are particularly applicable to binary signals, especially visibility signals. For example, point 312 can correspond to a pixel that is fully lit (e.g., no shading) such that there is no variance in the temporal ray-traced samples. As a result, it can be desirable to accept only data values that are the same or close to the data value of the pixel for filtering. Point 314 can correspond to a pixel associated with a dark region of a half-shadow, where there can still be modest variance. As a result, it can be desirable to accept data values that are somewhat far from the data value of the pixel for filtering. Point 310 can correspond to a pixel associated with a lighter region of a half-shadow, where there can be a large amount of variance. As a result, it can be desirable to accept most or all data values for filtering.

[0057] Using the disclosed approach can preserve contact hardening because the half-shadow region near the occluder is relatively small. For example, Figure 4 is a diagram showing an example of a frame 400 of a virtual environment 200 that can be generated using shadow denoising in accordance with some embodiments of the disclosure. The frame 400 can correspond to Figure 1 the output image 120 of FIG. 4. As can be seen, contact hardening is preserved in regions 430 and 432 near the occluder.

[0058] Spatial filtering of skipped (one or more) pixels

[0059] According to further aspects of the disclosure, a pixel of a frame can be skipped for spatial filtering based at least in part on determining that a mean value, a first moment, and / or a variance computed from temporal ray-traced samples associated with the pixel is greater than or equal to a first threshold, and / or less than or equal to a second threshold, and a count of values exceeds a third threshold.

[0060] For example, in cases where the variance is less than or equal to a threshold (e.g., zero) and the count of values exceeds a third threshold (e.g., 8 samples of frames), spatial filtering is skipped for the pixel, saving processing power. In Figure 3 In examples where the variance is 0, where the mean is 0 or 1. Thus, in some examples, the mean can be evaluated by the image filter 114 and filtering is skipped based on a determination that the mean is greater than or equal to a first threshold (e.g., 1) and / or based on a determination that the mean is less than or equal to a second threshold (e.g., 0). Additionally or alternatively, the mean can be evaluated by the image filter 114 and filtering can be performed based at least on a determination that the mean is less than the first threshold (e.g., 1) and / or based on a determination that the mean is greater than the second threshold (e.g., 0).

[0061] The mean is one example of a variable that can be evaluated to determine whether to skip filtering of a pixel. In other examples, one or more other variables can be evaluated with respect to one or more thresholds and / or a solution (e.g., a variance value) of a corresponding distribution function can be evaluated with respect to one or more thresholds. While the count is described as being evaluated for a pixel, generally, any variable and / or criteria corresponding to and / or indicative of the count can be used. In some examples, criteria corresponding to a solution (e.g., a variance) of a distribution function and a count can be consolidated into one or more values that can be evaluated with respect to one or more thresholds. Thus, there can be a variety of approaches to considering the criteria in determining whether to skip filtering of a pixel and / or whether to perform filtering of a pixel.

[0062] The present approach can be used for any suitable ray tracing effect or technique, such as for global illumination, environment occlusion, shadows, reflections, refractions, scattering phenomena, and dispersion phenomena. Thus, for example, while in some examples the ray tracing samples can correspond to visibility samples, in other examples the ray tracing samples can correspond to color brightness. Moreover, the present approach can be implemented in different rendering pipelines, as Figure 1 shown, which can or can not use the image combiner 116 to combine output from the 3D renderer 118.

[0063] Referring now to Figures 5-7 , each block of the methods 500, 600, and 700 and other methods described herein comprises a computational process that can be performed using any combination of hardware, firmware, and / or software. For instance, various functions can be carried out by a processor executing instructions stored in memory. The methods can also be embodied in computer-usable instructions stored on computer storage media. These methods can be provided by a standalone application, a service, or a plug-in to another product (standalone or in combination with another managed service), to name a few. Moreover, by way of example, with respect to the image rendering system 100 Figure 1These methods are described. However, these methods can additionally or alternatively be performed by any one system or any combination of systems, including but not limited to those described herein.

[0064] Figure 5 is a flowchart illustrating an example of a method 500 for defining a range of filter values based on at least an average of visibility values, in accordance with some embodiments of the present disclosure. The method 500 can be suitable for use in examples of any of the systems Figures 2-4 or other examples. At block B502, the method 500 includes determining an average of values corresponding to temporal ray tracing samples. For example, the temporal moment accumulator 108 can determine an average of values corresponding to temporal ray tracing samples of visibility of one or more pixels across multiple renderings of the virtual environment 200.

[0065] At block B504, the method 500 includes defining a range of filter values based on at least applying the average to a first moment and a second moment of a distribution function that decreases a range as a variance of the values decreases. For example, the variance determiner 112 can define a range of filter values of a spatio-temporal filter based on at least applying the average to at least a first moment and a second moment of a distribution function that decreases a range as a variance of the values decreases. In embodiments, the distribution function can correspond to Equations (2) and Figure 3 .

[0066] At block B506, the method 500 includes merging the values with the render using a filter based on at least the range of filter values. For example, the image filter 114 merges the values corresponding to temporal ray tracing samples of visibility with a rendering of the virtual environment 200 using a spatio-temporal filter based on the range of filter values.

[0067] Reference is now made to Figure 6 , Figure 6 is a flowchart illustrating an example of a method 600 for defining a second moment of a distribution in filtered ray tracing samples using a first moment of a distribution used to compute a variance, in accordance with some embodiments of the present disclosure. The method 600 can be suitable for use in examples of any of the systems Figures 2-4 or other examples. At block B602, the method 600 includes determining a first moment of a distribution used to compute a variance distribution from values corresponding to temporal ray tracing samples. For example, the temporal moment accumulator 108 can determine a first moment (e.g., an average) of a variance of values corresponding to temporal ray tracing samples of one or more pixels across multiple renderings of the virtual environment 200.

[0068] At block B604, the method 600 includes defining the second moment using the first moment and a variance of the second moment of the distribution to calculate the distribution. For example, the variance determiner 112 can define the second moment using the first moment from the first moment and the second moment of the variance of the time to calculate the variance (e.g., using Equation (2)).

[0069] At block B606, the method 600 includes determining one or more parameters of a filter based at least on the calculated variance. For example, the image filter 114 can determine one or more parameters of a filter based at least on the variance.

[0070] At block B608, the method 600 includes merging the temporal ray tracing samples with a rendering of the virtual environment using the filter. For example, the image filter 114 can merge the temporal ray tracing samples with a rendering of the virtual environment using the filter applied to the rendering at a location corresponding to one or more pixels (e.g., a pixel location).

[0071] Reference is now made to Figure 7 , Figure 7 is a flowchart illustrating an example of a method 700 for filtering ray tracing samples corresponding to a binary signal to apply data corresponding to an accumulated first moment of a distribution to a second moment of the distribution, in accordance with some embodiments of the present disclosure. The method 700 can apply to examples of any of Figures 2-4 or other examples. At block B702, the method 700 includes accumulating data corresponding to binary values of temporal ray tracing samples. For example, the temporal moment accumulator 108 can accumulate data corresponding to binary values of temporal ray tracing samples of the virtual environment 200 over a plurality of frames.

[0072] At block B704, the method 700 includes applying the data to a first moment and a second moment of a distribution in a variance calculation. For example, the variance determiner 112 can apply the data to the first moment and the second moment of the distribution in a calculation of a variance of the binary values.

[0073] At block B706, the method 700 includes merging the temporal ray tracing samples with a rendering using a filter based at least on the variance of the binary values. For example, the image filter 114 can merge the temporal ray tracing samples with a rendering of the virtual environment 200 using a filter applied to the rendering based at least on the variance of the binary values.

[0074] Example computing device

[0075] Figure 8is a block diagram of an example computing device 800 suitable for implementing some embodiments of the present disclosure. The computing device 800 can include an interconnection system 802 coupling the following components: a memory 804, one or more central processing units (CPUs) 806, one or more graphics processing units (GPUs) 808, a communication interface 810, input / output (I / O) ports 812, input / output components 814, a power supply 816, one or more presentation components 818 (e.g., display(s)), and one or more logic units 820.

[0076] Although Figure 8 various blocks of are shown as connected to each other via the interconnection system 802 having a bus, this is not intended to be limiting and is for clarity only. For example, in some embodiments, a presentation component 818 such as a display device can be considered an I / O component 814 (e.g., if the display is a touchscreen). As another example, a CPU 806 and / or GPU 808 can include memory (e.g., memory 804 can represent a storage device in addition to the memory of GPU 808, CPU 806, and / or other components). In other words, Figure 8 computing devices of are merely illustrative. There is no distinction, in terms of Figure 8 scope, between "workstation" "server" "laptop" "desktop" "tablet" "client device" "mobile device" "hand-held device" "game console" "electronic control unit (ECU)" "virtual reality system" and / or other device or system types as all of these can be considered within the scope of

[0077] The interconnection system 802 can represent one or more links or buses, such as an address bus, a data bus, a control bus, or a combination thereof. The interconnection system 802 can include one or more bus or link types, such as an industry standard architecture (ISA) bus, an extended industry standard architecture (EISA) bus, a video electronics standards board (VESA) bus, a peripheral component interconnect (PCI) bus, a peripheral component interconnect express (PCIe) bus, and / or another type of bus or link. In some embodiments, there are direct connections between components. As an example, a CPU 806 can be directly connected to a memory 804. Further, a CPU 806 can be directly connected to a GPU 808. Where there are direct or point-to-point connections between components, the interconnection system 802 can include a PCIe link to perform the connection. In these examples, a PCI bus need not be included in the computing device 800.

[0078] Memory 804 can include any of a wide variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 800. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media.

[0079] Computer storage media can include volatile and nonvolatile media, and removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules and / or other data types. For example, memory 804 can store computer readable instructions such as those representing programs and / or program elements, e.g., an operating system. Computer storage media can include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computing device 800. Computer storage media, as used herein, does not include signals per se.

[0080] Computer storage media can include computer-readable instructions, data structures, program modules, and / or other data types in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" can refer to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, computer storage media can include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the above should also be included within the scope of computer-readable media.

[0081] The CPUs 806 can be configured to execute at least some computer-readable instructions in order to control one or more components of the computing device 800 to perform one or more of the methods and / or processes described herein. Each of the CPUs 806 can include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a large number of software threads concurrently. The CPUs 806 can include any type of processors and can include different types of processors depending on the type of computing device 800 being implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 800, the processors can be Advanced RISC Machines (ARM) processors implemented using Reduced Instruction Set Computing (RISC) or x86 processors implemented using Complex Instruction Set Computing (CISC). The computing device 800 can include one or more CPUs 806 in addition to one or more microprocessors or supplemental co-processors such as math co-processors.

[0082] In addition to or instead of the CPUs 806, the GPUs 808 can be configured to execute at least some computer-readable instructions to control one or more components of the computing device 800 to perform one or more of the methods and / or processes described herein. One or more of the GPUs 808 can be integrated GPUs (e.g., one or more of which can be a discrete GPU) may include Figure 8In embodiments, one or more of the GPUs 808 can be co-processors to one or more of the CPUs 806. The GPUs 808 can be used by the computing device 800 to render graphics (e.g., 3D graphics) or to perform general purpose computing. For example, the GPUs 808 can be used for general purpose computing on GPUs (GPGPU). The GPUs 808 can include hundreds or thousands of cores capable of processing hundreds or thousands of software threads concurrently. The GPUs 808 can generate pixel data for an output image in response to rendering commands (e.g., rendering commands received from the CPUs 806 via a host interface). The GPUs 808 can include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPGPU data. The display memory can be included as part of the memory 804. The GPUs 808 can include two or more GPUs operating in parallel (e.g., via a link). The link can connect the GPUs directly (e.g., using NVLINK) or can connect the GPUs through a switch (e.g., using an NVSwitch). When combined together, each GPU 808 can generate different portions of pixel data or GPGPU data for output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU can include its own memory or can share memory with other GPUs.

[0083] In addition to or instead of the CPUs 806 and / or GPUs 808, the logic unit(s) 820 can be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 800 to perform one or more of the methods and / or processes described herein. In embodiments, the CPU(s) 806, GPU(s) 808, and / or logic unit(s) 820 can execute any combination of methods, processes, and / or portions thereof discretely or jointly. One or more of the logic units 820 can be one or more of the CPUs 806 and / or GPUs 808 and / or integrated in one or more of the CPUs 806 and / or GPUs 808, and / or one or more of the logic units 820 can be discrete components or otherwise external to the CPUs 806 and / or GPUs 808. In embodiments, one or more of the logic units 820 can be co-processors to one or more of the CPUs 806 and / or one or more of the GPUs 808.

[0084] Examples of logic units 820 include one or more processing cores and / or components thereof, such as tensor cores (TCs), tensor processing units (TPUs), pixel vision cores (PVCs), vision processing units (VPUs), graphics processing clusters (GPCs), texture processing clusters (TPCs), streaming multi-processors (SMs), tree traversal units (TTUs), artificial intelligence accelerators (AIAs), deep learning accelerators (DLAs), arithmetic logic units (ALUs), application-specific integrated circuits (ASICs), floating point units (FPUs), input / output (I / O) elements, peripheral component interconnects (PCIs) or peripheral component interconnect express (PCIe) elements, and / or the like.

[0085] Communication interface 810 can include one or more receivers, transmitters, and / or transceivers that enable computing device 800 to communicate with other computing devices via electronic communication networks, including wired and / or wireless communications. Communication interface 810 can include components and functionality enabling communication over any of a number of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication over Ethernet or InfiniBand), low power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.

[0086] I / O ports 812 can enable computing device 800 to be logically coupled to other devices including I / O components 814, presentation components 818, and / or other components, some of which can be built into (e.g., integrated with) computing device 800. Illustrative I / O components 814 include a microphone, mouse, keyboard, joystick, game pad, game controller, dish satellite antenna, scanner, printer, wireless device, etc. I / O components 814 can provide a natural user interface (NUI) that processes audio, speech, or other physiological inputs generated by a user. In some examples, the inputs can be transmitted to an appropriate network element for further processing. The NUI can implement any combination of speech recognition, stylus recognition, facial recognition, biometric recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, and touch recognition (as described in more detail below) associated with a display of the computing device 800. Computing device 800 Figure 8 Depth cameras, infrared cameras, RGB cameras, touchscreen technology, and combinations of these, such as stereoscopic camera systems, for gesture detection and recognition. In addition, computing device 800 can include an accelerometer or a gyroscope (e.g., as part of an inertial measurement unit (IMU)) that enables motion detection. In some examples, the output of the accelerometer or gyroscope can be used by computing device 800 to render an immersive augmented reality or virtual reality.

[0087] The power supply 816 can include a hard-wired power supply, a battery power supply, or a combination thereof. The power supply 816 can supply power to the computing device 800 to enable the components of the computing device 800 to operate.

[0088] The presentation component 818 can include a display (e.g., a monitor, a touchscreen, a television screen, a heads-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The presentation component 818 can receive data from other components (e.g., the GPU 808, the CPU 806, etc.) and output the data (e.g., as images, video, sound, etc.).

[0089] Example Network Environment

[0090] A network environment suitable for use in implementing embodiments of the present disclosure can include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) can be implemented on one or more instances of the computing device(s) 800— e.g., each device can include similar components, features, and / or functionality of the computing device(s) 800. Figure 9

[0091] Components of the network environment can communicate with each other via network(s), which can be wired, wireless, or both. The network(s) can include multiple networks or one of multiple networks. For example, the network(s) can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks such as the Internet, and / or the public switched telephone network (PSTN) and / or one or more private networks. In the case where the network(s) include a wireless telecommunication network, components such as base stations, communication towers, or even access points (among other components) can provide wireless connectivity.

[0092] Compatible network environments can include one or more peer-to-peer network environments (in which case servers can not be included in the network environment) and one or more client-server network environments (in which case one or more servers can be included in the network environment). In a peer-to-peer network environment, functionality described herein with respect to servers can be implemented on any number of client devices.

[0093] ​In at least one embodiment, a network environment can include one or more cloud-based network environments, distributed computing environments, combinations thereof, and the like. A cloud-based network environment can include a framework layer, a work scheduler, a resource manager, and a distributed file system implemented on one or more servers, which can include one or more core network servers and / or edge servers. The framework layer can include a framework that supports one or more applications of a software layer and / or an application layer. The software or applications can include network-based service software or applications, respectively. In embodiments, one or more client devices can use the network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer can be, but is not limited to, a type of free and open-source software web application framework, such as can use a distributed file system for large-scale data processing (e.g., “big data”).

[0094] A cloud-based network environment can provide cloud computing and / or cloud storage that performs any combination of the computing and / or data storage functions described herein (or one or more portions thereof). Any of these different functions can be distributed across multiple locations from central or core servers (e.g., one or more data centers that can be distributed across states, regions, countries, the Earth, and the like). Core servers can designate at least a portion of a function to an edge server if a connection to a user (e.g., a client device) is relatively close to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), can be public (e.g., available to many organizations), and / or combinations thereof (e.g., a hybrid cloud environment).

[0095] A client device can include at least some of the components, features, and functionality of the example computing device(s) 800 described herein. Figure 9 By way of example, and not limitation, a client device can be implemented as a personal computer (PC), a laptop computer, a mobile device, a smart phone, a tablet computer, a smart watch, a wearable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality headset, a global positioning system (GPS) or device, a video player, a camera, a surveillance device or system, a vehicle, a ship, a spacecraft, a virtual machine, a drone, a robot, a hand-held communication device, a hospital device, a gaming device or system, an entertainment system, a vehicle computer system, an embedded system controller, a remote control, an appliance, a consumer electronic device, a workstation, an edge device, any combination of these depicted devices, or any other suitable device.

[0096] Example data center

[0097] Figure 9An example data center 900 that can be used in at least one of the embodiments is shown. In at least one embodiment, the data center 900 may include a data center infrastructure layer 910, a framework layer 920, a software layer 930, and / or an application layer 940.

[0098] In at least one embodiment, such as ​ As shown, the data center infrastructure layer 910 may include a resource coordinator 912, grouped computing resources 914, and node computing resources (“nodes CR”) 916(1)-916(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 916(1)-916(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power modules and / or cooling modules, etc. In at least one embodiment, one or more nodes CR 916(1)-916(N) may be servers having one or more of the aforementioned computing resources.

[0099] In at least one embodiment, the grouped computing resources 914 may include individual groups (not shown) of node CRs housed in one or more racks, or a plurality of racks (also not shown) housed in data centers in various geographic locations. Individual groups of node CRs within the grouped computing resources 914 may include computing, networking, memory, or storage resources that can be configured or allocated to support groups of one or more workloads. In at least one embodiment, several node CRs 916, including CPUs, GPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, the one or more racks may also include any number of power modules, cooling modules, and network switches, in any combination.

[0100] In at least one embodiment, resource coordinator 922 may configure or otherwise control one or more nodes CR916(1)-916(N) and / or grouped computing resources 914. In at least one embodiment, resource coordinator 922 may include a software design infrastructure (“SDI”) management entity for data center 900. In at least one embodiment, resource coordinator may include hardware, software, or some combination thereof.

[0101] In at least one embodiment, such as ​As shown, framework layer 920 may include job scheduler 932, configuration manager 934, resource manager 936, and distributed file system 938. In at least one embodiment, framework layer 920 may include a framework of software 932 supporting software layer 930 and / or one or more applications 942 of application layer 940. In at least one embodiment, software 932 or application 942 may respectively include web-based service software or applications, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, framework layer 920 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark, which can utilize distributed file system 938 for large-scale data processing (e.g., "big data"). TM (Hereinafter referred to as "Spark"). In at least one embodiment, the job scheduler 932 may include a Spark driver to facilitate the scheduling of workloads supported by various layers of the data center 900. In at least one embodiment, the configuration manager 934 may be able to configure different layers, such as the software layer 930 and the framework layer 920, which includes Spark and a distributed file system 938 for supporting large-scale data processing. In at least one embodiment, the resource manager 936 is able to manage cluster or group computing resources mapped to or allocated to support the distributed file system 938 and the job scheduler 932. In at least one embodiment, the cluster or group computing resources may include grouped computing resources 914 on the data center infrastructure layer 910. In at least one embodiment, the resource manager 936 may coordinate with the resource coordinator 912 to manage these mapped or allocated computing resources.

[0102] In at least one embodiment, the software 932 included in the software layer 930 may include software used by at least a portion of nodes CR916(1)-916(N), grouped computing resources 914, and / or the distributed file system 938 of the framework layer 920. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0103] In at least one embodiment, one or more application programs 942 included in application layer 940 can include one or more types of application programs used by at least portions of node C.R.s 916(1)-916(N), grouped computing resources 914, and / or distributed file system 938 of framework layer 920. One or more types of application programs can include, but are not limited to, any number of genomics applications, cognitive computing and machine learning applications, including training or inferencing software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0104] In at least one embodiment, any of configuration manager 934, resource manager 936, and resource orchestrator 912 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible fashion. In at least one embodiment, self-modification actions can mitigate data center operators of data center 900 making possibly poor configuration decisions and can avoid underutilized and / or poorly performing portions of a data center.

[0105] In at least one embodiment, data center 900 can include tools, services, software, or other resources to train one or more machine learning models or use one or more machine learning models to predict or infer information according to one or more embodiments described herein. For example, in at least one embodiment, a machine learning model can be trained according to a neural network architecture by computing weight parameters using software and computing resources described above with respect to data center 900. In at least one embodiment, using trained machine learning models corresponding to one or more neural networks can be used to infer or predict information using weight parameters computed through one or more training techniques using resources described above with respect to data center 900.

[0106] The present disclosure can be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types. The present disclosure can be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general- purpose computers, more specialty computing devices, and the like. The present disclosure can also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.

[0107] As used herein, the recitation of “and / or” with respect to two or more elements should be interpreted to mean that one element or a combination of elements can be used. For example, “element A, element B, and / or element C” can include just element A, just element B, just element C, element A and element B, element A and element C, element B and element C, or elements A, B, and C. Further, “at least one of element A or element B” can include at least one of element A, at least one of element B, or at least one of each of element A and element B. Further still, “at least one of element A and element B” can include at least one of element A, at least one of element B, or at least one of each of element A and element B.

[0108] The subject matter of the present disclosure is now specifically described herein with the understanding that the description is intended to be illustrative and not limiting of the scope of the disclosure. The description is not intended to be exhaustive or to limit the scope of the disclosure to the precise form disclosed. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and / or “block” can be used herein to connote different elements of the methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.

Claims

1. A computer-implemented method comprising: determining data corresponding to temporal ray-tracing samples of one or more pixels across a plurality of renderings of a virtual environment; defining a range of values for a data value of a pixel located within a spatiotemporal filter coverage based on at least applying the data to a first moment and a second moment of a distribution function; determining an aggregate of the data value of the pixel using the spatiotemporal filter based on at least evaluating one of the data values with the range of values using the spatiotemporal filter; and generating an image of the virtual environment based on the aggregate of the data value.

2. The method of claim 1, wherein the data is applied to the first moment and the second moment of the distribution function to compute a variance in the temporal ray-tracing samples.

3. The method of claim 1, wherein the range of values is determined from a variance in the temporal ray-tracing samples.

4. The method of claim 1, wherein determining the aggregate includes excluding the data value from the aggregate based on at least the data value falling outside the range of values.

5. The method of claim 1, wherein the distribution function corresponds to a Bernoulli distribution.

6. The method of claim 1, wherein the determining the aggregate includes including the data value in the aggregate based on at least the data value falling within the range of values.

7. The method of claim 1, wherein the range is at a minimum when a mean of the temporal ray-tracing samples is at a maximum or a minimum.

8. The method of claim 1, wherein determining the aggregate includes computing a weight value for the data value in the aggregate using the range of values.

9. The method of claim 1, wherein the using the spatiotemporal filter is based on at least determining that a mean of the temporal ray-tracing samples is less than or equal to a second threshold value and a count of the values exceeds a third threshold value.

10. A computer-implemented method comprising: determining data representing a first moment of a distribution from values corresponding to temporal ray-tracing samples of one or more pixels across a plurality of renderings of a virtual environment; computing a variance from the first and second moments of the distribution based on at least using the data representing the first moment to determine a second moment of the distribution; determining one or more parameters of a filter based on at least the variance; and merging the temporal ray-tracing samples with a rendering of the virtual environment at a location corresponding to the one or more pixels using the filter applied to the rendering.

11. The method of claim 10, wherein the temporal ray-tracing samples correspond to a visibility of the one or more pixels with respect to one or more light sources in the virtual environment.

12. The method of claim 10, wherein the values correspond to binary signals.

13. The method of claim 10, wherein the first moment includes a mean of the values and the second moment includes a mean of squares of the values. ​ ​ 14. The method of claim 10, further comprising accumulating data representing the first moment to a buffer over the plurality of renders, wherein the determining data representing the first moment comprises retrieving the first moment from the buffer based on a location corresponding to the one or more pixels.

15. The method of claim 10, wherein the temporal ray tracing sample corresponds to an environmental occlusion of the one or more pixels relative to the virtual environment.

16. The method of claim 10, wherein the one or more parameters define a range of values for data values of pixels within a coverage of the filter and merging the temporal ray tracing sample with the render using the filter comprises aggregating the data values using the range of values.

17. A computer-implemented method comprising: accumulating data corresponding to binary values of temporal ray tracing samples of a virtual environment over a plurality of frames, the data representing a first moment of a distribution; using the data to compute a second moment of the distribution in a computation of a variance of the binary values; and merging the temporal ray tracing sample with a render of the virtual environment using a filter applied to the render based at least on a variance of the binary values.

18. The method of claim 17, wherein the temporal ray tracing sample corresponds to a visibility of one or more pixels relative to one or more light sources in the virtual environment.

19. The method of claim 17, wherein applying the data to the second moment comprises computing the second moment from the first moment.

20. The method of claim 17, wherein incorporating the temporal ray-tracing sample with the render further comprises: determining a range of values for data values of pixels processed using the filter based on the computation and merging the temporal ray tracing sample with the render using the filter based on the range of values.

21. The method of claim 17, further comprising generating the temporal ray tracing sample using at least one shadow ray for each of one or more pixels associated with the plurality of frames.

22. The method of claim 17, wherein the first moment corresponds to a mean and the second moment corresponds to a mean square.

23. The method of claim 17, wherein the virtual environment is generated by at least one of: a system to perform simulation operations; a system to perform simulation operations to test or validate autonomous machine applications; a system to perform deep learning operations; a system implemented using edge devices; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

Images