Denoising techniques for cyclic blur in ray tracing applications

Abstract

Cyclic blur can be used to render a frame of a virtual environment, where a radius of a filter for a pixel is based on a number of successfully accumulated frames corresponding to the pixel. To account for rejection of accumulated samples for the pixel, ray-tracing samples from a lower resolution version of the ray-tracing rendering can be used to increase an effective sample count for the pixel. Parallax can be used to control a speed of accumulation and an angle between a view vector corresponding to the pixel. A size of one or more dimensions of a filter applied to the pixel can be based on the angle of the view vector corresponding to the pixel, to cause reflections to elongate along an axis at a grazing angle. The one or more dimensions can be based on a direction of a specular reflection lobe associated with the pixel.

Description

Background Technology

[0001] Ray tracing is used to render images by tracing the paths of light in a virtual environment and simulating the effects of light encountering virtual objects. Various applications of ray tracing can simulate a wide range of optical effects—such as shadows, reflections and refractions, scattering, ambient occlusion, global illumination, and chromatic aberration (such as chromatic aberration). Ray tracing typically involves generating ray-traced samples by sampling lighting conditions that project light onto pixels in a virtual environment. These ray-traced samples can be combined and used to determine the pixel colors of an image. To conserve computational resources, lighting conditions can be sparsely sampled, resulting in noisy rendering data. To produce a final render that approximates a fully sampled scene, denoising filters can be applied to the noisy rendering data to reduce noise, and temporal accumulation can be used to increase the effective sample count by utilizing information from previous frames.

[0002] Denoising strategies involving temporal accumulation typically employ either post-accumulation or pre-accumulation. In post-accumulation, frames of noisy render data are first blurred, then temporally accumulated together with similar blurred frames that were previously blurred. In pre-accumulation, frames of noisy render data are first temporally accumulated, and then the temporally accumulated data is blurred. In either case, artifacts may occur because low-frequency and high-frequency inputs are not mixed in the blurring. Cyclic blur can be used to mix low-frequency and high-frequency inputs by iteratively accumulating frames of noisy render data with temporally accumulated data, then blurring the accumulated data to generate an output that is fed back to update the temporally accumulated data for subsequent iterations. However, because cyclic blur feeds back the blurred output used for blurring the next iteration, this method can be prone to over-blurring. Summary of the Invention

[0003] This disclosure relates to denoising techniques for cyclic blur in ray tracing applications. More specifically, this disclosure relates to various methods that can be used to improve denoising in ray tracing rendering (e.g., denoising using cyclic blur). Various disclosed concepts can be incorporated into blurring that utilizes time accumulation.

[0004] Compared to conventional systems, this disclosure provides cyclic blurring for rendering frames of a virtual environment, wherein the radius (or more generally, size or dimension) of at least one denoising filter for a pixel is based at least on the number of frames or samples successfully accumulated corresponding to that pixel. A hierarchical approach can be used to interpret the rejection of temporally accumulated samples relative to a pixel, where the effective sample count can be increased using ray-traced samples from a lower-resolution version of the ray-traced render, rather than using only ray-traced samples from the ray-traced render of that pixel. Parallax can be used to control the accumulation rate of the temporal accumulation of a pixel and the angle between the view vector corresponding to that pixel to account for how parallax is visually perceived at different viewpoints.

[0005] Instead of applying a denoising filter to pixels that are actually isotropic in world space, the magnitudes of one or more dimensions of the filter can be based at least on the angles of the view vector corresponding to the pixel, causing reflections to elongate along the axis at the grazing angle, thus producing a more realistic rendering. The dimensions can be based at least on the direction (e.g., the dominant direction) of the specular reflection lobes associated with the view vector and the corresponding surface normals. Attached Figure Description

[0006] The system and method for denoising techniques suitable for cyclic blurring are described in detail below with reference to the accompanying drawings, wherein:

[0007] Figure 1 This is a data flow diagram illustrating an example process for determining one or more filter parameters based at least on accumulated samples according to at least one embodiment of the present disclosure;

[0008] Figure 2A The present disclosure illustrates that at least one embodiment can be used after two accumulated frames. Figure 1 Example rendering of the virtual environment generated by the example process;

[0009] Figure 2B The present disclosure illustrates at least one embodiment that can be used after accumulating thirty-two frames. Figure 1 Example rendering of the virtual environment generated by the example process;

[0010] Figure 3 This is a flowchart illustrating an example of a method for determining one or more parameters of a filter based on the number of frames for rendering at least based on a virtual environment, according to at least one embodiment of the present disclosure;

[0011] Figure 4 This is a flowchart illustrating an example of a method for iteratively determining one or more parameters of a filter based at least on the number of renders corresponding to accumulated samples in a virtual environment, according to at least one embodiment of the present disclosure;

[0012] Figure 5 This is a flowchart illustrating an example of a method for determining one or more parameters of a filter based on at least one embodiment of the present disclosure for the number of renders corresponding to at least one sample of accumulated renders, at least based on a virtual environment;

[0013] Figure 6A This is a flowchart illustrating an example process for denoising using cyclic blur according to at least one embodiment of the present disclosure, the example process including generating enhanced samples based at least on the number of renders corresponding to the accumulated ray-traced samples in the virtual environment;

[0014] Figure 6B This is a flowchart illustrating an example process for denoising using pre-accumulation according to at least one embodiment of the present disclosure, the example process including generating enhanced samples based at least on the number of renders corresponding to the number of ray-traced samples in the accumulated renders of the virtual environment;

[0015] Figure 7A Examples of rendering virtual environments at different mip levels according to at least one embodiment of the present disclosure are shown;

[0016] Figure 7B Examples of mipmap levels and alignment between adjacent mipmap levels according to at least one embodiment of the present disclosure are shown;

[0017] Figure 8A An example rendering of a virtual environment that can be generated without the need for enhanced accumulated samples according to at least one embodiment of the present disclosure is shown;

[0018] Figure 8B An example rendering of a virtual environment generated using enhanced, accumulated samples according to at least one embodiment of this disclosure is shown;

[0019] Figure 9 This is a flowchart illustrating an example of a method for enhancing cumulative rendering based on at least one embodiment of the present disclosure, for enhancing cumulative rendering based at least on the number of renderings corresponding to the number of ray-traced samples of the virtual environment in the cumulative rendering of the virtual environment;

[0020] Figure 10 This is a flowchart illustrating an example of a method for iteratively enhancing an accumulated set of samples, according to at least one embodiment of the present disclosure, based on the number of renders corresponding to at least one ray-traced sample of a virtual environment;

[0021] Figure 11This is a flowchart illustrating an example of a method for enhancing cumulative rendering based at least on the number of renders corresponding to pixels in the cumulative rendering of a virtual environment, according to at least one embodiment of the present disclosure;

[0022] Figure 12 This is a data flow diagram illustrating an example process for denoising the rendering of a virtual environment according to at least one embodiment of the present disclosure;

[0023] Figure 13A This is a diagram illustrating an example of determining a blurred plane for specular reflection according to at least one embodiment of the present disclosure;

[0024] Figure 13B This is an illustration showing an example of determining a blur plane for a diffuse signal according to at least one embodiment of the present disclosure;

[0025] Figure 14A This is a diagram illustrating an example of determining a blur plane for denoising ray-traced samples according to at least one embodiment of the present disclosure;

[0026] Figure 14B This is an illustration showing an example of determining an anisotropic filter for denoising ray-traced samples according to at least one embodiment of the present disclosure;

[0027] Figure 15 This is a flowchart illustrating an example of a method for calculating the size of a filter based at least on the direction, normal, and angle of a view vector of a reflected specular lobe, according to at least one embodiment of the present disclosure.

[0028] Figure 16 This is a flowchart illustrating an example of a method for determining an anisotropic filter based at least on an isotropic filter extended at least based on a view vector, according to at least one embodiment of the present disclosure.

[0029] Figure 17 This is a flowchart illustrating an example of a method for calculating the size of a filter based at least on the direction, normal, and angle of light rays associated with the interaction of at least one embodiment of the present disclosure;

[0030] Figure 18 An example rendering of a virtual environment having view vectors that can be used to calculate parallax, according to at least one embodiment of the present disclosure, is shown.

[0031] Figure 19 This is a flowchart illustrating an example of a method for calculating cumulative velocity based at least on the angle between view vectors and the viewpoint associated with the view vectors, according to at least one embodiment of the present disclosure;

[0032] Figure 20 This is a flowchart illustrating an example of a method for accumulating ray-traced rendering using an accumulation velocity according to at least one embodiment of the present disclosure, the accumulation velocity being based at least on the parallax between a view and a viewpoint captured by ray-traced rendering;

[0033] Figure 21 This is a flowchart illustrating an example of a method for calculating cumulative speed based at least on the parallax between at least one embodiment of the present disclosure and at least one viewpoint associated with the rendering;

[0034] Figure 22 These are diagrams illustrating examples of ray tracing techniques according to some embodiments of the present disclosure, which can be used to generate ray-traced renderings;

[0035] Figure 23 This is a block diagram of an example computing device suitable for implementing some embodiments of the present disclosure; and

[0036] Figure 24 This is a block diagram of an example data center suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0037] Systems and methods related to denoising techniques suitable for cyclic blurring are disclosed. This disclosure provides various denoising techniques that can be used alone or in any combination thereof. Various disclosed concepts can be incorporated into blurring that utilizes time accumulation.

[0038] This disclosure provides aspects of cyclic blurring for rendering frames of a virtual environment, wherein the radius (or more generally, size or dimension) of at least one denoising filter for a pixel is based at least on the number of successfully accumulated frames corresponding to said pixel (e.g., temporally accumulated samples in a history buffer). For example, the number of successfully accumulated frames can be tracked per pixel and can be used to control or adjust the blur size of the denoising filters applied to those pixels (e.g., in one or more spatial filtering passes). These methods can be used to reduce the size of the denoising filters for a pixel as the number of accumulated samples or frames associated with those pixels increases, thereby reducing or preventing over-blurring.

[0039] This disclosure also provides a layered approach for interpreting the rejection of temporally accumulated samples relative to a pixel. Typically, when a pixel's temporally accumulated samples are rejected (e.g., due to de-occlusion of the pixel during ray tracing), the temporal accumulation of that pixel is reset. Consequently, when re-accumulating temporal data, the effective sample count for a given number of frames can be reduced. This can lead to artifacts in the rendered frames, particularly when the number of rays per pixel used to sample the pixel's virtual environment is low.

[0040] The disclosed method may supplement the effective number of rays per pixel used to sample the virtual environment for that pixel based at least on the number of successfully accumulated frames, renders, and / or samples corresponding to that pixel. For example, in a denoising pipeline, ray tracing samples from a lower-resolution version of the ray tracing render may be used instead of only using ray tracing samples from the ray tracing render for that pixel. In various embodiments, the resolution level (e.g., mipmap level) may be based at least on the number of successfully accumulated frames and may increase with the number of successfully accumulated frames (e.g., until the base resolution of the ray tracing render is reached). For example, the number of successfully accumulated frames may be tracked per pixel, and this number may be used to control or adjust the resolution of the ray tracing render, which is used in combination with or in lieu of the temporally accumulated data for those pixels (e.g., before or after one or more spatial filter passes). This method may be used in conjunction with cyclic blur, which may adjust the denoising filter size based at least on or not at least on the number of successfully accumulated frames. Other denoising strategies, such as post-accumulation or pre-accumulation, are also contemplated for implementation of these methods.

[0041] This disclosure also provides methods for calculating one or more dimensions of an anisotropic denoising filter (e.g., filter kernel) applied to a pixel when denoising ray-traced rendering. For example, instead of applying the denoising filter to a pixel that is actually isotropic in world space, the magnitudes of one or more dimensions of the filter may be based at least on the angle corresponding to the view vector of that pixel. This can be used to extend reflections along axes at grazing angles, resulting in more realistic rendering. This disclosure provides for calculating anisotropic filter kernels that can be extended such that the size of the filter kernel extends further along a first axis than along a second axis to more closely reflect the space occupied by BRDF lobes in the virtual environment. In various examples, one or more dimensions may also be based at least on the direction (e.g., dominant direction) of the specular lobes of the reflection associated with the view vector and the corresponding surface normal. These methods can be used in combination with cyclic blur, post-accumulation, pre-accumulation, and / or other denoising strategies that may not necessarily include temporal accumulation.

[0042] This disclosure also provides the use of parallax between frames or renders to control the accumulation rate of temporal accumulation (e.g., for specular accumulation). For example, surface motion can be used to reproject specular historical textures from a previous frame or render to a subsequent (e.g., current) frame or render. However, the motion of the reflection as light leaves the surface can differ from that surface motion, and therefore using only surface motion can result in artifacts in the rendered frames. The accuracy of the surface motion of a pixel can be based at least in part on the parallax between frames, and thus can be used to control the effect of surface motion in temporal accumulation.

[0043] In various embodiments, the disparity used to calculate the cumulative velocity of a pixel can be based at least on the angle between view vectors corresponding to the pixel. For example, disparity can correspond to the angle between a previous view vector and a current view vector for the same surface point. In at least one embodiment, the disparity of a pixel can be calculated as the ratio between the camera movement projection of one or more points to the screen plane and the distance to one or more points during rendering or between frames. The cumulative velocity can also be based at least on the viewpoint associated with the view vectors rendered by ray tracing, which can take into account how disparity is visually perceived at different viewpoints. These methods can be used in combination with cyclic blur, post-accumulation, pre-accumulation, and / or other denoising strategies.

[0044] The disclosed methods can be implemented, for example, using a denoiser that accepts radiation (e.g., radiation only and / or radiation among other information). According to various embodiments, diffuse and specular signals may or may not be separated. For example, radiation, or energy from a particular direction, can be separated from the material. In some embodiments, the final color of a pixel can be based at least on the multiplication or other combination of radiation applied to a physically-based light model including light properties (e.g., after denoising). As an example, for diffuse radiation, the final color can be based at least on the radiation multiplied by reflectivity, which essentially represents the primary color. While the disclosed methods are compatible with both specular and diffuse radiation, some embodiments can be adapted to accommodate only specular or diffuse radiation.

[0045] The disclosed method can be used to generate ray-traced samples (e.g., radiative ray-traced samples) using any suitable method for ray tracing, such as random ray tracing. Examples of random ray tracing techniques that can be used include those employing Monte Carlo or quasi-Monte Carlo sampling strategies. The disclosed method can take into account properties such as surface reflections projected by different types of surfaces (e.g., glossy surfaces). For this purpose, a normal distribution function (NDF) can be employed. A non-limiting example of an NDF described herein is a GGX NDF, but other types of NDFs can be used.

[0046] While this disclosure relates to roughness values ​​at a point on a surface, any suitable method can be used to define surface roughness, and any suitable method may include using one or more roughness values. For example, in the case where the NDF captures a GGX distribution, a single roughness value may be used. However, the examples provided herein can be adapted to other microfacet distribution models as needed. In various examples, this disclosure can be used with GGX, Beckman, Blinn-Phong, or other types of NDF distributions, which may also include non-standard distributions. In the case of defining multiple roughness values, a single roughness value may be determined and used by combining and / or analyzing the roughness values ​​as described herein, one of the roughness values ​​may be used, or those methods may be otherwise appropriately adapted.

[0047] The disclosed system may include or be included in one or more of the following: a system for performing simulation operations, a system for performing simulation operations to test or verify autonomous machine applications, a system for performing deep learning operations, a system implemented using edge devices, a system containing one or more virtual machines (VMs), a system implemented at least partially in a data center, or a system implemented at least partially using cloud computing resources.

[0048] Figure 1 This is a data flow diagram illustrating an example process 100 for determining one or more filter parameters based at least on accumulated samples, according to at least one embodiment of the present disclosure. Process 100 can be performed using, for example, an image renderer 102, a sample accumulator 104, a filter determiner 106, and a denoiser 108. It should be understood that such and other arrangements described herein are merely illustrative examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may be used in addition to or instead of the arrangements and elements shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or combined with other components, and in any suitable combination and location. The various functions described herein as being performed by entities can be performed by hardware, firmware, and / or software.

[0049] Image renderer 102 can be configured to render a virtual environment (e.g., Figure 22The image renderer 102 may employ a sample accumulator 104, a filter determiner 106, and a denoiser 108 to render the image of the virtual environment (2200). The sample accumulator 104 may be configured to accumulate (e.g., temporally relative to pixels) ray-traced samples (i.e., samples generated based on ray tracing of the virtual environment). The samples accumulated by the sample accumulator 104 may be used in different ways, for example, depending on whether post-accumulation, pre-accumulation, or cyclic blur is used to render the image. The filter determiner 106 may be configured to determine at least one size (e.g., geometry) of a filter (e.g., a spatial filter) and the corresponding filter weights for a point or corresponding pixel in the virtual environment. The denoiser 108 may be configured to apply the filter to pixels corresponding to points in the virtual environment (e.g., at the initial pixel location of the filter) to denoise the image representing the virtual environment (e.g., a ray-traced render or a frame).

[0050] Image renderer 102 may similarly use sample accumulator 104, filter determiner 106, and denoiser 108 to determine filters for other points in the virtual environment and corresponding pixels in the image to produce a resulting image. Thus, multiple filters can be used to denoise lighting condition data (e.g., radiance) associated with the image to produce a resulting image (e.g., at least one filter per pixel), and those filters can all be applied in a single draw call or channel (although multiple channels can be used for additional filters). The images described herein need not be fully rendered. As an example, one or more portions of the visual content of an image and / or its components may be rendered (e.g., radiance). Denoiser 108 may apply filters to image data representing one or more portions of the visual content of an image and / or its components to produce image data representing a rendered frame (e.g., using any number of denoising passes). Images may be shown to illustrate noise that can be filtered out in some examples of this disclosure.

[0051] Image renderer 102 can use lighting condition data for each pixel to render one or more corresponding pixels of an image. Typically, the accuracy of the lighting conditions calculated by the image renderer for a pixel relative to a light source can increase with the amount of principal rays and / or auxiliary rays used to sample the lighting conditions. However, the computational resources used to determine the lighting conditions can also increase with the number of rays, which may increase rendering time.

[0052] To conserve computational resources and reduce rendering time, the number of rays used to sample lighting conditions may be less than the number required for reflection quality to converge to an ideal ray tracing result. This can cause the image renderer 102 to generate lighting condition data that includes noisy image data, as indicated in the various images throughout the document. For example, the lighting conditions for each pixel of the screen may be based on a single ray tracing sample (e.g., one ray sample per pixel) or a finite number of other samples of the virtual environment's state (e.g., including a single camera or eye ray and a single reflected ray per incident ray).

[0053] To reduce noise in the lighting condition data of pixels, denoiser 108 can filter the lighting condition data (ray tracing samples, accumulated samples, etc.) an arbitrary number of times to generate a final rendered frame. In some examples, filter determiner 106 can determine the filter for each pixel of the screen and / or for rendering. Image renderer 102 can apply the filter to the lighting condition data corresponding to the image (e.g., pixel) location of the relevant pixel to render the resulting image. In various examples, one or more filters can be applied in parallel. Furthermore, each filter can be applied in one or more passes (e.g., as a separable filter). Additionally, while examples of information for determining the properties of the filters are provided, additional information that causes corresponding adjustments to the properties of the filters (e.g., size and / or filter weights) can be used.

[0054] Each filter described herein may include a filter kernel and may or may not include one or more filter directions. The filter kernel of a filter may specify a matrix (e.g., a rectangular array) that defines one or more convolutions for processing image data of an image to alter one or more properties of the image (e.g., the shading and color of pixels in the image) (e.g., by applying radiation). In some examples, the filter kernel may be applied to a separable filter, where the matrix can be represented using multiple submatrices or filters that can be applied to the image in multiple passes. When determining or computing the filter kernel of a separable filter, this disclosure contemplates that submatrices can be directly computed or derived from another matrix.

[0055] Each element of the filter kernel matrix can represent a corresponding pixel position. One of the pixel positions in the matrix can represent the initial pixel position corresponding to the pixel to which the filter is applied, typically located at the center of the matrix. For example, in the figures described herein, point p can correspond to the initial pixel position (e.g., and...). Figure 22(Point 2216 and / or pixel 2212). The filter orientation can define the alignment of the matrix of the image and / or pixels with respect to which the filter is applied. Thus, when a filter is applied to a pixel, other pixels at other pixel positions in the matrix used for the filter kernel can be aligned with the filter orientation relative to the initial pixel position.

[0056] Each element of the filter kernel matrix may include filter weights for a pixel location (which may be a combination of one or more sub-weights as described herein, such as multiplication). The matrix can be applied to an image using convolution, where the data value of each pixel in the image corresponding to a pixel location in the matrix can be added to the data values ​​of pixels corresponding to local neighbors in the matrix, weighted by filter values ​​(also called filter weights). Filter values ​​may be configured to blur pixels, for example by fitting a distribution to the size of the filter kernel (e.g., fitting to width and height), or otherwise modeling the filter weights or values ​​using a light-based model.

[0057] The data values ​​to which the filter is applied may correspond to the lighting condition data of a pixel (e.g., before or after previous denoising). Therefore, the denoiser 108 applies the matrix of the filter kernel to pixels such that the lighting condition data is at least partially shared among these pixels corresponding to the pixel locations of the filter kernel. When the filter kernel accurately defines which pixels can share the lighting condition data (e.g., via the size of the matrix and the filter orientation) and how much lighting condition data can be shared (e.g., via the filter weights), sharing the lighting condition data can mitigate noise caused by sparse sampling of lighting conditions in ray tracing. Accordingly, if the size, filter orientation, and / or filter weights of the filter kernel do not accurately reflect the lighting conditions of the virtual environment, the filter kernel may result in excessive blurring and / or unrealistic blurring, leading to unrealistic lighting conditions represented in the rendered image.

[0058] Sample accumulator 104 can be configured to accumulate (e.g., temporally, relative to pixels) ray tracing samples (i.e., samples generated by ray tracing based on a virtual environment). The denoising methods described herein may employ sample accumulator 104 to store the accumulated samples in a history buffer and / or texture (e.g., relative to a specific pixel). For example, image renderer 102 may iteratively generate a ray tracing renderer, and in each iteration, sample accumulator 104 may accumulate one or more samples into the history buffer. Sometimes, sample accumulator 104 may reject one or more accumulated samples for use in a denoising iteration (and / or subsequent iterations) of a rendered frame. For example, accumulated samples for pixels and / or ray tracing samples may be rejected from input signals (e.g., ray tracing renderers) that the system determines do not exist in the history buffer (e.g., using projection to map ray tracing samples to accumulated samples).

[0059] By way of example and not limitation, when one or more pixel and / or ray-traced samples correspond to an occlusion event or a non-occlusion event (e.g., as determined by sample accumulator 104), the accumulated samples maintained in the history buffer by sample accumulator 104 may be rejected and not used for those pixel and / or ray-traced samples. Occlusion and / or non-occlusion events may occur as the camera or viewpoint used for rendering the virtual environment moves and / or objects in the virtual environment move, causing occlusion and / or non-occlusion events. In various embodiments, sample accumulator 104 may analyze rendering data from ray tracing to determine such events, and if an event is identified for one or more pixel and / or one or more accumulated samples (e.g., non-occlusion is detected), the number of successfully accumulated frames may be reset (e.g., reset to zero) and accumulation may resume from the beginning (e.g., for that one or more accumulated samples).

[0060] As examples, the sample accumulator 104 may determine the non-occlusion of a ray-traced sample or pixel based at least on a comparison between the corresponding current depth value (e.g., z-value) and the corresponding previous depth value (e.g., determined using reprojection). Additionally or alternatively, the non-occlusion of a ray-traced sample or pixel may be determined at least based on a comparison of planar distances. Other suitable methods may be employed in various embodiments.

[0061] Image renderer 102, sample accumulator 104, filter determiner 106, and denoiser 108 may be configured differently depending on one or more denoising strategies employed. For post-accumulation, sample accumulator 104 may accumulate samples after the denoiser 108 performs blurring or denoising pass-through on the samples. For pre-accumulation, sample accumulator 104 may accumulate samples before performing blurring or denoising pass-through on the accumulated samples. The input signal for post-accumulation or pre-accumulation may include ray-traced rendering and / or ray-traced samples of the virtual environment (optionally after preprocessing, such as pre-blurring). The input signal may have a very high frequency, especially if it is sparsely sampled (e.g., only one ray per pixel). The input may also be referred to as ray-traced output and / or denoising input. After blurring, the input signal may be converted to a lower frequency signal. In post-accumulation, high-frequency inputs may be blurred and then used to accumulate to a low-frequency texture, and in pre-accumulation, the reverse may occur.

[0062] Cyclic blur, or blurring, can mix signals with a wide frequency range. After blurring is performed by denoiser 108, the signal may have a lower frequency and can be fed into a feedback loop (optionally, after additional processing and / or blurring) as input to the next frame and / or rendering, to be accumulated by sample accumulator 104 along with subsequent high-frequency noisy inputs. It can also be referred to as ray tracing rendering with background blur, and this background can be accumulated along with the current incoming input signal from the ray tracing process performed by image renderer 102. Because cyclic blur feeds back to a blurred output (e.g., blurred accumulated samples or background), which is then used for blurring, this method may tend to over-blur. This can be employed... Figure 1 The process 100 is used to reduce or prevent excessive blurring that could result in a sharper output frame.

[0063] Adaptive fuzzy radius

[0064] At a higher level, process 100 may include: image renderer 102 generating one or more outputs, such as virtual environments (e.g., Figure 22 The process 100 may further include: a sample accumulator 104 receiving one or more inputs (such as accumulated renders from previous iterations of ray-traced rendering and denoising from process 100), accumulating one or more inputs, and generating one or more outputs (such as accumulated renders). The process 100 may further include: a filter determiner 106 receiving one or more inputs (e.g., a count of the number of samples accumulated to the pixels of the accumulated render), and generating one or more outputs (e.g., one or more sizes of one or more filters). The process 100 may further include: a denoiser 108 receiving one or more inputs (e.g., one or more sizes), and generating one or more outputs (e.g., accumulated renders denoised using one or more filters).

[0065] like Figure 1As shown, due to its recursive nature, process 100 can utilize cyclic blurring to efficiently and timely reallocate spatial samples. For example, after the denoiser 108 performs blurring of the accumulated samples, the signal may have a lower frequency and can be fed into feedback loop 140 (optionally, after additional processing and / or blurring) as input to sample accumulator 104 for the next frame and / or rendering to be accumulated by sample accumulator 104 with subsequent high-frequency noisy inputs from image renderer 102. Process 100 can utilize sample accumulator 104 to increase the effective sample count of ray-traced rendering generated by image renderer 102. As a non-limiting example, if the frames per second (FPS) of the frames rendered by image renderer 102 are 30, and ray tracing of the virtual environment is performed using 8 ray tracing samples per pixel, then accumulation can effectively yield 240 ray tracing samples per second due to repeatedly blurring the same textures (e.g., in the history buffer).

[0066] To reduce or eliminate excessive blurring that may be caused by cyclic blurring, aspects of this disclosure may provide cyclic blurring for rendering frames of a virtual environment, wherein filter determiner 106 determines the radius (or more generally, size, or dimensions) of at least one denoising filter for a pixel based at least on frames that have been successfully accumulated for that pixel (e.g., temporally accumulated samples in a history buffer). For example, sample accumulator 104 may track the number of frames successfully accumulated per pixel and may be used to control or adjust the blurring size of the denoising filter applied by denoiser 108 to those pixels (e.g., in one or more spatial filter passes).

[0067] The disclosed method can reduce the size of the blur filter for a pixel based on the accumulated contribution to the final illumination integral of the resolved pixel. This allows the filter size to decrease over several denoising iterations as the contribution (e.g., the number of samples accumulated in the history buffer) increases. If the history buffer rejects a pixel, the accumulated contribution for that pixel can be eliminated for at least one iteration, and the size of the corresponding blur filter can be increased (e.g., without considering other potential factors that could affect the filter size).

[0068] In at least one embodiment, sample accumulator 104 may increment one or more counters to track the number of frames and / or samples successfully accumulated (e.g., per pixel). For example, the counters may be incremented for each iteration of a denoising iteration, each accumulation, each frame, or rendering (e.g., once for each successful accumulation for a pixel). If sample accumulator 104 rejects one or more accumulated samples for use in a denoising iteration, the counts may be reset (e.g., reset to zero) and accumulation may resume from the beginning (e.g., for that one or more accumulated samples). For example, the counts may be reset for pixels and / or ray tracing samples from input signals that sample accumulator 104 determines do not exist in the history buffer (e.g., ray tracing rendering) (e.g., using projection to map ray tracing samples to accumulated samples).

[0069] Figure 1 Image 150 provides a visualization of the number of frames successfully accumulated for a pixel. Darker pixels may correspond to fewer samples or frames accumulated for that pixel, and brighter pixels may correspond to more samples or frames accumulated for that pixel. For example, a black pixel may have a minimum count indicating frames that were not successfully accumulated (e.g., due to an occlusion event or a non-occlusion event), and a white pixel may have a maximum count indicating the maximum number of frames accumulated. By way of example and not limitation, the count may range from 0 to 32.

[0070] In at least one embodiment, the count can be used to calculate an adjustment factor, which is then used to calculate one or more dimensions of the filter (e.g., blur radius). The adjustment factor may correspond to the result of an equation or part thereof that uses the count as a variable. The adjustment factor may be proportional to or otherwise related to the number of accumulated frames and / or samples, and may be calculated per pixel. In some embodiments, the number of accumulated frames corresponding to one or more samples may be constrained to a maximum number of accumulated frames (e.g., 32). This number may be controlled by the end user and / or software and / or may be dynamic. Reducing the number of accumulated frames reduces lag. In at least one embodiment, the radius of the filter for a pixel may be maximum at reset (e.g., count = 0) and minimum when the maximum number of accumulated frames is reached (e.g., count = 32). As an example, the radius scale for a pixel may be calculated using equation (1):

[0071] 1 / (1+newCount), (1)

[0072] Where newCount can refer to the updated count of pixels in the denoising iteration after accumulation.

[0073] Now for reference Figure 2A and Figure 2B , Figure 2AThe present disclosure illustrates at least one embodiment that can be used after two cumulative frames. Figure 1 Example process 100 generates a virtual environment rendering 200A. Figure 2B The present disclosure illustrates at least one embodiment that can be used after thirty-two cumulative frames. Figure 1 The example process 100 generates a virtual environment rendering 200B. Although the count and filter size can vary per pixel, renderings 200A and 200B are examples where each pixel has the same count. As can be seen, in the case of two cumulative frames, rendering 200A has less detail than rendering 200B because the denoiser 108 uses a filter with a larger blur radius when blurring the rendering data to generate rendering 200A.

[0074] See now Figure 3-5 Each block of methods 300, 400, and 500, as well as other methods described herein, includes a computational process that can be performed using any combination of hardware, firmware, and / or software. For example, various functions can be implemented by a processor executing instructions stored in memory. The methods can also be embodied as computer-usable instructions stored on a computer storage medium. To name just a few examples, the methods can be provided by a standalone application, service, or managed service (alone or in combination with other managed services) or a plug-in to another product. Furthermore, by way of example, regarding... Figure 1 The system component description methods are as follows. However, these methods may be implemented additionally or alternatively by any system or any combination of systems, including but not limited to those described herein.

[0075] Figure 3 This is a flowchart illustrating an example of a method 300 for determining one or more parameters of a filter based at least on the number of frames rendered, according to at least one embodiment of the present disclosure. At block B302, method 300 includes: accumulating ray-traced samples to generate a second accumulated sample using at least one sample generated at least based on denoising a first accumulated sample. For example, sample accumulator 104 may accumulate ray-traced samples from a virtual environment of image renderer 102 using at least one sample generated at least based on denoising a first accumulated sample of the virtual environment by denoiser 108. This at least one sample may be provided from a previous iteration of process 100 by feedback loop 140, and the accumulation may generate a second accumulated sample to be provided to denoiser 108 in the current iteration of process 100.

[0076] At box B304, method 300 includes determining one or more parameters of a filter based at least on the number of rendered frames corresponding to the second accumulated samples. For example, filter determiner 106 may determine the blur radius of the filter based at least on the number of rendered frames (or samples) corresponding to the second accumulated samples.

[0077] In box B306, method 300 includes: generating a rendered frame based at least on samples blurred by a filter using a second accumulated sample. For example, the rendered frame may be generated based at least on samples blurred by a filter with a blur radius using a denoiser 108 using a filter with a blur radius.

[0078] Now for reference Figure 4 , Figure 4 This is a flowchart illustrating an example of a method 400 for iteratively determining one or more parameters of a filter based at least on the number of renderings of a virtual environment corresponding to accumulated samples, according to at least one embodiment of the present disclosure.

[0079] At box B402, method 400 includes: accumulating ray-traced samples using samples corresponding to samples from previous iterations to generate accumulated samples. For example, sample accumulator 104 can use samples from previous iterations of process 100 (provided by feedback loop 140) to accumulate ray-traced samples from the virtual environment of image renderer 102 to generate accumulated samples for the current iteration of process 100.

[0080] At box B404, method 400 includes determining one or more parameters of a filter based at least on the number of renders corresponding to at least one sample in the accumulated samples. For example, filter determiner 106 may determine the blur radius of the filter based at least on the number of renders corresponding to at least one sample in the accumulated samples.

[0081] At box B406, method 400 includes generating samples for subsequent iterations, at least based on samples accumulated using a filter blur. For example, samples for subsequent iterations may be generated at least based on samples accumulated using a filter blur with a blur radius by denoiser 108. In some examples, the samples may include the output of denoiser 108. However, in at least one embodiment, further processing may be performed after blurring by denoiser 108 to generate samples.

[0082] See now Figure 5 , Figure 5 This is a flowchart illustrating an example of a method 500 for determining one or more parameters of a filter based at least on the number of renders of a virtual environment corresponding to at least one sample of accumulated renders, according to at least one embodiment of the present disclosure.

[0083] At box B502, method 500 includes generating an accumulated render corresponding to a ray-traced render accumulated using a denoised accumulated render. For example, sample accumulator 104 may generate an accumulated render corresponding to a ray-traced render of a virtual environment from image renderer 102, the accumulated render being accumulated using a denoised accumulated render of the virtual environment generated using denoiser 108 in a previous iteration of process 100.

[0084] At box B504, method 500 includes determining one or more parameters of a filter based at least on the number of renders corresponding to at least one sample in the accumulated renders. For example, filter determiner 106 may determine the blur radius of the filter based at least on the number of renders corresponding to at least one sample in the accumulated renders.

[0085] At box B506, method 500 includes: generating a rendered frame based at least on a render using filter blur accumulation. For example, a rendered frame may be generated based at least on a render using filter blur accumulation with a blur radius by denoiser 108.

[0086] Hierarchical historical reconstruction

[0087] This disclosure also provides a hierarchical approach to interpreting the rejection of accumulated samples relative to a pixel. Typically, when a temporally accumulated sample for a pixel is rejected by the sample accumulator 104 (e.g., due to non-occlusion of the pixel during ray tracing performed by the image renderer 102), the temporal accumulation for that pixel is reset. Consequently, the effective sample count for a pixel can decrease over several frames because the temporal data is re-accumulated by the sample accumulator 104. This can lead to artifacts in the rendered frames, especially when the number of rays per pixel that the image renderer 102 uses to sample the virtual environment for that pixel is low.

[0088] The disclosed method can supplement the effective number of rays per pixel used for sampling the virtual environment pixel-wise, at least based on the number of frames corresponding to the successful accumulation of pixels. See now. Figure 6A and Figure 6B , Figure 6B This is a flowchart illustrating a process 600A for denoising using cyclic blur according to at least one embodiment of the present disclosure, the process 600A including generating enhanced samples based at least on the number of renderings of a virtual environment corresponding to accumulated ray-traced samples in the rendering. Figure 6B This is a flowchart illustrating a process 600B for denoising using pre-accumulation according to at least one embodiment of the present disclosure, the process 600B including generating enhanced samples based at least on the number of renderings of a virtual environment corresponding to the accumulated ray-traced samples in the rendering.

[0089] In processes 600A and 600B, the sample accumulator 104 can use ray tracing samples from a lower-resolution version of the ray tracing render, instead of only using ray tracing samples from the ray tracing render used for accumulation per pixel. At a higher level, process 600A may include an image renderer 102 generating one or more outputs, such as virtual environments (e.g., Figure 22 Ray tracing rendering in a virtual environment 2200. Process 100 may also include a sample accumulator 104 receiving one or more inputs (such as ray tracing rendering and denoised accumulated rendering) from previous iterations of process 600A, accumulating one or more inputs, and generating one or more outputs (such as accumulated rendering).

[0090] Process 600A may also include a resolution manager that receives one or more inputs (e.g., an accumulated render and a count of the number of samples accumulated to the accumulated render pixels) and generates one or more outputs (e.g., one or more samples augmented with at least one sample corresponding to the resolution level of the ray-traced render). Process 100 may further include: a filter determiner 106 receiving one or more inputs and generating one or more outputs, such as one or more parameters of one or more filters. Process 100 may further include: a denoiser 108 receiving one or more inputs (e.g., one or more parameters) and generating one or more outputs (e.g., an accumulated render denoised using one or more filters).

[0091] Process 600B may be similar to process 600A, except that sample accumulator 104 may accumulate samples before performing blurring or denoising propagation on the accumulated samples.

[0092] Due to the rejection of accumulated data, the history buffer may not include any available data for a pixel. Using the disclosed method, resolution manager 602 can apply a coarse resolution (e.g., the highest mipmap level) to pixels in iterations of process 600A or 600B. Resolution manager 602 can incrementally increase the resolution in each iteration (e.g., until a base-level resolution is reached) based on the number of frames and / or samples successfully accumulated for a pixel. Initially, resolution manager 602 can use corresponding composite samples of the pixel in the history buffer (e.g., together with unrejected accumulated samples of other pixels) to enhance the accumulated data in denoising iterations. After successful accumulation for a pixel, resolution manager 602 can use the corresponding composite samples of that pixel in conjunction with the corresponding accumulated data in the history buffer for that pixel to enhance the accumulated data in denoising iterations.

[0093] In various embodiments, resolution manager 602 may calculate a resolution level (e.g., mipmap level) for one or more samples and / or pixels based at least on the number of successfully accumulated frames, and may increase it as the number of successfully accumulated frames or samples increases (e.g., until a base resolution for ray tracing rendering is reached). For example, as described herein, sample accumulator 104 may track the number of successfully accumulated frames or samples (e.g., per pixel), which resolution manager 602 may use to control or adjust the resolution of ray tracing rendering from image renderer 102. In at least one embodiment, resolution manager 602 may additionally or alternatively use the surface roughness of a pixel to determine the resolution level of that pixel. For example, a resolution level may be calculated such that the resolution decreases as surface roughness increases.

[0094] Resolution manager 602 can combine or replace one or more portions of the temporally accumulated data from sample accumulator 104 (e.g., before or after one or more spatial filter passes) rendered using ray tracing at resolution. This approach can be used in the cyclic blurring method of process 600A, which may or may not adjust the denoising filter size based at least on the number of successfully accumulated frames (e.g., according to process 100). Other denoising strategies, such as post-accumulation or pre-accumulation, are also contemplated for implementation. For example, process 600B illustrates these methods for a pre-accumulation implementation.

[0095] In at least one embodiment, for example, regarding Figure 7A and Figure 7B This describes one or more mipmaps (texture maps) used to implement rendering versions at different resolution levels. Now refer to... Figure 7A and Figure 7B , Figure 7A Examples of rendering virtual environments at different mipmap levels according to at least one embodiment of the present disclosure are shown. Figure 7B Examples of mipmap levels MIP 0 to MIP 4 and alignment between adjacent mipmap levels are shown according to at least one embodiment of the present disclosure.

[0096] For example, in Figure 7A and Figure 7B In this context, MIP 0 can refer to the base resolution for ray tracing rendering, which can be output by the image renderer 102. As the mip level increases, the rendering resolution decreases. Figure 7BThe stacking 700 illustrates how the corresponding mipmaps can be visualized as a stack of overlapping images. An example of alignment 702 between adjacent resolutions / mip levels is shown on the right. According to various embodiments of this disclosure, any number of mipmaps and / or resolution levels can be used. In at least one embodiment, a mipmap can be generated for both radiance and view Z (which captures the Z coordinates of pixels in world space), where the view Z delta can be used to compute weights during upsampling.

[0097] In various embodiments, the resolution manager 602 may use a count of successfully accumulated frames or samples for one or more pixels to calculate an adjustment factor, which is then used to calculate the resolution level of the one or more pixels. This count may be the same count described with respect to process 100 and tracked by the sample accumulator 104, or it may be a different count. The adjustment factor may correspond to, for example, the result of an equation or part thereof that uses the count as a variable. The adjustment factor may be proportional to or otherwise related to the number of accumulated frames and / or samples, and may be calculated per pixel.

[0098] As an example, equation (2) can be used to calculate the mip level of a pixel:

[0099] MaxLevel*roughness*max(1–newCount / N,0), (2)

[0100] Roughness can refer to the surface roughness value of a pixel, and MaxLevel can refer to the renderer (e.g., ...). Figure 7A The lowest resolution mipmap level in MIP 4), and N can refer to multiple frames with active history reconstruction.

[0101] Using the disclosed method, resolution manager 602 can weigh the resolution of rays per pixel to increase the effective sample count of a pixel. Given, for example but not limited to, a base resolution at a MIP or a resolution level 0 corresponding to a ray per pixel (RPP), at MIP 1, resolution manager 602 can use a 2x2 pixel region for rendering a pixel to increase the effective sample count to 4 RPPs. At MIP 2, resolution manager 602 can use a 4x4 pixel region for rendering a pixel to increase the effective sample count to 16 RPPs. At MIP 3, resolution manager 602 can use an 8x8 pixel region for rendering a pixel to increase the effective sample count to 64 RPPs. Further, at MIP 4, resolution manager 602 can use a 16x16 pixel region for rendering a pixel to increase the effective sample count to 256 RPPs.

[0102] The resolution manager 602 can downsample the ray-traced rendering for iteration (e.g., by shrinking it in all directions by dividing it by 2) to generate any number of different resolutions, and the corresponding samples can be applied to the base resolution pixels in the denoising iteration. In some embodiments, those samples can be upsampled (e.g., using bilinear, bilateral, or higher-order upsampling) or otherwise combined to generate corresponding composite samples for the base resolution samples.

[0103] Now for reference Figure 8A and Figure 8B , Figure 8A A rendering 800A of a virtual environment that can be generated without enhanced accumulated samples according to at least one embodiment of the present disclosure is shown. Figure 8B A rendering 800B of a virtual environment that can be generated by enhancing accumulated samples according to at least one embodiment of the present disclosure is shown. For example, rendering 800B can be generated using process 600A or 600B.

[0104] See now Figure 9 , Figure 9 This is a flowchart illustrating an example of a method 900 for enhancing an accumulated rendering based at least on the number of renders corresponding to ray-traced samples of the virtual environment in an accumulated rendering of the virtual environment, according to at least one embodiment of the present disclosure. At block B902, method 900 includes: calculating a resolution level based at least on the number of renders corresponding to the number of pixels of ray-traced rendering in the accumulated rendering. For example, resolution manager 602 may calculate the resolution level (e.g., mipmap level) based at least on the number of renders corresponding to ray-traced samples of the virtual environment in the accumulated rendering of the virtual environment. The accumulated rendering may be generated using sample accumulator 104.

[0105] At box B904, method 900 includes: generating a rendered frame based at least on a rendering accumulated with at least one sample corresponding to the resolution level. For example, resolution manager 602 may generate enhanced samples based at least on a rendering accumulated with at least one sample corresponding to the resolution level and ray tracing samples.

[0106] At box B906, method 900 includes generating a rendered frame based at least on data corresponding to enhanced samples that are blurred using one or more filters. For example, the rendered frame may be generated based at least on data corresponding to enhanced samples that are blurred using one or more filters by denoiser 108.

[0107] See now Figure 10 , Figure 10This is a flowchart illustrating an example of a method 1000 for iteratively augmenting an accumulated sample set according to at least one embodiment of the present disclosure, based at least on the number of renders corresponding to at least one ray-traced sample of a virtual environment. At block B1002, method 1000 includes: accumulating ray-traced samples with samples from previous iterations to generate an accumulated sample set. For example, sample accumulator 104 may accumulate ray-traced samples from a virtual environment of image renderer 102 with samples from previous iterations across a plurality of iterations, wherein this accumulation will be used to generate the accumulated sample set.

[0108] At box B1004, method 1000 includes: calculating a resolution level based at least on the number of renders corresponding to at least one sample in the ray-traced samples. For example, resolution manager 602 may calculate the resolution level based at least on the number of renders corresponding to at least one sample in an accumulated set of samples.

[0109] At box B1006, method 1000 includes: augmenting an accumulated set of samples with one or more samples corresponding to a resolution level. For example, resolution manager 602 may augment the accumulated set of samples with one or more samples corresponding to a resolution level.

[0110] At box B1008, method 1000 includes generating a rendering based at least on data corresponding to an enhanced set of samples that has been blurred using one or more filters. For example, the rendering may be generated based at least on data corresponding to an accumulated set of samples enhanced with one or more samples that has been blurred using one or more filters by denoiser 108.

[0111] See now Figure 11 , Figure 11 This is a flowchart illustrating an example of a method 1100 for enhancing an accumulated rendering based at least on the number of renders corresponding to pixels in an accumulated rendering of a virtual environment, according to at least one embodiment of the present disclosure. At block B1102, method 1100 includes: calculating a resolution level based at least on the number of renders corresponding to pixels in an accumulated rendering of ray-traced renderings. For example, resolution manager 602 may calculate the resolution level based at least on the number of renders corresponding to pixels in an accumulated rendering of a virtual environment generated using image renderer 102.

[0112] In box B1104, method 1100 includes: enhancing the accumulated rendering with at least one sample corresponding to a pixel and a resolution level. For example, resolution manager 602 may enhance the accumulated rendering with at least one sample corresponding to a pixel and a resolution level to generate an enhanced accumulated rendering.

[0113] In box B1106, method 1100 includes generating a rendered frame based at least on data corresponding to an enhanced, accumulated render using one or more filters. For example, the rendered frame may be generated based at least on data corresponding to an enhanced, accumulated render using at least one sample, blurred by a denoiser 108 using one or more filters.

[0114] See now Figure 12 , Figure 12 This is a data flow diagram illustrating an example process 1200 for denoising the rendering of a virtual environment according to at least one embodiment of the present disclosure. Process 1200 can implement adaptive blur radius according to process 100 and history reconstruction according to process 600A.

[0115] As an example, process 1200 can be used for diffuse or mirror input signals and employ cyclic blurring. Further, as shown, process 1200 may include pre-blurring 1202 and post-blurring 1210. As an example, pre-blurring 1202, blurring 1208, and post-blurring 1210 can be based on Poisson disk sampling, but other types can be used. Pre-blurring 1202 can use a constant filter size, while post-blurring 1210 can be adaptive. Post-blurring 1210 may also consider the strength of the input signal after accumulation and the strength of the signal after blurring. Filter determiner 106 can compare these values, and the filter size (e.g., radius) can be based at least on the difference between these values. For example, the size can increase as a function of this difference, since a large difference may indicate a lack of convergence.

[0116] Process 1200 may include: one or more passed pre-blurs 1202 being applied to ray tracing rendering of the virtual environment (e.g., from image renderer 102). For example, the pre-blurs may use a constant or adaptive radius filter and may be used to resolve outliers in the samples of the ray tracing rendering. Process 1200 may also include: the pre-blurred ray tracing rendering being accumulated 1204 with denoised accumulated rendering from feedback loop 1240 (e.g., using sample accumulator 104). For example, accumulation 1204 may be performed using linear weights and may accumulate up to 32 or some other maximum number of frames.

[0117] Process 1200 may further include: performing history reconstruction from accumulated data (e.g., using resolution manager 602) and radius determination 1206 (e.g., using filter determiner 106). This may include: generating a mipmap and using the mipmap to augment the accumulated samples (e.g., in a history buffer used for accumulation). In at least one embodiment, the mipmap may be generated in a single pass in shared memory, e.g., using averaging. History reconstruction may be performed for each pixel of accumulated sample data that has been discarded or otherwise insufficient.

[0118] The process may further include: blurring 1202 of the enhanced accumulated samples (e.g., using denoiser 108). Blur 1202 may be determined at least based on the radius of one or more filters having corresponding radii. Process 1200 may additionally include: post-blurring 1210 of the blurred and enhanced accumulated samples (e.g., using denoiser 108). Post-blurring 1210 may use the same radius or a different radius determined for blur 1202. In at least one embodiment, the radius of the filter used by post-blurring 1210 may be based at least on the number of rendered frames corresponding to the pixel and the intensity increment between the pixel and the reprojection history from the historical reconstruction. The process may further include: temporal stabilization 1212 of the post-blurred rendering, which may utilize temporal anti-aliasing (TAA) with potentially wider variance clamping.

[0119] Noise reduction filter size

[0120] Now for reference Figure 13A and Figure 13B , Figure 13A This is an illustration of an example of determining a blur plane 1302A for specular reflection according to at least one embodiment of the present disclosure. Figure 13B This is an illustration of an example of determining a blur plane 1302B for a diffuse signal according to at least one embodiment of the present disclosure.

[0121] In at least one embodiment, the filter determiner 106 can use Figure 13A and Figure 13B The method shown determines the blur plane of the filter in world space. However, other methods can be used. Since diffuse signals can cover the entire hemisphere of the sampling space, the blur plane 1302B of point p can lie in a tangent plane constructed around the plane normal N. For specular signals, at least for near-spectral reflections, the blur plane 1302A can be constructed in a plane perpendicular to the view vector V, where... Figure 13A The vector R in the image can be the reflection of the inverse of the view vector V onto the normal vector N of surface 1306. This can actually be very close to screen space, such as... Figure 13A The screen plane 1304 is shown in the figure.

[0122] In some embodiments, the blurred plane may be based at least in part on the direction (e.g., the dominant direction) of the specular lobes associated with the view vector and the corresponding surface normal, as per [reference to...]. Figure 14A Further description. Figure 14A This is a diagram illustrating an example of determining a blur plane 1402 for denoising ray-traced samples according to at least one embodiment of the present disclosure. For example, Figure 14A This method can be used for mirror and / or diffuse signals.

[0123] Figure 14A The blurred plane 1402 for a filter targeting point p (e.g., the kernel center in world space) is shown to be at least partially based on the dominant direction 1404 of the GGX NDF, but other types of NDFs may be employed in various embodiments. As an example, the direction of the specular lobe can be calculated by trimming values ​​(e.g., random values) from the NDF importance samples, such as the visible normal distribution function (VNDF). This direction can be used, for example, to provide a generalized approach (e.g., compatible with both specular and diffuse signals) for defining the blurred plane 1402 perpendicular to the view vector, and more specifically perpendicular to the specular reflection direction (e.g., the dominant direction) 1406. In the case of a diffuse signal, if the surface roughness of surface 1306 is 0, the specular reflection direction 1406 can be the normal vector N, and the specular reflection direction 1406 (vector) around the normal remains the same normal vector N; therefore, the tangent plane constructing this case corresponds to the diffuse signal. The same approach can be used for near-spectral reflections.

[0124] This disclosure provides methods for calculating one or more dimensions of an anisotropic denoising filter (e.g., filter kernel) applied to a pixel when denoising ray-traced rendering. For example, filter determiner 106 may determine the magnitude of one or more filter dimensions based at least on the angle corresponding to the view vector of the pixel, rather than applying the denoising filter to a pixel that is actually isotropic in world space. This can be used to extend reflections along the axis at grazing angles, resulting in more realistic rendering. This disclosure provides for calculating anisotropic filter kernels that can be extended such that the dimensions of the filter kernel extend further along a first axis than along a second axis to more closely reflect the space occupied by BRDF lobes in the virtual environment. In various examples, one or more dimensions may also be based at least on the direction (e.g., dominant direction) of the specular reflection lobes and the corresponding surface normals associated with the view vector.

[0125] See now Figure 14B , Figure 14BThis is an illustration showing an example of determining an anisotropic filter for denoising ray-traced samples according to at least one embodiment of the present disclosure. Figure 14B An example is shown in which the anisotropic filter can be at least partially based on the dominant direction 1404 of the GGX NDF, but other types of NDFs may be used in various embodiments. At the grazing angle, the reflection extends along a certain axis. The disclosed method can be used to determine how much and how long the effect should be and how to find this axis.

[0126] As described herein, sampling can be defined on a blur plane 1402, which is perpendicular to the specular reflection direction (e.g., the dominant direction) 1406. The resulting filter can be anisotropic in screen space and isotropic in world space. Filter determiner 106 can construct the basis of the filter around the specular reflection direction 1406 using a tangent vector T and a basis vector B to define the filter direction and / or radius. When a generalized basis is used, vectors T and B can have the same unit length. The tangent vector T can be computed from the normal vector N and the specular reflection direction 1406 (e.g., at least based on the normalized cross product of these vectors). The basis vector B can be computed from the specular reflection direction 1406 and the tangent vector T (e.g., at least based on the cross product of the vectors). Vectors T and B can define an isotropic filter basis. To simulate reflection effects at grazing angles, the tangent vector T can be scaled at least based on the view vector V. In some embodiments, the surface roughness of point p can also be used as a scaling factor because diffuse or specular signals do not experience elongation when they approach diffuse signals due to the expansion of the hemisphere of the sampling space. In at least one embodiment, the tangent vector can be scaled and can be perpendicular to the surface normal, such that the elongation occurs along the surface normal.

[0127] See now Figure 15 , Figure 15 This is a flowchart illustrating an example of a method 1500 for calculating the size of a filter based at least on the direction, normal, and angle of a view vector of a specular reflection lobe, according to at least one embodiment of the present disclosure. At block B1502, method 1500 includes generating ray-tracing samples based at least on interactions in a virtual environment. For example, image renderer 102 may generate at least one ray-tracing sample based at least on the interaction between a view vector V in the virtual environment and a point p.

[0128] At box B1504, method 1500 includes calculating the size of a filter based at least on the direction of the specular reflection lobes of the interaction and the normal associated with the interaction, wherein the magnitude of the size is based at least on the angle of the view vector associated with the interaction. For example, filter determiner 106 may calculate the size of an anisotropic denoising filter (e.g., along the tangent vector T) based at least on the specular reflection direction of the interaction and the normal vector N corresponding to a point p in the virtual environment, wherein the magnitude of the size is based at least on the angle of the view vector V.

[0129] In box B1506, method 1500 includes: generating a rendered frame using a filter. For example, the rendered frame may be generated at least based on applying an anisotropic denoising filter to data corresponding to the at least one ray-traced sample, using denoiser 108.

[0130] See now Figure 16 , Figure 16 This is a flowchart illustrating an example of a method 1600 for determining an anisotropic filter based at least on an extended isotropic filter basis according to at least one embodiment of the present disclosure, the extended isotropic filter basis being based at least on the angle of the view vector. At block B1602, method 1600 includes calculating the isotropic filter basis based at least on the normal of the interaction between the direction of the specular reflection lobe and the view vector in the virtual environment. For example, filter determiner 106 may calculate the isotropic filter basis (e.g., including the tangent vector T and the basis vector B) based at least on the normal vector N of the interaction between the specular reflection direction 1406 and the view vector V in the virtual environment.

[0131] At box B1604, method 1600 includes determining an anisotropic filter based at least on extending the isotropic filter basis, the extension of the isotropic filter basis being based at least on an angle of a view vector associated with the interaction. For example, filter determiner 106 may determine the anisotropic filter based at least on extending the isotropic filter basis (e.g., tangent vector T), the extension of the isotropic filter basis being based at least on an angle of the view vector V.

[0132] In box B1606, method 1600 includes generating rendered frames using an anisotropic filter. For example, rendered frames of a virtual environment can be generated by filtering rendered data corresponding to interactions using an anisotropic filter, at least based on denoiser 108.

[0133] See now Figure 17 , Figure 17This is a flowchart illustrating an example of a method 1700 for calculating the size of a filter based at least on the direction, normal, and angle of a ray associated with an interaction of at least one embodiment of the present disclosure. At block B1702, method 1700 includes determining the size of the filter based at least on the direction of a specularly reflected lobe of an interaction of rays in a virtual environment, the normal of that interaction, and the angle of the ray. For example, filter determiner 106 may determine the size of the filter for point p based at least on the specular reflection direction 1406 of the interaction of a view vector V in the virtual environment at point p.

[0134] At box B1704, method 1700 includes: generating rendered frames using a filter. For example, rendered frames of a virtual environment can be generated by blurring rendered data corresponding to interactions, at least based on denoiser 108, using a filter.

[0135] cumulative velocity based on parallax

[0136] Aspects of the invention provide the ability to control the accumulation rate of temporal accumulation (e.g., for specular accumulation) using parallax between frames or renders. For example, surface motion can be used to reproject specular historical textures from a previous frame or render to a subsequent (e.g., current) frame or render. However, when light reflects from a surface, the motion of the reflection may differ from that surface motion, and therefore using surface motion alone can result in artifacts in the rendered frames. The accuracy of the surface motion of a pixel can be based at least in part on the parallax between frames, and thus can be used to control the effect of surface motion in temporal accumulation.

[0137] In various embodiments, the disparity used to calculate the cumulative velocity of a pixel can be based at least on the angle between view vectors corresponding to the pixel. For example, disparity can correspond to the angle between a previous view vector and a current view vector for the same surface point. In at least one embodiment, the disparity of a pixel can be calculated as the ratio between the camera motion projection of one or more points for rendering a screen plane or between frames and the distance to one or more points. The cumulative velocity can also be based at least on the viewpoint associated with the view vectors rendered by ray tracing, which can take into account how disparity is visually perceived at different viewpoints. These methods can be used in combination with cyclic blur, post-accumulation, pre-accumulation, and / or other denoising strategies.

[0138] See now Figure 18 , Figure 18 A rendering 1800 of a virtual environment, according to at least one embodiment of the present disclosure, is shown, having view vectors 1802A, 1802B, 1804A, and 1804B that can be used to calculate parallax. Figure 18In the diagram, view vectors 1802A and 1802B are examples of view vectors for the rendering state of the current frame and / or virtual environment, and view vectors 1804A and 1804B are examples of view vectors for the rendering state of a previous frame and / or virtual environment. View vectors 1802A and 1804A can be used to calculate the disparity of corresponding pixels, and view vectors 1802B and 1804B can be used to calculate the disparity of different corresponding pixels. In at least one embodiment, the disparity can be calculated at least based on the angle between the current view vector and the previous view vector (e.g., view vectors 1802A and 1804A or view vectors 1802B and 1804B). For example, the disparity of a pixel can be calculated at least in part as the ratio between the camera motion projection of a point to a rendering of the screen plane or between frames and the distance to said point. The camera motion of view vectors 1802A and 1804A may include a world space vector (as shown) between corresponding camera positions 1810A and 1810B, which can be projected onto the screen plane to determine the camera motion projection. In at least one embodiment, the view Z coordinates of the pixels in the current frame can be used as the distance to the point.

[0139] In various embodiments, the sample accumulator 104 can calculate disparity in world space or view space, and this disparity can be applied to one or more corresponding pixels to determine the accumulation rate. The disparity may be larger for pixels closer to the camera. For example, the disparity may be larger for pixels corresponding to view vectors 1802A and 1804A, and further for pixels corresponding to view vectors 1802B and 1804B. If the disparity is small, the actual accumulation determined by the sample accumulator 104 is more or less accurate based on surface motion. Therefore, the disparity can be used by the sample accumulator 104 to determine the accumulation rate for accumulation at the pixel level.

[0140] In at least one embodiment, the sample accumulator 104 may determine the accumulation rate based at least on parallax and on the dot product between the current view vector and previous view vectors (e.g., view vectors 1802A and 1804A or view vectors 1802B and 1804B). Surface roughness at the current point may also be a factor. If the roughness is close to 1, reflection can be decomposed into a diffuse signal, and the diffuse signal can always lie on the surface, such that, using a history buffer, accumulation can approach 100%. Regarding the viewing angle, the view is most sensitive to parallax when viewing the surface at a grazing angle. When viewed from above on the surface, the view vector is perpendicular to the surface, and the effect of parallax decreases. Accordingly, the viewing angle can be used as a measure of parallax sensitivity.

[0141] See now Figure 19 , Figure 19This is a flowchart illustrating an example of a method 1900 for calculating an accumulation velocity based at least on an angle between view vectors and a viewpoint associated with the view vectors, according to at least one embodiment of the present disclosure. At block B1902, method 1900 includes calculating an accumulation based at least on an angle between a first view vector and a second view vector and at least on a viewpoint associated with the first view vector. For example, sample accumulator 104 may calculate an accumulation velocity for at least one pixel based at least on an angle between a view vector 1802A corresponding to at least one pixel and a view vector 1804A corresponding to at least one pixel and at least on a viewpoint associated with view vector 1802A. View vector 1802A may be used by image renderer 102 to generate a first ray-traced render of a virtual environment, and view vector 1804A may be used by image renderer 102 to generate a second ray-traced render of the virtual environment.

[0142] At box B1904, method 1900 includes: using an accumulation rate to accumulate one or more samples to generate an accumulated render of the virtual environment. For example, sample accumulator 104 may use an accumulation rate for at least one pixel to accumulate one or more samples with a first accumulated render of the virtual environment to generate a second accumulated render of the virtual environment.

[0143] At box B1906, method 1900 includes: generating rendered frames using accumulated rendering. For example, a second accumulated rendering may be used to generate rendered frames.

[0144] See now Figure 20 , Figure 20 This is a flowchart illustrating an example of a method 2000 for accumulating ray-traced renderings using an accumulation velocity according to at least one embodiment of the present disclosure, the accumulation velocity being based at least on the parallax between a view and a perspective captured by the ray-traced rendering. At block B2002, method 2000 includes: calculating the accumulation velocity based at least on the parallax between ray-traced renderings of a virtual environment and at least on the perspective associated with one or more ray-traced renderings. For example, sample accumulator 104 may calculate the accumulation velocity based at least on the parallax between ray-traced renderings of a virtual environment and at least on the perspective associated with one or more ray-traced renderings (e.g., view vectors 1802A and / or 1802B).

[0145] In box B2004, method 2000 includes: using an accumulation rate to accumulate one or more samples to generate an accumulated render of the virtual environment. For example, sample accumulator 104 may use the accumulation rate to accumulate one or more samples to a first accumulated render of the virtual environment to generate a second accumulated render of the virtual environment.

[0146] In box B2006, method 2000 includes: using the second accumulated rendering to generate a rendered frame.

[0147] Now for reference Figure 21 , Figure 21 This is a flowchart illustrating an example of a method 2100 for calculating an accumulation rate based at least on the parallax between renders and the viewpoints associated with one or more renders, according to at least one embodiment of the present disclosure. At block B2102, method 2100 includes: accumulating ray-traced renders using an accumulation rate based at least on the parallax between views and the viewpoints captured by ray-traced renders. For example, sample accumulator 104 may accumulate ray-traced renders of a virtual environment using a pixel-based accumulation rate based at least on the parallax between views of a virtual environment represented by ray-traced renders and the viewpoints of the virtual environment captured by ray-traced renders.

[0148] In box B2104, method 2100 includes: generating one or more rendered frames using the data generated through accumulation.

[0149] Examples of ray tracing technology

[0150] See now Figure 22 , Figure 22 This is an illustration of an example of a ray tracing technique that an image renderer 102 can be used to generate ray tracing rendering, according to some embodiments of the present disclosure. Figure 22 A virtual environment 2200 is shown, including a camera 2202, a screen 2204, an object 2206, and a surface 2208.

[0151] Screen 2204 may be a virtual representation of a screen, which may or may not be the same or different resolution as the rendered frame or intermediate image or rendering described herein. Screen 2204 may include a matrix of virtual pixels or regions, whose pixels 2212 are individually labeled. To determine at least some lighting condition data for pixel 2212, image renderer 102 may project any number of rays (e.g., one or more) (such as ray 2214) through pixel 2212 of screen 2204 to sample the lighting conditions of pixel 2212. As an example, these rays may be referred to as camera rays, eye rays, incident rays, view vectors, or master rays.

[0152] Image renderer 102 can use camera rays to determine visible points in the environment that may be affected by object 2206 and / or at least one light source. For example, image renderer 102 can use ray 2214 to determine point 2216 (e.g., point p) on or near surface 2208. This can include the image renderer determining point 2216 as the location where ray 2214 intersects surface 2208 (or point 2216 may otherwise be based at least on that location). Although ray 2214 intersects surface 2208, in examples where more than one ray is projected, not all rays may intersect the surface.

[0153] From each point in the virtual environment 2200 determined by the image renderer 102 using rays projected through pixel 2212, any number of rays (e.g., one or more) (such as ray 2218) can be projected to model the reflection of ray 2214 at point 2216. The image renderer 102 can determine the direction of ray 2218 based at least on the normal of surface 2208 at point 2216. For example, the image renderer can define the NDF range 2220 of point 2216 based at least on the normal of surface 2208 at point 2216. The image renderer 102 can define the BRDF (e.g., using a smooth microfacet BRDF model) using the NDF, ray 2214, and the roughness value of surface 2208 associated with point 2216. For example, the NDF range 2220 can be defined using a function (NDF) that defines the possibility of microfacet alignment in a particular direction.

[0154] BRDF lobes can be defined using a function (BRDF) that uses NDF as a weighting function to scale the brightness of the reflection. Image renderer 102 can sample the BRDF lobes (e.g., using the BRDF randomly or using another sampling strategy) to determine rays 2218. As an example, these rays can be referred to as reflected rays or auxiliary rays. Although ray 2218 intersects object 2206, in examples where more than one ray is projected, not all rays will intersect object 2206. For example, a ray may intersect different objects or may not intersect any objects. In some embodiments, one or more additional rays may be projected to determine the lighting conditions of pixel 2212, for example, based on the interaction between ray 2218 and object 2206, and similar operations may be performed relative to the point of interaction.

[0155] Image renderer 102 can use individual rays (e.g., ray 2214, ray 2218, etc.) to determine ray-traced samples of the lighting conditions of pixel 2212. For example, rays can form any number of virtual light paths between pixel 2212 and one or more light sources, examples of which may include rays 2214 and ray 2218. Object 2206 can be an example of such a light source, or object 2206 can be another type of object, such as a non-emitting reflective object. In the case that object 2206 is not a light source, the virtual light path may also include one or more rays to the light source. Image renderer can determine at least some lighting condition data for pixel 2212 by combining (e.g., averaging) the lighting condition data derived from the individual ray-traced samples. Image renderer can similarly determine at least some lighting condition data for each pixel or region of screen 2204 (e.g., using any number of shadow rays, camera rays, and / or other ray types) to generate ray-traced rendering of virtual environment 2200.

[0156] As described herein, aspects of this disclosure provide a method for a filter determiner 108 to determine the size, shape, orientation, filter orientation, and / or filter weights of a filter kernel applied to pixels for denoising ray-traced samples of a graphics-rendered image. The size, orientation, and / or shape of the filter kernel, as well as the corresponding filter weights of the filter, can be determined and / or adjusted based on various information described herein (e.g., temporal ray-traced sample information from previous frames and / or image rendering data used by the image renderer for rendering output and / or the final frame or rendered image). Examples include radiance data (e.g., specular or diffuse), hit distance data, scene depth data, and world normals. Various types of filters are contemplated within the scope of this disclosure, including Poisson disk-based filters, Poisson-based filters, uniform dithering-based filters, bilateral filters, cross-bilateral filters, spatiotemporal filters, temporal filters, spatial filters, etc. Various numbers of samples can be used for the filter, examples including eight. Suitable footprints for the filter can include rectangles, squares, circles, ellipses, etc.

[0157] In various examples, filter determiner 108 may determine one or more filter sizes based at least on rays 2214, which may or may not have been used by the image renderer for sampling the lighting conditions for pixel 2212 (e.g., to determine corresponding image data). In some examples, image renderer 102 uses at least one ray-traced sample of at least one previous state of virtual environment 2200 (e.g., each state may correspond to a rendered frame representing virtual environment 2200) to determine the lighting conditions of pixel 2212 in one or more subsequent states of the virtual environment. For example, in addition to one or more spatial or current ray-traced samples used to determine the lighting conditions of pixel 2212, any number of temporal ray-traced samples may be used to determine the lighting conditions of pixel 2212 (e.g., using an accumulation or history buffer). In the case of using many temporal ray-traced samples, those samples may be counted (e.g., per pixel) or may be used in other ways to reduce the size of the filter and / or one or more geometries.

[0158] Using some of the methods described herein, at grazing angles, the filter can be extended along with the reflection itself, and in regions where ray 2214 is more perpendicular to surface 2208, the filter can become less extended because it can more closely correspond to the BRDF lobe at point 2216.

[0159] Example computing device

[0160] Figure 23This is a block diagram of an example computing device 2300 suitable for implementing some embodiments of the present disclosure. The computing device 2300 may include an interconnect system 2302 directly or indirectly coupled to the following devices: a memory 2304, one or more central processing units (CPUs) 2306, one or more graphics processing units (GPUs) 2308, a communication interface 2310, input / output (I / O) ports 2312, input / output components 2314, a power supply 2316, one or more presentation components 2318 (e.g., one or more displays), and one or more logic units 2320. In at least one embodiment, one or more computing devices 2300 may include one or more virtual machines (VMs), and / or any component thereof may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 2308 may include one or more vGPUs, one or more CPUs 2306 may include one or more vCPUs, and / or one or more logic units 2320 may include one or more virtual logic units. Thus, one or more computing devices 2300 may include discrete components (e.g., a full GPU dedicated to computing device 2300), virtual components (e.g., a portion of the GPU dedicated to computing device 2300), or a combination thereof.

[0161] although Figure 23 The various blocks are shown as connected via interconnect system 2302 using lines, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, presentation component 2318 (such as a display device) may be considered I / O component 2314 (e.g., if the display is a touchscreen). As another example, CPU 2306 and / or GPU 2308 may include memory (e.g., memory 2304 may represent a storage device other than the memory of GPU 2308, CPU 2306, and / or other components). In other words, Figure 23 The computing devices described are for illustrative purposes only. No distinction is made between such categories as “workstation,” “server,” “laptop computer,” “desktop computer,” “tablet computer,” “client device,” “mobile device,” “handheld device,” “game console,” “electronic control unit (ECU),” “virtual reality system,” and / or other device or system types, as all are considered within the scope of… Figure 23 Within the scope of computing devices.

[0162] Interconnect system 2302 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 2302 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Fast Peripheral Component Interconnect (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 2306 may be directly connected to memory 2304. Further, CPU 2306 may be directly connected to GPU 2308. In cases where there is a direct or point-to-point connection between components, interconnect system 2302 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required to be included in computing device 2300.

[0163] The memory 2304 may include any computer-readable medium from a variety of computer-readable media. The computer-readable medium may be any available medium accessible by the computing device 2300. The computer-readable medium may include volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, the computer-readable medium may include computer storage media and communication media.

[0164] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 2304 may store computer-readable instructions (e.g., representing (one or more) programs and / or (one or more) program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computing device 2300. As used herein, computer storage media does not include the signal itself.

[0165] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and includes any information transmission medium. The term "modulated data signal" can refer to a signal whose one or more characteristics are set or altered in a manner that encodes information in the signal. By way of example and not limitation, computer storage media can include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the above should also be included within the scope of computer-readable media.

[0166] CPU 2306 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 2300 to perform one or more of the methods and / or processes described herein. Each CPU 2306 may contain one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of handling numerous software threads simultaneously. CPU 2306 may contain any type of processor and may contain different types of processors depending on the type of computing device 2300 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 2300, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors (such as math coprocessors), computing device 2300 may also include one or more CPUs 2306.

[0167] In addition to or in lieu of one or more CPUs 2306, one or more GPUs 2308 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 2300 to perform one or more of the methods and / or processes described herein. One or more GPUs 2308 may be integrated GPUs (e.g., with one or more CPUs 2306) and / or one or more GPUs 2308 may be discrete GPUs. In embodiments, one or more GPUs 2308 may be coprocessors of one or more CPUs 2306. GPUs 2308 may be used by computing device 2300 for rendering graphics (e.g., 3D graphics) or performing general-purpose computing. For example, GPUs 2308 may be used for general-purpose computing on a GPU (GPGPU). GPUs 2308 may include hundreds or thousands of cores capable of handling hundreds or thousands of software threads simultaneously. GPUs 2308 may produce pixel data of an output image in response to rendering commands (e.g., rendering commands received from CPUs 2306 via a host interface). GPU 2308 may include graphics memory (e.g., display memory) for storing pixel data or any other suitable data (e.g., GPGPU data). Display memory may be included as part of memory 2304. GPU 2308 may include two or more GPUs operating in parallel (e.g., via links). The links may be directly connected to the GPUs (e.g., using NVLINK) or may be connected to the GPUs via a switch (e.g., using NVSwitch). When combined, each GPU 2308 may produce pixel data or GPGPU data for different portions of the output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may include its own memory or may share memory with other GPUs.

[0168] In addition to or in lieu of CPU 2306 and / or GPU 2308, logic unit 2320 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 2300 to perform one or more of the methods and / or processes described herein. In embodiments, one or more CPUs 2306, one or more GPUs 2308, and / or one or more logic units 2320 may execute any combination of methods, processes, and / or portions thereof discretely or jointly. One or more logic units 2320 may be a portion of one or more CPUs 2306 and / or GPUs 2308 and / or integrated into one or more CPUs 2306 and / or GPUs 2308, and / or one or more logic units 2320 may be discrete components or otherwise external to CPUs 2306 and / or GPUs 2308. In an embodiment, one or more of the logic units 2320 may be coprocessors of one or more of the CPU 2306 and / or one or more of the GPU 2308.

[0169] Examples of logic unit 2320 include one or more processing cores and / or components thereof, such as tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree lateral unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), arithmetic logic unit (ALU), application-specific integrated circuit (ASIC), floating-point unit (FPU), input / output (I / O) element, peripheral component interconnect (PCI) or fast peripheral component interconnect (PCIe) element, etc.

[0170] Communication interface 2310 may include one or more receivers, transmitters, and / or transceivers that enable computing device 2300 to communicate with other computing devices via electronic communication networks (including wired and / or wireless communications). Communication interface 2310 may include components and functions that enable communication over any of a plurality of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication over Ethernet or wirelessband), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.

[0171] I / O port 2312 enables computing device 2300 to be logically coupled to other devices including I / O component 2314, one or more presentation components 2318, and / or other components, some of which may be built into (e.g., integrated into) computing device 2300. Illustrative I / O component 2314 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, etc. I / O component 2314 can provide a natural user interface (NUI) for processing aerial gestures, voice, or other physiological input generated by the user. In some cases, input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, pen recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, aerial gestures, head and eye tracking, and touch recognition (as described in more detail below) associated with the display of computing device 2300. Computing device 2300 may include depth cameras for gesture detection and recognition, such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof. Additionally, computing device 2300 may include an accelerometer or gyroscope (e.g., as part of an inertial measurement unit (IMU)) that enables motion detection. In some examples, computing device 2300 may use the output of the accelerometer or gyroscope to render immersive augmented reality or virtual reality.

[0172] Power supply 2316 may include a hardwired power supply, a battery power supply, or a combination thereof. Power supply 2316 may provide power to computing device 2300 to enable the components of computing device 2300 to operate.

[0173] Presentation component 2318 may include a display (e.g., a monitor, touchscreen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. Presentation component 2318 may receive data from other components (e.g., GPU 2308, CPU 2306, etc.) and output said data (e.g., as images, videos, sounds, etc.).

[0174] Example Data Center

[0175] Figure 24 An example data center 2400 that may be used in at least one embodiment of this disclosure is shown. The data center 2400 may include a data center infrastructure layer 2410, a framework layer 2420, a software layer 2430, and / or an application layer 2440.

[0176] like Figure 24As shown, the data center infrastructure layer 2410 may include a resource coordinator 2412, grouped computing resources 2414, and node computing resources (“nodes CRs”) 2416(1)-2416(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CRs 2416(1)-2416(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power modules and / or cooling modules, etc. In some embodiments, one or more node CRs from nodes CRs 2416(1)-2416(N) may correspond to servers having one or more of the aforementioned computing resources. In addition, in some embodiments, nodes CRs 2416(1)-24161(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more nodes CRs 2416(1)-2416(N) may correspond to virtual machines (VMs).

[0177] In at least one embodiment, the grouped computing resources 2414 may include individual groups of node CRs 2416 housed within one or more racks (not shown), or multiple racks housed within a data center at different geographical locations (also not shown). Individual groups of node CRs 2416 within the grouped computing resources 2414 may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, several node CRs 2416, including CPUs, GPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0178] Resource coordinator 2422 may be configured or otherwise control one or more nodes CRs 2416(1)-2416(N) and / or groups of computing resources 2414. In at least one embodiment, resource coordinator 2422 may include a Software Design Infrastructure (“SDI”) management entity for data center 2400. Resource coordinator 2422 may include hardware, software, or some combination thereof.

[0179] In at least one embodiment, such as Figure 24As shown, framework layer 2420 may include a job scheduler 2418, a configuration manager 2434, a resource manager 2436, and / or a distributed file system 2438. Framework layer 2420 may include a framework for software 2418 supporting software layer 2430 and / or one or more applications 2442 supporting application layer 2440. Software 2418 or application 2442 may respectively contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 2420 may be, but is not limited to, free and open-source software web application frameworks (such as Apache Spark) that can utilize distributed file system 2438 for large-scale data processing (e.g., "big data"). TM (Hereinafter referred to as "Spark") is a type of resource. In at least one embodiment, the job scheduler 2418 may include Spark drivers to facilitate the scheduling of workloads supported by different layers of data center 2400. Configuration manager 2434 may be able to configure different layers, such as software layer 2430 and framework layer 2420 (which includes Spark and distributed file system 2438 for supporting large-scale data processing). Resource manager 2436 may be able to manage computing resources mapped to or allocated to clusters of distributed file system 2438 and job scheduler 2418 or to support clusters of distributed file system 2438 and job scheduler 2418. In at least one embodiment, clusters or groups of computing resources may include grouped computing resources 2414 in data center infrastructure layer 2410. Resource manager 1036 may coordinate with resource coordinator 2412 to manage these mapped or allocated computing resources.

[0180] In at least one embodiment, the software 2418 included in the software layer 2430 may include software used in at least a portion of the nodes CRs 2416(1)-2416(N), the grouped computing resources 2414, and / or the distributed file system 2438 of the framework layer 2420. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0181] In at least one embodiment, the application 2442 included in the application layer 2440 may include one or more types of applications used at least in part by nodes CRs 2416(1)-2416(N), grouped computing resources 2414, and / or the distributed file system 2438 of the framework layer 2420. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in combination with one or more embodiments.

[0182] In at least one embodiment, any of the configuration manager 2434, resource manager 2436, and resource coordinator 2412 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. Self-modification actions can free the data center operator of the data center 2400 from making potentially poor configuration decisions and may prevent underutilization and / or poor performance of the data center.

[0183] According to one or more embodiments described herein, data center 2400 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information. For example, one or more machine learning models may be trained by using the software and / or computing resources described above relative to data center 2400 to compute weight parameters according to a neural network architecture. In at least one embodiment, a trained or deployed machine learning model corresponding to one or more neural networks may be used to infer or predict information using the resources described above relative to data center 2400 by using weight parameters computed through one or more training techniques, such as, but not limited to, those described herein.

[0184] In at least one embodiment, the data center 2400 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the software and / or hardware resources described above may be configured to allow a user to train or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0185] Example network environment

[0186] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 23 This can be implemented on one or more examples of computing devices 2300—for example, each device may include similar components, features, and / or functions of one or more computing devices 2300. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of a data center 2400, examples of which are described herein. Figure 24 To describe in more detail.

[0187] Components of a network environment can communicate with each other via a network, which can be wired, wireless, or both. A network can include multiple networks or one of multiple networks. For example, a network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. Where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0188] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein for the server can be implemented on any number of client devices.

[0189] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework supporting software at the software layer and / or application at the application layer. The software or application may respectively include network-based service software or applications. In embodiments, one or more client devices may use the network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a free and open-source software network application framework that can use a distributed file system for large-scale data processing (e.g., "big data").

[0190] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). The core server may assign at least a portion of the functionality to the edge server if the connection to the user (e.g., a client device) is relatively close to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0191] One or more client devices may include the information described in this article. Figure 23 At least some of the components, features, and functions of one or more example computing devices 2300 described. By way of example and not limitation, the client device may be implemented as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming equipment or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these depicted devices, or any other suitable device.

[0192] This disclosure can be described in the general context of machine-usable instructions or computer code, including computer-executable instructions such as program modules, which are executed by a computer or other machine such as a personal digital assistant or other handheld device. Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a wide variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. This disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices linked via a communication network.

[0193] As used herein, the phrase "and / or" relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, "element A, element B, and / or element C" can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Furthermore, "at least one of element A or element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0194] This document describes in detail the subject matter of this disclosure to meet statutory requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the discloser has envisioned that the claimed subject matter may be embodied in other ways to include steps different from or similar combinations of steps described herein in conjunction with other current or future techniques. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of the method employed, these terms should not be construed as suggesting any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. A method for denoising a loop blur in a ray tracing application, comprising: generating a cumulative sample of a virtual environment based on a ray tracing sample of the virtual environment and a denoised sample of the virtual environment; determining a blur radius of a filter based on at least a number of rendered frames corresponding to the cumulative sample, wherein the number of rendered frames corresponding to the cumulative sample is determined based on at least one or more deocclusions detected in a process of generating the rendered frames; and generating a rendered frame based on at least blurring the cumulative sample using the filter with the blur radius.

2. The method of claim 1, wherein the determination of the blur radius uses an adjustment factor that decreases the blur radius as the number of rendered frames increases.

3. The method of claim 1, further comprising: generating the denoised sample based on at least accumulating another ray tracing sample of the virtual environment with another denoised sample of the virtual environment.

4. The method of claim 1, wherein the filter is applied to a pixel and the cumulative sample corresponding to the pixel is used in determining the blur radius based on at least the cumulative sample.

5. The method of claim 1, further comprising: enhancing the cumulative sample with a sample generated based on at least downsampling a set of ray tracing samples of the virtual environment prior to the blurring.

6. The method of claim 1, further comprising: calculating a cumulative velocity of a pixel based on at least a first view vector corresponding to the pixel, a second view vector corresponding to the pixel, and an angle between a view angle associated with the first view vector, wherein the accumulating uses the cumulative velocity calculated for the pixel.

7. The method of claim 1, wherein the filter is an anisotropic filter, the ray-tracing sample corresponds to an interaction between a view vector and a point in the virtual environment, and the method further comprises: calculating a size of the anisotropic filter based on at least a direction of the specular reflection lobe of the interaction and a normal corresponding to the point in the virtual environment.

8. The method of claim 1, wherein the cumulative sample is generated using a history buffer that includes the denoised sample.

9. The method of claim 1, wherein the generating comprises: accumulating ray tracing renderings of the virtual environment with denoised cumulative renderings of the virtual environment to generate a cumulative rendering that includes the cumulative sample.

10. A system for denoising loop blur in ray tracing applications, comprising: one or more processing units to perform operations comprising generating a rendering of a virtual environment over a plurality of iterations, each iteration comprising accumulating a ray tracing sample of the virtual environment with a sample of a previous iteration of the plurality of iterations, the accumulating to generate a cumulative sample; determining a blur radius of a filter based on at least a number of the renderings corresponding to at least one of the cumulative samples; and generating the sample for a subsequent iteration of the plurality of iterations based on at least blurring the cumulative sample using the filter with the blur radius; wherein the number of renderings corresponding to the at least one sample is determined based on at least one or more deocclusions detected in a process of generating the renderings.

11. The system of claim 10, wherein different ones of the cumulative samples correspond to different numbers of the renderings based on at least corresponding deocclusions detected in generating the renderings.

12. The system of claim 10, wherein the iterating further comprises: The count of the number of renders is updated based at least on the accumulation, and the blur radius is based at least on the count.

13. The system of claim 10, wherein the iterating further comprises: The count of the number of renders is updated based at least on the accumulation, wherein the count is used to determine a corresponding blur radius of a filter applied to a corresponding pixel by the blurring.

14. The system of claim 10, wherein the number comprises a count of the renders that have accumulated with respect to a pixel corresponding to the at least one sample to the accumulated samples.

15. A processor comprising: one or more circuits to generate an accumulated rendering corresponding to a ray-traced rendering of a virtual environment and a denoised accumulated rendering of the virtual environment; determine a blur radius of a filter based at least on a number of renders corresponding to at least one sample of the accumulated rendering; and generate a frame of a rendering based at least on blurring the accumulated rendering using the filter having the blur radius; wherein the number of renders corresponding to the at least one sample is determined based at least on one or more de-occlusions detected during generation of the renders.

16. The processor of claim 15, wherein the number comprises a count of the renders that have accumulated with respect to a pixel corresponding to the at least one sample to the accumulated rendering.

17. The processor of claim 15, further comprising augmenting the accumulated rendering, prior to the blurring, with at least samples generated based at least on down-sampling the ray-traced rendering.

18. The processor of claim 15, wherein the determination of the blur radius uses an adjustment factor that decreases the blur radius as the number of renders increases.

19. The processor of claim 15, wherein the processor is included in at least one of: a system to perform simulation operations; a system to perform deep learning operations; a system implemented using edge devices; a system that includes one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

Images