Dynamic global illumination method based on screen space probe

Abstract

The invention provides a dynamic global illumination method based on a screen space probe, which relates to the technical field of global illumination and comprises the steps of probe distribution and coding, light ray tracing and probe updating, light ray result reconstruction and result noise reduction. The method specifically comprises the following steps: firstly, acquiring a pixel world space position, and reversely deducing world space coordinates according to screen space coordinates through a variation matrix in rasterization and depth generated by a delay pipeline; and then generating a hemispherical cosine weight sampling direction by using a Hammerslay low-difference sequence, taking a value of a corresponding index in a buffer for each block of a screen during coloring, performing color reconstruction, performing noise reduction on the image after coloring is completed, and performing noise reduction by adopting a mode of filtering the image. The weight of a sample sampled during filtering is guided by adopting depth, a normal and brightness, and better dynamic diffuse reflection global illumination performance and effect are achieved through multi-level optimization.

Description

Technical Field

[0001] The present invention relates to the field of global illumination technology, and in particular to a dynamic global illumination method based on screen space probes. Background Art

[0002] In the gaming industry, global illumination is the core of visual presentation. In the traditional scene rendering process, environment map lighting is usually used directly as the source of indirect light for the scene, so that the parts of the scene illuminated by indirect light or quasi-light carry the color of the environment light. However, the scene lighting produced in this way obviously lacks authenticity, and it is difficult to have rich details in visual presentation. This is especially the case in night city scenes with complex self-luminous scenes. Therefore, in the latest gaming industry, we are paying more attention to global illumination, a technology designed to express indirect light.

[0003] Due to limited computing power, it is difficult to calculate global illumination using pure path tracing at this stage. The global illumination algorithms proposed in recent years and successfully implemented in games are more about using approximations to achieve close to real goals. Unlike offline rendering, global illumination solutions for real-time applications such as video games currently rely on lighting data that can be quickly read from spatial data structures, and are usually pre-calculated or limited to dynamic lighting that is slowly updated from static geometry. Including updating lightmaps, irradiance, and irradianceprobes, each representation needs to balance runtime flexibility, accuracy, and cost.

[0004] Dynamic Diffuse Global Illumination (DDGI) is a global illumination solution that has performed well in recent years and fully balances performance and performance. However, the original DDGI needs to take into account the details of the scene's specular reflection, so it uses octahedral mapping to encode the lighting results. This direct color recording method consumes more video memory.

[0005] Screen Space Global Illumination (SSGI) provides an effective idea: that is, for a certain perspective of the scene, most of the information can come from the screen itself. Using the world space position of the screen space pixel as the starting point of ray tracing is a good way. All SSGI information comes from the screen and cannot obtain off-screen information. When a significantly different bright surface enters the screen, it will cause unnatural changes in the scene GI. Another is that the results obtained under the condition of single pixel 1SPP are actually limited in authenticity. In many cases, powerful noise reduction algorithms are relied upon to correct results with large errors and poor visual performance. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a dynamic global illumination method based on screen space probes, which achieves better dynamic diffuse global illumination performance and effect through multi-level optimization.

[0007] A dynamic global illumination method based on screen space probes comprises the following steps:

[0008] Step 1: Probe distribution and coding;

[0009] Specifically, first obtain the pixel world space position, and reverse the world space coordinates according to the screen space coordinates through the change matrix in rasterization and the depth generated by the delay pipeline;

[0010] Implement probe placement in the Raytracing shader provided by RTX, use the first two components of the thread group as the block coordinate values ​​at the screen space probe resolution, and finally obtain the world space position of the screen space calculation probe distribution;

[0011] Step 2: Ray tracing and probe update;

[0012] Specifically, the hammerslay low-discrepancy sequence is used to generate the hemispherical cosine weight sampling direction. When there is a bright surface with a brightness greater than the set threshold within the set shading point range and its contribution to the shading point exceeds half, a portion of the samples are sampled in the direction of maximum brightness.

[0013] Using the hardware ray tracing architecture probe tracking and update, the screen space normal and depth map, the current thread ID and the main light direction are input into the ray generation shader, and then the world space coordinates of the shading point are restored according to the screen space UV and depth, and then the Hammersley sequence sampling direction is generated according to the shading point normal, and the starting point of the light is set to the world space coordinates of the shading point, and the direction is the generated sampling direction for the first tracking; when the first tracking misses, the first miss shader is executed to directly return the sampled sky color as the color, and the second miss shader changes the loaded shadowmask value to 1. When the second tracking misses, there is no object blocking the hit point and the main light, and the hit point color recorded during the first hit is directly returned; if a hit occurs in the second tracking, there is occlusion between the hit point and the main light, and the direct light result, which is 0, is directly returned in the case of simulating a catapult; the shadowMask value is set to 0.2 to return one-fifth of the hit point color and mix in part of the sky light color. The final color result is projected onto the spherical harmonic coefficients and stored in the probe data structure to complete the probe update.

[0014] Step 3: Light result reconstruction and result denoising;

[0015] Specifically, when shading, each block of the screen takes the value of the corresponding index in the buffer and reconstructs the color. This step is completed in the deferred rendering pipeline. First, the deferred pipeline is rendered normally to generate the corresponding Gbuffer. Then, the direct light shading result is obtained before the post-processing stage. Then, the spherical harmonic coefficients of the ray tracing result are taken to reconstruct the lighting color. The lighting color is multiplied by the basic color buffer in the Gbuffer, and then added to the direct light shading result to get the result.

[0016] After coloring is completed, the image is denoised by filtering the image. The sample weights sampled during filtering are guided by depth, normal and brightness. That is, when the difference between the normal of the surrounding samples, i.e. the first element of the guiding weight, and the depth, i.e. the second element of the guiding weight, and the depth of the central sample and the probe center sample exceeds the set threshold, the weight of the surrounding samples is reduced. The third element of the guiding weight is brightness.

[0017] The time domain noise reduction scheme is introduced to make the color of the same shading point in the previous and next frames stable; first calculate the motion vector of the current frame pixel, calculate the screen space coordinates of the previous frame pixel through this motion vector, obtain the color of the previous frame pixel, set the depth and normal difference thresholds, and judge that the color of the previous frame is credible within the threshold range, then mix the color of the previous frame at a ratio of 0.8. When the scene is stationary and the camera is moving; through the world space coordinates of the fragment, multiply the coordinates with the VP matrix of the previous frame to get the position of the fragment in the previous frame. This is the reprojection technology, that is, the information described above about locating the shading point of the previous frame by calculating the motion vector.

[0018] The beneficial effects of adopting the above technical solution are:

[0019] The present invention provides a dynamic global illumination method based on screen space probes. The present invention greatly improves the efficiency of the ray tracing global illumination scheme and achieves an effect close to path tracing by optimizing the three stages of screen space probe ray tracing global illumination.

[0020] In terms of probe distribution, we adopt the method of sampling screen depth and normal in blocks and performing multiple tests to obtain a reasonable block world coordinate space to place probes, so that the probes can better express the global illumination color of the area.

[0021] In terms of ray tracing architecture, NVIDIA's hardware ray tracing acceleration is adopted, and the hardware ray tracing core is used to build an acceleration structure to improve ray tracing efficiency. Ray Tracing is a function provided by modern APIs, such as DXR or Vulkan, which aims to use GPUs that provide hardware acceleration for ray tracing, usually as a supplement to rasterization rather than a replacement. Modern GPUs provide an independent core for ray tracing acceleration. This acceleration structure is basically opaque to users, but it can be driven by the corresponding shader to provide a convenient and fast solution for ray tracing on the PC side, which is the so-called hardware ray tracing.

[0022] In terms of denoising results, a comprehensive denoising method combining time domain and spatial domain is adopted to increase the visual performance and stability of the results, the importance sampling method is used to increase the sample accuracy, the normal depth and brightness are used to guide the sample weights, and then through time domain sample reuse, the final result is close to path tracing and has the efficiency of real-time operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a diagram of a ray tracing architecture in an embodiment of the present invention;

[0024] Figure 2 2 is a performance diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0026] A dynamic global illumination method based on screen space probes comprises the following steps:

[0027] Step 1: Probe distribution and coding;

[0028] Specifically: first obtain the world space position of the pixel. There are two existing methods. One is to directly use ray tracing to emit rays from the pixel to hit the scene surface and then calculate the position based on the camera and light stepping distance. The other is to use the change matrix in rasterization and the depth generated by the delay pipeline to reverse the world space coordinates according to the screen space coordinates. Since this solution requires rasterization pipeline process for shading except GI, GI is an additional calculation and there is no need to use ray tracing, so the second method is adopted.

[0029] The probe placement is implemented in the Raytracing shader provided by RTX. The first two components of the thread group are used as the block coordinate values ​​at the screen space probe resolution. The Raytracing shader can directly pass the resolution of the currently rendered rendertexture as a parameter. By left-multiplying the change matrix, the world space position of the screen space calculation probe distribution is finally obtained;

[0030] Step 2: Ray tracing and probe update;

[0031] Specifically: Use the hammerslay low-discrepancy sequence to generate the hemispherical cosine weighted sampling direction. When there is a bright surface with a brightness greater than the set threshold within the set shading point range and contributes more than half of the shading point, if light is emitted in a purely random manner, more errors or invalid samples will be obtained. The samples in the direction of maximum brightness are obviously more physically correct. Therefore, in order to make the picture more correct and visually better, we need to allocate a part of the samples to sample in the direction of maximum brightness instead of sampling all samples in the direction of the surface normal of the shading point (assuming that the normal direction is towards the occluder, the brightness will be significantly reduced).

[0032] Figure 1 This is a schematic diagram of the ray tracing architecture of this solution. The ray tracing is specifically as follows: using the tracing and updating of the hardware ray tracing architecture probe, inputting the screen space normal and depth map, the current thread id (screen space UV) and the main light direction into the ray generation shader, and then restoring the world space coordinates of the shading point according to the screen space UV and depth, and then generating the Hammersley sequence sampling direction according to the shading point normal, setting the ray starting point as the world space coordinates of the shading point, and the direction as the generated sampling direction for the first tracing; whether the first tracing hits determines whether the nearest hit shader or the first miss shader is executed. When the first tracing misses, the first miss shader is executed to directly return the sampled sky color as the color, and the second miss shader changes the loaded shadowmask value to 1. When the second tracing misses, there is no object blocking the hit point and the main light, and the hit point color recorded when the first hit is directly returned; if the second tracing hits, there is occlusion between the hit point and the main light, and the direct light result is directly returned in the case of simulating a bounce, which is 0; there are multiple bounces of light in the purely physical sense, so it will be visually dark if it is calculated in this way. In order to approximate the results of multiple bounces, the shadowMask value is set to 0.2 to return one-fifth of the color of the hit point and mix in part of the skylight color. The final color result is projected onto the spherical harmonic coefficients and stored in the probe's data structure to complete the probe update.

[0033] Step 3: Light result reconstruction and result denoising;

[0034] Specifically, when shading, each block of the screen takes the value of the corresponding index in the buffer and reconstructs the color. This step is completed in the deferred rendering pipeline. First, the deferred pipeline is rendered normally to generate the corresponding Gbuffer. Then, the direct light shading result is obtained before the post-processing stage. Then, the spherical harmonic coefficients of the ray tracing result are taken to reconstruct the lighting color. The lighting color is multiplied by the basic color buffer in the Gbuffer, and then added to the direct light shading result to get the result.

[0035] After the coloring is completed, the image is denoised by filtering the image. In order to retain the details, the sample weights sampled during filtering are guided by depth, normal and brightness. That is, when the difference between the normal of the surrounding samples, that is, the first element of the guiding weight, and the depth, that is, the second element of the guiding weight, and the depth of the center sample and the probe center sample exceeds the set threshold, the weight of the surrounding samples is reduced. The third element of the guiding weight is brightness. Most of the GI details come from the reflection of the bright surface in the scene. If a uniform weight filtering is performed simply, these bright sample colors will be assigned to the dark surface, resulting in loss of details. Therefore, adding brightness as an unconventional guide is conducive to retaining the bright surface details. When the normal difference exceeds 75° and the depth difference exceeds 0.01, the weight will be multiplied by 0.1 and 0.1 respectively. The weight coefficient within the range is proportional to the difference, and the control coefficient is between 1 and 0.1. When the brightness difference exceeds the threshold, the weight will be directly set to 0.

[0036] The time domain noise reduction scheme is introduced to make the color of the same shading point in the previous and next frames stable; first calculate the motion vector of the current frame pixel, calculate the screen space coordinates of the previous frame pixel through this motion vector, obtain the color of the previous frame pixel, set the depth and normal difference thresholds, and judge that the color of the previous frame is credible within the threshold range, then mix it into the previous frame color at a ratio of 0.8. When the scene is stationary and the camera moves. The same point in the scene may have different positions on the screen. If the pixels at the same position are still used for mixing, serious afterimages may occur. Since the world space coordinates of the fragment have not changed in this case, the coordinates are multiplied by the VP matrix of the previous frame through the world space coordinates of the fragment to obtain the position of the fragment in the previous frame. This is the reprojection technology, that is, the information described above about locating the shading point of the previous frame by calculating the motion vector.

[0037] Compared with the prior art, the technical solution proposed in the present invention supports fully dynamic scenes and dynamic lighting, has very efficient performance, and solves some defects of mainstream global illumination solutions to a certain extent. At the same time, it provides a time-space combined denoising solution to effectively and effectively denoise the GI results, achieving relatively good visual performance.

[0038] Figure 2 This is a performance diagram of the technical solution of the present invention. The GPU performance delay of the present invention is mainly affected by the number of probes. It can be seen that under the condition of a medium number of probes, an increase in the number of rays will not obviously lead to an increase in performance overhead, and the number of probes mainly affects the detail performance. The diffuse lighting performance itself allows a certain amount of detail loss. In scenes with fewer details, the performance of the present invention is more prominent.

[0039] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A dynamic global illumination method based on screen space probes, characterized in that: The following steps are involved: Step 1: Probe distribution and coding; Step 2: Ray tracing and probe update; Step 3: Light result reconstruction and result denoising.

2. A screen space probe based dynamic global illumination method according to claim 1, characterized in that: The step 1 is specifically as follows: First, the world space position of the pixel is obtained, and the world space coordinates are inferred from the screen space coordinates through the change matrix in rasterization and the depth generated by the delay pipeline.

3. A method for dynamic global illumination based on screen space probe according to claim 2, characterized in that: The probe placement is implemented in the Raytracing shader provided by RTX, and the first two components of the thread group are used as the block coordinate values ​​at the screen space probe resolution, and finally the world space position of the screen space calculation probe distribution is obtained.

4. The method for dynamic global illumination based on screen space probe according to claim 1, characterized in that: The step 2 is specifically as follows: using the hammerslay low-discrepancy sequence to generate the hemispherical cosine weight sampling direction, when there is a bright surface with a brightness greater than a set threshold within the set shading point range and contributes more than half of the shading point, a portion of the samples are separated and sampled in the direction of maximum brightness.

5. The method for dynamic global illumination based on screen space probe according to claim 4, characterized in that: Using the hardware ray tracing architecture probe tracking and update, the screen space normal and depth map, the current thread ID and the main light direction are input into the ray generation shader, and then the world space coordinates of the shading point are restored according to the screen space UV and depth, and then the Hammersley sequence sampling direction is generated according to the shading point normal, and the starting point of the light is set to the world space coordinates of the shading point, and the direction is the generated sampling direction for the first tracking; when the first tracking misses, the first miss shader is executed to directly return the sampled sky color as the color, and the second miss shader changes the loaded shadowmask value to 1. When the second tracking misses, there is no object blocking the hit point and the main light, and the hit point color recorded during the first hit is directly returned; if a hit occurs in the second tracking, there is occlusion between the hit point and the main light, and the direct light result, which is 0, is directly returned in the case of simulating a catapult; the shadowMask value is set to 0.2 to return one-fifth of the hit point color and mix in part of the sky light color. The final color result is projected onto the spherical harmonic coefficients and stored in the probe data structure to complete the probe update.

6. The method for dynamic global illumination based on screen space probe according to claim 1, characterized in that: The step 3 is specifically as follows: when shading, each block of the screen takes the value of the corresponding index in the buffer and performs color reconstruction. This step is completed in the deferred rendering pipeline. First, the deferred pipeline rendering is performed normally to generate the corresponding Gbuffer, and then the direct light shading result is obtained before the post-processing stage. Then, the spherical harmonic coefficients of the ray tracing result are taken to reconstruct the lighting color, and the lighting color is multiplied by the basic color buffer in the Gbuffer, and then added to the direct light shading result to obtain the result; After coloring is completed, the image is denoised by filtering the image. The sample weights sampled during filtering are guided by depth, normal, and brightness. That is, when the normal of the surrounding samples, which is the first element of the guiding weight, and the depth, which is the second element of the guiding weight, and the depth of the central sample and the probe center sample differ by more than the set threshold, the weights of the surrounding samples are reduced. The third element of the guiding weight is brightness.

7. The method for dynamic global illumination based on screen space probe according to claim 6, characterized in that: A time-domain noise reduction scheme is introduced to stabilize the color of the same shading point in the previous and next frames. First, the motion vector of the current frame pixel is calculated, and the screen space coordinates of the previous frame pixel are calculated through this motion vector to obtain the color of the previous frame pixel. The depth and normal difference thresholds are set, and the color of the previous frame is judged to be credible within the threshold range, and then the color of the previous frame is mixed in at a ratio of 0.

8. When the scene is stationary and the camera moves, the world space coordinates of the fragment are multiplied by the VP matrix of the previous frame to obtain the position of the fragment in the previous frame. This is the reprojection technology, that is, the information of locating the shading point of the previous frame by calculating the motion vector as described above.

Images