A dynamic global illumination method based on screen-space probes

Abstract

The application provides a dynamic global illumination method based on a screen space probe, relates to the technical field of global illumination, and comprises probe distribution and coding, ray tracing and probe updating, ray result reconstruction and result noise reduction; specifically, first, a pixel world space position is acquired, world space coordinates are reversely deduced according to screen space coordinates through a change matrix in rasterization and depth generated by a delay pipeline; then, a hammer slay low-difference sequence is used to generate a hemispherical cosine weight sampling direction, a corresponding index value in the buffer is taken and color reconstruction is performed for each block of the screen during shading, and after the shading is completed, the image is denoised, the image is denoised in a filtering manner, and the sample weight during filtering is guided by depth, a normal and brightness, so that better dynamic diffuse reflection global illumination performance and effects are achieved through multi-level optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of global illumination, in particular to a dynamic global illumination method based on a screen space probe. BACKGROUND

[0002] In the game industry, global illumination is the core of picture performance. In the traditional scene rendering process, the environment map lighting is usually directly used as the source of scene indirect light, so that the parts of the scene illuminated by non-direct light or quasi-time light are colored with environment light. However, the scene lighting produced in this way obviously lacks realism and is difficult to have rich detail performance in visual performance. This situation is particularly poor in night city scenes, which have relatively complex self-luminous scenes. Therefore, in the latest game industry, we pay more attention to global illumination, which is a technology aimed at representing indirect light.

[0003] Due to limited computing power, the use of pure path tracing means to calculate global illumination is currently difficult to achieve at this stage. In recent years, the global illumination algorithm proposed and successfully landed in games is more of an approximate way to achieve a close-to-real goal. Unlike offline rendering, the global illumination solution for real-time applications such as video games currently basically relies on lighting data that can be quickly read from spatial angle data structures, and is usually pre-calculated or limited to dynamic lighting slowly updated from static geometry. This includes updating lightmaps, irradiance, and irradiance probes. Each representation needs to balance runtime flexibility, accuracy, and cost.

[0004] Dynamic Diffuse Global Illumination (DDGI) is a good global illumination scheme in recent years that fully balances performance and performance. However, the original DDGI needs to consider the details of the scene's specular reflection part, so it uses octree mapping to encode the lighting results. This direct color recording method is relatively memory-consuming.

[0005] Screen Space Global Illumination (SSGI) provides an effective idea: most of the information for a certain view of the scene can come from the screen itself. Using the world space position of the screen space pixel as the starting point of the ray tracing is a good way. All information of SSGI comes from the screen and cannot obtain information outside the screen. When there are obviously different bright surfaces entering the screen, it will cause unnatural changes in the scene GI. Another is that the result obtained under the condition of single-pixel 1SPP is actually limited in reality. A powerful denoising algorithm is often used to correct large errors, and the visual performance of the result is poor. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides a dynamic global illumination method based on a screen space probe, which achieves better dynamic diffuse global illumination performance and effect through multi-level optimization.

[0007] A dynamic global illumination method based on a screen space probe, comprising the following steps:

[0008] Step 1: distribution and coding of probes;

[0009] Specifically, first, the pixel world space position is obtained, the world space coordinates are back calculated from the screen space coordinates through the change matrix in rasterization and the depth generated by the deferred pipeline;

[0010] The probe placement is realized in the Raytracing shader provided by RTX, the first two components of the thread group are used as the block coordinate values under the screen space probe resolution, and finally the world space position of the screen space calculation probe distribution is obtained;

[0011] Step 2: ray tracing and probe updating;

[0012] Specifically, the hemispherical cosine weight sampling direction is generated using the hammer slay low difference sequence, when there is a bright surface with a brightness greater than a set threshold in the set shading point range and the shading point contributes more than half, a part of the sample is deviated to the maximum brightness direction for sampling;

[0013] The tracking and updating of the probe using the hardware light tracking architecture are performed, the screen space normal and depth map, the current thread id and the main light direction are input into the ray generation shader, then the shading point world space coordinates are restored according to the screen space UV and the depth, the hammer slay sequence sampling direction is generated according to the shading point normal, the starting point of the light is set as the shading point world space coordinates, the direction is the generated sampling direction for the first tracking; when the first tracking does not hit, the first miss shader directly returns the sampled sky color as the color, the second miss shader changes the shadowmask value to 1, when the second tracking does not hit, there is no object blocking between the hit point and the main light, the color of the hit point recorded in the first hit is directly returned; if the second tracking hits, there is an occlusion between the hit point and the main light, the direct light result, that is, 0, is directly returned in the simulation of a bounce; the shadowMask value is set to 0.2 to return one fifth of the hit point color and mix part of the sky color, and finally the color result is projected to the spherical harmonic coefficient and stored in the data structure of the probe to complete the probe updating.

[0014] Step 3: ray result reconstruction and result denoising;

[0015] Specifically, when coloring, each block of the screen takes the value corresponding to the index in the buffer and performs color reconstruction, which is completed in the deferred rendering pipeline, first normally walking the deferred pipeline rendering, generating the corresponding Gbuffer, then obtaining the direct light coloring result before the post-processing stage, then taking the spherical harmonic coefficient of the light tracing result to reconstruct the illumination color, multiplying the illumination color with the base color buffer in the Gbuffer, and then adding to the direct light coloring result to obtain the result.

[0016] After coloring, the image is denoised by filtering the image, and the sample weight is guided by depth, normal and brightness during filtering, that is, when the normal of the surrounding sample, that is, the first element of the guide weight, and the depth, that is, the second element of the guide weight, and the difference between the depth of the center sample and the center sample of the probe exceed the set threshold, the weight of the surrounding sample is reduced, and the third element of the guide weight is brightness.

[0017] The time domain denoising scheme is introduced to stabilize the color of the same shading point in the front and rear frames; first, the motion vector of the current frame pixel is calculated, the screen space coordinates of the last frame pixel are calculated through the motion vector, the color of the last frame pixel is obtained, the depth and normal difference threshold is set, and the color of the last frame is mixed in the threshold range. Within the range, the color of the last frame is mixed in the proportion of 0.8. In the case of a scene not moving and a camera moving; through the world space coordinates of the fragment, the coordinates are multiplied by the VP matrix of the last frame to obtain the position of the fragment in the last frame picture, which is the re-projection technology, that is, the information of locating the shading point of the last frame by calculating the motion vector described above.

[0018] The beneficial effects produced by the above technical solutions are:

[0019] The application provides a dynamic global illumination method based on a screen space probe, which optimizes three stages of screen space probe ray tracing global illumination, greatly improves the efficiency of ray tracing global illumination scheme and achieves an effect close to path tracing.

[0020] In the probe distribution, the screen depth and normal are sampled and tested multiple times to obtain a reasonable block world coordinate space to place the probe, so that the probe can better express the regional global illumination color.

[0021] On the light ray tracing architecture, the hardware light ray tracing acceleration launched by Nvidia is adopted, the acceleration structure is constructed by using the hardware light ray tracing core to improve the light ray tracing efficiency, Ray Tracing is a function provided by modern API, such as DXR or Vulkan, which aims to use the GPU providing hardware acceleration for light ray tracing, and is usually used as a supplement to rasterization instead of replacement. Modern GPU provides an independent core for light ray tracing acceleration, which is basically opaque to users, but can be driven by corresponding shaders, providing a convenient and fast solution for PC light ray tracing, which is so-called hardware light tracing.

[0022] On the result denoising, a comprehensive denoising method combining time domain and space domain is adopted to increase the visual performance and stability of the result, importance sampling method is used to increase the sample accuracy, normal depth and brightness are used to guide the sample weight, and finally the result is close to path tracing with the efficiency of real-time running. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The figure is a light ray tracing architecture in the embodiment of the application.

[0024] Figure 2 The figure is a performance diagram in the embodiment of the application. DETAILED DESCRIPTION

[0025] The specific embodiments of the application will be further described in detail below in combination with the drawings and examples. The following examples are used to illustrate the application, but not to limit the scope of the application.

[0026] A dynamic global illumination method based on a screen space probe, comprising the following steps:

[0027] Step 1: distribution and coding of the probe;

[0028] Specifically, first, the pixel world space position is obtained, there are two methods in the existing method, one is to directly use the light tracing method to emit a ray from the pixel to hit the scene surface, and then calculate the position according to the camera and the light step distance, the other is to generate the depth according to the screen space coordinates through the change matrix in the rasterization and the delay pipeline, since the shading except GI needs to go through the rasterization pipeline process, GI as an additional calculation, there is no need to use light tracing, therefore, the second method is adopted.

[0029] The probe placement is implemented in the Raytracing shader provided by RTX, the first two components of the thread group are taken as the block coordinate values in the screen space probe resolution, and the Raytracing shader can directly take the resolution of the current rendering rendertexture as a parameter. By left multiplying the change matrix, the world space position of the screen space calculation probe distribution is finally obtained;

[0030] Step 2: ray tracing and probe updating;

[0031] Specifically, the hemispherical cosine weight sampling direction is generated using the hammer slay low difference sequence. When there is a bright surface with a brightness greater than a set threshold in a set shading point range and the shading point contributes more than half, if a pure random way is used to emit light, more errors or invalid samples will be obtained, and the sample in the maximum brightness direction is obviously more physically correct. Therefore, in order to make the picture more correct and visually better, a part of the samples need to be biased towards the maximum brightness direction sampling instead of all the samples being directed towards the shading point surface normal direction (assuming the normal direction is towards the occluder, which will obviously reduce the brightness).

[0032] Figure 1 The ray tracing architecture schematic diagram of the present scheme is shown in the figure. The ray tracing specifically includes: inputting the screen space normal and depth map, the current thread id (screen space UV), and the main light direction into the ray generation shader using the tracking and updating of the probe of the hardware light tracing architecture, then restoring the shading point world space coordinates according to the screen space UV and the depth, generating the hammer slay sequence sampling direction according to the shading point normal, setting the starting point of the light ray as the shading point world space coordinates, and the direction as the generated sampling direction to perform the first tracking; whether the first tracking hits determines whether to execute the nearest hit shader or the first miss shader. When the first tracking does not hit, the first miss shader is executed to directly return the sampled sky color as the color, the second miss shader changes the shadowmask value to 1, and when the second tracking does not hit, there is no object blocking between the hit point and the main light, and the color of the hit point recorded when the first hit is directly returned; if the second tracking hits, there is an occlusion between the hit point and the main light, and the direct light result, that is, 0, is directly returned in the simulation of one bounce; the physically meaningful light ray exists multiple bounces, and therefore the visual result will be dark only by such calculation. In order to approximate the multiple bounce result, the shadowMask value is set to 0.2 to return one fifth of the hit point color and mix part of the sky color, and finally the color result is projected to the spherical harmonic coefficient and stored in the data structure of the probe to complete the probe updating.

[0033] Step 3: ray result reconstruction and result denoising;

[0034] Specifically, when coloring, each block of the screen takes the value corresponding to the index in the buffer and performs color reconstruction, which is completed in the deferred rendering pipeline, first normally walking the deferred pipeline rendering, generating the corresponding Gbuffer, then obtaining the direct light coloring result before the post-processing stage, then taking the spherical harmonic coefficient of the light tracing result to reconstruct the lighting color, multiplying the lighting color with the base color buffer in the Gbuffer, and then adding it to the direct light coloring result to obtain the result.

[0035] After coloring, the image is denoised by filtering the image, and in order to preserve details, the sample weight is guided by depth, normal and brightness during filtering, that is, when the normal of the surrounding sample, that is, the first element of the guide weight, and the depth, that is, the second element of the guide weight, and the difference between the depth of the central sample and the depth of the probe central sample exceed the set threshold, the weight of the surrounding sample is reduced, and the third element of the guide weight is brightness. Most of the GI details come from the reflection of the bright surface in the scene. If uniform weight filtering is simply performed, the color of these bright samples will be distributed to the dark surface, resulting in loss of details. Therefore, adding brightness as an unconventional guide is beneficial to preserving bright surface details. When the normal difference exceeds 75° and the depth difference exceeds 0.01, the weight will be multiplied by 0.1 and 0.1 respectively, and the weight coefficient in the range is proportional to the difference, and the control coefficient is between 1 and 0.1. When the brightness difference exceeds the threshold, the weight is directly set to 0.

[0036] The time domain denoising scheme is introduced to stabilize the color of the same coloring point in the previous and subsequent frames. First, the motion vector of the current frame pixel is calculated, and the screen space coordinates of the pixel in the previous frame are calculated through the motion vector. The color of the pixel in the previous frame is obtained, the depth and normal difference thresholds are set, and the color of the previous frame is mixed in a ratio of 0.8 within the threshold range. In the case of a stationary scene and a moving camera, the same point in the scene may have different positions on the screen. If the same position pixel is still used for mixing, serious ghosting phenomenon may occur. Since the world space coordinates of the fragment do not change in this case, the coordinates are multiplied by the VP matrix of the previous frame to obtain the position of the fragment in the previous frame. This is the re-projection technology, that is, the information of the coloring point in the previous frame is located by calculating the motion vector as described above.

[0037] Compared with the prior art, the technical scheme provided by the present application supports completely dynamic scenes and dynamic lighting, has very high performance, and to some extent solves some defects of mainstream global illumination schemes. At the same time, a space-time combined denoising scheme is provided to efficiently and effectively denoise the GI result, and a relatively good visual performance is achieved.

[0038] Figure 2 As a performance performance diagram of the technical solution of the present application, the GPU performance delay of the present application is mainly affected by the number of probes. It can be seen that under the condition of medium number of probes, the increase of the number of light rays does not obviously lead to the rise of performance overhead, and the number of probes mainly affects the detail performance. The diffuse reflection lighting performance itself allows a certain loss of details. In the scene with less details, the performance of the present application is more prominent.

[0039] The above description is only the preferred embodiment of the present disclosure and the explanation of the applied technical principles. Those skilled in the art should understand that the scope of the application involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the embodiments of the present disclosure (but not limited to) having similar functions.

Claims

1. A dynamic global illumination method based on screen space probes, characterized in that: The following steps are involved: Step 1: Probe distribution and coding; Step 2: Ray tracing and probe update; Step 3: Light result reconstruction and result denoising; The step 1 is specifically as follows: firstly, obtaining the pixel world space position, and inferring the world space coordinates according to the screen space coordinates through the change matrix in rasterization and the depth generated by the delay pipeline; Implement probe placement in the Raytracing shader provided by RTX, using the first two components of the thread group as the block coordinates at the screen-space probe resolution, ultimately obtaining the world-space position of the screen-space computation probe distribution. Step 2 specifically includes: using the Hammerslay low-discrepancy sequence to generate hemispherical cosine weighted sampling directions; when there is a bright surface with brightness greater than a set threshold within the set shading point range and its contribution to the shading point exceeds half, a portion of samples is separated and sampled in the direction of maximum brightness; Using the hardware ray tracing architecture probe tracking and update, the screen space normal and depth map, the current thread ID and the main light direction are input into the ray generation shader, and then the world space coordinates of the shading point are restored according to the screen space UV and depth, and the Hammersley sequence sampling direction is generated according to the shading point normal. The starting point of the light is set to the world space coordinate of the shading point, and the direction is the generated sampling direction for the first tracking; when the first tracking misses, the first miss shader is executed to directly return the sampled sky color as the color, and the second miss shader changes the loaded shadowmask value to 1. When the second tracking misses, there is no object blocking the hit point and the main light, and the hit point color recorded during the first hit is directly returned; if a hit occurs in the second tracking, there is occlusion between the hit point and the main light. In the case of simulating a bounce, the direct light result, which is 0, is directly returned; the shadowMask value is set to 0.2 to return one-fifth of the hit point color and mix in part of the sky light color. The final color result is projected onto the spherical harmonic coefficient and stored in the probe data structure to complete the probe update; The step 3 is specifically as follows: when shading, each block of the screen takes the value of the corresponding index in the buffer and performs color reconstruction. This step is completed in the deferred rendering pipeline. First, the deferred pipeline rendering is performed normally to generate the corresponding Gbuffer. Then, before the post-processing stage, the direct light shading result is obtained. Then, the spherical harmonic coefficients of the ray tracing result are taken to reconstruct the lighting color. The lighting color is multiplied by the base color buffer in the Gbuffer, and then added to the direct light shading result to obtain the result. After the shading is completed, the image is denoised by filtering the image. The sample weights sampled during filtering are guided by depth, normal and brightness. That is, when the differences between the normal, depth, center sample depth and probe center sample of the surrounding samples exceed the set threshold, the weights of the surrounding samples are reduced, where the normal of the surrounding samples is the first element of the guidance weight, the depth is the second element of the guidance weight, and the brightness is the third element of the guidance weight.

2. The method for dynamic global illumination based on screen space probe according to claim 1, characterized in that: A time-domain noise reduction scheme is introduced to stabilize the color of the same shading point in the previous and next frames. First, the motion vector of the current frame pixel is calculated, and the screen space coordinates of the previous frame pixel are calculated through this motion vector to obtain the color of the previous frame pixel. The depth and normal difference thresholds are set. If the color of the previous frame is judged to be credible within the threshold range, the color of the previous frame is mixed in at a ratio of 0.

8. When the scene is stationary and the camera moves, the world space coordinates of the fragment are multiplied by the VP matrix of the previous frame to obtain the position of the fragment in the previous frame. This is the reprojection technology, which locates the information of the shading point of the previous frame by calculating the motion vector.

Images