A Hybrid Screen-Spatial Global Illumination Method Based on Resampling and Probes

Abstract

This invention provides a hybrid screen-space global illumination method based on resampling and probes, relating to the field of global illumination technology. The method first pre-computes and bakes light field probes in an offline environment; then renders the geometry buffer and updates the light field probes; generates initial samples using screen-space ray tracing; performs temporal and spatial resampling; obtains indirect diffuse lighting results; performs deferred shading using the generated geometry buffer, indirect diffuse lighting, light source data, and shadow information; and samples indirect diffuse lighting using the screen-space coordinates of pixels, overlaying it with the direct lighting results to obtain the final shading result.

Description

Technical Field

[0001] This invention relates to the field of global illumination technology, and in particular to a hybrid screen-space global illumination method based on resampling and probes. Background Technology

[0002] With the rapid popularization and development of interactive 3D digital applications, users have increasingly higher demands for visual presentation and lighting effects. Global Illumination (GI) is a crucial component of realistic rendering, essentially simulating lighting effects in the real world. GI primarily focuses on two aspects: direct lighting and indirect lighting. Direct lighting refers to the lighting effects produced when light originates from a light source and directly illuminates the surface of an object; while indirect lighting takes into account the lighting effects produced by light rays bouncing between object surfaces. Currently, there are several methods that can achieve global illumination in real-time environments.

[0003] Pre-computed global illumination methods simulate complex lighting effects in a scene by calculating the lighting effects of static scenes and static light sources in an offline environment, encoding and storing them in a lightmap, and loading and retrieving them at runtime. The drawbacks of this method are its long pre-computation time and its applicability only to static scenes and static light sources; it cannot support dynamic scenes or light sources.

[0004] Probe-based global illumination methods capture and encode irradiance information within a spherical area at a specific location in the scene. During shading, the illumination information from probes near the shading point is interpolated to achieve indirect lighting effects. Depending on the data structure and update time of the probes, different global illumination effects can be achieved. Specifically, the light field probe records the scene's surface properties (albedo, world space normal, and world space position) within the spherical area during the pre-computation phase. At runtime, it uses the stored surface properties and light source information for re-illumination, updating its own irradiance distribution with the updated illumination data to achieve global illumination effects under dynamic light sources. The drawback of this method is the difficulty in balancing the number of probes with global illumination accuracy: achieving high-precision global illumination requires increasing the probe density, but this increases the offline pre-computation time and the overhead of runtime probe updates.

[0005] Ray tracing-based global illumination methods achieve high-precision global illumination effects by simulating the propagation path of light rays in a scene. These methods typically require a data structure to represent scene information, such as a Sign Distance Field (SDF), Hierarchical Z-buffer (HiZ), or Bounding Volume Hierarchy (BVH). The disadvantages of these methods include high runtime overhead, requiring a low sample-per-pixel (spp) count to meet real-time requirements, resulting in slow convergence and significant noise issues necessitating complex denoising processors for post-processing. Furthermore, some methods rely on specific hardware units, namely ray tracing units. Screen-space ray tracing methods use hierarchical Z-buffers as scene descriptions. Their advantage lies in the fact that algorithm complexity is independent of scene complexity, resulting in relatively stable runtime overhead. However, this method cannot capture scene information outside the scene itself.

[0006] Global illumination methods based on light field probes cannot achieve high-precision global illumination effects because they require baking the scene's geometry during the pre-computation stage and re-illuminating during the runtime stage. These overheads limit the number and density of light field probes.

[0007] Ray tracing-based global illumination methods can achieve high-precision global illumination effects. However, on the one hand, hardware-accelerated ray tracing relies on specific hardware units and has poor compatibility; on the other hand, the limited frame rendering time makes it difficult for ray tracing methods to converge, requiring the use of very complex denoisers to eliminate image noise. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a hybrid screen-space global illumination method based on resampling and probes, which can provide high-quality and high-precision global illumination effects.

[0009] A hybrid screen-space global illumination method based on resampling and probes includes the following steps:

[0010] Step 1: Pre-calculate and bake the light field probe in an offline environment;

[0011] Step 1.1: Place probe groups; Place evenly spaced light field probe groups in the scene to cover the scene area where global illumination is needed;

[0012] Step 1.2: Baking the cube texture;

[0013] For each probe, three cube maps are rendered with the probe's location as the center, storing the surface albedo, world space normal, and world space coordinates respectively.

[0014] Step 1.3: Sample the cube map;

[0015] For each probe, a certain number of sampling directions are uniformly generated within the spherical area, and the sampling directions are used to sample three cube maps respectively. The surface properties obtained by sampling are stored in the corresponding buffers respectively. There are three buffers of the same size, which store the albedo, world space normal and world space coordinates of the corresponding surface points respectively. Each set of albedo, world space normal and world space coordinates represents a geometric point on the surface, which is called a surface element (Surfel).

[0016] Step 2: Render the geometry buffer;

[0017] Step 2.1: Prepare material, geometry, light source, and texture data, and initiate the drawing call;

[0018] Step 2.2: Using the Multiple Render Targets (MRT) technique, albedo, world space normal, metallicity, and roughness are stored in different channels of a set of geometry buffers (G-Buffers).

[0019] Step 3: Update the light field probe;

[0020] Step 3.1: Select the probe to be updated;

[0021] In each frame, 10% of the probes that have never been updated are selected and added to the queue to be updated, meaning that all probes are updated once every 10 frames.

[0022] Step 3.2: Calculate the direct lighting of the surface element;

[0023] For each facet in the probe, its stored surface properties are read, and the shading result of the facet is calculated using the light source and shadow information of the current frame and the Lambert lighting model, thereby obtaining the direct lighting result of the facet; where the Lambert lighting model is an empirical model for simulating diffuse reflection lighting on rough surfaces.

[0024] Step 3.3: Calculate the indirect lighting of the surface element;

[0025] In each light field probe, a second-order spherical harmonic function is used to describe the illumination information within the spherical area at the probe location;

[0026] For each surface element in the probe, the second-order spherical harmonic function in the light field probe of the previous frame is sampled using the world space position of the surface element, and the sampling results are weighted and mixed to obtain the indirect lighting result of the surface element.

[0027] Step 3.4: Calculate the spherical harmonic coefficients of the probe;

[0028] The direct illumination result obtained in step 3.2 is mixed with the indirect illumination result obtained in step 3.3 to obtain the outgoing radiance of the surface element. The outgoing radiance is then projected onto the second-order spherical harmonic function, and the coefficients of the second-order spherical harmonic function are stored in a buffer specifically for storing coefficients.

[0029] Step 4: Generate initial samples using screen-space ray tracing;

[0030] Step 4.1: Generate the HiZ buffer;

[0031] Based on the set maximum level, the compute shader generates different levels of HiZ buffers from low to high. At level 0, the depth buffer of the current frame is directly sampled. In each subsequent level, the four depth values ​​around the corresponding pixel in the HiZ buffer of the previous level are sampled, and the depth value closest to the camera position is used as the value in the current level.

[0032] Step 4.2: Prepare lighting data;

[0033] For each screen pixel, sample the geometry buffer of the current frame to obtain the world space normal and depth, and combine them with the screen space coordinates of the pixel to reconstruct the world space position corresponding to the pixel.

[0034] Step 4.3: Generate sampling direction;

[0035] A random sequence is generated using a random seed and Hammersley sampling method, and then the Cosine-Weighted sampling algorithm within the hemispherical area is used to generate the light sampling direction within the hemisphere.

[0036] Step 4.4: Ray stepping;

[0037] First, a tentative step is taken using the minimum step size, sampling the lowest-level HiZ buffer and comparing it with the current ray depth. If the difference between the ray depth and the sampling depth is less than a set threshold, the ray is considered to intersect the surface; otherwise, there is no intersection. After the ray intersects the surface, it backs down one level according to the current sampling HiZ buffer level and continues to step, repeating this process until it intersects with the lowest-level HiZ buffer. At this point, the screen space coordinates of the intersecting pixel are returned. If there is no intersection, the step size is increased, and the step is continued, sampling a higher-level HiZ buffer as the current surface depth value, until an intersection occurs or the ray exceeds the screen range. If the ray intersects the scene surface and the returned pixel screen space coordinates are valid, proceed to step 4.5; otherwise, proceed to step 4.6. The starting position of the ray is called the visible point, and the intersection point of the ray and the scene is called the sampling point.

[0038] Step 4.5: Tracking successful, generate initial samples using screen space information;

[0039] For each screen pixel, using the pixel screen space coordinates from step 4.4, the geometric buffer information of the current frame is sampled to obtain the geometric information of the sampling point, namely the world space position and normal direction of the sampling point; the screen space coordinates of the sampling point are reprojected onto the previous frame, and the final shading result of the previous frame is sampled to finally obtain the outgoing radiance of the sampling point; the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point are written into the initial reservoir buffer.

[0040] Step 4.6: Tracking failed; use the light field probe to generate an initial sample.

[0041] Using the world space location of the viewpoint, find the 8 nearest light field probes around that location, and sample the spherical harmonic function using the sampling direction of the viewpoint to obtain the incident radiance in that direction; use the difference between the direction from the viewpoint to the probe and the normal direction of the viewpoint as a weight to perform a weighted mixture of the 8 incident radiances; assuming that the sampling point is located at a certain distance from the viewpoint along the sampling direction, construct the geometric information of the sampling point, and finally write the geometric information of the viewpoint and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer;

[0042] Step 5: Time resampling;

[0043] Step 5.1: Reprojection: For each pixel on the screen, use the motion vector to project it into the previous frame and find the corresponding pixel in the previous frame;

[0044] Step 5.2: Validity check: Read and calculate the depth difference and normal difference of the pixels between the current frame and the previous frame. If the two differences are less than the set threshold, the reprojection is considered successful and proceed to step 5.3; otherwise, it fails and proceed to step 5.4.

[0045] Step 5.3: Reprojection successful, perform temporal resampling; the newly generated reservoir for each frame is called the initial reservoir; the reservoir cached from the previous frame is called the temporal reservoir. Read the initial reservoir of the current frame and the temporal reservoir of the previous frame, use the brightness value of the emitted radiance as the weight, update the temporal reservoir using samples from the initial reservoir, and write the updated reservoir back to the temporal reservoir; during the update, the number of reused frames of the sample needs to be compared with the maximum number of reused frames. If the number of reused frames of the sample is greater than the maximum number of reused frames, then directly read the initial reservoir of the current frame and write it directly to the corresponding position in the temporal reservoir;

[0046] Step 5.4: Reprojection failed, so temporal resampling was not performed. The initial reservoir of the current frame was read and directly written to the corresponding position in the temporal reservoir.

[0047] Step 6: Spatial resampling;

[0048] Step 6.1: Select the number of iterations: For each pixel on the screen, read the corresponding time reservoir, obtain the number of frames that the sample has been reused, and compare it with a threshold. If it is less than the threshold, iterate 5 times; otherwise, iterate 3 times. In each iteration, repeat steps 6.2-6.4.

[0049] Step 6.2: Randomly select an adjacent reservoir: Read the time reservoir of the current pixel, and in each iteration, randomly select a pixel within a certain radius and read the time reservoir of that pixel;

[0050] Step 6.3: Validity Check: Load the geometric attributes of the current visible pixel and the selected visible pixel, and compare the positional differences and normal differences. If both differences are less than the set threshold, the two visible pixels are considered to come from the same surface, and the samples stored at the corresponding positions can be spatially reused, proceeding to Step 6.4; otherwise, the two visible pixels are considered to come from different surfaces, and the samples stored at the corresponding positions cannot be spatially reused, returning to Step 6.2.

[0051] Step 6.4: Perform spatial reuse: Use the samples in the temporal reservoir of the selected pixel to update the temporal reservoir of the current pixel, and write the updated reservoir back to the spatial reservoir buffer at the corresponding position;

[0052] Step 7: Obtain the results of indirect diffuse reflection illumination;

[0053] Step 7.1: Read the sample information in the time reservoir, multiply the radiance of the sample by the reservoir weight, BRDF, and cosine term to obtain the indirect diffuse reflection illumination result.

[0054] Step 7.2: Perform spatial noise reduction to eliminate residual noise in the result. Use a separable bilateral filter with depth difference and normal difference as filter weights. First, perform a horizontal filter, then a vertical filter to eliminate residual noise in the indirect diffuse illumination and obtain the final indirect diffuse illumination result.

[0055] Step 8: Delayed shading is performed using the geometric buffer generated in Step 2, the indirect diffuse lighting generated in Step 7, and the light source data and shadow information; the indirect diffuse lighting is sampled using the screen space coordinates of the pixels and superimposed with the direct lighting results to obtain the final shading result.

[0056] The beneficial effects of adopting the above technical solution are as follows:

[0057] This invention provides a hybrid screen-space global illumination method based on spatiotemporal resampling and light field probes. Building upon screen-space ray tracing, this invention introduces dynamically updated light field probes to supplement the initial sample generation and further applies spatiotemporal resampling to improve sample quality and accelerate convergence. This solves the problem of low accuracy in light field probe-based global illumination methods and improves the slow convergence speed of ray tracing-based global illumination methods. Attached Figure Description

[0058] Figure 1 This is an overall flowchart of the hybrid screen space global illumination method in an embodiment of the present invention;

[0059] Figure 2 This is a result diagram of the original screen space global illumination in an embodiment of the present invention;

[0060] Figure 3 This is a diagram showing the result of original screen space global illumination plus spatiotemporal resampling in an embodiment of the present invention;

[0061] Figure 4 This is a result diagram of the hybrid screen space global illumination method in an embodiment of the present invention;

[0062] Figure 5 This is the result of path tracing in an embodiment of the present invention. Detailed Implementation

[0063] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0064] A hybrid screen-space global illumination method based on resampling and probes, such as Figure 1 As shown, the following steps are included:

[0065] Step 1: Pre-calculate and bake the light field probe in an offline environment;

[0066] Step 1.1: Place probe groups; Place evenly spaced light field probe groups in the scene to cover the scene area that needs to generate global illumination; In this embodiment, the spacing between probes is 4 meters under the default settings.

[0067] Step 1.2: Baking the cube texture;

[0068] For each probe, three cube maps are rendered with the probe's location as the center, storing the surface albedo, world space normal, and world space coordinates respectively. In this embodiment, the cube map resolution is 256*256*6 under the default settings.

[0069] Step 1.3: Sample the cube map;

[0070] For each probe, a certain number of sampling directions are uniformly generated within the spherical area, with a default of 128. The sampling directions are then used to sample three cube maps, and the sampled surface properties are stored in their respective buffers. There are three buffers of the same size, which store the albedo, world space normal, and world space coordinates of the corresponding surface points. Each set of albedo, world space normal, and world space coordinates represents a geometric point on the surface, which is called a Surfel.

[0071] Step 2: Render the geometry buffer;

[0072] Step 2.1: Prepare material, geometry, light source, and texture data, and initiate the drawing call;

[0073] Step 2.2: Using the Multiple Render Targets (MRT) technique, albedo, world space normal, metallicity, and roughness are stored in different channels of a set of geometry buffers (G-Buffers).

[0074] Step 3: Update the light field probe;

[0075] Step 3.1: Select the probe to be updated;

[0076] In each frame, 10% of the probes that have never been updated are selected and added to the queue to be updated, meaning that all probes are updated once every 10 frames.

[0077] Step 3.2: Calculate the direct lighting of the surface element;

[0078] For each facet in the probe, its stored surface properties are read, and the shading result of the facet is calculated using the light source and shadow information of the current frame and the Lambert lighting model, thereby obtaining the direct lighting result of the facet; where the Lambert lighting model is an empirical model for simulating diffuse reflection lighting on rough surfaces.

[0079] Step 3.3: Calculate the indirect lighting of the surface element;

[0080] In each light field probe, a second-order spherical harmonic function is used to describe the illumination information within the spherical range of the probe location; the spherical harmonic function is a spherical basis function used to describe the distribution of a certain signal within the spherical range, and is widely used in global illumination methods.

[0081] For each surface element in the probe, the second-order spherical harmonic function in the light field probe of the previous frame is sampled using the world space position of the surface element, and the sampling results are weighted and mixed to obtain the indirect lighting result of the surface element; thus achieving an infinite bounce lighting effect.

[0082] Step 3.4: Calculate the spherical harmonic coefficients of the probe;

[0083] The direct illumination result obtained in step 3.2 is mixed with the indirect illumination result obtained in step 3.3 to obtain the outgoing radiance of the surface element. The outgoing radiance is then projected onto the second-order spherical harmonic function, and the coefficients of the second-order spherical harmonic function are stored in a buffer specifically for storing coefficients.

[0084] Step 4: Generate initial samples using screen-space ray tracing;

[0085] Step 4.1: Generate the HiZ buffer;

[0086] Based on the set maximum level, the compute shader generates different levels of HiZ buffers from low to high. At level 0, the depth buffer of the current frame is directly sampled. In each subsequent level, the four depth values ​​around the corresponding pixel in the HiZ buffer of the previous level are sampled, and the depth value closest to the camera position is used as the value in the current level.

[0087] Step 4.2: Prepare lighting data;

[0088] For each screen pixel, sample the geometry buffer of the current frame to obtain the world space normal and depth, and combine them with the screen space coordinates of the pixel to reconstruct the world space position corresponding to the pixel.

[0089] Step 4.3: Generate sampling direction;

[0090] A low-discrepancy random sequence is generated using a random seed and Hammersley sampling method, and then the Cosine-Weighted sampling algorithm within the hemispherical area is used to generate the light sampling direction within the hemisphere.

[0091] Step 4.4: Ray stepping;

[0092] First, a tentative step is performed using the minimum step size, sampling the lowest-level HiZ buffer and comparing it with the current ray depth. If the difference between the ray depth and the sampling depth is less than a set threshold, the ray is considered to intersect the surface; otherwise, there is no intersection. After the ray intersects the surface, it backs down one level based on the current sampling HiZ buffer level and continues stepping, repeating this process until it intersects with the lowest-level HiZ buffer. At this point, the screen space coordinates of the intersecting pixel are returned. If there is no intersection, the step size is increased, and the stepping continues, sampling a higher-level HiZ buffer as the current surface depth value, until an intersection occurs or the ray exceeds the screen range. In this embodiment, the ray steps 50 times by default. If the ray intersects the scene surface and the returned pixel screen space coordinates are valid, proceed to step 4.5; otherwise, proceed to step 4.6. The starting position of the ray is called the visible point, and the intersection point of the ray and the scene is called the sampling point.

[0093] Step 4.5: Tracking successful, generate initial samples using screen space information;

[0094] For each screen pixel, using the pixel screen space coordinates from step 4.4, the geometric buffer information of the current frame is sampled to obtain the geometric information of the sampling point, namely the world space position and normal direction of the sampling point; the screen space coordinates of the sampling point are reprojected onto the previous frame, and the final shading result of the previous frame is sampled to achieve an infinite bounce global illumination effect, and finally the outgoing radiance of the sampling point is obtained; the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point are written into the initial reservoir buffer;

[0095] Step 4.6: Tracking failed; use the light field probe to generate an initial sample.

[0096] Using the world space location of the viewpoint, find the 8 nearest light field probes around that location, and sample the spherical harmonic function using the sampling direction of the viewpoint to obtain the incident radiance in that direction; use the difference between the direction from the viewpoint to the probe and the normal direction of the viewpoint as a weight to perform a weighted mixture of the 8 incident radiances; assuming that the sampling point is located at a certain distance from the viewpoint along the sampling direction, construct the geometric information of the sampling point, and finally write the geometric information of the viewpoint and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer;

[0097] Step 5: Time resampling;

[0098] Step 5.1: Reprojection: For each pixel on the screen, use the motion vector to project it into the previous frame and find the corresponding pixel in the previous frame;

[0099] Step 5.2: Validity check: Read and calculate the depth difference and normal difference of the pixels between the current frame and the previous frame. If the two differences are less than the set threshold, the reprojection is considered successful and proceed to step 5.3; otherwise, it fails and proceed to step 5.4.

[0100] Step 5.3: Reprojection successful, perform temporal resampling; the newly generated reservoir for each frame is called the initial reservoir; the reservoir cached from the previous frame is called the temporal reservoir. Read the initial reservoir of the current frame and the temporal reservoir of the previous frame, use the brightness value of the emitted radiance as the weight, update the temporal reservoir using samples from the initial reservoir, and write the updated reservoir back to the temporal reservoir; during the update, the number of reused frames of the sample needs to be compared with the maximum number of reused frames. If the number of reused frames of the sample is greater than the maximum number of reused frames, then directly read the initial reservoir of the current frame and write it directly to the corresponding position in the temporal reservoir;

[0101] Step 5.4: Reprojection failed, so temporal resampling was not performed. The initial reservoir of the current frame was read and directly written to the corresponding position in the temporal reservoir.

[0102] Step 6: Spatial resampling;

[0103] Step 6.1: Select the number of iterations: For each pixel on the screen, read the corresponding time reservoir, obtain the number of frames that the sample has been reused, and compare it with a threshold. If it is less than the threshold, iterate 5 times; otherwise, iterate 3 times. In each iteration, repeat steps 6.2-6.4.

[0104] Step 6.2: Randomly select an adjacent reservoir: Read the time reservoir of the current pixel, and in each iteration, randomly select a pixel within a certain radius and read the time reservoir of that pixel;

[0105] Step 6.3: Validity Check: Load the geometric attributes of the current visible pixel and the selected visible pixel, and compare the positional differences and normal differences. If both differences are less than the set threshold, the two visible pixels are considered to come from the same surface, and the samples stored at the corresponding positions can be spatially reused, proceeding to Step 6.4; otherwise, the two visible pixels are considered to come from different surfaces, and the samples stored at the corresponding positions cannot be spatially reused, returning to Step 6.2.

[0106] Step 6.4: Perform spatial reuse: Use the samples in the temporal reservoir of the selected pixel to update the temporal reservoir of the current pixel, and write the updated reservoir back to the spatial reservoir buffer at the corresponding position;

[0107] Step 7: Obtain the results of indirect diffuse reflection illumination;

[0108] Step 7.1: Read the sample information in the time reservoir, multiply the radiance of the sample by the reservoir weight, BRDF, and cosine term to obtain the indirect diffuse reflection illumination result.

[0109] Step 7.2: Perform spatial noise reduction to eliminate residual noise in the result. Use a separable bilateral filter with depth difference and normal difference as filter weights. First, perform a horizontal filter, then a vertical filter to eliminate residual noise in the indirect diffuse illumination and obtain the final indirect diffuse illumination result.

[0110] Step 8: Delayed shading is performed using the geometric buffer generated in Step 2, the indirect diffuse lighting generated in Step 7, and the light source data and shadow information; the indirect diffuse lighting is sampled using the screen space coordinates of the pixels and superimposed with the direct lighting results to obtain the final shading result.

[0111] This invention improves the screen-space-based global illumination method by introducing a light field probe method to increase the number of effective initial samples and by introducing a spatiotemporal resampling method to improve sample quality and increase convergence speed. This invention can provide high-quality, high-precision, and high-stability diffuse global illumination effects under both static scenes and dynamic light sources. Furthermore, by introducing the light field probe method to generate effective initial samples, it can also mitigate the bias caused by spatiotemporal resampling, resulting in a more physically accurate image. Figures 2-5 The images from different solutions are shown below. Figure 2 This is the result of the original screen-space global illumination. Figure 3 This is the result of adding spatiotemporal resampling to the original screen-space global illumination. Figure 4 For the results of this solution, the closest path tracing is... Figure 5The result.

[0112] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A hybrid screen-space global illumination method based on resampling and probes, characterized in that, The following steps are involved: Step 1: Pre-calculate and bake the light field probe in an offline environment; Step 2: Render the geometry buffer; Step 3: Update the light field probe; Step 4: Generate initial samples using screen-space ray tracing; Step 4 includes the following steps: Step 4.1: Generate the HiZ buffer; Based on the set maximum level, the compute shader generates different levels of HiZ buffers from low to high. At level 0, the depth buffer of the current frame is directly sampled. In each subsequent level, the four depth values ​​around the corresponding pixel in the HiZ buffer of the previous level are sampled, and the depth value closest to the camera position is used as the value in the current level. Step 4.2: Prepare lighting data; For each screen pixel, sample the geometry buffer of the current frame to obtain the world space normal and depth, and combine them with the screen space coordinates of the pixel to reconstruct the world space position corresponding to the pixel. Step 4.3: Generate sampling direction; A random sequence is generated using a random seed and Hammersley sampling method, and then the Cosine-Weighted sampling algorithm within the hemispherical area is used to generate the light sampling direction within the hemisphere. Step 4.4: Ray stepping; First, a tentative step is taken using the minimum step size, sampling the lowest-level HiZ buffer and comparing it with the current ray depth. If the difference between the ray depth and the sampling depth is less than a set threshold, the ray is considered to intersect the surface; otherwise, there is no intersection. After the ray intersects the surface, it backs down one level according to the current sampling HiZ buffer level and continues to step, repeating this process until it intersects with the lowest-level HiZ buffer. At this point, the screen space coordinates of the intersecting pixel are returned. If there is no intersection, the step size is increased, and the step is continued, sampling a higher-level HiZ buffer as the current surface depth value, until an intersection occurs or the ray exceeds the screen range. If the ray intersects the scene surface and the returned pixel screen space coordinates are valid, proceed to step 4.5; otherwise, proceed to step 4.

6. The starting position of the ray is called the visible point, and the intersection point of the ray and the scene is called the sampling point. Step 4.5: Tracking successful, generate initial samples using screen space information; For each screen pixel, using the pixel screen space coordinates from step 4.4, the geometric buffer information of the current frame is sampled to obtain the geometric information of the sampling point, namely the world space position and normal direction of the sampling point; the screen space coordinates of the sampling point are reprojected onto the previous frame, and the final shading result of the previous frame is sampled to finally obtain the outgoing radiance of the sampling point; the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point are written into the initial reservoir buffer. Step 4.6: Tracking failed; use the light field probe to generate an initial sample. Using the world space location of the viewpoint, find the 8 nearest light field probes around that location, and sample the spherical harmonic function using the sampling direction of the viewpoint to obtain the incident radiance in that direction; use the difference between the direction from the viewpoint to the probe and the normal direction of the viewpoint as a weight to perform a weighted mixture of the 8 incident radiances; assuming that the sampling point is located at a certain distance from the viewpoint along the sampling direction, construct the geometric information of the sampling point, and finally write the geometric information of the viewpoint and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer; Step 5: Time resampling; Step 5 includes the following steps: Step 5.1: Reprojection: For each pixel on the screen, use the motion vector to project it into the previous frame and find the corresponding pixel in the previous frame; Step 5.2: Validity check: Read and calculate the depth difference and normal difference of the pixels between the current frame and the previous frame. If the two differences are less than the set threshold, the reprojection is considered successful and proceed to step 5.3; otherwise, it fails and proceed to step 5.

4. Step 5.3: Reprojection successful, perform temporal resampling; the newly generated reservoir for each frame is called the initial reservoir; the reservoir cached from the previous frame is called the temporal reservoir; read the initial reservoir of the current frame and the temporal reservoir of the previous frame, use the brightness value of the emitted radiance as the weight, update the temporal reservoir using the samples in the initial reservoir, and write the updated reservoir back to the temporal reservoir; during the update, the number of reused frames of the sample needs to be compared with the maximum number of reused frames. If the number of reused frames of the sample is greater than the maximum number of reused frames, then directly read the initial reservoir of the current frame and write it directly to the corresponding position in the temporal reservoir. Step 5.4: Reprojection fails, no time resampling is performed; the initial reservoir of the current frame is read and directly written to the corresponding position in the time reservoir; Step 6: Spatial resampling; Step 6 includes the following steps: Step 6.1: Select the number of iterations: For each pixel on the screen, read the corresponding time reservoir, obtain the number of frames that the sample has been reused, and compare it with the threshold; if it is less than the threshold, iterate 5 times; otherwise, iterate 3 times; in each iteration, repeat steps 6.2-6.

4. Step 6.2: Randomly select an adjacent reservoir: Read the time reservoir of the current pixel, and in each iteration, randomly select a pixel within a certain radius and read the time reservoir of that pixel; Step 6.3: Validity check: Load the geometric attributes of the current visible pixel and the selected visible pixel, and compare the positional difference and normal difference; if both differences are less than the set threshold, then the two visible pixels are considered to come from the same surface, and the samples stored in the corresponding positions can be spatially reused, and proceed to step 6.4; otherwise, the two visible pixels are considered to come from different surfaces, and the samples stored in the corresponding positions cannot be spatially reused, and return to step 6.2; Step 6.4: Perform spatial reuse: Use the samples in the temporal reservoir of the selected pixel to update the temporal reservoir of the current pixel, and write the updated reservoir back to the spatial reservoir buffer at the corresponding position; Step 7: Obtain the results of indirect diffuse reflection illumination; Step 8: Delayed shading is performed using the geometric buffer generated in Step 2, the indirect diffuse lighting generated in Step 7, and the light source data and shadow information; the indirect diffuse lighting is sampled using the screen space coordinates of the pixels and superimposed with the direct lighting results to obtain the final shading result.

2. The hybrid screen-space global illumination method based on resampling and probes according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step 1.1: Place probe groups; Place evenly spaced light field probe groups in the scene to cover the scene area where global illumination is needed; Step 1.2: Baking the cube texture; For each probe, three cube maps are rendered with the probe's location as the center, storing the surface albedo, world space normal, and world space coordinates respectively. Step 1.3: Sample the cube map; For each probe, a certain number of sampling directions are uniformly generated within the spherical area, and the sampling directions are used to sample the three cube maps respectively. The surface attributes obtained by sampling are stored in the corresponding buffers respectively. There are three buffers of the same size, which store the albedo, world space normal and world space coordinates of the corresponding surface points respectively. Each set of albedo, world space normal and world space coordinates represents a geometric point on the surface, which is called a surface element Surfel.

3. The hybrid screen-space global illumination method based on resampling and probe according to claim 1, characterized in that, Step 2 includes the following steps: Step 2.1: Prepare material, geometry, light source, and texture data, and initiate the drawing call; Step 2.2: Using the Multiple Render Targets (MRT) technique, albedo, world space normal, metallicity, and roughness are stored in different channels of a set of geometry buffers (G-Buffers).

4. The hybrid screen-space global illumination method based on resampling and probe according to claim 1, characterized in that, Step 3 includes the following steps: Step 3.1: Select the probe to be updated; In each frame, 10% of the probes that have never been updated are selected and added to the queue to be updated, meaning that all probes are updated once every 10 frames. Step 3.2: Calculate the direct lighting of the surface element; For each facet in the probe, its stored surface properties are read, and the shading result of the facet is calculated using the light source and shadow information of the current frame and the Lambert lighting model, thereby obtaining the direct lighting result of the facet; where the Lambert lighting model is an empirical model for simulating diffuse lighting on rough surfaces. Step 3.3: Calculate the indirect lighting of the surface element; In each light field probe, a second-order spherical harmonic function is used to describe the illumination information within the spherical area at the probe location; For each surface element in the probe, the second-order spherical harmonic function in the light field probe of the previous frame is sampled using the world space position of the surface element, and the sampling results are weighted and mixed to obtain the indirect lighting result of the surface element. Step 3.4: Calculate the spherical harmonic coefficients of the probe; The direct illumination result obtained in step 3.2 is mixed with the indirect illumination result obtained in step 3.3 to obtain the outgoing radiance of the surface element. The outgoing radiance is then projected onto a second-order spherical harmonic function, and the coefficients of the second-order spherical harmonic function are stored in a buffer specifically for storing coefficients.

5. The hybrid screen-space global illumination method based on resampling and probe according to claim 1, characterized in that, Step 7 includes the following steps: Step 7.1: Read the sample information in the time reservoir, multiply the radiance of the sample by the reservoir weight, BRDF, and cosine term to obtain the indirect diffuse reflection illumination result; Step 7.2: Perform spatial noise reduction to eliminate residual noise in the result; use a separable bilateral filter with depth difference and normal difference as filter weights, that is, first perform a horizontal filter and then a vertical filter to eliminate residual noise in indirect diffuse lighting and obtain the final indirect diffuse lighting result.

Images