Hybrid screen space global illumination method based on resampling and probe

Abstract

The invention provides a hybrid screen space global illumination method based on resampling and a probe, and relates to the technical field of global illumination, and the method comprises the steps: firstly, in an offline environment, pre-calculating a baking light field probe; rendering a geometric buffer area, and updating a light field probe; using screen space ray tracing to generate an initial sample; performing time resampling and space resampling; obtaining an indirect diffuse reflection illumination result; carrying out delayed coloring by utilizing the generated geometric buffer area, indirect diffuse reflection illumination, light source data and shadow information; and sampling indirect diffuse reflection illumination by using the screen space coordinates of the pixels, and superposing the indirect diffuse reflection illumination with a direct illumination result, thereby obtaining a final coloring result.

Description

Technical Field

[0001] The present invention relates to the field of global illumination technology, and in particular to a hybrid screen space global illumination method based on resampling and probes. Background Art

[0002] With the rapid popularization and development of interactive 3D digital applications, users have higher and higher requirements for picture performance and lighting effects. Global illumination technology (GI) is an important part of realistic rendering. It is essentially a simulation of lighting effects in the real world. Global illumination technology is mainly concerned with direct lighting and indirect lighting. Direct lighting refers to the lighting effect produced by the light source directly shining on the surface of the object; while indirect lighting refers to the lighting effect produced by the light bouncing between the surfaces of the object. There are currently a series of methods that can achieve global illumination in real-time environments.

[0003] The pre-calculated global illumination method simulates the complex lighting effects in the scene by calculating the lighting effects of static scenes and static light sources in an offline environment, encoding them in a certain way and storing them in a light map, and loading and reading them at runtime. The disadvantage of this method is that the pre-calculation time is long, and it is only applicable to static scenes and static light sources, and cannot support dynamic scenes or light sources.

[0004] The probe-based global illumination method captures and encodes the irradiance information within the spherical range at a certain position in the scene; and in the shading process, the illumination information in the probe near the shading point is interpolated to obtain the indirect illumination effect. According to the data structure and update time stored in the probe, the global illumination effect in different situations can be achieved. Among them, the light field probe will record the surface properties of the scene, namely albedo, world space normal and world space position, within the spherical range during the pre-calculation stage; and use the stored surface properties and light source information for re-illumination at runtime, and use the updated illumination data to update its own radiance distribution, so as to achieve the global illumination effect under dynamic light sources. The disadvantage of this method is that it is difficult to balance the number of probes and the accuracy of global illumination: to achieve high-precision global illumination effects, the density of probes needs to be increased, but this will increase the time of offline pre-calculation of probes and the overhead of updating probes at runtime.

[0005] Tracing-based global illumination methods can achieve high-precision global illumination effects by simulating the propagation path of light in the scene. Such methods usually require the use of some data structure to express scene information, such as Sign Distance Field (SDF), Hierarchical Z-buffer (HiZ), Bounding Volume Hierarchy (BVH), etc. The disadvantage of such methods is that the runtime overhead is high. Therefore, in order to meet the real-time requirements, only a low number of samples per pixel (spp) can be used, so the convergence speed is slow and very serious noise problems will occur, which requires the use of very complex denoisers for subsequent processing; and some of these methods also rely on specific hardware units, namely ray tracing units. The screen-space-based ray tracing method uses a hierarchical Z buffer as a scene description. Its advantage is that the algorithm complexity is independent of the scene complexity and has a relatively stable runtime overhead, but this method cannot capture scene information outside the scene.

[0006] The global illumination method based on light field probes needs to bake the geometric information of the scene in the pre-calculation stage and re-illuminate at runtime. These overheads limit the number and density of light field probes, and therefore cannot achieve high-precision global illumination effects.

[0007] The global illumination method based on ray tracing can achieve high-precision global illumination effects. However, on the one hand, ray tracing based on hardware acceleration depends on specific hardware units and has poor compatibility. On the other hand, the limited frame rendering time makes it difficult for the ray tracing method to converge, and a very complex denoiser is required to eliminate image noise. Summary of the invention

[0008] In view of the deficiencies in the prior art, the present invention provides a hybrid screen space global illumination method based on resampling and probes, which can provide high-quality and high-precision global illumination effects.

[0009] A hybrid screen space global illumination method based on resampling and probes includes the following steps:

[0010] Step 1: Precompute and bake light field probes in an offline environment.

[0011] Step 1.1: Place probe groups; Place evenly spaced light field probe groups in the scene to cover the scene areas where global illumination needs to be generated;

[0012] Step 1.2: Bake the cubemap;

[0013] For each probe, three cube maps are rendered with the probe location as the center, storing the surface albedo, world space normal and world space coordinates respectively;

[0014] Step 1.3: Sample the cubemap;

[0015] For each probe, a certain number of sampling directions are uniformly generated within the spherical range, and the three cubemaps are sampled using the sampling directions, and the sampled surface properties are stored in the corresponding buffers respectively; there are three buffers of the same size, which store the albedo, world space normal and world space coordinates of the corresponding surface points respectively; each set of albedo, world space normal and world space coordinates represents a geometric point on the surface, which is called a surfel;

[0016] Step 2: Render the geometry buffer;

[0017] Step 2.1: Prepare materials, geometry, light sources, texture data, and initiate a drawing call;

[0018] Step 2.2: Use the multiple rendering target technology MRT to store the albedo, world space normal, metalness, and roughness in different channels of a set of geometry buffers G-Buffer;

[0019] Step 3: Update the light field probe;

[0020] Step 3.1: Select the probe to be updated;

[0021] In each frame, 10% of the probes that have never been updated are selected to be added to the queue to be updated, that is, all probes are updated once every 10 frames;

[0022] Step 3.2: Calculate the direct illumination of the surface element;

[0023] For each face element in the probe, read its stored surface properties, use the light source information and shadow information of the current frame, and use the Lambert illumination model to calculate the shading result of the face element, so as to obtain the direct illumination result of the face element; the Lambert illumination model is an empirical model used to simulate diffuse illumination on rough surfaces;

[0024] Step 3.3: Calculate the indirect lighting of the surface element;

[0025] In each light field probe, a second-order spherical harmonic function is used to describe the illumination information within the spherical range of the probe position;

[0026] For each face element in the probe, the second-order spherical harmonics in the light field probe of the previous frame are sampled using the face element's world space position, and the sampling results are weighted mixed to obtain the indirect lighting result of the face element;

[0027] Step 3.4: Calculate the spherical harmonic coefficients of the probe;

[0028] The direct illumination result obtained in step 3.2 is mixed with the indirect illumination result obtained in step 3.3 to finally obtain the outgoing radiance of the face element, and the outgoing radiance is projected into the second-order spherical harmonic function, and the coefficients of the second-order spherical harmonic function are stored in a buffer specially used for storing coefficients;

[0029] Step 4: Generate initial samples using screen space ray tracing;

[0030] Step 4.1: Generate HiZ buffer;

[0031] According to the set maximum level, different levels of HiZ buffers are generated from low to high using compute shaders; at level 0, the depth buffer of the current frame is directly sampled; at each subsequent level, the four depth values ​​around the corresponding pixel in the HiZ buffer of the previous level are sampled, and the depth value closest to the camera position is used as the value in the current level;

[0032] Step 4.2: Prepare light data;

[0033] For each screen pixel, sample the geometry buffer of the current frame, obtain the world space normal and depth, and reconstruct the world space position corresponding to the pixel in combination with the screen space coordinates of the pixel;

[0034] Step 4.3: Generate sampling direction;

[0035] Use random seeds and Hammersley sampling method to generate a random sequence, and then use the Cosine-Weighted sampling algorithm within the hemisphere to generate the light sampling direction within the hemisphere;

[0036] Step 4.4: Light stepping;

[0037] First, use the minimum step size to perform a tentative step, sample the lowest level HiZ buffer, and compare it with the current light depth. If the difference between the light depth and the sampling depth is less than the set threshold, it is considered that the light intersects with the surface at this time; otherwise, there is no intersection; after the light intersects with the surface, it goes back one level according to the currently sampled HiZ buffer level, and then continues to step, repeating this process until it intersects with the lowest level HiZ buffer, at which time the screen space coordinates of the intersecting pixels are returned; if there is no intersection, increase the step size, continue to step, and sample a higher level HiZ buffer as the depth value of the surface at this time, until it intersects or exceeds the screen range; if the light intersects with the scene surface and the returned pixel screen space coordinates are legal, jump to step 4.5; otherwise, jump to step 4.6; the starting point of the light is called the visible point, and the intersection point of the light and the scene is called the sampling point;

[0038] Step 4.5: Tracking is successful, and the initial samples are generated using the screen space information;

[0039] For each screen pixel, use the screen space coordinates of the pixel point in step 4.4 to sample the geometric buffer information of the current frame, and obtain the geometric information of the sampling point, that is, the world space position and normal direction of the sampling point; and reproject the screen space coordinates of the sampling point to the previous frame, sample the final shading result of the previous frame, and finally obtain the outgoing radiance of the sampling point; write the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer;

[0040] Step 4.6: If tracking fails, use the light field probe to generate initial samples;

[0041] Using the world space position of the visible point, find the 8 light field probes closest to the position, and use the sampling direction of the visible point to sample the spherical harmonics respectively to obtain the incident radiance in that direction; and use the difference between the direction of the visible point pointing to the probe and the normal direction of the visible point as the weight to perform weighted mixing on the 8 incident radiances; assuming that the sampling point is located at a certain distance from the visible point along the sampling direction, construct the geometric information of the sampling point, and finally write the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer;

[0042] Step 5: Time resampling;

[0043] Step 5.1: Reprojection: For each pixel on the screen, project it into the previous frame using the motion vector to find the corresponding pixel in the previous frame;

[0044] Step 5.2: Validity check: read and calculate the depth difference and normal difference between the current frame and the previous frame. If the two differences are less than the set threshold, the reprojection is considered successful and step 5.3 is executed; otherwise, it fails and step 5.4 is executed.

[0045] Step 5.3: Reprojection is successful, and time resampling is performed; the reservoir regenerated for each frame is called the initial reservoir; the reservoir cached from the previous frame is called the time reservoir. Read the initial reservoir of the current frame and the time reservoir of the previous frame, use the brightness value of the outgoing radiance as the weight, use the samples in the initial reservoir to update the time reservoir, and write the updated reservoir back to the time reservoir; when updating, the number of reused frames of the sample needs to be compared with the maximum number of reused frames. If the number of reused frames of the sample is greater than the maximum number of reused frames, directly read the initial reservoir of the current frame and write it directly to the corresponding position of the time reservoir;

[0046] Step 5.4: Reprojection fails, no temporal resampling is performed. Read the initial reservoir of the current frame and write it directly to the corresponding position of the temporal reservoir.

[0047] Step 6: spatial resampling;

[0048] Step 6.1: Select the number of iterations: For each pixel on the screen, read the corresponding time reservoir, obtain the number of frames that the sample has been reused, and compare it with the threshold. If it is less than the threshold, iterate 5 times; otherwise, iterate 3 times. In each iteration, repeat steps 6.2-6.4;

[0049] Step 6.2: Randomly select an adjacent reservoir: read the temporal reservoir of the current pixel, and in each iteration, randomly select a pixel within a certain radius and read the temporal reservoir of the pixel;

[0050] Step 6.3: Validity test: Load the geometric properties of the current pixel visible point and the selected pixel visible point, and compare the position difference and normal difference. If both differences are less than the set threshold, it is considered that the two pixel visible points come from the same surface, and the samples stored in the corresponding positions can be spatially reused, and execute step 6.4; otherwise, it is considered that the two pixel visible points come from different surfaces, and the samples stored in the corresponding positions cannot be spatially reused, and return to step 6.2;

[0051] Step 6.4: Perform spatial multiplexing: Use the samples in the time reservoir of the selected pixel to update the time reservoir of the current pixel, and write the updated reservoir back to the spatial reservoir buffer at the corresponding position;

[0052] Step 7: Get the indirect diffuse lighting result;

[0053] Step 7.1: Read the sample information in the time reservoir, multiply the radiance of the sample by the reservoir weight, BRDF, and cosine term to obtain the indirect diffuse lighting result.

[0054] Step 7.2: Perform spatial denoising to eliminate the remaining noise in the result. Use a separable bilateral filter, use the depth difference and normal difference as the filter weights, that is, first perform a horizontal filter, and then perform a vertical filter, so as to eliminate the remaining noise in the indirect diffuse reflection illumination and obtain the final indirect diffuse reflection illumination result;

[0055] Step 8: Use the geometry buffer generated in step 2, the indirect diffuse lighting generated in step 7, and the light source data and shadow information for delayed shading; use the screen space coordinates of the pixel to sample the indirect diffuse lighting and superimpose it with the direct lighting result to obtain the final shading result.

[0056] The beneficial effects of adopting the above technical solution are:

[0057] The present invention provides a hybrid screen space global illumination method based on spatiotemporal resampling and light field probes. On the basis of screen space ray tracing, the present invention introduces a dynamically updated light field probe to supplement the generation of initial samples, and further applies the spatiotemporal resampling method to improve the quality of the samples and achieve the effect of accelerated convergence. The problem of low precision of the global illumination method based on light field probes is solved, and the problem of slow convergence speed of the global illumination method based on ray tracing is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is an overall flow chart of the hybrid screen space global illumination method in an embodiment of the present invention;

[0059] Figure 2 A result diagram of the original screen space global illumination in an embodiment of the present invention;

[0060] Figure 3 This is a result diagram of original screen space global illumination + spatiotemporal resampling in an embodiment of the present invention;

[0061] Figure 4 This is a result diagram of the hybrid screen space global illumination method in an embodiment of the present invention;

[0062] Figure 5 It is the result of path tracing in the embodiment of the present invention. DETAILED DESCRIPTION

[0063] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0064] A hybrid screen-space global illumination method based on resampling and probes, such as Figure 1 As shown, the following steps are included:

[0065] Step 1: Precompute and bake light field probes in an offline environment.

[0066] Step 1.1: Place probe groups; Place evenly spaced light field probe groups in the scene to cover the scene area where global illumination needs to be generated; in this embodiment, under the default setting, the spacing between probes is 4 meters.

[0067] Step 1.2: Bake the cubemap;

[0068] For each probe, three cube maps are rendered with the probe location as the center, storing the surface albedo, world space normal and world space coordinates respectively; in this embodiment, under the default setting, the resolution of the cube map is 256*256*6.

[0069] Step 1.3: Sample the cubemap;

[0070] For each probe, a certain number of sampling directions are uniformly generated within the spherical range, with a default of 128. The three cubemaps are sampled using the sampling directions, and the sampled surface properties are stored in corresponding buffers. There are three buffers of the same size, which store the albedo, world space normal, and world space coordinates of the corresponding surface points. Each set of albedo, world space normal, and world space coordinates represents a geometric point on the surface, which is called a surfel.

[0071] Step 2: Render the geometry buffer;

[0072] Step 2.1: Prepare materials, geometry, light sources, texture data, and initiate a drawing call;

[0073] Step 2.2: Use the multiple rendering target technology MRT to store the albedo, world space normal, metalness, and roughness in different channels of a set of geometry buffers G-Buffer;

[0074] Step 3: Update the light field probe;

[0075] Step 3.1: Select the probe to be updated;

[0076] In each frame, 10% of the probes that have never been updated are selected to be added to the queue to be updated, that is, all probes are updated once every 10 frames;

[0077] Step 3.2: Calculate the direct illumination of the surface element;

[0078] For each face element in the probe, read its stored surface properties, use the light source information and shadow information of the current frame, and use the Lambert illumination model to calculate the shading result of the face element, so as to obtain the direct illumination result of the face element; the Lambert illumination model is an empirical model used to simulate diffuse illumination on rough surfaces;

[0079] Step 3.3: Calculate the indirect lighting of the surface element;

[0080] In each light field probe, a second-order spherical harmonic function is used to describe the illumination information within the spherical range of the probe position; the spherical harmonic function is a spherical basis function used to describe the distribution of a certain signal within the spherical range and is widely used in global illumination methods.

[0081] For each face element in the probe, the second-order spherical harmonics in the light field probe of the previous frame are sampled using the face element's world space position, and the sampling results are weighted mixed to obtain the indirect lighting result of the face element, thus achieving the lighting effect of infinite bounces;

[0082] Step 3.4: Calculate the spherical harmonic coefficients of the probe;

[0083] The direct illumination result obtained in step 3.2 is mixed with the indirect illumination result obtained in step 3.3 to finally obtain the outgoing radiance of the face element, and the outgoing radiance is projected into the second-order spherical harmonic function, and the coefficients of the second-order spherical harmonic function are stored in a buffer specially used for storing coefficients;

[0084] Step 4: Generate initial samples using screen space ray tracing;

[0085] Step 4.1: Generate HiZ buffer;

[0086] According to the set maximum level, different levels of HiZ buffers are generated from low to high using compute shaders; at level 0, the depth buffer of the current frame is directly sampled; at each subsequent level, the four depth values ​​around the corresponding pixel in the HiZ buffer of the previous level are sampled, and the depth value closest to the camera position is used as the value in the current level;

[0087] Step 4.2: Prepare light data;

[0088] For each screen pixel, sample the geometry buffer of the current frame, obtain the world space normal and depth, and reconstruct the world space position corresponding to the pixel in combination with the screen space coordinates of the pixel;

[0089] Step 4.3: Generate sampling direction;

[0090] Use random seeds and Hammersley sampling method to generate low-discrepancy random sequences, and then use the Cosine-Weighted sampling algorithm within the hemisphere to generate the light sampling direction within the hemisphere;

[0091] Step 4.4: Light stepping;

[0092] First, use the minimum step size to perform a tentative step, sample the lowest level HiZ buffer, and compare it with the current light depth. If the difference between the light depth and the sampling depth is less than the set threshold, it is considered that the light intersects with the surface at this time; otherwise, there is no intersection; after the light intersects with the surface, it goes back one level according to the currently sampled HiZ buffer level, and then continues to step, repeating this process until it intersects with the lowest level HiZ buffer, at which time the screen space coordinates of the intersecting pixels are returned; if there is no intersection, increase the step size, continue to step, and sample a higher level HiZ buffer as the depth value of the surface at this time, until it intersects or exceeds the screen range; in this embodiment, under the default settings, the number of light stepping times is 50 times. If the light intersects with the scene surface and the returned pixel screen space coordinates are legal, jump to step 4.5; otherwise, jump to step 4.6; the starting point of the light is called the visible point, and the intersection point of the light and the scene is called the sampling point;

[0093] Step 4.5: Tracking is successful, and the initial samples are generated using the screen space information;

[0094] For each screen pixel, use the screen space coordinates of the pixel point in step 4.4 to sample the geometric buffer information of the current frame to obtain the geometric information of the sampling point, that is, the world space position and normal direction of the sampling point; and reproject the screen space coordinates of the sampling point to the previous frame, sample the final shading result of the previous frame, so as to achieve the global illumination effect of infinite bounces, and finally obtain the outgoing radiance of the sampling point; write the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer;

[0095] Step 4.6: If tracking fails, use the light field probe to generate initial samples;

[0096] Using the world space position of the visible point, find the 8 light field probes closest to the position, and use the sampling direction of the visible point to sample the spherical harmonics respectively to obtain the incident radiance in that direction; and use the difference between the direction of the visible point pointing to the probe and the normal direction of the visible point as the weight to perform weighted mixing on the 8 incident radiances; assuming that the sampling point is located at a certain distance from the visible point along the sampling direction, construct the geometric information of the sampling point, and finally write the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer;

[0097] Step 5: Time resampling;

[0098] Step 5.1: Reprojection: For each pixel on the screen, project it into the previous frame using the motion vector to find the corresponding pixel in the previous frame;

[0099] Step 5.2: Validity check: read and calculate the depth difference and normal difference between the current frame and the previous frame. If the two differences are less than the set threshold, the reprojection is considered successful and step 5.3 is executed; otherwise, it fails and step 5.4 is executed.

[0100] Step 5.3: Reprojection is successful, and time resampling is performed; the reservoir regenerated for each frame is called the initial reservoir; the reservoir cached from the previous frame is called the time reservoir. Read the initial reservoir of the current frame and the time reservoir of the previous frame, use the brightness value of the outgoing radiance as the weight, use the samples in the initial reservoir to update the time reservoir, and write the updated reservoir back to the time reservoir; when updating, the number of reused frames of the sample needs to be compared with the maximum number of reused frames. If the number of reused frames of the sample is greater than the maximum number of reused frames, directly read the initial reservoir of the current frame and write it directly to the corresponding position of the time reservoir;

[0101] Step 5.4: Reprojection fails, no temporal resampling is performed. Read the initial reservoir of the current frame and write it directly to the corresponding position of the temporal reservoir.

[0102] Step 6: spatial resampling;

[0103] Step 6.1: Select the number of iterations: For each pixel on the screen, read the corresponding time reservoir, obtain the number of frames that the sample has been reused, and compare it with the threshold. If it is less than the threshold, iterate 5 times; otherwise, iterate 3 times. In each iteration, repeat steps 6.2-6.4;

[0104] Step 6.2: Randomly select an adjacent reservoir: read the temporal reservoir of the current pixel, and in each iteration, randomly select a pixel within a certain radius and read the temporal reservoir of the pixel;

[0105] Step 6.3: Validity test: Load the geometric properties of the current pixel visible point and the selected pixel visible point, and compare the position difference and normal difference. If both differences are less than the set threshold, it is considered that the two pixel visible points come from the same surface, and the samples stored in the corresponding positions can be spatially reused, and execute step 6.4; otherwise, it is considered that the two pixel visible points come from different surfaces, and the samples stored in the corresponding positions cannot be spatially reused, and return to step 6.2;

[0106] Step 6.4: Perform spatial multiplexing: Use the samples in the time reservoir of the selected pixel to update the time reservoir of the current pixel, and write the updated reservoir back to the spatial reservoir buffer at the corresponding position;

[0107] Step 7: Get the indirect diffuse lighting result;

[0108] Step 7.1: Read the sample information in the time reservoir, multiply the radiance of the sample by the reservoir weight, BRDF, and cosine term to obtain the indirect diffuse lighting result.

[0109] Step 7.2: Perform spatial denoising to eliminate the remaining noise in the result. Use a separable bilateral filter, use the depth difference and normal difference as the filter weights, that is, first perform a horizontal filter, and then perform a vertical filter, so as to eliminate the remaining noise in the indirect diffuse reflection illumination and obtain the final indirect diffuse reflection illumination result;

[0110] Step 8: Use the geometry buffer generated in step 2, the indirect diffuse lighting generated in step 7, and the light source data and shadow information for delayed shading; use the screen space coordinates of the pixel to sample the indirect diffuse lighting and superimpose it with the direct lighting result to obtain the final shading result.

[0111] The present invention improves the global illumination method based on screen space, increases the number of effective initial samples by introducing the light field probe method, and improves the quality of samples and the convergence speed by introducing the spatiotemporal resampling method. The present invention can provide high-quality, high-precision, and high-stability diffuse global illumination effects under the conditions of static scenes and dynamic light sources. In addition, by introducing the light field probe method to generate effective initial samples, the deviation caused by the spatiotemporal resampling of samples can also be improved, making the picture effect more physically correct. Figure 2-Figure 5 The screen results under different schemes are shown respectively. Figure 2 is the result map of the original screen space global illumination, Figure 3 This is the result of the original screen space global illumination + space-time resampling. Figure 4 As a result of this solution, the closest path tracking is Figure 5result.

[0112] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A hybrid screen space global illumination method based on resampling and probes, characterized in that: The following steps are involved: Step 1: Precompute and bake light field probes in an offline environment. Step 2: Render the geometry buffer; Step 3: Update the light field probe; Step 4: Generate initial samples using screen space ray tracing; Step 5: Time resampling; Step 6: spatial resampling; Step 7: Get the indirect diffuse lighting result; Step 8: Use the geometry buffer generated in step 2, the indirect diffuse lighting generated in step 7, and the light source data and shadow information for delayed shading; use the screen space coordinates of the pixel to sample the indirect diffuse lighting and superimpose it with the direct lighting result to obtain the final shading result.

2. A hybrid screen space global illumination method based on resampling and probe according to claim 1, characterized in that: The step 1 specifically comprises the following steps: Step 1.1: Place probe groups; Place evenly spaced light field probe groups in the scene to cover the scene areas where global illumination needs to be generated; Step 1.2: Bake the cubemap; For each probe, three cube maps are rendered with the probe location as the center, storing the surface albedo, world space normal and world space coordinates respectively; Step 1.3: Sample the cubemap; For each probe, a certain number of sampling directions are uniformly generated within the spherical range, and the sampling directions are used to sample the three cube maps respectively, and the sampled surface properties are stored in the corresponding buffers respectively; there are three buffers of the same size, which store the albedo, world space normal and world space coordinates of the corresponding surface points respectively; each set of albedo, world space normal and world space coordinates represents a geometric point on the surface, which is called a surfel.

3. The hybrid screen space global illumination method based on resampling and probe according to claim 1, characterized in that: The step 2 comprises the following steps: Step 2.1: Prepare materials, geometry, light sources, texture data, and initiate a drawing call; Step 2.2: Using the multiple rendering target technology MRT, the albedo, world space normal, metalness, and roughness are stored in different channels of a set of geometric buffers G-Buffer.

4. The hybrid screen space global illumination method based on resampling and probe according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1: Select the probe to be updated; In each frame, 10% of the probes that have never been updated are selected to be added to the queue to be updated, that is, all probes are updated once every 10 frames; Step 3.2: Calculate the direct illumination of the surface element; For each face element in the probe, read its stored surface properties, use the light source information and shadow information of the current frame, and use the Lambert illumination model to calculate the shading result of the face element, so as to obtain the direct illumination result of the face element; the Lambert illumination model is an empirical model used to simulate diffuse illumination on rough surfaces; Step 3.3: Calculate the indirect lighting of the surface element; In each light field probe, a second-order spherical harmonic function is used to describe the illumination information within the spherical range of the probe position; For each face element in the probe, the second-order spherical harmonics in the light field probe of the previous frame are sampled using the face element's world space position, and the sampling results are weighted mixed to obtain the indirect lighting result of the face element; Step 3.4: Calculate the spherical harmonic coefficients of the probe; The direct illumination result obtained in step 3.2 is mixed with the indirect illumination result obtained in step 3.3 to finally obtain the outgoing radiance of the face element, and the outgoing radiance is projected into the second-order spherical harmonic function, and the coefficients of the second-order spherical harmonic function are stored in a buffer specifically used to store coefficients.

5. The hybrid screen space global illumination method based on resampling and probe according to claim 1, characterized in that: The step 4 comprises the following steps: Step 4.1: Generate HiZ buffer; According to the set maximum level, different levels of HiZ buffers are generated from low to high using compute shaders; at level 0, the depth buffer of the current frame is directly sampled; at each subsequent level, the four depth values ​​around the corresponding pixel in the HiZ buffer of the previous level are sampled, and the depth value closest to the camera position is used as the value in the current level; Step 4.2: Prepare light data; For each screen pixel, sample the geometry buffer of the current frame, obtain the world space normal and depth, and reconstruct the world space position corresponding to the pixel in combination with the screen space coordinates of the pixel; Step 4.3: Generate sampling direction; Use random seeds and Hammersley sampling method to generate a random sequence, and then use the Cosine-Weighted sampling algorithm within the hemisphere to generate the light sampling direction within the hemisphere; Step 4.4: Ray stepping; First, use the minimum step size to perform a tentative step, sample the lowest level HiZ buffer, and compare it with the current light depth. If the difference between the light depth and the sampling depth is less than the set threshold, it is considered that the light intersects with the surface at this time; otherwise, there is no intersection; after the light intersects with the surface, it goes back one level according to the currently sampled HiZ buffer level, and then continues to step, repeating this process until it intersects with the lowest level HiZ buffer, at which time the screen space coordinates of the intersecting pixels are returned; if there is no intersection, increase the step size, continue to step, and sample a higher level HiZ buffer as the depth value of the surface at this time, until it intersects or exceeds the screen range; if the light intersects with the scene surface and the returned pixel screen space coordinates are legal, jump to step 4.5; otherwise, jump to step 4.6; the starting point of the light is called the visible point, and the intersection point of the light and the scene is called the sampling point; Step 4.5: Tracking is successful, and the initial samples are generated using the screen space information; For each screen pixel, use the screen space coordinates of the pixel point in step 4.4 to sample the geometric buffer information of the current frame, and obtain the geometric information of the sampling point, that is, the world space position and normal direction of the sampling point; and reproject the screen space coordinates of the sampling point to the previous frame, sample the final shading result of the previous frame, and finally obtain the outgoing radiance of the sampling point; write the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer; Step 4.6: If tracking fails, use the light field probe to generate initial samples; Using the world space position of the visible point, find the 8 nearest light field probes around the position, and use the sampling direction of the visible point to sample the spherical harmonics respectively to obtain the incident radiance in that direction; and use the difference between the direction of the visible point pointing to the probe and the normal direction of the visible point as the weight to perform weighted mixing on the 8 incident radiances; assuming that the sampling point is located at a certain distance from the visible point along the sampling direction, construct the geometric information of the sampling point, and finally write the geometric information of the visible point and the sampling point, the random seed for generating the sampling direction, and the outgoing radiance of the sampling point into the initial reservoir buffer.

6. The hybrid screen space global illumination method based on resampling and probe according to claim 1, characterized in that: The step 5 comprises the following steps: Step 5.1: Reprojection: For each pixel on the screen, project it into the previous frame using the motion vector to find the corresponding pixel in the previous frame; Step 5.2: Validity check: read and calculate the depth difference and normal difference between the current frame and the previous frame. If the two differences are less than the set threshold, the reprojection is considered successful and step 5.3 is executed; otherwise, it fails and step 5.4 is executed; Step 5.3: Reprojection is successful, and time resampling is performed; the reservoir regenerated for each frame is called the initial reservoir; the reservoir cached in the previous frame is called the time reservoir; read the initial reservoir of the current frame and the time reservoir of the previous frame, use the brightness value of the outgoing radiance as the weight, use the sample in the initial reservoir to update the time reservoir, and write the updated reservoir back to the time reservoir; when updating, it is necessary to compare the number of multiplexed frames of the sample with the maximum number of multiplexed frames. If the number of multiplexed frames of the sample is greater than the maximum number of multiplexed frames, directly read the initial reservoir of the current frame and write it directly to the corresponding position of the time reservoir; Step 5.4: If reprojection fails, no time resampling is performed; read the initial reservoir of the current frame and write it directly to the corresponding position of the time reservoir.

7. The hybrid screen space global illumination method based on resampling and probe according to claim 1, characterized in that: The step 6 comprises the following steps: Step 6.1: Select the number of iterations: For each pixel on the screen, read the corresponding time reservoir, obtain the number of frames that the sample has been reused, and compare it with the threshold; if it is less than the threshold, iterate 5 times; otherwise, iterate 3 times; in each iteration, repeat steps 6.2-6.4; Step 6.2: Randomly select an adjacent reservoir: read the temporal reservoir of the current pixel, and in each iteration, randomly select a pixel within a certain radius and read the temporal reservoir of the pixel; Step 6.3: Validity check: load the geometric properties of the current pixel visible point and the selected pixel visible point, and compare the position difference and normal difference; if both differences are less than the set threshold, it is considered that the two pixel visible points come from the same surface, and the samples stored in the corresponding positions can be spatially reused, and execute step 6.4; otherwise, it is considered that the two pixel visible points come from different surfaces, and the samples stored in the corresponding positions cannot be spatially reused, and return to step 6.2; Step 6.4: Perform spatial multiplexing: Use the samples in the temporal reservoir of the selected pixel to update the temporal reservoir of the current pixel, and write the updated reservoir back to the spatial reservoir buffer at the corresponding position.

8. The hybrid screen space global illumination method based on resampling and probe according to claim 1, characterized in that: The step 7 comprises the following steps: Step 7.1: Read the sample information in the time reservoir, multiply the radiance of the sample by the reservoir weight, BRDF, and cosine term to obtain the indirect diffuse lighting result; Step 7.2: Perform spatial denoising to eliminate the remaining noise in the result; use a separable form of bilateral filtering, use depth difference and normal difference as filtering weights, that is, first perform a horizontal filter, and then perform a vertical filter, so as to eliminate the remaining noise in the indirect diffuse lighting and obtain the final indirect diffuse lighting result.

Images