A model hybrid rendering method based on virtual lighting environment and light query

Abstract

The present invention discloses a model hybrid rendering method based on a virtual lighting environment and light query, comprising the following steps: 1) using a light source to directly illuminate the surrounding environment, and recording a rendering image generated by the direct illumination of the light source, thereby obtaining information on the light source perspective; 2) rendering the environment from the perspective of the light source to obtain a luminous area seen from the perspective of the light source, and saving world space position information, normal information, and flux color information into a texture; 3) using the normal information saved in step 2 to fit the luminous area into a virtual planar light source, and calculating indirect lighting under the virtual planar light source through a truncated cosine distribution BRDF; 4) fusing the indirect lighting and direct lighting, and using a light query in a GLSL shader to perform shadow rendering on the environment, so that the lighting effect of the rendered environment is more realistic and the shadow effect is more realistic.

Description

Technical Field

[0001] The present invention relates to computer graphics rendering technology, in particular to a model hybrid rendering method based on virtual lighting environment and light query. Background Art

[0002] Model rendering is a very important step in graphics processing. In a virtual environment, in order to make the rendering effect more realistic, the rendering target needs to interact with the surrounding virtual environment to achieve better lighting effects.

[0003] Global illumination refers to a rendering technique that takes into account both direct lighting from light sources in the scene and indirect lighting reflected from other objects in the scene. When we use common engine rendering models, there is generally only direct lighting (diffuse reflection, highlights), and indirect lighting needs to be added using an additional shader language. Different indirect lighting corresponds to different solutions. Virtual light sources are one of the solutions for indirect lighting. Using virtual lighting requires generating depth images. If shadow images are used for shadow rendering at this time, it will cause additional burden, so another shadow generation method is needed.

[0004] Ray tracing is a feature that all mainstream graphics APIs now have. This means that when we use ray tracing related functions, we do not need to manually simulate light. We can directly call the hardware ray tracing function, and ray tracing technology has a clearer way of rendering shadows than traditional rasterization. Summary of the Invention

[0005] The purpose of the present invention is to provide a model hybrid rendering method based on a virtual lighting environment and light query, so that the lighting effect of the rendering environment is more realistic and the shadow effect is more realistic.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A model hybrid rendering method based on a virtual lighting environment and light query includes the following steps:

[0008] 1) Use the light source to directly illuminate the surrounding environment and record the rendering image produced by the direct illumination of the light source to obtain the information of the light source perspective;

[0009] 2) Render the environment from the perspective of the light source to obtain the luminous area seen from the light source, and save the world space position information, normal information and flux color information into the texture;

[0010] 3) Use the normal information saved in step 2 to fit the luminous area into a virtual planar light source, and use the truncated cosine distribution BRDF to calculate the indirect lighting under the virtual planar light source;

[0011] 4) Combine indirect lighting and direct lighting and use ray queries in GLSL shaders to shadow the environment.

[0012] Furthermore, the information of the light source viewing angle in step 1) includes a model matrix, a perspective matrix and a position of the model in a three-dimensional coordinate system.

[0013] Furthermore, the step 2) specifically includes the following steps:

[0014] 2.1) First, render the environment from the light source's perspective. Obtain the model's vertex position and normal from the loaded model of the vertex shader for rendering the entire scene. Then, obtain the normal from the light source's perspective by illuminating it from the light source's perspective. Finally, obtain the world position from the light source's perspective by multiplying the model matrix and the model's position in the light source's three-dimensional coordinate system.

[0015] 2.2) First, get the normal information through FS_normal in the vertex shader, and get the worldposition coordinates under the known camera perspective from FS_position in the vertex shader, then get the flux color information of the environment space through the texture information in the texture, and finally save the normal information, flux color information and worldposition under the camera perspective into the texture.

[0016] Furthermore, the information of the light source viewing angle in step 1) includes a model matrix, a perspective matrix and a position of the model in a three-dimensional coordinate system.

[0017] Furthermore, the virtual planar light source in step 3) is formed by inversely combining the depth value dp, the world space position xp, the normal np and the reflected radiation flux Φp of each pixel p.

[0018] Furthermore, the step 3) specifically includes the following steps:

[0019] 3.1) Calculate the light space coordinate FragPosInLightSpace value to be used in the fragment shader by multiplying the known perspective matrix under the light source's perspective by the world position under the camera's perspective;

[0020] 3.2) Use the truncated cosine distribution BRDF to calculate the indirect lighting under the virtual planar light source. The indirect lighting effect is calculated by formula (8). In the indirect lighting stage based on the virtual planar light source, the lighting effect calculated by each virtual planar light source is multiplied by the distance weight. The total lighting effect and the lighting effect of each sampling are added together to get the lighting effect of the secondary virtual light source:

[0021]

[0022] Where: P is the polygon, D is the polygon illumination integral after truncated cosine transform, I L is the lighting effect, d wi is the differential of the solid angle, L is the light source in the solid angle ω i Directional irradiance.

[0023] Furthermore, the process of calculating the indirect illumination under the virtual planar light source using the truncated cosine distribution BRDF in step 3.2) is as follows:

[0024] 3.2.1) First, set the depth buffer generated by the light source rendering the environment as a shadow map. For each pixel in the shadow map, the light reflected from the impact surface is additionally stored. Then, the information stored in step 2) is divided into blocks and fitted into several small planes and the plane information is stored.

[0025] 3.2.2) Find the contribution integral of each surface of the rendered point, expressed as:

[0026] I=∫ P L i (w i )ρ(w V ,w i )cosθ i dw i (1)

[0027] Where: ρ is the BRDF function, P is the polygon, I is the contribution integral of the illumination integral, θ is the angle between the normal of the shading point and the illumination direction of the light source, d wi is the differential of the solid angle;

[0028] According to the diffuse BRDF contribution formula (2), after the first spherical transformation, the shading point is shaded using LTC to fit the cosine-weighted BRDF. In the case of indirect lighting, only the diffuse BRDF is considered and set to a commonly used constant intensity value, specifically a custom ratio of the light intensity to π;

[0029] I=ρL∫ P cosθ i dw i (2)

[0030] Where: ρ is the commonly used BRDF fixed value, P is the polygon, θ is the angle between the normal of the shading point and the direction of the light source, d wi is the differential of the solid angle;

[0031] By setting the intensity of each polygon light source to a constant value, formulas (3) to (5) are derived:

[0032] I=∫ P L(wi)D(wi)dwi (3)

[0033] I=L∫ P D(wi)dwi (4)

[0034] I=L*E(p0) (5)

[0035] Where: E represents the virtual polygon, P0 is the incident flux or energy;

[0036] 3.2.3) Assuming that a 2D color texture is used to represent the radiance emitted by the light source, formula (3) is rewritten as formula (6), and then formulas (7) and (8) are derived:

[0037] A≈∫ P L(wi)D(wi)dwi=I D I L (6)

[0038] I D ≈∫ P D(wi)dwi (7)

[0039]

[0040] Where: P is the polygon, A is the illumination integral result, I D is the result of integrating the shape of the original light source, I L is the integral result of polygon light color, d wi is the differential of the solid angle;

[0041] Furthermore, in step 4), the ray tracing effect is incrementally added to the lighting effect obtained by formula (6), or a rasterization hybrid method is used to achieve visibility of the main light.

[0042] Furthermore, the step 4) specifically includes the following steps:

[0043] 4.1) Reference the ray tracing and ray query extensions in the Vulkan graphics API, transmit the acceleration structure topLevelAS through the uniform, and perform ray tracing through the acceleration structure;

[0044] 4.2) By recording the rendered frames, random numbers are simulated in the glsl shader program to achieve a random sampling effect. The position of a virtual plane light source is determined by random sampling. Each virtual plane light source is processed with several points. The light is initialized by the rayQueryInitializeEXT function, and the rayQueryGetIntersectionTypeEXT function is used to determine whether the light emitted by the point determined by the sampling of the virtual plane light source as the initial point intersects with the shading point. If no intersection occurs, the shading point is defined as being in shadow.

[0045] 4.3) Each light source on the virtual plane is sampled several times, and the sampling results are recorded. Ray tracing is used to determine whether the results are used to determine the indirect lighting effect for the shading point. The rendering result of each shading point is the accumulation of the illumination effects of different light sources. After accumulation, the shadow effect under global illumination is rendered.

[0046] Furthermore, the acceleration structure of step 4.1) is divided into two levels of acceleration structures: a bottom layer and a top layer. The bottom layer acceleration structure contains triangles or axis-aligned bounding boxes of custom geometry that constitute the scene; the top layer acceleration structure contains references to a set of bottom layer acceleration structures, each reference containing shading and transformation information of the reference.

[0047] Furthermore, the underlying acceleration structure adopts multiple sets of geometric figures of a given type.

[0048] The present invention has the following beneficial effects:

[0049] The present invention first renders the rendering environment according to the light source angle, then uses the saved world space position information, normal information and flux color information, and fits the luminous area into a virtual planar light source using the normal information. The indirect lighting under the virtual planar light source is calculated using the truncated cosine distribution BRDF. Ray tracing is integrated into rasterization through related technologies such as ray query, and shadow rendering is performed on the environment, making the lighting effect of the rendering environment more realistic and the shadow effect more realistic, thus solving the problems of traditional virtual animation rendering lighting scenes being not realistic enough and the high overhead of indirect shadow rendering. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 : Two indirect pixel lights xp and xq of the present invention and their corresponding two RSM pixels p and q;

[0051] Figure 2 : The reflection shadow map sampling of the present invention;

[0052] Figure 3 : A ray query flow chart under the Vulkan graphics API of the present invention;

[0053] Figure 4 : Construction of the virtual plane light source of the present invention;

[0054] Figure 5 : Shadow diagram under the virtual plane light source of the present invention;

[0055] Figure 6 : The model rendering effect diagram combining virtual plane light source and light query of the present invention. DETAILED DESCRIPTION

[0056] The specific contents of the present invention are further explained in detail below with reference to the embodiments, but are not intended to limit the present invention.

[0057] like Figure 1-6 As shown, a graphics rendering method based on hybrid virtual lighting and light query includes the following steps:

[0058] 1) Use the light source to directly illuminate the surrounding environment once, record the rendering image produced by the direct illumination of the light source, and thus obtain the model matrix, perspective matrix and model position in the three-dimensional coordinate system of the light source perspective;

[0059] 2) Render the environment from the perspective of the light source to obtain the area seen from the light source's perspective, and save the world space position information, normal information, and flux color information to the texture. The world space position can not only be recalculated from the pixel coordinates and depth value, but also can be used directly to save valuable pixel shader instructions. The specific process is as follows;

[0060] 2.1) First, render the environment from the perspective of the light source, obtain the vertex position and normal of the model in the loaded model of the vertex shader of the overall scene rendering, then obtain the normal of the light source perspective through the illumination of the light source perspective, and obtain the world coordinate point worldposition from the light source perspective by multiplying the model matrix and the position of the model in the three-dimensional coordinate system of the light source perspective. The world coordinate point worldposition is used for the calculation of the tangent space. The specific process is: subtract the world coordinate point worldposition from the known camera perspective, and calculate the result with the normal of the shading point to obtain the tangent space vLTCMatrix matrix based on the shading point. Any calculation is multiplied by this matrix, so that when rendering textures with more details, the rendering result is better;

[0061] 2.2) Get the normal information through FS_normal in the vertex shader, get the worldposition coordinates under the known camera perspective from FS_position in the vertex shader for storage, then get the flux color information of the environment space through the texture information (including diffuse and specular) in the texture, and finally save the normal information, flux color information and worldposition under the camera perspective in the texture;

[0062] 3) Use the normal information saved in step 2 to fit the luminous area into a virtual planar light source. In the reflection shadow map, for each pixel p, the light reflected from the impact surface is additionally stored. The depth value dp, world space position xp, normal np and reflected radiant flux Φp of each pixel p are inversely synthesized into a virtual planar light source. Each pixel is interpreted as a small area light source that illuminates the scene. The specific operations are as follows:

[0063] 3.1) Calculate the light source space coordinate FragPosInLightSpace value to be used in the fragment shader by multiplying the known perspective matrix under the light source perspective by the worldposition under the camera perspective;

[0064] 3.2) The fragment shader calculates the angle θ by using the normal line under the light source's perspective and the world position cache, and then uses the truncated cosine distribution BRDF to calculate the indirect lighting. The specific process is:

[0065] 3.2.1) First, set the depth buffer generated by the light source rendering the environment as a shadow map. For each pixel in the shadow map, the light reflected from the impact surface is additionally stored. Then, the information stored in step 2 is divided and fitted into several small planes and the plane information is stored.

[0066] 3.2.2) Find the contribution integral of each surface of the rendered point, expressed as:

[0067] I=∫ P L i (w i )ρ(w V ,w i )cosθ i dw i (1)

[0068] Where: ρ is the BRDF function, P is the polygon, I is the contribution integral of the illumination integral, θ is the angle between the normal of the shading point and the illumination direction of the light source, d wiis the differential of the solid angle, L is the light source in the solid angle ω i Directional irradiance;

[0069] According to the diffuse BRDF contribution formula (2), after the first spherical transformation, the cosine-weighted BRDF is fitted using LTC to color the shading point;

[0070] I=ρL∫ P cosθ i dw i (2)

[0071] Where: ρ is the commonly used BRDF fixed value, P is the polygon, I is the contribution integral result of the illumination integral, θ is the angle between the normal line of the shading point and the illumination direction of the light source, d wi is the differential of the solid angle, L is the light source in the solid angle ω i Directional irradiance;

[0072] By setting the intensity of each polygon light source to a constant value, formulas (3) to (5) are derived:

[0073] I=∫ P L(wi)D(wi)dwi (3)

[0074] I=L∫ P D(wi)dwi (4)

[0075] I=L*E(p0) (5)

[0076] Where: E represents the virtual polygon, P0 is the incident flux or energy, D is the polygon illumination integral after truncated cosine transform, and L is the light source in the solid angle ω i Directional irradiance;

[0077] 3.2.3) Assuming that a 2D color texture is used to represent the radiance emitted by the light source, formula (3) is rewritten as formula (6), and then formulas (7) and (8) are derived:

[0078] A≈∫ P L(wi)D(wi)dwi=I D I L (6)

[0079] I D ≈∫ P D(wi)dwi (7)

[0080]

[0081] Where: P is the polygon, D is the polygon illumination integral after truncated cosine transform, A is the illumination integral result, I Dis the result of integrating the shape of the original light source, I L is the integral result of polygon light color, d wi is the differential of the solid angle, L is the light source in the solid angle ω i Directional irradiance;

[0082] The indirect lighting effect is calculated by formula (8). The lighting effect of each sample will undergo a weighted propagation. The total lighting effect and the lighting effect of each sample are added together to get the lighting effect of the secondary virtual light source:

[0083] 4) Combine indirect lighting and direct lighting, use ray query in GLSL shader to render shadows on rendered objects, incrementally add ray tracing effect to the indirect lighting effect under the planar light source obtained by formula (6) in step 3.2.3), or use rasterization hybrid method to achieve visibility of the main light, use ray tracing for auxiliary query, and process the query results to achieve the effect of rendering shadows. The specific implementation method is as follows:

[0084] 4.1) Reference the ray tracing and ray query extensions in the Vulkan graphics API, transfer the acceleration structure topLevelAS through uniform, and perform ray tracing effects through references to the acceleration structure; ray tracing performs ray intersection on optimized data structures built based on scene information; the acceleration structure is divided into two levels of hierarchy, the lower layer (i.e., the underlying acceleration structure) contains triangles or axis-aligned bounding boxes of custom geometry that constitute the scene, and since each underlying acceleration structure corresponds to multiple drawing calls in the rasterization pipeline, each underlying construction can adopt multiple sets of geometries of a given type; the upper layer (i.e., the top-level acceleration structure) contains references to a set of underlying acceleration structures, each reference contains shading and transformation information for the reference, and constructing any type of acceleration structure will produce an opaque, implementation-defined format in memory; the underlying acceleration structure is only referenced from the top-level acceleration structure; the top-level acceleration structure is bound as a descriptor or accessed from the shader through the device address;

[0085] 4.2) By recording the rendered frames, random numbers are simulated in the glsl shader program to achieve a random sampling effect. The position of a virtual plane light source is determined by random sampling. Each plane is processed with several points. The light is initialized by the rayQueryInitializeEXT function, and the rayQueryGetIntersectionTypeEXT function is used to determine whether the light emitted from the point determined by sampling on the virtual plane as the initial point intersects with the shaded point. If no intersection occurs, the shaded point is defined as being in shadow.

[0086] 4.3) Each light source on the virtual plane is sampled several times, and the sampling results are recorded. Ray tracing is used to determine whether the shaded point should be shaded with indirect lighting. The rendering result of each shaded point is the accumulation of the illumination effects of different light sources. After accumulation, shadow rendering under global illumination is achieved.

Claims

1. A model hybrid rendering method based on virtual lighting environment and light query, characterized in that: The steps include: 1) Use the light source to directly illuminate the surrounding environment and record the rendering image produced by the direct illumination of the light source to obtain the information of the light source perspective; 2) Render the environment from the light source's perspective to obtain the luminous area seen from the light source's perspective, and save the world space position information, normal information, and flux color information into the texture; 3) Use the normal information saved in step 2) to fit the luminous area into a virtual planar light source, and use the truncated cosine distribution BRDF to calculate the indirect lighting under the virtual planar light source. The specific process is: 3.1) Calculate the light space coordinate FragPosInLightSpace value to be used in the fragment shader by multiplying the known perspective matrix under the light source's perspective by the world position under the camera's perspective; 3.2) Use the truncated cosine distribution BRDF to calculate the indirect lighting under the virtual planar light source. The indirect lighting effect is calculated by formula (8). In the indirect lighting stage based on the virtual planar light source, the lighting effect calculated by each virtual planar light source is multiplied by the distance weight. The total lighting effect and the lighting effect of each sampling are added together to get the lighting effect of the secondary virtual light source: Where: P is the polygon, D is the polygon illumination integral after truncated cosine transform, I L is the lighting effect, d wi is the differential of the solid angle, L is the light source in the solid angle ω i Directional irradiance; 4) Combine indirect lighting and direct lighting and use ray queries in GLSL shaders to shadow the environment.

2. The model hybrid rendering method based on virtual lighting environment and light query according to claim 1, characterized in that: The information of the light source viewing angle in step 1) includes the model matrix, the perspective matrix and the position of the model in the three-dimensional coordinate system.

3. The model hybrid rendering method based on virtual lighting environment and light query according to claim 2, characterized in that: The step 2) specifically includes the following steps: 2.1) First, render the environment from the light source's perspective. Obtain the model's vertex position and normal from the loaded model of the vertex shader for rendering the entire scene. Then, obtain the normal from the light source's perspective by illuminating it from the light source's perspective. Finally, obtain the world position from the light source's perspective by multiplying the model matrix and the model's position in the light source's three-dimensional coordinate system. 2.2) First, get the normal information through FS_normal in the vertex shader, and get the worldposition coordinates under the known camera perspective from FS_position in the vertex shader, then get the flux color information of the environment space through the texture information provided by the texture, and finally save the normal information, flux color information and worldposition under the camera perspective to the texture.

4. The model hybrid rendering method based on virtual lighting environment and light query according to claim 1, characterized in that: The virtual planar light source in step 3) is formed by inversely combining the depth value dp, the world space position xp, the normal np and the reflected radiation flux Φp of each pixel p.

5. The model hybrid rendering method based on virtual lighting environment and light query according to claim 1, characterized in that: The process of calculating the indirect illumination under the virtual planar light source using the truncated cosine distribution BRDF in step 3.2) is as follows: 3.2.1) First, set the depth buffer generated by the light source rendering the environment as a shadow map. For each pixel in the shadow map, the light reflected from the impact surface is additionally stored. Then, the information stored in step 2) is divided into blocks and fitted into several small planes and the plane information is stored. 3.2.2) Find the contribution integral of each surface of the rendered point, expressed as: I=∫ P L i (In i )ρ(in V ,In i )cosθ i dw i (1) Where: ρ is the BRDF function, P is the polygon, I is the contribution integral of the illumination integral, θ is the angle between the normal of the shading point and the illumination direction of the light source, d wi is the differential of the solid angle; According to the diffuse BRDF contribution formula (2), after the first spherical transformation, the shading point is shaded using LTC to fit the cosine-weighted BRDF. In the case of indirect lighting, only the diffuse BRDF is considered and set to a commonly used constant intensity value, specifically a custom ratio of the light intensity to π; I=ρL∫ P cosθ i dw i (2) Where: ρ is the commonly used BRDF fixed value, P is the polygon, θ is the angle between the normal of the shading point and the direction of the light source, d wi is the differential of the solid angle; By setting the intensity of each polygon light source to a constant value, formulas (3) to (5) are derived: I∫ P L(yes)D(yes)two (3) I=L∫ P D(wi)dwi (4) I=L*E(p0) (5) Where: E represents the virtual polygon, P0 is the incident flux or energy; 3.2.3) Assuming that a 2D color texture is used to represent the radiance emitted by the light source, formula (3) is rewritten as formula (6), and then formulas (7) and (8) are derived: A≈∫ P L(wi)D(wi)dwi=I D I L (6) I D ≈∫ P D(wi)dwi (7) Where: P is the polygon, A is the illumination integral result, I D is the result of integrating the shape of the original light source, I L is the integral result of polygon light color, d wi is the differential of the solid angle.

6. The model hybrid rendering method based on virtual lighting environment and light query according to claim 5, characterized in that: In step 4), the ray tracing effect is incrementally added to the lighting effect obtained by formula (6), or a rasterization hybrid method is used to achieve visibility of the main light.

7. The model hybrid rendering method based on virtual lighting environment and light query according to claim 6, characterized in that: The step 4) specifically includes the following steps: 4.1) Reference the ray tracing and ray query extensions in the Vulkan graphics API, transmit the acceleration structure topLevelAS through the uniform, and perform ray tracing through the acceleration structure; 4.2) By recording the rendered frames, random numbers are simulated in the glsl shader program to achieve a random sampling effect. The position of a virtual plane light source is determined by random sampling. Each virtual plane light source is processed with several points. The light is initialized by the rayQueryInitializeEXT function, and the rayQueryGetIntersectionTypeEXT function is used to determine whether the light emitted by the point determined by the sampling of the virtual plane light source as the initial point intersects with the shading point. If no intersection occurs, the shading point is defined as being in shadow. 4.3) Each light source on the virtual plane is sampled several times, and the sampling results are recorded. Ray tracing is used to determine whether the results are used to determine the indirect lighting effect for the shading point. The rendering result of each shading point is the accumulation of the illumination effects of different light sources. After accumulation, the shadow effect under global illumination is rendered.

8. The model hybrid rendering method based on virtual lighting environment and light query according to claim 7, characterized in that: The acceleration structure of step 4.1) is divided into two levels of acceleration structures: the bottom level and the top level. The bottom level acceleration structure contains triangles or axis-aligned bounding boxes of custom geometry that constitute the scene; the top level acceleration structure contains references to a set of bottom level acceleration structures, each reference containing shading and transformation information of the reference.

9. The model hybrid rendering method based on virtual lighting environment and light query according to claim 8, characterized in that: The underlying acceleration structure employs multiple sets of geometries of a given type.

Images