Lighting rendering method, device, computer equipment and storage medium

Abstract

The present application relates to a lighting rendering method, device, computer equipment and storage medium. The method includes: when the light source in a virtual scene changes, determining the light source change information; determining the current light source projection coefficient corresponding to the changed light source according to the light source change information; determining the indirect lighting value of the target pixel in the virtual scene according to the lighting transfer parameters corresponding to the target pixel in the virtual scene and the current light source projection coefficient; determining the direct lighting value corresponding to the target pixel under the changed light source; and performing lighting rendering on the target pixel according to the direct lighting value and the indirect lighting value. The use of this method can effectively improve the lighting rendering effect under dynamic lighting in virtual scenes.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a lighting rendering method, device, computer equipment and storage medium. Background Art

[0002] In image rendering, in order to achieve realistic scene lighting effects, it is necessary to specify global illumination in the renderer. Global illumination represents the combined effect of direct and indirect lighting. For example, global illumination can be achieved through ray tracing, ambient occlusion, light probes, etc. When light is emitted from the light source, it will be reflected and refracted several times after hitting obstacles, and the surfaces and corners of objects in the virtual scene will have a sense of light.

[0003] In the related art, static lighting scenes are processed and static lighting information of objects in the static lighting scenes is pre-baked for direct use by the scene at runtime. However, this method is not suitable for dynamic lighting scenes and will result in poor lighting rendering effects in dynamic lighting scenes. Summary of the invention

[0004] Based on this, it is necessary to provide a lighting rendering method, device, computer equipment and storage medium that can effectively improve the lighting rendering effect in dynamic lighting scenes in response to the above technical problems.

[0005] A lighting rendering method, the method comprising:

[0006] When the light source in the virtual scene changes, light source change information is determined;

[0007] Determining a current light source projection coefficient corresponding to the changed light source according to the light source change information;

[0008] Determining an indirect illumination value of the target pixel in the virtual scene according to an illumination transfer parameter corresponding to the target pixel and a current light source projection coefficient;

[0009] Determine the direct illumination value corresponding to the target pixel point in the virtual scene under the changed light source;

[0010] Perform illumination rendering on the target pixel point according to the direct illumination value and the indirect illumination value.

[0011] A lighting rendering device, comprising:

[0012] A light source determination module, used to determine light source change information when the light source in the virtual scene changes;

[0013] A projection coefficient updating module, used to determine the current light source projection coefficient corresponding to the changed light source according to the light source change information;

[0014] An indirect illumination determination module, used to determine the indirect illumination value of the target pixel in the virtual scene according to the illumination transfer parameter corresponding to the target pixel in the virtual scene and the current light source projection coefficient;

[0015] A direct illumination determination module, used to determine the direct illumination value corresponding to the target pixel under the changed light source;

[0016] A lighting rendering module is used to perform lighting rendering on the target pixel point according to the direct lighting value and the indirect lighting value.

[0017] In one embodiment, the projection coefficient update module is also used to determine the illumination transformation matrix based on the light source change information; determine the initial light source projection coefficient before the light source in the virtual scene changes; and determine the current light source projection coefficient corresponding to the changed light source based on the illumination transformation matrix and the initial light source projection coefficient.

[0018] In one embodiment, the light source change information is the change information of the changed light source compared to the original light source preset in the virtual scene; the projection coefficient update module is also used to obtain the initial light source projection coefficient obtained by projecting the original light source onto the projection basis function; according to the illumination transformation matrix, the initial light source projection coefficient is updated to obtain the current light source projection coefficient corresponding to the changed light source.

[0019] In one embodiment, the light source change information is the change information generated by the changed light source compared to the historical light source before the change in the virtual scene; the projection coefficient update module is also used to obtain the historical light source projection coefficient corresponding to the historical light source; according to the illumination transformation matrix, the historical light source projection coefficient is updated to obtain the current light source projection coefficient corresponding to the changed light source.

[0020] In one embodiment, the indirect lighting determination module is also used to determine the indirect lighting value of the target pixel point belonging to a static object in the virtual scene based on the current light source projection coefficient and the lighting transfer parameters stored in the matching light map of the target pixel point.

[0021] In one embodiment, the indirect lighting determination module is also used to search for a texel matching the target pixel belonging to a static object in the virtual scene in the light map of the static object based on a texture mapping relationship; obtain lighting transfer parameters corresponding to the target pixel from the light map according to the matching texel; and determine the indirect lighting value of the target pixel in the virtual scene according to the current light source projection coefficient and the lighting transfer parameters corresponding to the target pixel.

[0022] In one embodiment, the indirect lighting determination module is also used to determine the indirect lighting value of the target pixel point belonging to the dynamic object in the virtual scene according to the current light source projection coefficient and the lighting transfer parameters stored in the matching lighting probe.

[0023] In one embodiment, the indirect lighting determination module is also used to determine the lighting probe that matches the target pixel point of the dynamic object in the virtual scene; update the light brightness corresponding to each direction on the lighting probe according to the current light source projection coefficient and the lighting transfer parameters stored in the lighting probe; interpolate the updated light brightness corresponding to each lighting probe; and determine the indirect lighting value of the target pixel point in the virtual scene according to the interpolated light brightness and the normal direction of the target pixel point.

[0024] In one embodiment, the above-mentioned lighting rendering device also includes a lighting transfer parameter baking module, which is used to project the original light source to multiple projection basis functions, and use each projection basis function after projection as a corresponding virtual light source; for each pixel point of the static object in the virtual scene, the lighting transfer parameters corresponding to each virtual light source at the pixel point are determined based on ray tracing, and stored in the corresponding light map; for the lighting probe in the virtual scene, the lighting transfer parameters corresponding to each virtual light source in the lighting probe are determined based on ray tracing, and stored in the lighting probe.

[0025] In one embodiment, the lighting transfer parameter baking module is also used to obtain, for each pixel point of a static object in the virtual scene, the light brightness corresponding to the pixel point after the projection of each virtual light source; starting from the pixel point, emitting rays with the light brightness toward the hemispherical surface pointed by the normal of the pixel point; in the virtual scene, sampling the reflection brightness and light attenuation of each ray after reflection from the hemispherical surface; determining the lighting transfer parameters corresponding to the pixel point based on the reflection light brightness and the light source brightness corresponding to each ray, and storing the lighting transfer parameters in the corresponding light map.

[0026] In one embodiment, the lighting transfer parameter baking module is also used to obtain the light brightness corresponding to each surface of the lighting probe after each virtual light source is projected on the lighting probe in the virtual scene; based on the light brightness, with the center point of the lighting probe as the starting point, rays are emitted to a sphere with a preset radius centered on the center point; in the virtual scene, the reflection brightness and light attenuation of each ray after reflection from the spherical surface are sampled; based on the reflection brightness and light attenuation corresponding to each ray, the lighting transfer parameters corresponding to each lighting probe are determined, and the lighting transfer parameters are stored in the lighting probe.

[0027] In one embodiment, the virtual scene is a virtual game scene, and the light source in the virtual game scene is a distant light source; the light source determination module is also used to monitor the light source direction of the distant light source in the virtual game scene when running the virtual game scene; when the light source direction changes, the light source change information is determined according to the changed light source direction and the initial light source direction of the distant light source.

[0028] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0029] When the light source in the virtual scene changes, light source change information is determined;

[0030] Determining a current light source projection coefficient corresponding to the changed light source according to the light source change information;

[0031] Determining an indirect illumination value of the target pixel in the virtual scene according to an illumination transfer parameter corresponding to the target pixel in the virtual scene and a current light source projection coefficient;

[0032] Determine the direct illumination value corresponding to the target pixel under the changed light source;

[0033] Perform illumination rendering on the target pixel point according to the direct illumination value and the indirect illumination value.

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0035] When the light source in the virtual scene changes, light source change information is determined;

[0036] Determining a current light source projection coefficient corresponding to the changed light source according to the light source change information;

[0037] Determining an indirect illumination value of the target pixel in the virtual scene according to an illumination transfer parameter corresponding to the target pixel in the virtual scene and a current light source projection coefficient;

[0038] Determine the direct illumination value corresponding to the target pixel under the changed light source;

[0039] Perform illumination rendering on the target pixel point according to the direct illumination value and the indirect illumination value.

[0040] A computer program product or a computer program, wherein the computer program product or the computer program comprises computer instructions, wherein the computer instructions are stored in a computer-readable storage medium; a processor of the computer device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, the following steps are implemented:

[0041] When the light source in the virtual scene changes, light source change information is determined;

[0042] Determining a current light source projection coefficient corresponding to the changed light source according to the light source change information;

[0043] Determining an indirect illumination value of the target pixel in the virtual scene according to an illumination transfer parameter corresponding to the target pixel in the virtual scene and a current light source projection coefficient;

[0044] Determine the direct illumination value corresponding to the target pixel under the changed light source;

[0045] Perform illumination rendering on the target pixel point according to the direct illumination value and the indirect illumination value.

[0046] The above-mentioned illumination rendering method, device, computer equipment and storage medium, when the light source in the virtual scene changes, first determine the light source change information, and then determine the current light source projection coefficient corresponding to the changed light source based on the light source change information, so that the current light source projection coefficient can be accurately and effectively updated in real time. Then, according to the illumination transfer parameters corresponding to the target pixel point in the virtual scene and the current light source projection coefficient, the indirect illumination value of the target pixel point in the virtual scene is determined. In this way, the indirect illumination value of the target pixel point under the changed light source can be accurately calculated. By determining the direct illumination value corresponding to the target pixel point under the changed light source, and according to the direct illumination value and the indirect illumination value, the target pixel point is illuminated and rendered, so that the light source that changes in real time can be accurately and efficiently illuminated and rendered, thereby effectively improving the illumination rendering effect under dynamic illumination in the virtual scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1A diagram of an application environment of a lighting rendering method in an embodiment;

[0048] Figure 2 is a schematic flow chart of a lighting rendering method in one embodiment;

[0049] Figure 3 is a schematic diagram of a lighting effect of a light source in one embodiment;

[0050] Figure 4 is a schematic diagram of the lighting effect of a light source in another embodiment;

[0051] Figure 5 A schematic diagram of lighting transfer parameters for each surface corresponding to a lighting probe in one embodiment;

[0052] Figure 6 is a schematic diagram of a scene screen in an embodiment;

[0053] Figure 7 It is a schematic diagram of projecting a scene picture in one embodiment;

[0054] Figure 8 is a flowchart of a lighting rendering method in a specific embodiment;

[0055] Fig. 9 A schematic diagram of a rendering effect of a scene screen in one embodiment;

[0056] Fig.10 A schematic diagram of a rendering effect of a scene screen in another embodiment;

[0057] Fig.11 A schematic diagram of the effects of using three different methods for lighting rendering in one embodiment;

[0058] Fig.12 is a structural block diagram of a lighting rendering device in one embodiment;

[0059] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] The image processing method in a three-dimensional scene provided by the present application can be applied to a computer device. The computer device can be a terminal or a server. It is understandable that the image processing method in a three-dimensional scene provided by the present application can be applied to a terminal, can also be applied to a server, can also be applied to a system including a terminal and a server, and can be implemented through the interaction between the terminal and the server.

[0062] The illumination rendering method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. Specifically, the terminal 102 can obtain scene data corresponding to the virtual scene from the server 104, and the scene data includes light source, light source change information, light source projection coefficient and other lighting data. When the light source in the virtual scene changes during the operation of the virtual scene, the terminal 102 determines the light source change information. The terminal 102 then determines the current light source projection coefficient corresponding to the changed light source based on the light source change information; determines the indirect lighting value of the target pixel in the virtual scene based on the lighting transfer parameters corresponding to the target pixel and the current light source projection coefficient; determines the direct lighting value corresponding to the target pixel under the changed light source; and performs lighting rendering on the target pixel based on the direct lighting value and the indirect lighting value.

[0063] Among them, the terminal 102 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 102 and the server 104 can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0064] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of technical network systems require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0065] Computer Vision Technology (Computer Vision, CV) Computer vision is a science that studies how to make machines "see". To put it more concretely, it refers to machine vision such as using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition. It can be understood that the illumination rendering method of the present application is based on computer vision technology to perform image processing on the target pixel points of the image to be rendered in the virtual scene, so as to effectively improve the illumination rendering effect under dynamic lighting scenes.

[0066] In one embodiment, Figure 2 As shown, a lighting rendering method is provided, and the method is applied to a computer device as an example for explanation. The computer device can be specifically Figure 1 It is understood that the method can also be applied to a system including a terminal and a server and implemented through the interaction between the terminal and the server. In this embodiment, the following steps are included:

[0067] S202: When the light source in the virtual scene changes, determine light source change information.

[0068] It can be understood that virtual scenes refer to digital scenes outlined by computers through digital communication technology, including two-dimensional virtual scenes and three-dimensional virtual scenes. Virtualization technology can be used to truly simulate various material forms, spatial relationships and other information in the world. Among them, three-dimensional virtual scenes can display the form of objects more beautifully, and can also display the virtual reality world more intuitively. For example, objects in a three-dimensional virtual scene can include at least one of terrain, houses, trees, and people. With the increasing application of virtual scenes, such as game development and video production,

[0069] For a virtual scene, a three-dimensional simulation environment can be displayed on the screen through a computer three-dimensional graphic display, and all objects in the virtual scene can be described by three-dimensional scene data. For example, the three-dimensional scene data can be loaded into the three-dimensional scene to display the three-dimensional simulation environment. For example, the three-dimensional scene data includes at least one of model data, texture data, illumination data, vector data, terrain data, grid data, etc. Among them, the illumination data includes at least one of a light source and light source change information, etc.

[0070] The terminal can perform a series of rendering processes on the scene images in the virtual scene so that the picture content in the virtual scene can be displayed on the screen in the form of a two-dimensional image. Among them, in computer graphics, rendering refers to the process of projecting the object model in the three-dimensional scene into a digital image in two dimensions according to the set environment, material, lighting and rendering parameters. Among them, lighting rendering is the process of converting the three-dimensional light energy transfer process into a two-dimensional image. The entities in the virtual scene and environment are represented in three-dimensional form, which is closer to the real world and is easy to manipulate and transform. After a series of rendering processes such as vertex rendering, shading rendering and lighting rendering are performed on the pixels to be displayed in the virtual scene, the presentation effect of the virtual scene can be made more refined and beautiful.

[0071] The light source refers to an object that can emit light by itself and is emitting light, such as the sun, electric lights, burning substances, etc. It can be understood that the light source in the virtual scene is a series of lighting data that can truly model the lighting effect of the light source in reality.

[0072] The light source in the virtual scene is calculated by the light distribution and color distribution of the illuminated object surface, and the light effect of the object is displayed by making the illuminated object and the surrounding environment have a contrast in light and color. Different from the light source in reality, the light source in the virtual scene can be a virtual light source node without shape or outline.

[0073] For example, the lighting effect of the illuminated object can be calculated based on the position of the light source in the virtual scene, while the viewer cannot see the light source itself. Similar to the light source in reality, when the colored light source in the virtual scene is projected onto the surface of a colored object, the final rendered color depends on the reflection and absorption of the light. In particular, the shadow effect when the light source in the virtual scene shines on the object in the virtual scene can also be constructed based on the pre-built 3D geometric model in the virtual scene.

[0074] In a virtual scene, the propagation of light in the environment can be simulated to achieve a simulated lighting effect. Among them, light can propagate in a straight line in a medium. When encountering obstacles such as objects during the propagation process, light may be reflected, scattered, diffusely reflected, absorbed, etc. Among them, objects of different materials have different reflection pass rates on their surfaces. For example, when light propagates to a mirror, it will be reflected, and its emission pass rate is higher. When light propagates to a desktop, it will be diffusely reflected, and its emission pass rate will be attenuated.

[0075] It is understandable that when the light source in the virtual scene changes, the ambient lighting effect in the virtual scene will also change accordingly. The light source change information refers to the change information generated by the light source after the change compared to the light source before the change in the virtual scene. For example, the light source change information may include at least one of the light source position change information, light source intensity change information, and light source direction change information.

[0076] Specifically, when the terminal is displaying a virtual scene or running an application including a virtual scene, when it detects that the light source in the virtual scene has changed, it obtains the current light source information, compares the current light source information with the light source information before the light source changed, and thus determines the light source change information.

[0077] S204: Determine a current light source projection coefficient corresponding to the changed light source according to the light source change information.

[0078] Among them, the light source projection coefficient may refer to the illumination intensity coefficient corresponding to the light source. It can be understood that there is a linear relationship between the light source and the light source projection coefficient, and different light sources correspond to different projection coefficients. For example, the light source in the virtual scene can be regarded as a virtual light source, that is, a set of light source functions. When this virtual light source is projected onto the projection basis function, a set of coefficients will be obtained, which is the light source projection coefficient.

[0079] It is understandable that the lighting in the virtual scene is a linear system, and the lighting effect produced by the linear combination of light sources is also a linear combination of the lighting effects of individual light sources. Figure 3As shown, FIG32 includes light source 3a and light source 3b, FIG32a includes light source 3a, and FIG32b includes light source 3b. The lighting effect in FIG32 is the lighting effect of FIG32a and FIG32b superimposed. Figure 4 As shown, FIG42 includes light source 4a, and FIG42a includes half the light brightness of light source 4a. The lighting effect in FIG42 is twice the lighting effect in FIG42a. It can be seen that FIG32 includes one times the light source 3a and one times the light source 3b, and FIG42 includes two times the light source 4a, wherein the multiple of the light source is the light source projection coefficient corresponding to each light source.

[0080] When the terminal detects that the light source in the virtual scene has changed, the terminal updates the lighting in the current virtual scene according to the light source change information after the light source has changed. Specifically, the terminal updates the lighting data corresponding to the target pixel point in the virtual scene according to the light source change information. That is, when the light source changes, the terminal needs to recalculate the current light source projection coefficient corresponding to the current light source in the virtual scene according to the light source change information.

[0081] S206, determining the indirect illumination value of the target pixel point in the virtual scene according to the illumination transfer parameter corresponding to the target pixel point in the virtual scene and the current light source projection coefficient.

[0082] The pixel point may refer to the center position of a pixel. It can be understood that a pixel may be an area in the form of a small square, and a small square may correspond to a pixel, and the center of a pixel is the pixel point. An image is usually composed of multiple pixels.

[0083] The three-dimensional virtual scene needs to go through a series of renderings to make its content appear on the screen in a two-dimensional form. The three-dimensional virtual scene can be displayed through continuous scene screen images, which are composed of multiple pixels.

[0084] It can be understood that the target pixel refers to the pixel corresponding to the image to be rendered in the virtual scene, that is, the pixel that needs to be rendered currently. Among them, the pixel information such as the color value, brightness value, depth value, etc. of the target pixel can be determined by the scene data in the virtual scene. For example, the brightness value and color value of the target pixel are related to the light density of the environment in which it is located, the number of virtual objects that block the light around it, etc.

[0085] The illumination transfer parameters may include an illumination transfer vector, which is a vector used to convert incident light into a transferred incident light including self-occlusion and mutual reflection, and can be specifically used to determine the irradiance per unit area.

[0086] It can be understood that after the light source is projected into the virtual environment, a global illumination effect can be generated. Among them, global illumination is a special term in 3D software. Light has the properties of reflection and refraction. In the real natural environment, the sunlight during the day is reflected and refracted countless times before it hits the ground, so the ground seen by the human eye is clear. In 3D software, global illumination can not only calculate the bright and dark sides of objects, but also calculate various light effects such as reflection and refraction of light in the virtual environment.

[0087] In reality, the illumination received by the surface of an object does not all come from the light source, but also includes the illumination reflected by other objects. The illumination from the light source is direct illumination, while the illumination from other objects is indirect illumination. The global illumination effect includes the effects of direct illumination and indirect illumination, that is, global illumination = direct illumination + indirect illumination.

[0088] Direct illumination refers to the brightness of light that is directly irradiated on an object and then reflected to a virtual observation point or camera. Indirect illumination refers to the brightness of light that is first irradiated on other objects and, after one or more bounces, finally reaches the surface of the observed object and then is reflected to a virtual observation point or camera.

[0089] Among them, light illuminance refers to the light intensity, that is, the radiance per unit projection area. The unit projection area is a plane perpendicular to the light, and the radiant flux per unit area is called radiance. Among them, radiant flux represents the total amount of light energy flowing through a certain area per second. The same light energy irradiates on the surface of objects of different areas and its brightness is also different. The angle of the surface of the object will cause the incident light to spread out and reduce the radiance.

[0090] Among them, the direct illumination value refers to the color value of the object in the virtual environment after receiving direct illumination from the light source, that is, the illumination intensity value. Similarly, the indirect illumination value refers to the color value of the object in the virtual environment after receiving indirect illumination.

[0091] After the terminal determines the current light source projection coefficient corresponding to the changed light source according to the light source change information after the light source changes, it determines the indirect lighting value of the target pixel in the virtual scene according to the lighting transfer parameters corresponding to the target pixel and the current light source projection coefficient.

[0092] Specifically, before rendering the target pixel, the terminal may pre-calculate the lighting transfer parameters corresponding to each object in the virtual scene, which may specifically include the lighting transfer parameters corresponding to each static object in the virtual scene and the lighting transfer parameters corresponding to the lighting probe.

[0093] It is understandable that the indirect illumination value of the target pixel in the virtual scene can be directly calculated from the illumination transfer parameter and the current light source projection coefficient.

[0094] S208, determining the direct illumination value corresponding to the target pixel under the changed light source.

[0095] It is understandable that when the light source in the virtual environment changes, the direct light intensity received by each object in the virtual environment will also change. The direct lighting and shadows in the virtual scene can be dynamically calculated using the runtime rendering pipeline, thereby achieving real-time calculation of direct lighting.

[0096] Specifically, the terminal can determine the brightness of the light irradiated by the changed light source onto the target pixel point according to the light source change information, and calculate the direct illumination value corresponding to the target pixel point under the changed light source in real time according to the changed light brightness, the object color corresponding to the target pixel point, and the angle between the target pixel point and the virtual observation point. Specifically, a preset method can be used to calculate the brightness contribution of the light after the light of the changed light source is projected onto the target pixel point, and use it as the color value of the pixel, so as to obtain the direct illumination value corresponding to the target pixel point under the changed light source.

[0097] Among them, when the light source does not directly illuminate the target pixel point, the direct illumination value corresponding to the target pixel point can be zero, that is, the target pixel point only includes the indirect illumination value after multiple reflections of the light from the light source to the virtual environment. It can be understood that there are multiple target pixels to be rendered, and for each pixel point, it can include only direct illumination value or indirect illumination value, or it can include both direct illumination value and indirect illumination value. Among them, when the direct illumination value is a valid value, the indirect illumination value can be zero; when the indirect illumination value is a valid value, the direct illumination value can be zero.

[0098] In one embodiment, the terminal further determines the shadow corresponding to each target pixel according to the direct illumination value corresponding to the target pixel. Specifically, for the calculation of the shadow value, for each pixel, a ray is emitted to the light source. If the ray intersects with the object, the object is in the shadow, and the shadow value is 0. If the ray does not intersect with the object, the shadow value is 1. For the case where there may be a penumbra at the edge of the shadow, the shadow value is between 0 and 1.

[0099] S210: Perform illumination rendering on the target pixel point according to the direct illumination value and the indirect illumination value.

[0100] Among them, lighting rendering refers to the lighting calculation processing of the target pixel points in the rendered image during the rendering process, so that the final target pixel points have a lighting effect.

[0101] It can be understood that by calculating the direct illumination value and the indirect illumination value of each target pixel in the virtual environment, the global illumination information of each target pixel in the virtual environment is obtained.

[0102] Specifically, after the terminal calculates the direct illumination value and the indirect illumination value corresponding to the target pixel to be rendered, it performs illumination rendering on the target pixel according to the direct illumination value and the indirect illumination value corresponding to each target pixel to obtain the final picture rendering result.

[0103] In the above-mentioned illumination rendering method, when the light source in the virtual scene changes, the terminal first determines the light source change information, and then determines the current light source projection coefficient corresponding to the changed light source according to the light source change information, so that the current light source projection coefficient corresponding to the current light source can be accurately and effectively updated in real time. The terminal then determines the indirect illumination value of the target pixel in the virtual scene according to the current light source projection coefficient and the illumination transfer parameter corresponding to the target pixel. In this way, the indirect illumination value of the target pixel under the changed light source can be accurately calculated. The terminal then determines the direct illumination value corresponding to the target pixel under the changed light source, and performs illumination rendering on the target pixel according to the direct illumination value and the indirect illumination value, thereby being able to accurately and efficiently perform illumination rendering processing on the light source that changes in real time, thereby effectively improving the illumination rendering effect in dynamic illumination scenes.

[0104] In one embodiment, based on the light source change information, the current light source projection coefficient corresponding to the changed light source is determined, including: determining the illumination transformation matrix based on the light source change information; determining the initial light source projection coefficient before the light source in the virtual scene changes; and determining the current light source projection coefficient corresponding to the changed light source based on the illumination transformation matrix and the initial light source projection coefficient.

[0105] It can be understood that when the light source changes, the corresponding light source projection coefficient will also change. The light source change information corresponding to the light source change is specifically determined based on the difference between the light source after the change and the light source before the change.

[0106] Among them, the transformation matrix is ​​a linear algebra that can perform linear transformations. Any linear transformation can be represented by a matrix in a consistent form that is easy to calculate, and multiple transformations can also be easily connected together by multiplying matrices. For example, in three-dimensional computer graphics, affine transformations and perspective projections in projection space can use homogeneous coordinates to represent multi-dimensional linear transformations.

[0107] It can be understood that the illumination transformation matrix refers to a transformation matrix used to represent light changes. The initial light source projection coefficient refers to the light source projection coefficient corresponding to the light source before the light source changes.

[0108] Specifically, after the terminal determines the light source change information, it first updates the illumination transformation matrix after the light source changes according to the light source change information. The terminal then obtains the initial light source projection coefficient corresponding to the light source before the light source changes, and then calculates the current light source projection coefficient corresponding to the changed light source according to the updated illumination transformation matrix and the initial light source projection coefficient. In this way, the current light source projection coefficient corresponding to the current light source in the virtual scene can be calculated in real time, so that the light source projection coefficient under the dynamic light source can be quickly updated.

[0109] In one embodiment, the light source change information is the change information of the changed light source compared with the original light source preset in the virtual scene; determining the initial light source projection coefficient before the light source in the virtual scene changes includes: obtaining the initial light source projection coefficient obtained by projecting the original light source onto the projection basis function. Determining the current light source projection coefficient corresponding to the changed light source according to the illumination transformation matrix and the initial light source projection coefficient includes: updating the initial light source projection coefficient according to the illumination transformation matrix to obtain the current light source projection coefficient corresponding to the changed light source.

[0110] The original light source preset in the virtual scene is a fixed original light source preset in the virtual scene before the virtual scene is displayed. The preset original light source will not change.

[0111] In mathematics, basis functions refer to elements of a specific basis in function space. In function space, each continuous function can be expressed as a linear combination of basis functions. Basis functions are also called mixing functions, and they can be mixed together to serve as interpolation functions. It can be understood that the projection basis function is used to input the light source into the basis function in order to calculate the reflection of the light source through the basis function.

[0112] In one embodiment, the projection basis function may specifically be a spherical harmonic function. The spherical harmonic function is a harmonic function that satisfies the Laplace equation restricted to a unit sphere in a spherical coordinate system, and can be used to represent the directional distribution on a sphere. In graphics, the spherical harmonic function is a generalized Fourier transform defined on a sphere, and discrete spherical functions can be parameterized to simulate low-frequency ambient light illumination. For example, functions related to direction such as BRDF functions (Bidirectional Reflectance Distribution Function) used to represent the surface of an object in a virtual scene, environment maps, illumination transmission, and visibility distribution. The spherical harmonic function can be used to capture illumination, and then perform re-illumination processing to calculate the global illumination in a virtual environment in real time. In other embodiments, the projection basis function may also use a spherical Gaussian function, a custom piecewise function, etc. as the projection basis function, which is not limited in this application.

[0113] It can be understood that the light source in the virtual scene has a corresponding light source function. Projecting the light source into the projection basis function means projecting the light source function onto the preset projection basis function.

[0114] By projecting the light source into the projection basis function, for each point on the surface, the incident light from all directions will contribute to its brightness. To calculate the final outgoing light intensity of a single point requires a lot of light sampling. The specific solution of the rendering equation can be regarded as an integral on a hemisphere.

[0115] When the light source in the virtual scene changes, the terminal compares the changed light source with the original light source preset in the virtual scene to obtain light source change information. Then, the terminal obtains the initial light source projection coefficient corresponding to the original light source. The initial light source projection coefficient is the projection coefficient obtained by projecting the original light source onto the projection basis function.

[0116] The terminal determines the corresponding illumination transformation matrix according to the light source change information, and then updates the initial light source projection coefficient according to the illumination transformation matrix. Specifically, the terminal can perform dot product processing on the illumination change matrix and the initial light source projection coefficient, so as to obtain the updated projection coefficient. The terminal uses the obtained updated projection coefficient as the current light source projection coefficient corresponding to the changed light source in the virtual scene, thereby being able to quickly calculate the current light source projection coefficient corresponding to the current light source in real time.

[0117] In one embodiment, the light source change information is the change information of the light source after the change compared with the historical light source before the change in the virtual scene; determining the initial light source projection coefficient before the light source in the virtual scene changes includes: obtaining the historical light source projection coefficient corresponding to the historical light source. Determining the current light source projection coefficient corresponding to the light source after the change according to the illumination transformation matrix and the initial light source projection coefficient includes: updating the historical light source projection coefficient according to the illumination transformation matrix to obtain the current light source projection coefficient corresponding to the light source after the change.

[0118] The historical light source before the change in the virtual scene refers to the light source corresponding to the current light source after the change before the change. For example, if the light source at the current moment has changed compared to the light source at the previous moment, then the light source at the current moment is the light source after the change, and the light source at the previous moment is the historical light source before the change. It can be understood that the historical light source projection coefficient is the historical light source projection coefficient that has been calculated and corresponds to the historical light source. Specifically, it can be the historical light source projection coefficient obtained by projecting the historical light source onto the projection basis function.

[0119] When the light source in the virtual scene changes, the terminal compares the changed light source with the historical light source before the change in the virtual scene to obtain light source change information, and then directly obtains the calculated historical light source projection coefficient corresponding to the historical light source.

[0120] The terminal then determines the corresponding illumination transformation matrix according to the light source change information to update the historical light source projection coefficient according to the illumination transformation matrix. Specifically, the terminal can perform dot product processing on the illumination change matrix and the historical light source projection coefficient, which is similar to transforming the matrix and the vector (a set of projection coefficients) to obtain updated projection coefficients. The terminal uses the obtained updated projection coefficients as the current light source projection coefficients corresponding to the changed light source in the virtual scene.

[0121] In this embodiment, by updating the historical light source projection coefficient before the change in real time, the historical light source projection coefficient at the last moment can be accurately and effectively updated to accurately and in real time calculate the lighting information after the light source changes.

[0122] In one embodiment, the indirect lighting value of the target pixel in the virtual scene is determined based on the lighting transfer parameters corresponding to the target pixel in the virtual scene and the current light source projection coefficient, including: for the target pixel that is a static object in the virtual scene, the indirect lighting value of the target pixel in the virtual scene is determined based on the current light source projection coefficient and the lighting transfer parameters of the target pixel stored in a matching light map.

[0123] It can be understood that a static object in a virtual scene refers to an object that is fixed in the virtual scene, and its position and direction etc. will not change.

[0124] Among them, light map, or Light Map, refers to a picture of lighting information pre-generated by using the global illumination algorithm for static target objects in the virtual scene, which is used to represent the lighting visual effect of static objects. Light map, similar to color rendering, is used to perform lighting rendering processing for the corresponding static objects to render the corresponding lighting effects, thereby simulating the impact of lighting on objects in real scenes.

[0125] The light map includes lighting information corresponding to multiple texels, that is, pixel values ​​or color values. Texel is the abbreviation of texture element, which is the basic unit in the texture space of computer graphics. Texels can be mapped to appropriate output image pixels through texture mapping technology. The texels in the embodiment of the present application correspond to pixels, that is, the texels in the light map correspond to the pixels on the surface of static objects. The lighting information of the pixels can be used to render the objects.

[0126] Specifically, a light map may correspond to an object surface map corresponding to a static object in a virtual scene. In another embodiment, a light map may correspond to one or more static objects in a virtual scene, for example, multiple object surface maps on the same plane. The use of light map technology can generate lighting information for real-time rendered drawing objects offline, improving image quality while ensuring performance.

[0127] Among them, the light map corresponding to the static object in the virtual scene stores the pre-calculated light transfer parameters corresponding to the texels of the static object. For example, the light transfer parameters corresponding to the static object can be pre-calculated based on the PRT technology (Precomputed Radiance Transfer, a rendering technology for pre-calculated radiance transmission), and then the light transfer parameters are stored in the corresponding light map. Therefore, when the terminal calculates the indirect lighting of the static object, it can directly obtain the corresponding light transfer parameters from the light map for lighting calculation processing.

[0128] When the light source in the virtual scene changes, the terminal determines the current light source projection coefficient corresponding to the changed light source based on the light source change information after the change, and then obtains the pre-calculated lighting transfer parameters corresponding to the target pixel point to calculate the indirect lighting value of the target pixel point in the virtual scene based on the lighting transfer parameters and the current light source projection coefficient.

[0129] Specifically, when the target pixel points to be rendered include the target pixel points of static objects in the virtual scene, for the target pixel points belonging to the static objects in the virtual scene, the terminal obtains the lighting transfer parameters corresponding to the target pixel points from the target pixel points in the matching light map. Then, the terminal calculates the indirect lighting value of the target pixel points in the virtual scene based on the current light source projection coefficient and the obtained lighting transfer parameters.

[0130] In this embodiment, the lighting transfer parameters corresponding to the static objects in the virtual scene are pre-baked and stored in the corresponding light map. When the terminal calculates the indirect lighting of the static objects, it can directly obtain the corresponding lighting transfer parameters from the light map for lighting calculation processing, thereby being able to accurately and quickly calculate the indirect lighting of the static objects.

[0131] In one embodiment, for a target pixel point that is a static object in a virtual scene, the current light source projection coefficient and the lighting transfer parameters of the target pixel point stored in a matching light map are used to determine the indirect lighting value of the target pixel point in the virtual scene, including: for the target pixel point that is a static object in the virtual scene, based on the texture mapping relationship, searching for a texel that matches the target pixel point in the light map of the static object; based on the matching texel, obtaining the lighting transfer parameters corresponding to the target pixel point from the light map; and determining the indirect lighting value of the target pixel point in the virtual scene based on the current light source projection coefficient and the lighting transfer parameters corresponding to the target pixel point.

[0132] Among them, texture mapping is the process of mapping texture pixels in texture space to pixels in screen space. Texture mapping relationship is the mapping relationship between texels in texture space and pixels in screen space. Through texture mapping, an image can be pasted on the surface of a three-dimensional object to enhance the sense of reality. It can be combined with lighting calculation, image blending and other technologies to form many very beautiful effects.

[0133] Specifically, when calculating the indirect illumination value of the target pixel points belonging to the static object in the virtual scene, for each target pixel point, the terminal first obtains the light map corresponding to the static object, and then searches for the texel matching the target pixel point in the light map corresponding to the static object according to the texture mapping relationship. The texel of the static object in the light map corresponds to the pixel point on the surface of the static object.

[0134] Then, the terminal obtains the pre-stored lighting transfer parameters corresponding to each target pixel from the light map according to the matching texels. The terminal then performs dot product processing on the current light source projection coefficient and the lighting transfer parameters corresponding to each target pixel, and can calculate the indirect lighting value of each target pixel in the virtual scene.

[0135] In this embodiment, the lighting transfer parameters of each static object in the virtual scene are pre-baked and stored in the corresponding light map. Therefore, when the virtual scene is running, the indirect lighting value of the static object can be quickly calculated in real time, and the indirect lighting value of the static object after the light source changes can be accurately and quickly calculated.

[0136] In one embodiment, the indirect lighting value of the target pixel in the virtual scene is determined based on the lighting transfer parameters corresponding to the target pixel in the virtual scene and the current light source projection coefficient, including: for the target pixel belonging to the dynamic object in the virtual scene, the indirect lighting value of the target pixel in the virtual scene is determined based on the current light source projection coefficient and the lighting transfer parameters stored in the matching lighting probe.

[0137] It can be understood that a dynamic object in a virtual scene refers to a non-fixed object in the virtual scene, that is, a moving object that can move in the virtual scene, and at least one of its position and direction is changeable.

[0138] Among them, the light probe is a light detector pre-placed in the virtual scene, which can be a sphere or a polygon, such as a cube. The light probe can capture and use the relevant information of the light passing through the empty space of the scene, which can be used to provide high-quality lighting information including indirect reflected light for dynamic objects in the virtual scene, and provide high-precision detailed lighting information for static objects.

[0139] Similar to lightmaps, light probes store baked information about the lighting in the virtual scene, including pre-calculated lighting transfer parameters. The difference is that lightmaps store information about the lighting that hits the surfaces of static objects in the virtual scene, while light probes store information about the light that passes through the empty space in the scene.

[0140] Specifically, when the target pixel to be rendered includes the target pixel of a dynamic object in the virtual scene, for the target pixel belonging to the dynamic object in the virtual scene, the terminal obtains the lighting probe matching the target pixel, and obtains the corresponding lighting transfer parameter from the lighting probe. The terminal then calculates the indirect lighting value of the target pixel in the virtual scene based on the current light source projection coefficient and the lighting transfer parameter stored in the lighting probe matching the target pixel. Then, the terminal calculates the indirect lighting value of the target pixel in the virtual scene based on the current light source projection coefficient and the obtained lighting transfer parameter.

[0141] In this embodiment, the lighting transfer parameters corresponding to the lighting probes in the virtual scene are pre-baked and stored in the corresponding lighting probes. When the terminal calculates the indirect lighting of dynamic objects, it can directly obtain the corresponding lighting transfer parameters from the lighting probes for lighting calculation processing, thereby being able to accurately and quickly calculate the indirect lighting of dynamic objects.

[0142] In one embodiment, for a target pixel point belonging to a dynamic object in a virtual scene, the indirect lighting value of the target pixel point in the virtual scene is determined based on a current light source projection coefficient and lighting transfer parameters stored in a matching lighting probe, including: for a target pixel point belonging to a dynamic object in the virtual scene, a lighting probe matching the target pixel point of the dynamic object is determined; based on the current light source projection coefficient and lighting transfer parameters stored in the lighting probe, the light brightness corresponding to each direction on the lighting probe is updated; the updated light brightness corresponding to each lighting probe is interpolated; and based on the interpolated light brightness and the normal direction of the target pixel point, the indirect lighting value of the target pixel point in the virtual scene is determined.

[0143] It can be understood that there are multiple directions on the light probe, and each direction has a corresponding pre-baked light brightness. That is, for each direction on the light probe, the corresponding light transfer parameters for each direction are pre-calculated.

[0144] Among them, the normal line refers to a straight line that is always perpendicular to a plane. In geometry, a normal line refers to a line on a plane that is perpendicular to the tangent line of a curve at a certain point. For a solid surface, the normal line has a direction, and the positive direction of the normal line is usually from the inside of the solid to the outside, and vice versa. In this embodiment, the normal direction of the target pixel point refers to the direction of the normal line from the target pixel point to the virtual observation point or camera.

[0145] Specifically, in the process of calculating the indirect lighting value of the target pixel points belonging to the dynamic object in the virtual scene, the terminal first obtains the lighting probe matching the dynamic object for each target pixel point. Specifically, the lighting probe within the preset distance from the dynamic object can be obtained. Then the lighting transfer parameters stored in each lighting probe are obtained. The terminal then calculates the light brightness in each direction of each lighting probe based on the lighting transfer parameters stored in each lighting probe, and the current projection coefficients corresponding to each probe surface. Specifically, the calculated light brightness can be an indirect lighting value.

[0146] Furthermore, the terminal then interpolates the light brightness corresponding to the illumination probes matching the dynamic object to obtain the interpolated light brightness. The terminal then calculates the indirect illumination value corresponding to the target pixel on the dynamic object according to the normal direction of the target pixel on the dynamic object, thereby being able to accurately calculate the indirect illumination value of each target pixel in the virtual scene.

[0147] In this embodiment, the lighting transfer parameters of each lighting probe in the virtual scene are pre-baked and stored in the corresponding lighting probe. Therefore, when the virtual scene is running, the indirect lighting value of the dynamic object can be quickly calculated in real time, and the indirect lighting value of the dynamic object after the light source changes can be accurately and quickly calculated.

[0148] In one embodiment, when the light source in the virtual scene changes, before determining the light source change information, the above-mentioned lighting rendering method also includes: projecting the original light source to multiple projection basis functions, and using each projection basis function after projection as the corresponding virtual light source; for the pixel points of static objects in the virtual scene, determining the lighting transfer parameters corresponding to each virtual light source at the pixel points based on ray tracing, and storing them in the corresponding light map; for the lighting probes in the virtual scene, determining the lighting transfer parameters corresponding to each virtual light source in the lighting probes based on ray tracing, and storing them in the lighting probes.

[0149] Among them, ray tracing, also known as path tracing, is a technology used in computer graphics to render virtual scenes. It refers to tracing the propagation path of light in a virtual scene, and the light path of the light during the propagation process can be obtained. The principle is to send a ray from the viewpoint. When the ray intersects with the surface of the object, a random direction is selected according to the material properties of the surface, and another ray is sent out for tracking sampling, and iterates until the ray hits the light source or escapes from the scene. Then the contribution of the ray is calculated as the color value of the pixel. For example, the Monte Carlo method can be used to calculate the contribution of the ray.

[0150] Therefore, as long as the number of iterations is large enough and the rendering time is long enough, the final rendered image can converge to a lighting rendering result with high accuracy. Ray tracing can render a highly realistic scene effect, but it also requires relatively high computational overhead.

[0151] In geometric optics, the radiant brightness of light reflected from the surface of an object does not change during optical propagation, that is, the illumination transfer parameters do not change. Therefore, by pre-baking the illumination transfer parameters of the surfaces of objects in the virtual scene, the computational overhead in the real-time rendering process can be effectively reduced to improve the efficiency of the rendering process.

[0152] It can be understood that before rendering the target pixel points to be rendered in the virtual scene, the lighting transfer parameters corresponding to the pixel points of the static objects in the virtual scene and the lighting transfer parameters corresponding to the lighting probes can be pre-baked. Specifically, the lighting transfer parameters can be baked based on the basis function light source in the virtual scene. Among them, the basis function light source, that is, the virtual light source, that is, after the original light source is projected onto the preset projection basis function, each basis function is used as a virtual light source, and the projected basis function is the basis function light source.

[0153] The illumination transfer parameters can be pre-baked by ray tracing based on the projection basis function. The projection basis function can be a spherical harmonic function. Since the spherical harmonic function has the characteristics of rotation invariance and high projection performance, when the light source changes, the illumination transfer parameters will not change, and only the light source projection coefficient corresponding to the light source will change with the change of the light source. Therefore, the spherical harmonic function can be used to effectively bake the illumination transfer parameters in the form of hemispherical surface integral.

[0154] The terminal can project the original light source onto multiple projection basis functions, use each projection basis function as the corresponding virtual light source, and then use ray tracing to calculate the brightness information of the virtual light source on the surface of each object in the virtual scene. In the process of pre-calculating the light transfer parameters, the terminal only samples the indirect light, that is, only samples the rays that have bounced more than twice.

[0155] Then, each virtual light source is projected onto the surface of each object in the virtual scene, and each pixel point on the surface of each object in the virtual scene receives the brightness information corresponding to each virtual light source.

[0156] Specifically, in the process of baking the lighting transfer parameters, the terminal first projects the preset original light source onto a plurality of different preset projection basis functions, for example, it can be projected onto three different projection basis functions. Among them, a plurality means at least two or more. It can be understood that projecting the original light source onto different projection basis functions will produce an approximately dynamic light source, and each projection basis function after projection is a virtual light source, so each projection basis function after projection can be used as a corresponding virtual light source. Each projection basis function after projection is projected as a virtual light source and then projected into the virtual scene, and each pixel point on the surface of each object in the virtual scene can receive the brightness information of each virtual light source, so that the lighting information corresponding to the approximate dynamic light source can be sampled in the virtual scene.

[0157] After receiving the brightness information of each virtual light source, each pixel point on the surface of each object in the virtual scene emits a reflected ray from the pixel point of each object. Then, ray tracing is performed on the reflected ray to capture the brightness contribution of the reflected ray in the virtual scene, thereby determining the illumination transfer parameters corresponding to each pixel point on the surface of the object.

[0158] Specifically, the terminal can bake out the corresponding lighting transfer parameters for the static objects and lighting probes in the virtual scene. For each pixel of the static object in the virtual scene, the terminal determines the lighting transfer parameters corresponding to each virtual light source at each pixel based on ray tracing, and stores them in the corresponding light map. For the lighting probes in the virtual scene, the terminal determines the lighting transfer parameters corresponding to each virtual light source in the lighting probe based on ray tracing, and stores them in the corresponding lighting probes.

[0159] In this embodiment, the lighting transfer parameters corresponding to the static objects and lighting probes in the virtual scene are calculated in advance and stored in the corresponding light maps and lighting probes respectively. Therefore, when performing real-time lighting rendering processing, the lighting transfer parameters corresponding to the target pixel points can be directly obtained for indirect lighting calculation, thereby effectively reducing the computing consumption in the real-time rendering process and effectively improving the lighting rendering effect.

[0160] In one embodiment, for each pixel point of a static object in a virtual scene, the lighting transfer parameters corresponding to each virtual light source at the pixel point are determined based on ray tracing, and stored in a corresponding light map, including: for each pixel point of a static object in the virtual scene, obtaining the light brightness corresponding to each virtual light source at the pixel point after projection; starting from the pixel point, emitting rays with light brightness toward the hemispherical surface pointed by the normal of the pixel point; in the virtual scene, sampling the reflection brightness and light attenuation of each ray after reflection from the hemispherical surface; based on the reflection light brightness and light source brightness corresponding to each ray, determining the lighting transfer parameters corresponding to the pixel point, and storing the lighting transfer parameters in the corresponding light map.

[0161] Among them, the brightness of the light after the pixel is projected by the original light source refers to the light brightness information received by each pixel on the static object in the virtual scene after the original light source in the virtual scene is projected to the preset projection basis function, that is, the brightness of the pixel color after the pixel is irradiated by the light source.

[0162] It can be understood that, taking the spherical harmonic function as an example of a projection basis function, the spherical harmonic function is an orthogonal basis defined on the unit sphere. After the light source is projected onto the projection basis function, the projection basis function can be used to represent the directional distribution on the sphere. The hemispherical surface pointed to by the normal of the pixel point refers to the orthogonal basis on the unit hemispherical surface corresponding to each pixel point on the projection basis function.

[0163] Specifically, when the terminal pre-calculates the illumination transfer parameters corresponding to the static objects in the virtual scene, after each virtual light source is projected in the virtual scene, each pixel on the static object will receive the light brightness corresponding to each virtual light source, which may be the light brightness of each virtual light source after multiple reflections.

[0164] Then, for each pixel on the static object, the terminal takes the world space position corresponding to the center of each pixel as the starting point, and based on the brightness of the received light, emits light to the hemisphere in the direction of the normal of the pixel, that is, emits rays with light brightness.

[0165] The terminal then samples in the virtual scene, calculates the reflection brightness and light attenuation of the ray after it is reflected by the hemispherical surface, and then calculates the light brightness contribution of the ray after multiple bounces based on the reflection brightness and light attenuation. Specifically, when the reflected ray intersects with an object in the virtual scene, the light brightness contribution of the ray after multiple bounces on the current projection basis function is calculated. For each ray, the light brightness contribution corresponding to each ray is calculated, and then the light brightness contribution is fitted to the corresponding projection basis function, and then the lighting transfer parameters on the corresponding projection basis function are calculated based on the light brightness contribution.

[0166] It can be understood that after the terminal projects the original light source onto multiple projection basis functions, for each basis function, it samples the brightness contribution of the ray after multiple bounces in the virtual scene, and calculates the illumination transfer parameters on the corresponding projection basis function. Then, the illumination transfer parameters on each projection basis function are fitted to obtain the final illumination transfer parameters corresponding to the pixel point.

[0167] After the terminal calculates the lighting transfer parameters corresponding to each pixel point on the static object in the virtual scene, it stores the lighting transfer parameters to the corresponding texel position in the light map according to the texture mapping relationship.

[0168] In this embodiment, after the original light source is projected onto multiple projection basis functions, rays are emitted from each pixel point to the hemisphere pointed to by the normal of the pixel point, and then the brightness contribution of each ray after multiple reflections is sampled in the virtual scene, so that the lighting transfer parameters corresponding to each pixel point on the static object in the virtual scene can be accurately calculated.

[0169] In one embodiment, for the lighting probes in the virtual scene, the lighting transfer parameters corresponding to each virtual light source in the lighting probe are determined based on ray tracing and stored in each lighting probe, including: for the lighting probes in the virtual scene, the light brightness corresponding to each surface of the lighting probe after the projection of each virtual light source is obtained; based on the light brightness, with the center point of the lighting probe as the starting point, rays are emitted to a sphere with a preset radius centered on the center point; in the virtual scene, the reflection brightness and light attenuation of each ray after reflection from the spherical surface are sampled; based on the reflection brightness and light attenuation corresponding to each ray, the lighting transfer parameters corresponding to each lighting probe are determined, and the lighting transfer parameters are stored in the lighting probe.

[0170] It can be understood that each lighting probe may be preset with a corresponding plurality of projection basis functions, which may specifically be spherical basis functions.

[0171] When the terminal pre-calculates the lighting transfer parameters corresponding to the lighting probe in the virtual scene, after each virtual light source is projected in the virtual scene, the faces in multiple directions of each lighting probe in the virtual scene will receive the light brightness corresponding to each virtual light source. For each face in each direction of the lighting probe, the terminal then uses the center point of the lighting probe as the starting point, and emits rays to a sphere with a preset radius centered on the center point according to the light brightness received by the lighting probe.

[0172] The brightness contribution of each ray is calculated in the same way as for each pixel on a static object. After the brightness contribution corresponding to each ray is calculated, the brightness contribution is fitted to the corresponding projection basis function to calculate the corresponding lighting transfer parameters based on the brightness contribution.

[0173] Specifically, for each ray and its corresponding brightness value, the angle of the ray is calculated based on the direction of the ray and the direction of each surface on the illumination probe. When the angle of the ray is less than 90 degrees, the brightness contribution of the ray after multiple bounces on the current projection basis function is calculated. For each ray, the brightness contribution corresponding to each ray is calculated, and then the brightness contribution is fitted to the projection basis function of the corresponding surface, and then the illumination transfer parameters on the projection basis function of the corresponding surface are calculated based on the brightness contribution.

[0174] For example, Figure 5 As shown, a regular cube is taken as an example, such as an Ambient Cube, which is used to represent faces in multiple directions on the light probe. Among them, the ambient light cube includes six faces, namely the x face, -x face, y face, -y face, z face and -z face, each face corresponds to a projection basis function, and each basis function stores the corresponding lighting transfer parameters, which can be specifically a lighting transfer vector Transfer Vector. That is, the x face corresponds to the lighting transfer vector (x), the -x face corresponds to the lighting transfer vector (-x), the y face corresponds to the lighting transfer vector (y), the -y face corresponds to the lighting transfer vector (-y), the z face corresponds to the lighting transfer vector (z), and the -z face corresponds to the lighting transfer vector (-z).

[0175] After the terminal calculates the lighting transfer parameters corresponding to each surface on the lighting probe in the virtual scene, the lighting transfer parameters are stored on the lighting probe.

[0176] In this embodiment, after the original light source is projected onto multiple projection basis functions, rays are emitted from the center point of each lighting probe to the spherical surface, and then the brightness contribution of each ray after multiple reflections is sampled in the virtual scene, so that the lighting transfer parameters of multiple surfaces on the lighting probe in the virtual scene can be accurately calculated.

[0177] In one embodiment, the virtual scene is a virtual game scene, and the light source in the virtual game scene is a distant light source; when the light source in the virtual scene changes, the light source change information is determined, including: when running the virtual game scene, the light source direction of the distant light source in the virtual game scene is monitored; when the light source direction changes, the light source change information is determined according to the changed light source direction and the initial light source direction of the distant light source.

[0178] It can be understood that the virtual game scene refers to the virtual scene deployed in the game application, which can be a three-dimensional virtual scene. Among them, the distant light source can specifically refer to the light source with a longer range presented in the picture, that is, the light source far away from the object in the virtual scene, such as the sunlight or sky light in the virtual game scene. For example, the sky light in the game scene is usually calculated based on the method of simulating atmospheric scattering to calculate the scattered light in the environment, and can render the sky and scenery under different sun positions and atmospheric characteristics. Usually the final color value is calculated by the integral formula, and the integral interval is a continuous interval, so the approximate result can be obtained by calculating the cumulative sum by discrete sampling.

[0179] Specifically, a game application is deployed in the terminal, and the game application includes a virtual game scene. When the virtual game scene in the terminal is running, the terminal monitors the light source direction of the distant light source in the virtual game scene in real time.

[0180] When the terminal detects that the direction of the light source in the virtual game scene has changed, the light source change information is determined based on the changed light source direction and the initial light source direction of the distant light source. For example, when the light source rotates, the angle of rotation of the light source is determined based on the rotation direction. Then the terminal calculates the rotation matrix corresponding to the spherical harmonics according to the rotation angle, and updates the light source projection coefficient in the virtual game scene in real time according to the rotation matrix and the initial light source projection coefficient in the virtual game scene, so as to calculate the current light source projection coefficient corresponding to the target pixel to be displayed. Then, the target pixel is illuminated and rendered according to the current light source projection coefficient calculated in real time.

[0181] For example, Figure 6 As shown in FIG. 6a, a scene screen in an embodiment, it can be seen that the distant light source in the scene screen is sunlight. In the virtual scene, there is a house on the left and two walls on the right, which are red and green respectively, and they will reflect the corresponding indirect light in the virtual scene. When the distant light source changes, for example, when the sunlight in the virtual scene rotates and changes, the light received by each object in the virtual scene will also change. Figure 7 As shown, Figure 6 The scene picture 7a after the sunlight in the virtual scene rotates. Figure 7 It can be seen that when the wall receives brighter light, the corresponding reflected light on the left side of the house wall is also brighter.

[0182] In this embodiment, the light source projection coefficient in the scene is updated in real time according to the light source change information of the dynamic light source in the virtual game scene, so as to dynamically calculate the direct lighting value and the indirect lighting value in the virtual game scene. This can effectively realize real-time dynamic lighting rendering of the scene as the light source changes dynamically, thereby presenting an effect close to the real natural environment, thereby effectively improving the lighting rendering effect under the dynamic light source in the virtual scene.

[0183] In a specific embodiment, Figure 8 As shown, a specific lighting rendering method is provided, which specifically includes the following steps:

[0184] S802: When the light source in the virtual scene changes, determine light source change information.

[0185] S804, determining a lighting transformation matrix according to the light source change information; determining a current light source projection coefficient corresponding to the changed light source according to the lighting transformation matrix and the initial light source projection coefficient before the light source in the virtual scene changes.

[0186] S806 , for a target pixel belonging to a static object in the virtual scene, search for a texel matching the target pixel in the light map of the static object based on a texture mapping relationship.

[0187] S808, obtaining a lighting transfer parameter corresponding to the target pixel from the light map according to the matched texel.

[0188] S810, determining an indirect illumination value of the target pixel in the virtual scene according to a current light source projection coefficient and an illumination transfer parameter corresponding to the target pixel.

[0189] S812, for a target pixel point belonging to a dynamic object in the virtual scene, determine a lighting probe that matches the target pixel point of the dynamic object.

[0190] S814: Update the light brightness corresponding to each direction on the light probe according to the current light source projection coefficient and the light transfer parameters stored in the light probe.

[0191] S816, interpolate the updated light brightness corresponding to each light probe; determine the indirect lighting value of the target pixel in the virtual scene according to the interpolated light brightness and the normal direction of the target pixel.

[0192] S818, determining the direct illumination value corresponding to the target pixel under the changed light source.

[0193] S820: Perform illumination rendering on the target pixel point according to the direct illumination value and the indirect illumination value.

[0194] When the terminal detects that the light source in the virtual scene has changed, the light source projection coefficient corresponding to the changed light source will also change. The terminal determines the illumination transformation matrix based on the light source change information. For example, the light source rotation matrix can be determined based on the angle of the light source change. The terminal then updates the current light source projection coefficient corresponding to the current light source in the virtual scene based on the illumination transformation matrix and the initial light source projection coefficient before the light source changes.

[0195] It can be understood that the lighting in the virtual scene is a linear system. The light source in the virtual scene can be represented by a function, and the lighting in the virtual scene is calculated by projecting the light source function onto the light source projection basis function.

[0196] Specifically, the expression of the illumination linear system can be as follows:

[0197]

[0198] Among them, x, y, z are the basis functions of the light source projection in the virtual scene, and a, b, c are the light source projection coefficients projected onto the basis functions.

[0199] For example, for distant light sources, CubeMap can be used to represent them in the virtual scene, so that they can be projected onto the basis function during illumination calculation. The projection expression of the distant light source can be:

[0200] (sh0,sh1,sh2...)=(x,y,z...)=a*sh0+b*sh1+c*sh2+...

[0201] Among them, sh0, sh1 and sh2 are basis functions, corresponding to x, y and z respectively. After the light source is projected, the lighting of the virtual scene is an approximate effect, which can be used for smooth lighting such as indirect light. When the light source rotates, the basis function does not change, only the corresponding light source projection coefficients a, b, c change. Therefore, f(x), f(y), and f(z) in the above formula do not change. Specifically, it can be calculated by pre-baking, that is, the corresponding lighting transfer parameter TransferVector, and the light source projection coefficients a, b, c need to be calculated in real-time according to the corresponding light source in the virtual scene.

[0202] It can be understood that in practical applications, spherical harmonic basis functions of any order can be used to fit the light source according to application requirements. When the order is higher, the fitting of complex light sources is more accurate, but the more orders, the more computing resources are consumed. In this embodiment, only indirect lighting baking based on PRT technology can be considered. Specifically, a third-order spherical harmonic basis function, that is, 9 basis functions, or a fourth-order spherical harmonic basis function, that is, 16 basis functions, can be used to meet the fitting requirements of distant light sources.

[0203] For example, Fig. 9 As shown in FIG. 1 , a scene picture 92 in an embodiment is shown, and the scene picture 92 includes a distant light source 92a. Fig.10 As shown, the projection Fig.10 a is for Fig. 9 From the schematic diagram of the projection of the distant light source 92a, it can be seen that the brightness of the light source part and the area around the light source is relatively high.

[0204] Taking the distant light source in the virtual scene as an example, in the case of day and night changes, the distant light source rotates around the scene. Among them, the selection of the projection basis function affects the baking of the lighting transfer parameters and the projection of the light source. Since the light source projection is an approximation of the distant light source, during the rotation of the distant light source, if the projection basis function used does not have the characteristics of rotational invariance, the coefficient jitter will occur, thereby causing the scene lighting jitter. In this embodiment, the spherical harmonic function is used as the projection basis function of the light source. Since the spherical harmonic function has the characteristics of rotational invariance and high projection performance, it can more accurately calculate the lighting information in the virtual scene, thereby effectively improving the final lighting rendering effect and runtime performance.

[0205] Specifically, the terminal can project the original light source onto the projection basis function in advance according to the state information of the original light source preset in the virtual scene, and then calculate the initial projection coefficient based on the Monte Carlo method. Then, in the process of rendering the virtual scene, when the light source changes, for example, in the rotation of the light source, after calculating the illumination rotation matrix according to the rotation angle, it is only necessary to perform a matrix and vector product once according to the initial projection coefficient to update the current light source projection coefficient after the light source changes. Compared with the method of performing a large amount of sampling and re-projecting to calculate the light source projection coefficient each time, the method of this embodiment can effectively save the consumption of computing resources.

[0206] Among them, before the terminal renders the target pixel to be rendered in the virtual scene, it is necessary to pre-bake the lighting transfer parameters corresponding to each pixel of the static object in the virtual scene, as well as the lighting transfer parameters corresponding to the lighting probe. Specifically, the lighting transfer parameters can be baked based on the PRT method based on path tracing. In traditional light energy transmission calculations, the lighting contribution of the entire distant light source to the scene is calculated. In this embodiment, only the brightness information of indirect lighting in the virtual scene is considered. Specifically, only low-frequency indirect lighting is calculated in the lighting transfer parameter TransferVector. Specifically, a path integral-based method can be adopted, and the brightness representation of indirect lighting in the virtual scene can be as follows:

[0207]

[0208]

[0209] Among them, L o Indicates the brightness of light emitted from a position on an object in a certain direction, L e is the brightness of the light source or self-luminous object, P(p n ) is the brightness contribution of the light after N bounces, T(p n ) is the attenuation of light brightness after N bounces.

[0210] In the process of baking lighting transfer parameters, the light sources in the virtual scene are projected onto the spherical harmonic basis functions, and each projection basis function calculates the corresponding brightness contribution according to the above steps.

[0211] In one embodiment, in order to speed up the baking performance of the lighting transfer parameters, the brightness information of multiple projection basis functions can share the same light ray ejection path and be packaged together for calculation, thereby being able to bake the lighting transfer parameters corresponding to the indirect lighting in the virtual scene more quickly.

[0212] Among them, for static objects in the virtual scene, the lighting baking data of each static object, that is, the lighting transfer parameters, are stored in the light map corresponding to each static object. Each texel in the light map stores the corresponding brightness information, that is, each texel stores a corresponding lighting transfer parameter information.

[0213] Specifically, after the original light source in the virtual scene is projected onto the spherical harmonic basis function, each pixel on the surface of the static object in the virtual scene will receive the corresponding brightness information. Then, for each pixel on the static object, that is, corresponding to the texel in the light map, the world space position corresponding to the center of the texel is used as the starting point to emit light to the hemisphere in the direction of the normal. Then sampling is performed in the scene to calculate the brightness attenuation of the ray after multiple bounces. When the light source intersects with an object in the scene, the brightness contribution of each ray on the current basis function is calculated, and then the brightness contribution of each ray is fitted, and then the corresponding lighting transfer parameters are calculated based on the brightness contribution obtained by fitting on each basis function.

[0214] The expression for fitting the illumination transfer parameters by emitting light to the hemispherical surface can be as follows:

[0215]

[0216] Where T represents the light transfer parameter, L(w) is the brightness of the light incident in the direction w, and θ is the angle between the incident direction and the normal. When using cosine weights to project light, its pdf(w i )=cos(θ)*π, and the final fitting expression is obtained after simplification.

[0217] Among them, for the lighting probes in the virtual scene, AmbientCube can be used to represent the faces in multiple directions on the lighting probes. Each face corresponds to a projection basis function, and a lighting transfer parameter is stored on the basis function corresponding to each face. The baking data of dynamic objects in the virtual scene is stored in the lighting probe, and each face in the lighting probe stores the incident light information in various directions.

[0218] Specifically, the baking of the light probe takes the position of the light probe as the starting point, and evenly emits rays to the entire sphere. For each ray, the calculation method is the same as that of the light map, accumulating the brightness contribution corresponding to the ray on each basis function, and then fitting it to the corresponding basis function. For a ray and its corresponding brightness value, the angle is calculated based on the direction of the ray and the direction of each surface. When the angle is less than 90 degrees, it is accumulated into the lighting transfer parameters of the corresponding surface.

[0219] When the terminal renders a virtual scene, for example, when running an application that includes a virtual scene, the terminal needs to update the light source projection coefficient corresponding to the changed light source according to the real-time light source information. Taking the light source as a distant light source, such as sunlight, first, it is necessary to update the coefficient of the distant light source projected onto the basis function according to the skylight and sunlight of the current scene, and then for static objects in the virtual scene, the stored pre-baked lighting transfer parameters of the corresponding pixel points are obtained from the corresponding light map, and then the final brightness contribution, that is, the indirect lighting value of the static object, can be directly calculated based on the current light source projection coefficient and the obtained lighting transfer parameters, and then mapped to the surface of the static object according to the texture mapping relationship.

[0220] The projection of distant light sources to basis functions can be sampled and calculated using the Monte Carlo method. Specifically, starting from the center of the world coordinate system in the virtual scene, rays are uniformly emitted in all directions to solve the coefficients of the projection basis function. The expression for solving the light source projection coefficient of the projection basis function can be as follows:

[0221]

[0222] Where a represents the spherical harmonic coefficient that needs to be projected onto a certain basis function, i.e., the light source projection coefficient. s represents the entire sphere, w represents the solid angle, f(w) represents the brightness value in the direction of the solid angle, and a(w) is the corresponding spherical harmonic basis function. i The parameter representing the solid angle of the current sampling, and the pdf is the corresponding weight. When uniform spherical sampling is used, the pdf is 1 / 4π.

[0223] For spherical harmonic basis functions, we can first project the basis functions based on the default direction, and then calculate the illumination transformation matrix when the light source changes. For example, when the light source rotates, we calculate the rotation matrix corresponding to the spherical harmonics, and update the rotated matrix based on the default initial projection coefficients and the rotation matrix, so that the light source projection coefficients can be updated more quickly. The rotation matrix of spherical harmonics can be as follows:

[0224]

[0225] For dynamic objects in the virtual scene, multiple lighting probes around the dynamic objects are obtained, and the lighting transfer parameters corresponding to multiple faces in the lighting probes are obtained, which can be specifically 6 faces. Then, based on the calculated current light source projection coefficient and the lighting transfer parameters corresponding to each face in the lighting probe, the light brightness of each face is updated, that is, the indirect lighting value corresponding to each face in each lighting probe is calculated. Then, the light brightness of the multiple surrounding lighting probes is linearly interpolated, and then based on the interpolated light brightness and the normal direction of the target pixel on the dynamic object, the final brightness of the target pixel in this direction is calculated, thereby obtaining the indirect lighting value corresponding to the target pixel.

[0226] The specific calculation code can be shown as follows:

[0227] float3 AmbientLinghting(const float3 sortNormal){

[0228] float3 nSquared=worldNormal*worldNormal;

[0229] int3 isNegative=(worldNormal<0.0);

[0230] Return nSquared.x*cAmbientCube[isNegative.x]+nSquared.y*cAmbientCube[isNegative.y+2]+nSquared.z*cAmbientCube[isNegative.z+4];}

[0231] Among them, the brightness information of the six surfaces on the light probe can be stored in the order of (PositiveX, NegativeX, PositiveY, NegativeY, PositiveZ, NegativeZ).

[0232] In another embodiment, the indirect illumination value of the dynamic object can also be calculated according to the illumination probe by using the spherical harmonic method. Similar to the method of using the ambient cube, the projection coefficient on each basis function in the illumination probe is first calculated in real time according to the pre-calculated illumination transfer parameters and the current light source projection coefficient stored in the illumination probe, and the spherical harmonic representation of the brightness of the illumination probe is obtained. Then, according to the normal direction of the target pixel on the dynamic object, the diffuse spherical harmonic conversion parameter corresponding to the target pixel is calculated, and finally, the dot product of the diffuse spherical harmonic conversion parameter and the spherical harmonic coefficient in the spherical harmonic representation corresponding to the illumination probe is performed, so that the final indirect illumination value corresponding to the target pixel of the dynamic object can be accurately calculated.

[0233] In one embodiment, by splitting direct illumination and indirect illumination, a PRT-based method is used to dynamically calculate the direct illumination value and the indirect illumination value corresponding to the target pixel in the virtual scene. Fig.11 As shown, Fig.11 In Fig.11 a is a lighting rendering effect diagram in this embodiment, in which direct lighting and indirect lighting are split and the direct lighting and indirect lighting are respectively calculated based on the PRT method. Fig.11 In Fig.11 b is a lighting rendering effect diagram that uses a PRT-based method to mix direct lighting and indirect lighting. Fig.11 In Fig.11 c is the lighting rendering effect diagram using only the PRT-based method to calculate indirect lighting. It can be seen that by splitting direct lighting and indirect lighting, the overall indirect lighting in the virtual scene can be dynamically updated, and the high-frequency direct lighting and shadow information are more accurate, thus making the overall lighting effect in the virtual scene better.

[0234] The present application also provides an application scenario, which is an open world type three-dimensional game scene, and the game scene applies the above-mentioned lighting rendering method. Specifically, the open world type three-dimensional game scene includes a virtual scene, and the virtual scene includes static objects and dynamic objects and lighting probes, and also includes distant light sources that simulate the real world, such as sunlight and skylight. Among them, the lighting transfer parameters corresponding to each static object and lighting probe in the virtual scene can be pre-baked and stored in the corresponding light map and lighting probe respectively.

[0235] Among them, a game application corresponding to the three-dimensional game scene is deployed in the terminal. When the game application in the terminal is running, the light source state of the distant light source in the three-dimensional game scene is detected in real time. When the distant light source in the virtual scene changes, the light source change information is determined. Then, the current light source projection coefficient corresponding to the changed distant light source is determined according to the light source change information.

[0236] Then, for the target pixels to be displayed in the virtual scene, for the target pixels belonging to static objects in the virtual scene, the pre-calculated matching lighting transfer parameters are obtained from the corresponding light map, and the indirect lighting value of the target pixel in the virtual scene is calculated based on the current light source projection coefficient and the lighting transfer parameters. For the target pixels belonging to dynamic objects in the virtual scene, the pre-calculated matching lighting transfer parameters are obtained from the corresponding lighting probe, and the indirect lighting value of the target pixel in the virtual scene is calculated based on the current light source projection coefficient and the lighting transfer parameters.

[0237] The terminal also calculates the direct illumination value corresponding to the target pixel under the changed light source based on the light source change information, thereby being able to calculate the dynamic direct illumination value and indirect illumination value in the virtual scene in real time. The terminal then performs illumination rendering on the target pixel based on the direct illumination value and the indirect illumination value. This enables accurate and efficient illumination rendering of the distant light source that changes in real time in the game scene, thereby effectively improving the illumination rendering effect under the dynamic distant light source in the virtual scene.

[0238] The present application also provides an application scenario, which is a three-dimensional environment demonstration scenario, such as a tourism environment demonstration scenario, a building demonstration scenario, and a virtual training scenario, etc. The three-dimensional environment demonstration scenario applies the above-mentioned illumination rendering method under the three-dimensional scene. The three-dimensional environment demonstration scene for various environments includes a pre-set virtual scene, and the virtual scene includes static objects, dynamic objects, and illumination probes.

[0239] Before displaying the three-dimensional environment demonstration scene, the lighting transfer parameters corresponding to each static object and lighting probe in the virtual scene are pre-baked and stored in the corresponding light map and light probe respectively.

[0240] The terminal can be pre-loaded with scene data corresponding to the 3D environment demonstration scene, or it can obtain the required scene data from the server in real time. When the terminal is demonstrating the 3D environment demonstration scene, the light source status in the virtual scene is detected in real time. When the light source in the virtual scene changes, the current light source projection coefficient corresponding to the target pixel point to be displayed in the virtual scene is updated in real time according to the light source change information.

[0241] Then, for the target pixel points that belong to static objects in the virtual scene, the pre-calculated matching lighting transfer parameters are obtained from the corresponding light map, and the indirect lighting value of the target pixel point in the virtual scene is calculated based on the current light source projection coefficient and the lighting transfer parameters. For the target pixel points that belong to dynamic objects in the virtual scene, the pre-calculated matching lighting transfer parameters are obtained from the corresponding lighting probe, and the indirect lighting value of the target pixel point in the virtual scene is calculated based on the current light source projection coefficient and the lighting transfer parameters.

[0242] At the same time, the terminal also calculates the direct illumination value corresponding to the target pixel under the changed light source based on the light source change information, thereby being able to calculate the dynamic direct illumination value and indirect illumination value in the virtual scene in real time. The terminal then performs illumination rendering on the target pixel based on the direct illumination value and the indirect illumination value. This not only effectively saves the computing resources of indirect illumination, but also accurately performs illumination rendering processing on the light source that changes in real time in the virtual scene, effectively improving the illumination rendering effect under the dynamic light source in the virtual scene.

[0243] It should be understood that although Figure 2 and Figure 8 The steps in the flowchart are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Moreover, Figure 2 and Figure 8 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0244] In one embodiment, Fig.12 As shown, a lighting rendering device 1200 is provided, which can be a part of a computer device using a software module or a hardware module, or a combination of the two. The device specifically includes: a light source determination module 1202, a projection coefficient update module 1204, an indirect lighting determination module 1206, a direct lighting determination module 1208 and a lighting rendering module 1210, wherein:

[0245] The light source determination module 1202 is used to determine light source change information when the light source in the virtual scene changes.

[0246] The projection coefficient updating module 1204 is used to determine the current light source projection coefficient corresponding to the changed light source according to the light source change information.

[0247] The indirect illumination determination module 1206 is used to determine the indirect illumination value of the target pixel point in the virtual scene according to the illumination transfer parameter corresponding to the target pixel point in the virtual scene and the current light source projection coefficient.

[0248] The direct illumination determination module 1208 is used to determine the direct illumination value corresponding to the target pixel under the changed light source.

[0249] The lighting rendering module 1210 is used to perform lighting rendering on the target pixel point according to the direct lighting value and the indirect lighting value.

[0250] In one embodiment, the projection coefficient update module 1204 is also used to determine the lighting transformation matrix based on the light source change information; determine the initial light source projection coefficient before the light source in the virtual scene changes; and determine the current light source projection coefficient corresponding to the changed light source based on the lighting transformation matrix and the initial light source projection coefficient.

[0251] In one embodiment, the light source change information is the change information generated by the changed light source compared to the original light source preset in the virtual scene; the projection coefficient update module 1204 is also used to obtain the initial light source projection coefficient obtained by projecting the original light source onto the projection basis function; according to the illumination transformation matrix, the initial light source projection coefficient is updated to obtain the current light source projection coefficient corresponding to the changed light source.

[0252] In one embodiment, the light source change information is the change information generated by the changed light source compared to the historical light source before the change in the virtual scene; the projection coefficient update module 1204 is also used to obtain the historical light source projection coefficient corresponding to the historical light source; according to the illumination transformation matrix, the historical light source projection coefficient is updated to obtain the current light source projection coefficient corresponding to the changed light source.

[0253] In one embodiment, the indirect lighting determination module 1206 is also used to determine the indirect lighting value of the target pixel point in the virtual scene that belongs to a static object according to the current light source projection coefficient and the lighting transfer parameters stored in the matching light map of the target pixel point.

[0254] In one embodiment, the indirect lighting determination module 1206 is also used to search for texels matching the target pixel points belonging to static objects in the virtual scene in the light map of the static object based on the texture mapping relationship; obtain the lighting transfer parameters corresponding to the target pixel points from the light map according to the matching texels; and determine the indirect lighting value of the target pixel points in the virtual scene according to the current light source projection coefficient and the lighting transfer parameters corresponding to the target pixel points.

[0255] In one embodiment, the indirect lighting determination module 1206 is also used to determine the indirect lighting value of the target pixel point belonging to the dynamic object in the virtual scene according to the current light source projection coefficient and the lighting transfer parameters stored in the matching lighting probe.

[0256] In one embodiment, the indirect lighting determination module 1206 is also used to determine the lighting probe that matches the target pixel point of the dynamic object in the virtual scene; update the light brightness corresponding to each direction on the lighting probe according to the current light source projection coefficient and the lighting transfer parameters stored in the lighting probe; interpolate the updated light brightness corresponding to each lighting probe; and determine the indirect lighting value of the target pixel point in the virtual scene according to the interpolated light brightness and the normal direction of the target pixel point.

[0257] In one embodiment, the above-mentioned lighting rendering device also includes a lighting transfer parameter baking module, which is used to project the original light source to multiple projection basis functions, and use each projection basis function after projection as a corresponding virtual light source; for each pixel point of a static object in the virtual scene, the lighting transfer parameters corresponding to each virtual light source at the pixel point are determined based on ray tracing, and stored in the corresponding light map; for the lighting probe in the virtual scene, the lighting transfer parameters corresponding to each virtual light source in the lighting probe are determined based on ray tracing, and stored in the lighting probe.

[0258] In one embodiment, the lighting transfer parameter baking module is also used to obtain the light brightness corresponding to each pixel point of the static object in the virtual scene after the projection of each virtual light source; starting from the pixel point, emitting rays with light brightness to the hemisphere pointed by the normal of the pixel point; in the virtual scene, sampling the reflection brightness and light attenuation of each ray after reflection from the hemisphere; based on the reflection light brightness and light source brightness corresponding to each ray, determining the lighting transfer parameters corresponding to the pixel point, and storing the lighting transfer parameters in the corresponding light map.

[0259] In one embodiment, the lighting transfer parameter baking module is also used to obtain the light brightness corresponding to each surface of the lighting probe after each virtual light source is projected on the lighting probe in the virtual scene; based on the light brightness, with the center point of the lighting probe as the starting point, rays are emitted to a sphere with a preset radius centered on the center point; in the virtual scene, the reflection brightness and light attenuation of each ray after reflection from the spherical surface are sampled; based on the reflection brightness and light attenuation corresponding to each ray, the lighting transfer parameters corresponding to each lighting probe are determined, and the lighting transfer parameters are stored in the lighting probe.

[0260] In one embodiment, the virtual scene is a virtual game scene, and the light source in the virtual game scene is a distant light source; the light source determination module 1202 is also used to monitor the light source direction of the distant light source in the virtual game scene when running the virtual game scene; when the light source direction changes, the light source change information is determined according to the changed light source direction and the initial light source direction of the distant light source.

[0261] For the specific definition of the lighting rendering device, please refer to the definition of the lighting rendering method above, which will not be repeated here. Each module in the above lighting rendering device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0262] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a lighting rendering method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0263] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0264] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0265] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0266] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0267] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0268] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0269] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A lighting rendering method, characterized in that: The method comprises: When the light source in the virtual scene changes, determining light source change information; the light source change information includes at least one of light source position change information, light source intensity change information, and light source direction change information; Determine an illumination transformation matrix according to the light source change information; determine an initial light source projection coefficient before the light source in the virtual scene changes; determine a current light source projection coefficient corresponding to the changed light source according to the illumination transformation matrix and the initial light source projection coefficient; the illumination transformation matrix is ​​a transformation matrix used to represent light changes; Determining an indirect illumination value of the target pixel in the virtual scene according to an illumination transfer parameter corresponding to the target pixel in the virtual scene and a current light source projection coefficient; Determine the direct illumination value corresponding to the target pixel under the changed light source; Perform illumination rendering on the target pixel point according to the direct illumination value and the indirect illumination value.

2. The method according to claim 1, characterized in that The light source change information is the change information of the changed light source compared with the original light source preset in the virtual scene; The determining of the initial light source projection coefficient before the light source in the virtual scene changes includes: Obtaining initial light source projection coefficients obtained by projecting the original light source onto a projection basis function; The step of determining the current light source projection coefficient corresponding to the changed light source according to the illumination transformation matrix and the initial light source projection coefficient comprises: The initial light source projection coefficient is updated according to the illumination transformation matrix to obtain the current light source projection coefficient corresponding to the changed light source.

3. The method according to claim 1, characterized in that The light source change information is the change information of the light source after the change compared with the historical light source before the change in the virtual scene; The determining of the initial light source projection coefficient before the light source in the virtual scene changes includes: Obtaining a historical light source projection coefficient corresponding to the historical light source; The step of determining the current light source projection coefficient corresponding to the changed light source according to the illumination transformation matrix and the initial light source projection coefficient comprises: The historical light source projection coefficient is updated according to the illumination transformation matrix to obtain the current light source projection coefficient corresponding to the changed light source.

4. The method according to claim 1, characterized in that The step of determining the indirect illumination value of the target pixel in the virtual scene according to the illumination transfer parameter corresponding to the target pixel in the virtual scene and the current light source projection coefficient comprises: For a target pixel point belonging to a static object in the virtual scene, an indirect lighting value of the target pixel point in the virtual scene is determined according to the current light source projection coefficient and the lighting transfer parameter of the target pixel point stored in a matching light map.

5. The method according to claim 4, characterized in that The step of determining, for a target pixel point belonging to a static object in the virtual scene, an indirect illumination value of the target pixel point in the virtual scene according to the current light source projection coefficient and an illumination transfer parameter of the target pixel point stored in a matching illumination map, comprises: For a target pixel point belonging to a static object in the virtual scene, searching for a texel matching the target pixel point in a light map of the static object based on a texture mapping relationship; According to the matched texels, obtaining a lighting transfer parameter corresponding to the target pixel from the light map; According to the current light source projection coefficient and the lighting transfer parameter corresponding to the target pixel, an indirect lighting value of the target pixel in the virtual scene is determined.

6. The method according to claim 1, characterized in that The step of determining the indirect illumination value of the target pixel in the virtual scene according to the illumination transfer parameter corresponding to the target pixel in the virtual scene and the current light source projection coefficient comprises: For a target pixel point belonging to a dynamic object in the virtual scene, an indirect illumination value of the target pixel point in the virtual scene is determined according to the current light source projection coefficient and the illumination transfer parameters stored in the matching illumination probe.

7. The method according to claim 6, characterized in that The step of determining, for a target pixel point belonging to a dynamic object in the virtual scene, an indirect illumination value of the target pixel point in the virtual scene according to the current light source projection coefficient and an illumination transfer parameter stored in a matching illumination probe, comprises: For a target pixel point belonging to a dynamic object in the virtual scene, determining a lighting probe that matches the target pixel point of the dynamic object; According to the current light source projection coefficient and the lighting transfer parameters stored in the lighting probe, update the light brightness corresponding to each direction on the lighting probe; Interpolating the updated light brightness corresponding to each of the light probes; According to the interpolated light brightness and the normal direction of the target pixel, the indirect lighting value of the target pixel in the virtual scene is determined.

8. The method according to claim 1, characterized in that When the light source in the virtual scene changes, before determining the light source change information, the method further includes: Projecting the original light source to a plurality of projection basis functions, and using each projection basis function after projection as a corresponding virtual light source; For pixel points of static objects in the virtual scene, determining illumination transfer parameters corresponding to each of the virtual light sources at the pixel points based on ray tracing, and storing the parameters in the corresponding lightmap; For the lighting probe in the virtual scene, the lighting transfer parameters corresponding to each of the virtual light sources in the lighting probe are determined based on ray tracing and stored in the lighting probe.

9. The method according to claim 8, characterized in that The step of determining, for pixel points of static objects in the virtual scene, illumination transfer parameters corresponding to the pixel points of each virtual light source based on ray tracing, and storing the parameters in the corresponding illumination map comprises: For each pixel point of a static object in the virtual scene, obtaining the light brightness corresponding to the pixel point after projection of each virtual light source; Taking the pixel point as a starting point, emitting a ray having the brightness of the light toward the hemispherical surface pointed by the normal line of the pixel point; In the virtual scene, sampling the reflection brightness and brightness attenuation of each ray after being reflected by the hemispherical surface; Based on the reflection brightness and the light attenuation corresponding to each of the rays, the lighting transfer parameters corresponding to the pixel point are determined, and the lighting transfer parameters are stored in the corresponding light map.

10. The method according to claim 8, characterized in that The step of determining, for the illumination probe in the virtual scene, illumination transfer parameters corresponding to each of the virtual light sources in the illumination probe based on ray tracing, and storing the parameters in the illumination probe includes: For the illumination probe in the virtual scene, obtaining the light brightness corresponding to each surface of the illumination probe after projection of each virtual light source; Based on the brightness of the light, starting from the center point of the illumination probe, emitting rays to a sphere with a preset radius centered on the center point; In the virtual scene, sampling the reflection brightness and brightness attenuation of each ray after being reflected by the spherical surface; Based on the reflection brightness and the light attenuation corresponding to each of the rays, the lighting transfer parameters corresponding to each of the lighting probes are determined, and the lighting transfer parameters are stored in the lighting probes.

11. The method according to any one of claims 1 to 10, characterized in that: The virtual scene is a virtual game scene, and the light source in the virtual game scene is a distant light source; When the light source in the virtual scene changes, determining the light source change information includes: When running a virtual game scene, monitoring the light source direction of a distant light source in the virtual game scene; When the light source direction changes, light source change information is determined according to the changed light source direction and the initial light source direction of the distant view light source.

12. A lighting rendering device, characterized in that: The device comprises: A light source determination module, used to determine light source change information when a light source in a virtual scene changes; the light source change information includes at least one of light source position change information, light source intensity change information, and light source direction change information; A projection coefficient updating module is used to determine an illumination transformation matrix according to the light source change information; determine an initial light source projection coefficient before the light source in the virtual scene changes; determine a current light source projection coefficient corresponding to the changed light source according to the illumination transformation matrix and the initial light source projection coefficient; the illumination transformation matrix is ​​a transformation matrix used to represent light changes; An indirect illumination determination module, used to determine the indirect illumination value of the target pixel in the virtual scene according to the illumination transfer parameter corresponding to the target pixel in the virtual scene and the current light source projection coefficient; A direct illumination determination module, used to determine the direct illumination value corresponding to the target pixel under the changed light source; A lighting rendering module is used to perform lighting rendering on the target pixel point according to the direct lighting value and the indirect lighting value.

13. The device according to claim 12, characterized in that The light source change information is the change information of the changed light source compared with the original light source preset in the virtual scene; The projection coefficient updating module is also used for: Obtaining initial light source projection coefficients obtained by projecting the original light source onto a projection basis function; The initial light source projection coefficient is updated according to the illumination transformation matrix to obtain the current light source projection coefficient corresponding to the changed light source.

14. The device according to claim 12, characterized in that The light source change information is the change information of the light source after the change compared with the historical light source before the change in the virtual scene; The projection coefficient updating module is also used for: Obtaining a historical light source projection coefficient corresponding to the historical light source; The historical light source projection coefficient is updated according to the illumination transformation matrix to obtain the current light source projection coefficient corresponding to the changed light source.

15. The device according to claim 12, characterized in that The indirect illumination determination module is further used for: For a target pixel point belonging to a static object in the virtual scene, an indirect lighting value of the target pixel point in the virtual scene is determined according to the current light source projection coefficient and the lighting transfer parameter of the target pixel point stored in a matching light map.

16. The device according to claim 15, characterized in that The indirect illumination determination module is further used for: For a target pixel point belonging to a static object in the virtual scene, searching for a texel matching the target pixel point in a light map of the static object based on a texture mapping relationship; According to the matched texels, obtaining a lighting transfer parameter corresponding to the target pixel from the light map; According to the current light source projection coefficient and the lighting transfer parameter corresponding to the target pixel, an indirect lighting value of the target pixel in the virtual scene is determined.

17. The device according to claim 12, characterized in that The indirect illumination determination module is further used for: For a target pixel point belonging to a dynamic object in the virtual scene, an indirect illumination value of the target pixel point in the virtual scene is determined according to the current light source projection coefficient and the illumination transfer parameters stored in the matching illumination probe.

18. The device according to claim 17, characterized in that The indirect illumination determination module is further used for: For a target pixel point belonging to a dynamic object in the virtual scene, determining a lighting probe that matches the target pixel point of the dynamic object; According to the current light source projection coefficient and the lighting transfer parameters stored in the lighting probe, update the light brightness corresponding to each direction on the lighting probe; Interpolating the updated light brightness corresponding to each of the light probes; According to the interpolated light brightness and the normal direction of the target pixel, the indirect lighting value of the target pixel in the virtual scene is determined.

19. The device according to claim 12, characterized in that The device also includes a lighting transfer parameter baking module, which is used to: Projecting the original light source to a plurality of projection basis functions, and using each projection basis function after projection as a corresponding virtual light source; For pixel points of static objects in the virtual scene, determining illumination transfer parameters corresponding to each of the virtual light sources at the pixel points based on ray tracing, and storing the parameters in the corresponding lightmap; For the lighting probe in the virtual scene, the lighting transfer parameters corresponding to each of the virtual light sources in the lighting probe are determined based on ray tracing and stored in the lighting probe.

20. The device according to claim 19, characterized in that The lighting transfer parameter baking module is also used to: For each pixel point of a static object in the virtual scene, obtaining the light brightness corresponding to the pixel point after projection of each virtual light source; Taking the pixel point as a starting point, emitting a ray having the brightness of the light toward the hemispherical surface pointed by the normal line of the pixel point; In the virtual scene, sampling the reflection brightness and brightness attenuation of each ray after being reflected by the hemispherical surface; Based on the reflection brightness and the light source brightness of the light attenuation corresponding to each of the rays, the lighting transfer parameters corresponding to the pixel point are determined, and the lighting transfer parameters are stored in the corresponding light map.

21. The device according to claim 19, characterized in that The lighting transfer parameter baking module is also used to: For the illumination probe in the virtual scene, obtaining the light brightness corresponding to each surface of the illumination probe after projection of each virtual light source; Based on the brightness of the light, starting from the center point of the illumination probe, emitting rays to a sphere with a preset radius centered on the center point; In the virtual scene, sampling the reflection brightness and brightness attenuation of each ray after being reflected by the spherical surface; Based on the reflection brightness and the light attenuation corresponding to each of the rays, the lighting transfer parameters corresponding to each of the lighting probes are determined, and the lighting transfer parameters are stored in the lighting probes.

22. The device according to any one of claims 12 to 21, characterized in that The virtual scene is a virtual game scene, and the light source in the virtual game scene is a distant light source; The light source determination module is also used for: When running a virtual game scene, monitoring the light source direction of a distant light source in the virtual game scene; When the light source direction changes, light source change information is determined according to the changed light source direction and the initial light source direction of the distant view light source.

23. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

24. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

25. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images