Lighting rendering method and apparatus, and terminal and storage medium

Abstract

A lighting rendering method and apparatus, and a terminal and a storage medium, which belong to the technical field of computers. The method comprises: acquiring light data of a plurality of lighting probes in a scene (201); on the basis of the light data, determining a first ray and a second ray among a plurality of rays, and first indirect lighting data, wherein the first ray is a ray that hits pixels in the current screen space of the scene, the second ray is a ray that does not hit pixels in the current screen space, and the first indirect lighting data is indirect lighting data of the hit pixels (202); acquiring elements that are hit by the second ray inside the current screen space and outside the current screen space, and acquiring second indirect lighting data from a lighting cache library of the elements, wherein the second indirect lighting data is indirect lighting data of the elements (203); and performing lighting rendering on the scene on the basis of the first indirect lighting data and the second indirect lighting data (204). The method improves the accuracy of lighting rendering.

Description

Lighting rendering method, device, terminal and storage medium

[0001] This application claims priority to the Chinese patent application filed on August 3, 2023, with application number 202310972605.6 and invention name “Lighting rendering method, device, terminal and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a lighting rendering method, device, terminal, and storage medium. Background Art

[0003] With the development of computer technology, global illumination technology has been widely used in fields such as gaming, film, and virtual reality to achieve lighting and shadow effects in related scenes. Global illumination technology is an advanced rendering technique that considers both direct lighting from light sources and indirect lighting reflected from other objects, thereby significantly improving the realism of the scene.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a lighting rendering method, device, terminal, and storage medium that can improve the accuracy of lighting rendering. The technical solution is as follows:

[0006] In one aspect, a lighting rendering method is provided, the method comprising:

[0007] Obtain light data of multiple light probes in the scene, wherein the light data includes starting points and directions of multiple rays on the light probes;

[0008] determining, based on the light data, a first ray, a second ray, and first indirect lighting data among the multiple rays, wherein the first ray is a ray that hits a pixel in a current screen space of the scene, the second ray is a ray that does not hit a pixel in the current screen space, and the first indirect lighting data is indirect lighting data of the hit pixel;

[0009] Obtaining elements hit by the second ray within the current screen space and outside the current screen space, and obtaining second indirect lighting data from a lighting cache of the elements, where the second indirect lighting data is the indirect lighting data of the elements;

[0010] Perform lighting rendering on the scene based on the first indirect lighting data and the second indirect lighting data.

[0011] In another aspect, a lighting rendering device is provided, comprising:

[0012] An acquisition module, configured to acquire light data of a plurality of light probes in a scene, wherein the light data includes starting points and directions of a plurality of rays on the light probes;

[0013] a detection module, configured to determine, based on the light data, a first ray, a second ray, and first indirect lighting data among the multiple rays, wherein the first ray is a ray that hits a pixel in a current screen space of the scene, the second ray is a ray that does not hit a pixel in the current screen space, and the first indirect lighting data is indirect lighting data of the hit pixel;

[0014] The acquisition module is further configured to acquire elements hit by the second ray within the current screen space and outside the current screen space, and acquire second indirect lighting data of the elements from a lighting cache, where the second indirect lighting data is the indirect lighting data of the elements;

[0015] A rendering module is configured to perform lighting rendering on the scene based on the first indirect lighting data and the second indirect lighting data.

[0016] On the other hand, a terminal is provided, comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded and executed by the processor to implement the lighting rendering method in the embodiment of the present application.

[0017] On the other hand, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the lighting rendering method in the embodiment of the present application.

[0018] On the other hand, a computer program product is provided, which includes a computer program code, the computer program code being stored in a computer-readable storage medium, the processor of a terminal reading the computer program code from the computer-readable storage medium, and the processor executing the computer program code, so that the terminal performs the lighting rendering method described in any of the above implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] FIG1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0021] FIG2 is a flow chart of a lighting rendering method provided in an embodiment of the present application;

[0022] FIG3 is a flow chart of another lighting rendering method provided by an embodiment of the present application;

[0023] FIG4 is a flowchart of a method for obtaining a pixel cache library provided in an embodiment of the present application;

[0024] FIG5 is a schematic diagram of light bounce in a scene provided by an embodiment of the present application;

[0025] FIG6 is a schematic diagram of a starting point of a ray provided in an embodiment of the present application;

[0026] FIG7 is a schematic diagram of an iterative process provided in an embodiment of the present application;

[0027] FIG8 is a flowchart of a method for obtaining a primitive cache library provided in an embodiment of the present application;

[0028] FIG9 is a schematic diagram of a graphic element provided in an embodiment of the present application;

[0029] FIG10 is a schematic diagram of the relationship between a picture element and a pixel provided in an embodiment of the present application;

[0030] FIG11 is a heat map of a graphic element distribution provided by an embodiment of the present application;

[0031] FIG12 is a schematic diagram of discrete graphic elements in a scenario provided by an embodiment of the present application;

[0032] FIG13 is a flow chart of a method for obtaining a voxel cache library provided in an embodiment of the present application;

[0033] FIG14 is a schematic diagram of a three-dimensional texture map provided in an embodiment of the present application;

[0034] FIG15 is a schematic diagram of a multi-sized voxel provided in an embodiment of the present application;

[0035] FIG16 is a schematic diagram of the dimensions of a cone and voxels provided in an embodiment of the present application;

[0036] FIG17 is a schematic diagram of a global illumination framework provided by an embodiment of the present application;

[0037] FIG18 is a schematic diagram of another framework of global illumination provided by an embodiment of the present application;

[0038] FIG19 is a schematic diagram of a lighting setting interface provided in an embodiment of the present application;

[0039] FIG20 is a schematic diagram of a lighting effect in a scene provided by an embodiment of the present application;

[0040] FIG21 is a schematic diagram showing a comparison of lighting effects provided by an embodiment of the present application;

[0041] FIG22 is a schematic diagram of adjusting a lighting effect provided by an embodiment of the present application;

[0042] FIG23 is a schematic diagram of a scene after rendering and illumination provided by an embodiment of the present application;

[0043] FIG24 is a block diagram of a lighting rendering device provided in an embodiment of the present application;

[0044] Figure 25 is a block diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0046] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0047] In the present application, the term "at least one" means one or more, and the term "plurality" means two or more.

[0048] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the light data and indirect lighting data involved in this application were obtained with full authorization.

[0049] The following is an introduction to the professional terms involved in this application:

[0050] Virtual scene: It is a virtual scene displayed (or provided) when the application is running on the terminal. The virtual scene can be a simulation environment of the real world, a semi-simulation and semi-fictitious virtual environment, or a purely fictitious virtual environment. The virtual scene can be any of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiment of the present application does not limit the dimension of the virtual scene. For example, the virtual scene may include the sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. The terminal user can control the movement of virtual objects in the virtual scene.

[0051] The following is an introduction to the implementation environment involved in this application:

[0052] The lighting rendering method provided in the embodiment of the present application is executed by the terminal. The implementation environment of the lighting rendering method provided in the embodiment of the present application is introduced below. Referring to Figure 1, Figure 1 is a schematic diagram of an implementation environment of a lighting rendering method provided in the embodiment of the present application, and the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are directly or indirectly connected via wired or wireless communication, and the present application does not limit this. In some embodiments, a target application is installed on the terminal 101, and the target application can be an application for displaying a scene, and the scene can be a virtual scene. The terminal 101 is used to render lighting in the scene. The server 102 is used to provide background services for the target application installed by the terminal 101.

[0053] In some embodiments, the terminal 101 can be a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device, smart home appliance, vehicle-mounted terminal, aircraft, VR (Virtual Reality) device, AR (Augmented Reality) device, etc., but is not limited thereto. In some embodiments, the server 102 is an independent server, or a server cluster or distributed system composed of multiple servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some embodiments, the server 102 is primarily responsible for computing tasks, and the terminal 101 is responsible for secondary computing tasks; alternatively, the server 102 is responsible for secondary computing services, and the terminal 101 is responsible for primary computing tasks; alternatively, the server 102 and the terminal 101 use a distributed computing architecture for collaborative computing.

[0054] Refer to Figure 2, which is a flowchart of a lighting rendering method provided in an embodiment of the present application. The method includes the following steps.

[0055] 201. The terminal obtains light data of multiple light probes in the scene, where the light data includes starting points and directions of multiple rays on the light probes.

[0056] In the embodiments of the present application, the scene may be a virtual scene. A light source exists in the scene, and the light source can realistically simulate the lighting effects of a real-world light source. In the scene, a simulated global illumination effect can be achieved by simulating the propagation of light in the environment.

[0057] In the embodiment of the present application, a lighting probe is a light detector placed in a scene, which can also be understood as a three-dimensional sensing point that can sense light. Accordingly, the terminal can obtain light data in the scene through the lighting probe.

[0058] 202. The terminal determines a first ray, a second ray, and first indirect lighting data among multiple rays based on the light data, where the first ray is a ray that hits a pixel in the current screen space of the scene, the second ray is a ray that does not hit a pixel in the current screen space, and the first indirect lighting data is the indirect lighting data of the hit pixel.

[0059] In an embodiment of the present application, the terminal uses the ray that hits the pixel as the first ray and the ray that does not hit the pixel as the second ray, and obtains the indirect lighting data of the pixel hit by the first ray. The terminal detects rays in the scene based on the light data to obtain the first ray and the second ray.

[0060] In the embodiments of the present application, the illumination data of a pixel includes indirect illumination data and direct illumination data. The direct illumination data of a pixel is the illumination data generated by the light source in the scene directly irradiating the pixel. The indirect illumination data of a pixel refers to the illumination data generated by the light source first irradiating other pixels and then, after one, two, or more reflections, finally reaching the pixel. In other words, the indirect illumination data is generated based on light reflections.

[0061] 203. The terminal obtains elements hit by the second ray in the current screen space and outside the current screen space, and obtains second indirect lighting data from a lighting cache of the element, where the second indirect lighting data is the indirect lighting data of the element.

[0062] In an embodiment of the present application, a scene includes one or more elements, each of which includes at least one of a primitive and a voxel. A primitive is a small plane used to describe the surface of an object, that is, the smallest unit used to describe the surface of an object, and a primitive has a certain area. A voxel is a small element used to describe the volume of an object, that is, the smallest unit used to describe the volume of an object. In addition, a scene may include one or more objects.

[0063] In an embodiment of the present application, the current screen space refers to the space corresponding to the screen used to display the scene, and the space outside the current screen space refers to the space outside the current screen space in the scene. When the scene is displayed through the current screen space, part of the surface of the scene is located within the current screen space and can be displayed through the current screen space, while another part of the surface of the scene is located outside the current screen space and cannot be displayed through the current screen space.

[0064] In the embodiment of the present application, the second ray does not hit the pixel in the current screen space, but the second ray may hit the element in the current screen space and the element outside the current screen space.

[0065] 204. The terminal performs lighting rendering on the scene based on the first indirect lighting data and the second indirect lighting data.

[0066] Lighting rendering refers to the process of generating realistic images through computer simulation of the propagation and reflection of light in a scene. The purpose of lighting rendering is to make the generated images more realistic and natural, allowing viewers to better understand and perceive the objects in the scene. Lighting rendering involves adding light sources and tracing and calculating light to simulate real-world lighting effects, thereby enhancing the realism and fidelity of the scene. Scenes are typically composed of three-dimensional models with their own materials and textures. When light is reflected in the scene, it is affected by the materials and textures of the model's surface, resulting in different reflection and refraction effects. By calculating these effects, the color and brightness of each pixel can be determined, thus generating a realistic image.

[0067] In an embodiment of the present application, after the terminal performs illumination rendering on the scene based on the first indirect illumination data and the second indirect illumination data, a scene rendered with the indirect illumination data is obtained. Optionally, the terminal performs illumination rendering on the scene in combination with the first indirect illumination data, the second indirect illumination data, and the direct illumination data to achieve global illumination rendering of the scene. Alternatively, the terminal performs illumination rendering on the scene rendered with the direct illumination data based on the first indirect illumination data and the second indirect illumination data to achieve global illumination rendering of the scene.

[0068] Global illumination techniques, which are commonly used in related technologies, typically calculate the direct illumination data for each pixel before determining the indirect illumination data for each light. They then use screen-space ray tracing to obtain the indirect illumination data for that pixel, thereby achieving global illumination rendering within the screen space. However, this technique only obtains indirect illumination data reflected within the screen, making the indirect illumination data incomplete and reducing its accuracy, leading to inaccurate lighting rendering of the scene.

[0069] An embodiment of the present application provides a lighting rendering method, which uses lighting probes to determine the rays that hit the pixels and the rays that miss the pixels, and obtains the indirect lighting data of the hit pixels. For the rays that miss the pixels, the elements hit by these rays in the current screen space and outside the current screen space are obtained, and the indirect lighting data of the hit elements are obtained. In this way, the method performs lighting rendering through the indirect lighting data of the pixels and the indirect lighting data of the elements, which can achieve the effective supplementation of the indirect lighting data of the elements to the indirect lighting data of the pixels, and the indirect lighting data of the elements also includes the indirect lighting data outside the current screen space, which further improves the comprehensiveness and accuracy of the indirect lighting data, and then performs lighting rendering on the scene based on such indirect lighting data, which can improve the accuracy of lighting rendering.

[0070] The above describes the basic process of illumination rendering based on the embodiment of Figure 2. The following further describes the process of illumination rendering based on the embodiment of Figure 3. Referring to Figure 3, Figure 3 is a flow chart of an illumination rendering method provided in an embodiment of the present application, which includes the following steps.

[0071] 301. The terminal generates multiple light probes for the scene.

[0072] In the embodiments of this application, screen-space probes are used as an example for illustration. The number and positions of multiple light probes can be set and modified as needed. Multiple light probes can be evenly distributed or adaptively distributed, without specific limitation here.

[0073] In the embodiments of this application, Light Probes are light detectors placed in the scene. Light Probes can also be understood as individual three-dimensional sensing points that can sense the light data passing through them and can be used to provide high-quality lighting data, including indirect lighting, for objects in the virtual scene. It should be noted that Light Probes can also be generated based on world space. After determining the probe lighting data at each Light Probe position, the lighting data for any point in the scene can be obtained based on the probe lighting data of at least one Light Probe closest to that point and interpolation.

[0074] 302. The terminal obtains light data of multiple light probes in the scene, where the light data includes starting points and directions of multiple rays on the light probes.

[0075] In an embodiment of the present application, a terminal generates rays for multiple lighting probes to obtain light data for the multiple lighting probes. Optionally, the terminal generates rays based on an importance sampling method to obtain light data for the multiple lighting probes. Importance sampling methods include, but are not limited to, cos-weighted (an importance sampling method), brdf (bidirectional reflectance distribution function), and history lighting methods, which are not described in detail here.

[0076] In some embodiments, the Light Probe is a sphere. Accordingly, the starting points of the multiple rays on the Light Probe are the center of the Light Probe. Alternatively, the starting points of the multiple rays are evenly distributed on the surface of the Light Probe, located at multiple locations on the surface of the Light Probe. The multiple locations can be directly above, directly below, directly to the left, directly to the right, etc. The directions of the multiple rays can be set and changed as needed and are not specifically limited here.

[0077] 303. The terminal determines a first ray, a second ray, and first indirect lighting data among multiple rays based on the light data, where the first ray is a ray that hits a pixel in the current screen space of the scene, the second ray is a ray that does not hit a pixel in the current screen space, and the first indirect lighting data is the indirect lighting data of the hit pixel.

[0078] In some embodiments, the terminal uses a screen space algorithm to detect rays in the scene to obtain a first ray and a second ray. It should be noted that the ray has a first preset distance, and the terminal uses the pixels hit by the first ray within the first preset distance as the pixels hit by the first ray. The pixels hit by the first ray within the first preset distance are pixels on the first ray that are between the vertex of the first ray and the first target position, as well as pixels on the first ray that are located at the first target position. The first target position refers to a position whose distance from the vertex of the first ray is equal to the first preset distance.

[0079] Accordingly, the terminal determines the coordinate information of the pixel hit by the first ray. The terminal obtains the indirect lighting data of the pixel based on the coordinate information of the pixel. Optionally, the terminal obtains the coordinate information of the hit pixel based on the starting point coordinate information, direction and first hit distance of the first ray; that is, the starting point coordinate information is extended in this direction, and the extended distance is equal to the first hit distance, and the coordinate information obtained after the extension is the coordinate information of the hit pixel. The first hit distance refers to the distance between the starting point of the first ray and the hit pixel. The first hit distance can be obtained in the process of detecting rays using a screen space algorithm, or obtained by other methods. The embodiments of the present application do not limit this.

[0080] In an embodiment of the present application, the first indirect lighting data is obtained from a pixel cache, which stores a plurality of pixels in a scene and indirect lighting data for the plurality of pixels. In some embodiments, the pixel cache stores coordinate information and indirect lighting data for the plurality of pixels. Accordingly, the terminal obtains the indirect lighting data for the hit pixel from the pixel cache based on the coordinate information of the hit pixel. In some embodiments, the pixel cache acquisition process is implemented by the embodiment of FIG4 below and will not be further described here.

[0081] It should be noted that the lighting received by the surface of an object in reality does not all come from the light source, but also includes lighting reflected from other objects. The lighting from the light source is direct lighting, while the lighting from other objects is indirect lighting. Global illumination effects include direct and indirect lighting effects. Direct lighting data refers to the brightness of light that is directly irradiated by the light source on the object and reflected to the virtual observation point or camera viewpoint. Indirect lighting data refers to the brightness of light that is first irradiated by the light source on other objects, and after one, two, or more bounces, it finally reaches the surface of the observed object and is then reflected to the virtual observation point or camera viewpoint.

[0082] In some embodiments, after performing ray detection, the terminal reorganizes the data of multiple rays to obtain the first ray and the second ray (missing ray), and repackages and compresses the multiple second rays together; in this way, the multiple second rays are compressed and placed in a continuous space, which facilitates subsequent detection based on the second rays.

[0083] 304. The terminal determines, based on the second ray, within and outside the current screen space, the primitive hit by the second ray, and obtains indirect lighting data of the hit primitive from a primitive cache.

[0084] In some embodiments, the terminal uses an SDF (Signed Distance Field) or Ray Query method to detect primitives in the scene, thereby determining the primitives hit by the second ray. It should be noted that the ray has a second preset distance, and the terminal regards the primitives hit by the second ray within the second preset distance as the primitives hit by the second ray. The primitives hit by the second ray within the second preset distance refer to the primitives on the second ray that are between the vertex of the second ray and the second target position, as well as the primitives on the second ray that are located at the second target position. The second target position refers to a position whose distance from the vertex of the second ray is equal to the second preset distance.

[0085] In an embodiment of the present application, the terminal determines the coordinate information of the hit primitive based on the second ray, and then obtains the indirect lighting data of the hit primitive from the primitive cache based on the coordinate information. Optionally, the terminal obtains the coordinate information of the hit primitive based on the starting point coordinate information, direction and second hit distance of the second ray. That is, the starting point coordinate information is extended in this direction, and the extended distance is equal to the second hit distance. The coordinate information obtained after the extension is the coordinate information of the hit primitive. Among them, the coordinate information includes but is not limited to the center point coordinates, radius, normal, etc. of the primitive. The second hit distance refers to the distance between the starting point of the second ray and the hit primitive. The second hit distance can be obtained in the process of detecting the primitive, or obtained in other ways. The embodiment of the present application does not limit this.

[0086] In embodiments of the present application, the primitive cache stores multiple primitives in a scene and their indirect lighting data. In some embodiments, the primitive cache stores coordinate information and indirect lighting data for the multiple primitives. In some embodiments, the primitive cache acquires the primitives using the embodiment described in FIG8 below and is not further described here.

[0087] In some embodiments, the terminal further determines the hit rate of the pixel based on the ratio of the number of first rays to the total number of rays. The total number of rays refers to the sum of the number of first rays and the number of second rays. If the hit rate is not greater than the first reference hit rate, step 304 is executed. If the hit rate is greater than the first reference hit rate, step 304 is no longer executed, that is, lighting rendering is performed based on the first indirect lighting data, and the indirect lighting data of the primitive is no longer considered. This embodiment improves the flexibility of lighting rendering and improves rendering efficiency while ensuring the data volume of indirect lighting data.

[0088] 305. The terminal determines a third ray among the multiple second rays, where the third ray is a ray that does not hit the primitive.

[0089] In an embodiment of the present application, after the terminal performs detection based on the second ray, it reorganizes the data of multiple second rays to obtain multiple third rays, and repackages and compresses the multiple third rays together. In this way, the multiple third rays are compressed and placed in a continuous space, which facilitates subsequent detection based on the third ray.

[0090] 306. The terminal determines, based on the third ray, the voxels hit by the third ray in the current screen space and outside the current screen space, and obtains indirect lighting data of the hit voxels from the voxel cache.

[0091] In some embodiments, the terminal uses SDF, Ray Query or Cone Trace (cone ray tracing) methods to detect voxels in the scene, thereby determining the voxels hit by the third ray. It should be noted that the ray has a third preset distance, and the terminal uses the voxels hit by the third ray within the third preset distance as the voxels hit by the third ray. The voxels hit by the third ray within the third preset distance refer to: the voxels on the third ray that are between the vertex of the third ray and the third target position, and the pixels on the third ray that are located at the third target position. Among them, the third target position refers to the position whose distance from the vertex of the third ray is equal to the third preset distance.

[0092] In an embodiment of the present application, the terminal determines the coordinate information of the hit voxel based on the third ray, and then obtains the indirect lighting data of the hit voxel from the voxel cache based on the coordinate information. Optionally, the terminal obtains the coordinate information of the hit voxel based on the starting point coordinate information, direction and third hit distance of the third ray. That is, the starting point coordinate information is extended in this direction, and the extended distance is equal to the third hit distance. The coordinate information obtained after the extension is the coordinate information of the hit voxel. The third hit distance refers to the distance between the starting point of the third ray and the hit voxel. The third hit distance can be obtained in the process of detecting voxels, or obtained by other methods. The embodiment of the present application does not limit this.

[0093] In embodiments of the present application, a voxel cache stores multiple voxels in a scene and indirect lighting data for each voxel. In some embodiments, the voxel cache stores coordinate information and indirect lighting data for each voxel. In some embodiments, the voxel cache is accessed using the embodiment of FIG. 13 , which is not further described here.

[0094] In some embodiments, the terminal further determines the hit rate of the primitive based on the ratio of the number of third rays to the number of second rays. If the hit rate is not greater than the second reference hit rate, step 306 is executed. If the hit rate is greater than the second reference hit rate, step 306 is not executed, and lighting rendering is performed based on the first indirect lighting data and the indirect lighting data of the primitive, without considering the indirect lighting data of the voxel. This embodiment improves the flexibility of lighting rendering and improves rendering efficiency while ensuring the amount of indirect lighting data.

[0095] In this embodiment of the present application, steps 304-306 are used to obtain the elements hit by the second ray within and outside the current screen space, and to obtain second indirect lighting data from the element's lighting cache. The second indirect lighting data includes at least one of the indirect lighting data for the primitives hit by the second ray and the indirect lighting data for the voxels hit by the second ray. In this implementation, primitives are first detected based on the second ray, and voxels are then detected for rays that do not hit primitives. This avoids repeated detection of primitives and voxels by the second ray, ensures comprehensive detection, and thus improves the comprehensiveness and accuracy of the acquired indirect lighting data.

[0096] It should be noted that the above steps 304-306 are only an optional implementation method for obtaining the elements hit by the second ray in the current screen space and outside the current screen space, and obtaining the second indirect lighting data from the lighting cache library of the elements. The terminal can also implement this process through other optional methods.

[0097] In one implementation, the process of obtaining, by the terminal, elements hit by the second ray within and outside the current screen space and obtaining second indirect lighting data from a lighting cache of the elements includes at least one of the following: determining, by the terminal, primitives hit by the second ray within and outside the current screen space, and obtaining indirect lighting data for the hit primitives from the primitive cache. The terminal also determines, by the terminal, voxels hit by the second ray within and outside the current screen space, and obtaining indirect lighting data for the hit voxels from the voxel cache.

[0098] In one possible implementation, when the elements shown in the embodiment of the present application include only primitives, the terminal detects the primitives only based on the second rays and obtains indirect lighting data for the primitives. When the elements shown in the embodiment of the present application include only voxels, the terminal detects the voxels only based on the second rays and obtains indirect lighting data for the voxels. When the elements shown in the embodiment of the present application include primitives and voxels, the primitives and voxels are respectively detected based on the second rays to obtain indirect lighting data for the primitives and voxels; this implementation improves the flexibility and efficiency of detection, thereby improving the flexibility and efficiency of obtaining indirect lighting data.

[0099] In another implementation, the terminal first detects voxels and then detects primitives. Accordingly, based on the second ray, the terminal detects voxels within and outside the current screen space, obtains the voxels hit by the second ray, and obtains indirect lighting data of the hit voxels from the voxel cache; determines a fourth ray among the multiple second rays, where the fourth ray is a ray that misses a voxel; based on the fourth ray, detects primitives within and outside the current screen space, obtains the primitives hit by the fourth ray, and obtains indirect lighting data of the hit primitives from the primitive cache. In this implementation, voxels are first detected based on the second ray, and primitives are then detected for the missed ray. This avoids repeated detection of primitives and voxels by the second ray, ensures comprehensiveness of detection, and thereby improves the comprehensiveness and accuracy of the acquired indirect lighting data.

[0100] 307. The terminal performs lighting interpolation on pixels in the current screen space based on the first indirect lighting data and the second indirect lighting data to obtain interpolated indirect lighting data, where the second indirect lighting data includes indirect lighting data of primitives and indirect lighting data of voxels.

[0101] In an embodiment of the present application, the terminal performs lighting interpolation on pixels in the current screen space to obtain diffuse indirect lighting data and glossy indirect lighting data in the current screen space.

[0102] In some embodiments, before performing illumination interpolation, the terminal also performs spatiotemporal filtering on the illumination probes. The spatiotemporal filtering includes filtering in time and space to reduce noise and accelerate convergence.

[0103] It should be noted that, if the elements shown in the embodiment of the present application only include primitives, steps 305-306 are not performed, and the second indirect illumination data includes the indirect illumination data of the primitives, but does not include the indirect illumination data of the voxels. If the elements shown in the embodiment of the present application only include voxels, step 304 is not performed, and the second indirect illumination data includes the indirect illumination data of the voxels, but does not include the indirect illumination data of the primitives.

[0104] 308. The terminal performs denoising on the third indirect lighting data, and performs lighting rendering on the scene based on the denoised third indirect lighting data, where the third indirect lighting data includes the first indirect lighting data and indirect lighting data obtained by interpolation.

[0105] In some embodiments, the terminal performs denoising processing through a denoiser.

[0106] In the embodiment of the present application, the process of performing illumination rendering of the scene based on the first indirect illumination data and the interpolated indirect illumination data is implemented through step 308. This implementation removes noise data through noise reduction processing, improves the accuracy of the indirect illumination data, and thus improves the quality of the rendered image.

[0107] It should be noted that the denoising process in step 308 is an optional implementation method. After executing step 307, the terminal can also perform lighting rendering on the scene based on the first indirect lighting data and the interpolated indirect lighting data to improve rendering efficiency.

[0108] Correspondingly, step 307 is also an optional implementation method of the process of lighting rendering the scene based on the first indirect lighting data and the second indirect lighting data. The terminal can also assign the second indirect lighting data of the element in the current screen space to the element at the corresponding position, and then render the scene based on the indirect lighting data of the pixel.

[0109] In which, the terminal implements global illumination based on indirect illumination data and direct illumination data. Accordingly, illumination rendering of the scene includes the following steps: the terminal obtains direct illumination data of the scene; determines global illumination data based on the first indirect illumination data, the second indirect illumination data and the direct illumination data; and performs illumination rendering of the scene based on the global illumination data. In which, the global illumination data comprehensively considers the direct illumination data of the scene, the indirect illumination data of the pixels hit by the first ray in the current screen space, and the indirect illumination data of the elements hit by the second ray in the current screen space and outside the current screen space. Therefore, the global illumination data is illumination data close to the real scene, has higher accuracy, and can more realistically represent the illumination effect of the real scene. Therefore, based on illumination data close to the real scene, efficient real-time global rendering of the scene is achieved. In some embodiments, the terminal superimposes the third indirect illumination data with the direct illumination data to obtain global illumination data, and performs illumination rendering of the scene based on the global illumination data.

[0110] In an embodiment of the present application, the lighting rendering process can be implemented by a rendering engine on the terminal, which can provide multiple light source scenes, multiple rendering materials, etc., thereby realizing global lighting rendering of the scene.

[0111] An embodiment of the present application provides a lighting rendering method, which first determines the rays that hit pixels and the rays that miss pixels in the current screen space through lighting probes, and obtains indirect lighting data of the hit pixels. For rays that miss pixels, the primitives or voxels and other elements hit by these rays in the current screen space and outside the current screen space are obtained, and the indirect lighting data of the hit elements are obtained. In this way, the method performs lighting rendering through the indirect lighting data of the hit pixels and the indirect lighting data of the elements, which can achieve the effective supplementation of the indirect lighting data of the elements to the indirect lighting data of the pixels, and the indirect lighting data of the elements also includes the indirect lighting data of the elements hit outside the current screen space, further improving the comprehensiveness and accuracy of the indirect lighting data, and then performing lighting rendering on the scene based on such indirect lighting data, which can improve the accuracy of lighting rendering.

[0112] 4 is a flowchart of a method for obtaining a pixel cache library provided in an embodiment of the present application. The method includes the following steps.

[0113] 401. The terminal obtains scene data of a previous frame of a current frame of a scene, and determines a screen map of the current frame based on the scene data and light bounce data corresponding to the current frame, where the screen map includes depth data of multiple pixels.

[0114] In the embodiments of the present application, a scene is a virtual scene within a video frame. As the virtual scene moves or rotates, or as the virtual camera moves or rotates, different images are formed. Light bounce data is used to indicate light bounce data on multiple pixels, including the light bounce angle and bounce distance.

[0115] The terminal renders the scene data of the previous frame based on the light bounce data corresponding to the current frame to obtain the current frame's image texture. The current frame's image texture consists of multiple pixels and includes depth data for the multiple pixels.

[0116] For example, see FIG5 , which is a schematic diagram of light bounce in a scene provided by an embodiment of the present application.

[0117] 402. The terminal performs ray tracing based on the screen map to obtain indirect lighting data of multiple pixels in the current frame, and stores the indirect lighting data in a pixel cache.

[0118] Optionally, the terminal performs ray tracing based on the HZB (Hierarchical Z-buffer) method. The generation of HZB supports not only the furthest strategy but also the closest strategy, which can be set as needed. For example, the code corresponding to the ray tracing process is:

[0119] Stackless ray walk of min-Z pyramid;

[0120] mip=0;

[0121] while(level>-1)

[0122] step through current cell;

[0123] if(above Z plane)++level;

[0124] if(below Z plane)--level.

[0125] This code indicates a minimum-depth, unstacked pyramid ray-walking method. The level is initially set to 0. When the level is greater than -1, the current cell is stepped through. If it is above the Z plane, the level is increased; if it is below the Z plane, the level is decreased. Furthermore, a schematic diagram of the starting point of the ray corresponding to the code for this ray tracing process is shown in Figure 6.

[0126] In an embodiment of the present application, an iterative method is used for ray tracing. The terminal sets multiple levels of pixel precision, and the multiple levels of pixel precision correspond to different numbers of pixels, such as multiple levels of pixel precision, and the precision corresponds to 1, 2, and 4 pixels from high to low. First, ray tracing is performed based on the first level of pixel precision to determine whether it intersects with the pixel; if not, ray tracing is performed based on the second level of pixel precision to determine whether it intersects with the pixel, and the second level of pixel precision is lower than the first level; if not, ray intersection is performed based on the next level of pixel precision, and the above process is repeated, gradually reducing the pixel precision until an intersection occurs. Since the intersection at this time is an intersection with multiple pixels included in a certain level, the pixel precision is improved based on the direction and starting point of the ray at the time of intersection to determine which specific pixel is intersected.

[0127] For example, see Figure 7, which is a schematic diagram of an iterative process provided in an embodiment of the present application. This iterative process corresponds to the code shown in Figure 6. The multiple levels of pixel precision correspond to pixel precisions corresponding to 1 pixel, 2 pixels, and 4 pixels, respectively, and are named mip0, mip1, and mip2. As can be seen from the figure, the terminal first performs ray tracing based on mip0 pixel precision, resulting in no pixel intersection. It then performs ray tracing based on mip1 pixel precision, which also results in no pixel intersection. Ray tracing then performs ray tracing based on mip2 pixel precision, resulting in an intersection. However, since the pixel precision at this point corresponds to four pixels, it is necessary to determine which of the four pixels the intersection occurs. Therefore, based on the starting point and direction of the ray corresponding to the mip2 pixel precision, ray tracing is first performed based on mip1 pixel precision, resulting in two intersecting pixels. Ray tracing is then performed again based on mip0 pixel precision, resulting in one intersecting pixel. Accordingly, the levels of pixel precision corresponding to this iterative process are: mip0-mip1-mip2-mip1-mip0-mip0.

[0128] In an embodiment of the present application, a pixel cache library is obtained through the above steps 401-402, and screen space lighting cache (Screen Space Lighting) is implemented, which facilitates the subsequent acquisition of indirect lighting data of pixels from the pixel cache library, improves the efficiency of obtaining indirect lighting data, and thus improves the efficiency of lighting rendering.

[0129] It should be noted that steps 401-402 are only an optional implementation method for obtaining the pixel cache library. The pixel cache library can also be obtained through other optional implementation methods, which will not be described in detail here.

[0130] Referring to Figure 8, Figure 8 is a flow chart of a method for obtaining a primitive cache library provided by an embodiment of the present application. The method includes the following steps.

[0131] 801. The terminal discretizes a scene into multiple primitives, performs ray tracing on the multiple primitives, obtains indirect lighting data of the multiple primitives, and stores the indirect lighting data of the multiple primitives in a first buffer of a primitive buffer library.

[0132] Among them, the primitive is essentially a storage structure of a surface cache (Surface Cache). The primitive discretizes the entire scene surface, and the surface information of the center point is approximately diffused as the information of the entire primitive. The primitive metadata include center point coordinates, radius, normal, base color (Base Color), etc. Optionally, in order to meet the needs of various feature development, parameters such as primitive identifier (Prim ID), instance identifier (Instance ID), BRDF (Bidirectional Reflectance Distribution Function) can also be added to the primitive metadata. Optionally, the amount of data content stored in the primitive metadata does not exceed the reference threshold. For example, referring to Figure 9, Figure 9 is a schematic diagram of a primitive provided in an embodiment of the present application, wherein the primitive is circular, which is a small plane used to describe the surface of an object, and the primitive metadata include the center point coordinates P, radius r, normal PN, etc. of the primitive.

[0133] The terminal performs ray tracing on the primitive based on the primitive metadata and light source information in the scene to obtain indirect lighting data of the primitive. Optionally, the terminal uses a BVH (Bounding Volume Hierarchy) method to perform ray tracing on the primitive.

[0134] Optionally, the terminal generates primitives based on the screen space. Accordingly, discretizing the scene into multiple primitives includes: dividing the scene into multiple blocks, that is, dividing each preset number of pixels in the screen space into a block, and the pixels in the multiple blocks are covered by primitives. Each block records the target pixel with the smallest primitive coverage among its multiple pixels, and updates the primitives based on the primitive coverage. Accordingly, for each block, the target pixel with the smallest primitive coverage in the block is determined, and when the primitive coverage of the target pixel does not meet the coverage threshold, the primitive is generated at the target pixel until the primitive coverage of multiple pixels in the block respectively meets the coverage threshold. That is, when the primitive coverage of the target pixel does not meet the coverage threshold, after the primitive is generated at the target pixel, for each tile, the step of "determining the target pixel with the smallest primitive coverage in the tile, and generating the primitive at the target pixel when the primitive coverage of the target pixel does not meet the coverage threshold" is repeated, so that after one or more repeated executions, it is ensured that the primitive coverage of multiple pixels in the tile respectively meets the coverage threshold.

[0135] Among them, generating a primitive at a target pixel refers to generating a new primitive at the target pixel. In the process of generating primitives, based on the screen space position of the point, the base color, normal and other information of the primitive are obtained from the first buffer (GBuffer), and the world space coordinates of the point are reconstructed in combination with the depth information, and the minimum primitive coverage of the block is updated at the same time. If the minimum primitive coverage corresponding to each block is large enough, the generation is stopped. In an embodiment of the present application, the minimum primitive coverage recorded by these blocks is traversed during each generation process, and through the iterative update of the primitives, it is ensured that every pixel in the screen space is covered and that too many primitives are not generated.

[0136] Among them, the primitive coverage refers to the number of primitives covered on a pixel. If a pixel is covered by 3 primitives, the primitive coverage is 3. For example, see Figure 10, which is a schematic diagram of the relationship between primitives and pixels provided in an embodiment of the present application. For example, see Figure 11, which is a heat map of primitive distribution provided in an embodiment of the present application. Among them, the heat represents the density of the distribution of the primitives, and different shades of color represent different heats.

[0137] 802. The terminal converts the scene into a three-dimensional grid, where the three-dimensional grid includes multiple grids.

[0138] The terminal divides the space corresponding to the scene into grids of equal size, thereby obtaining a three-dimensional grid. Each grid can be a cube or a cuboid. Each grid corresponds to a storage space for storing primitive data, such as a primitive index. Optionally, the terminal uses a spatial hashing method to store primitive data.

[0139] 803. The terminal stores the primitive index of the primitive corresponding to each grid in each grid, where the primitive index is used to obtain indirect lighting data of the primitive from the first buffer; and stores the three-dimensional grid in the second buffer of the primitive buffer library.

[0140] In some embodiments, to reduce the waste of storage resources, the terminal filters out the grids that do not store primitive indexes from the multiple grids in the second buffer and removes the grids that do not store primitive indexes. In the embodiment of the present application, for spatial grid division, since primitives are attached to the surface of the object, many grids are empty. Therefore, by adding a grid compression filter (pass), grids with no internal information are discarded to reduce bandwidth and reduce storage resource usage.

[0141] For example, see Figure 12, which is a schematic diagram of discrete graphic elements in a scene provided by an embodiment of the present application. The upper left corner is a schematic diagram of graphic elements of the scene shown in Figure 12, and the scene is discretized into multiple graphic elements.

[0142] In the embodiments of this application, the example of first obtaining indirect lighting data for a primitive and then storing multiple primitives in a primitive cache is used for illustration. In other embodiments, primitive metadata is first stored in a first buffer, primitive indexes are stored in a three-dimensional grid, and the three-dimensional grid is stored in a second buffer. Ray tracing is then performed on the stored primitives to obtain indirect lighting data for the primitives and store it in the primitive cache. This is not specifically limited here.

[0143] In an embodiment of the present application, a primitive cache library is obtained through the above steps 801-803, and a lighting cache (Discreted Surfels) of discrete primitives is realized, thereby facilitating the subsequent acquisition of indirect lighting data of the primitives from the primitive cache library, improving the efficiency of obtaining indirect lighting data, and thereby improving the efficiency of lighting rendering.

[0144] It should be noted that steps 801 to 803 are only an optional implementation method for obtaining the primitive cache library. The primitive cache library can also be obtained through other optional implementation methods, which will not be described in detail here.

[0145] Referring to Figure 13, Figure 13 is a flow chart of a method for obtaining a voxel cache library provided in an embodiment of the present application. The method includes the following steps.

[0146] 1301. The terminal voxelizes the scene to obtain multiple voxels of each object in the scene.

[0147] Voxelization is the process of converting an object's surface geometry into a voxel representation. The result is a voxel grid consisting of multiple voxels of the object. Each voxel contains not only surface information about the object model but also describes its internal properties. Voxelization can convert unordered point cloud data into an ordered representation, which helps simplify or reshape the data for further analysis and processing.

[0148] Optionally, the terminal voxelizes the scene using a rasterization method. In the process of voxelizing the scene using the rasterization method, the terminal performs three rasterization renderings on the camera along the x, y, and z axes of the scene. In some embodiments, during rasterization rendering, properties such as occlusion culling and backface culling are disabled to obtain multiple voxels for each object in the scene, and the multiple voxels are used to construct the object.

[0149] 1302. The terminal embeds the normal data, depth data, and albedo data of the object into multiple voxels of the object to obtain a three-dimensional texture map of the object.

[0150] Optionally, the terminal embeds the normal data, depth data and albedo data of the object into multiple voxels of the object to obtain a three-dimensional texture map of the object, including: storing the normal data, depth data and albedo data of the object into a three-dimensional texture in a voxelized pixel shader, thereby embedding the normal data, depth data and albedo data of the object into multiple voxels of the object to obtain a three-dimensional texture map of the object. Optionally, the terminal implements this process through UAV (Unordered Access View). For example, see Figure 14, which is a schematic diagram of a three-dimensional texture map provided in an embodiment of the present application. Among them, normal data, depth data and albedo data are embedded in voxels.

[0151] Among them, albedo, also known as reflectivity, is a physical quantity that represents the degree of reflection of light by an object. It is the ratio of the energy of light reflected from the surface of the object to the energy of light incident on the surface of the object. Albedo can be used to represent the optical properties of the object's surface, such as the object's color, degree of diffuse reflection, and degree of specular reflection.

[0152] 1303. The terminal adjusts the sizes of the multiple voxels included in the three-dimensional texture map based on the distance between the voxels and the camera viewpoint, and the size of each voxel is positively correlated with the distance corresponding to the voxel.

[0153] In the embodiment of the present application, the size of each voxel is positively correlated with the distance corresponding to the voxel, which means that the closer the distance to the camera viewpoint is, the smaller the voxel size is, and the farther the distance to the camera viewpoint is, the larger the voxel size is.

[0154] In the embodiments of the present application, if the voxelized data were stored using a simple 3D texture map, the video memory usage would be very large. Therefore, a multi-level texture map (clip map) method is used to adjust the voxel size so that the closer the voxel is to the camera viewpoint, the smaller the radius, and the farther the voxel is from the camera, the larger the radius, thereby achieving higher precision near the camera and lower precision far away. For example, see Figure 15, which is a schematic diagram of multi-sized voxels provided in the embodiments of the present application.

[0155] 1304. The terminal performs ray tracing on the resized voxels to obtain indirect lighting data of the multiple voxels, and stores the indirect lighting data of the multiple voxels in a voxel cache.

[0156] Optionally, the terminal uses cone tracing to perform ray tracing on the resized voxels to obtain indirect lighting data for the multiple voxels. Accordingly, the process of the terminal performing ray tracing on the resized voxels to obtain indirect lighting data for the multiple voxels includes the following steps: the terminal divides the hemispherical space corresponding to the starting point of the ray into multiple cones, each cone corresponding to a cone angle and multiple levels of step distances, and the multiple levels of step distances are used to indicate different distances of ray emission; for each cone, from the multiple sizes of the voxels, determines the target size corresponding to the multiple levels of step distances, and obtains the indirect lighting data for the voxels of the target size.

[0157] Cone tracing is a generalized method for the ray collection process. It uses cones instead of single rays, generalizing the hemispherical space into several cones to approximate indirect lighting data. It also integrates well with voxels. It calculates the area to be sampled based on the cone's propagation distance and the cone's angle, allowing for rapid selection of the stepping distance level and the tracing step size. When a ray is emitted from the camera, it ensures that the cone covers a specific area of ​​pixels on the screen, rather than a single point. Since the ray has become a cone and has a volume, the intersection calculation between the ray and the object not only determines whether the cone intersects the object, but also determines how much of the cone intersects. Because the ray has a volume, a single ray may intersect multiple objects. In this case, the lighting calculation results for these multiple objects will affect the final pixel color, thus achieving an anti-aliasing effect. This embodiment sets different stepping distances and selects a size corresponding to the stepping distance to obtain indirect lighting data for voxels of corresponding sizes. This improves efficiency while ensuring accuracy and achieving an anti-aliasing effect.

[0158] For example, see Figure 16, which is a schematic diagram of the size of a cone and voxel provided in an embodiment of the present application. Wherein, C0 represents the starting point of the ray, Cd represents the direction of the ray, θ represents the angle of the cone trace, t represents the current overall step distance, and d represents the cone diameter corresponding to a single step. As can be seen from the figure, as the number of steps increases, t is increasing, d is also increasing, and the overall iteration is getting faster and faster. Accordingly, you can choose which level of step distance to sample at according to the size of d.

[0159] For example, referring to the following formula (1), the terminal determines the target size through the formula (1): d = 2t × tan (θ ÷ 2) (1).

[0160] Correspondingly, as the number of steps increases, the step distance levels also increase, and the step distance of the next level is larger than the step distance of the current level. For each step distance level, the cone diameter corresponding to that level is determined, and then the target size that matches the cone diameter among the multiple voxel sizes is determined, that is, the target size corresponding to the step distance of that level is obtained.

[0161] In an embodiment of the present application, the process of obtaining the voxel cache library is implemented through the above steps 1301-1304, and the voxelized lighting cache (Scene Voxelize) is realized, which facilitates the subsequent acquisition of indirect lighting data of voxels from the voxel cache library, improves the efficiency of obtaining indirect lighting data, and thus improves the lighting rendering efficiency.

[0162] It should be noted that steps 1301 to 1304 are only an optional implementation method for obtaining the voxel cache library. The terminal may also obtain the voxel cache library through other optional implementation methods, which will not be described in detail here.

[0163] It should be noted that in the embodiment of the present application, the process of obtaining the pixel cache library, the primitive cache library and the voxel cache library is completed through the terminal for lighting rendering, so that global lighting rendering of the scene can be achieved through one terminal.

[0164] The lighting rendering method provided in the embodiments of the present application can be named SmartGI (scalable dynamic global illumination based on hybrid architecture) technology, which can realize fully dynamic global illumination technology on mobile terminals. This technology is a global illumination (GI) framework based on a hybrid architecture, which abstracts global illumination into two parts: the surface cache and the final gather. It also seamlessly integrates SSGI, VoxelGI, SurfelGI and other technologies for lighting caching. Among them, SSGI is a screen-space global illumination technology that first calculates the direct lighting data of the pixel before calculating the indirect lighting data of each light. Then, the indirect lighting data of the shading point is collected through screen-space ray tracing technology to achieve global illumination effects. This technology cannot obtain light reflection information of objects outside the screen, resulting in poor accuracy of indirect lighting data. VoxelGI is a technology that voxelizes the scene and then uses the voxelized information as a lighting cache. However, the accuracy of voxelization is difficult to control. If the accuracy is not enough, it is difficult to express fine lighting effects. If the accuracy is too high, it will face storage and performance pressures. SurfelGI is a technology that discretizes scenes into primitives and then uses these discrete primitives for lighting caching. This technology mainly generates primitives based on the screen space seen by the camera. This technology cannot generate valid primitive information where the camera does not see, resulting in incorrect lighting effects. The method provided in the embodiment of the present application solves the above-mentioned technical problems by seamlessly integrating multiple implementation methods. The method provided in the embodiment of the present application supports a variety of software and hardware ray tracing solutions, such as HZB-based ray tracing (raymarching), multi-level texture-based cone tracing, SDF-based ray tracing, and GPU (Graphics Processing Unit)-based ray query technology, etc., to achieve scalable dynamic global illumination.

[0165] For example, refer to Figure 17, which is a schematic diagram of the framework of global illumination provided by an embodiment of the present application. Among them, the fully dynamic global illumination technology includes an illumination cache part, an illumination acquisition part and an illumination combination part. In the illumination cache part, the scene structure is described in a simple and low-cost form, and is used to cache illumination data, including screen space illumination, discrete primitives and illumination data corresponding to scene voxels. In the illumination acquisition part, the illumination acquisition process is abstracted, and a variety of software and hardware ray tracing solutions can be supported for screen space tracking, cached primitive tracking, voxel tracking, etc., which can solve performance and compatibility issues. In the illumination combination part, global rendering of the scene is achieved through indirect illumination texture and direct illumination texture. And the indirect illumination texture includes indirect illumination data of the diffuse reflection part and the highlight reflection part.

[0166] For another example, see Figure 18, which is a schematic diagram of another framework of global illumination provided by an embodiment of the present application. Among them, when rendering indirect lighting for a scene, the process mainly includes the following parts. First, a screen space probe is generated, which can be a uniform probe or an adaptive probe. Then the BRDF function is calculated to generate rays for the screen space probe to obtain light data; this process is implemented by the importance sampling method, which includes methods such as BRDF and historical lighting. Then, ray tracing is performed in screen space, which can be implemented through various strategies. In this process, the hit distance and hit rate can also be obtained. Then, primitives and voxels are detected to obtain indirect lighting data for the primitives and voxels. Among them, screen space tracing includes the farthest HZB and the nearest HZB in the HZB technology. In this process, the hit distance and hit rate can also be obtained. After screen space tracing, primitive tracing and voxel tracing are performed based on compact rays. Primitive tracing includes the BVH ray tracing method; further, the moving ray detection method and the SDF ray tracing method are used to find primitives. Voxel tracing involves voxelizing the scene. Further, moving ray detection, SDF ray tracing, and voxel cone tracing are employed. This process also provides hit distances and hit rates. RSM (Reflective Shadow Maps) lighting is then generated, using ray traced depth maps, ray hit distances, and ray hit rates. Filtering is then performed, including temporal and spatial filtering of light probes. Finally, noise removal, or denoising, is performed. This denoising process includes denoising the irradiance in all screen-space directions, denoising the diffuse indirect lighting data and specular indirect lighting data for the entire screen space, and denoising the historical indirect lighting data corresponding to the temporal filter. Furthermore, screen cache tracing is performed through the reflection channel. During the lighting combination stage, the engine's transparent mode is used to process lighting rendering based on direct and indirect lighting data. Direct lighting data is used to render scene colors. Indirect lighting data includes specular and diffuse lighting textures.

[0167] For example, see Figure 19, which is a schematic diagram of a lighting setting interface provided by an embodiment of the present application. Among them, the lighting setting interface is a ue5 (unreal engine 5) interface, and a selection control 1901 for the lighting algorithm is displayed on the lighting setting interface. The selection control 1901 is used to set the algorithm for global illumination, such as selecting the SmartGI algorithm. For example, see Figure 20, which is a schematic diagram of the lighting effect in a scene provided by an embodiment of the present application. Among them, the scene shown in a in Figure 20 has only the effect of skylight, and the scene shown in b in Figure 20 has the effect of global illumination. By comparison, it can be seen that the lighting effect of the scene shown in b in Figure 20 is better.

[0168] Compared with the problems of long-term baking of textures, high storage pressure for large scene data, and inability to change scenes and lighting in static global illumination technology, the method provided in the embodiment of the present application realizes fully dynamic global illumination, and the lighting rendering of the scene can achieve zero-cost baking, zero pre-calculated data, cross-platform, high performance, real-time dynamic lighting and dynamic scene effects.

[0169] For example, see Figure 21, which is a schematic diagram comparing lighting effects provided by an embodiment of the present application. Figure a in Figure 21 is a schematic diagram of lighting for a scene obtained using static global illumination technology, and Figure b in Figure 21 is a schematic diagram of lighting for a scene obtained using the method provided by an embodiment of the present application. From the comparison, it can be seen that the method provided by an embodiment of the present application can achieve the same effect as the static global illumination technology even without using static global illumination technology to bake the light map offline. That is, the lighting effect rendered by the method provided by an embodiment of the present application is good.

[0170] The method provided in the embodiment of the present application also supports lighting effects that change over time. For example, see Figure 22, which is a schematic diagram of adjusting lighting effects provided in the embodiment of the present application. A slider is displayed on the terminal, and the terminal adjusts the lighting effects in the scene in response to the adjustment operation of the slider. For example, the lighting effects are different during the day and at night, and the lighting effects are different in the morning and in the afternoon. That is, the method provided in the embodiment of the present application realizes dynamic and flexible adjustment of lighting effects based on time.

[0171] The method provided in the embodiments of the present application can also support higher-precision frame rates. For example, it can support 60fps, 90fps, and 120fps. For example, see Figure 23, which is a schematic diagram of a scene after rendering and illumination provided in the embodiments of the present application. The frame rate of the scene image reaches 60fps.

[0172] An embodiment of the present application provides a lighting rendering method, which uses lighting probes to determine the rays that hit the pixels and the rays that miss the pixels, and obtains the indirect lighting data of the hit pixels. For the rays that miss the pixels, the elements hit by these rays in the current screen space and outside the current screen space are obtained, and the indirect lighting data of the hit elements are obtained. In this way, the method performs lighting rendering through the indirect lighting data of the pixels and the indirect lighting data of the elements, which can achieve the effective supplementation of the indirect lighting data of the elements to the indirect lighting data of the pixels, and the indirect lighting data of the elements also includes the indirect lighting data outside the current screen space, which further improves the comprehensiveness and accuracy of the indirect lighting data, and then performs lighting rendering on the scene based on such indirect lighting data, which can improve the accuracy of lighting rendering.

[0173] FIG24 is a block diagram of a lighting rendering device according to an embodiment of the present application. Referring to FIG24 , the device includes:

[0174] An acquisition module 2401 is used to acquire light data of multiple light probes in the scene, where the light data includes the starting points and directions of multiple rays on the light probes.

[0175] a detection module 2402 configured to determine, based on the light data, a first ray, a second ray, and first indirect lighting data among a plurality of rays, wherein the first ray is a ray that hits a pixel in a current screen space of the scene, the second ray is a ray that does not hit a pixel in the current screen space, and the first indirect lighting data is indirect lighting data of the hit pixel;

[0176] The acquisition module 2401 is further configured to acquire elements hit by the second ray in the current screen space and outside the current screen space, and acquire second indirect lighting data from a lighting cache of the elements, where the second indirect lighting data is the indirect lighting data of the elements;

[0177] The rendering module 2403 is configured to perform lighting rendering on the scene based on the first indirect lighting data and the second indirect lighting data.

[0178] In some embodiments, the acquisition module 2401 is configured to perform at least one of the following:

[0179] Determine the primitives hit by the second ray in the current screen space and outside the current screen space, and obtain indirect lighting data of the hit primitives from the primitive cache;

[0180] The voxels hit by the second ray are determined in the current screen space and outside the current screen space, and the indirect lighting data of the hit voxels are obtained from the voxel cache.

[0181] In some embodiments, the second ray is multiple, and the acquisition module 2401 is used to:

[0182] Determine, within the current screen space and outside the current screen space, a primitive hit by the second ray, and obtain indirect lighting data of the hit primitive from a primitive cache; determine a third ray among the plurality of second rays, the third ray being a ray that does not hit a primitive; determine, within the current screen space and outside the current screen space, a voxel hit by the third ray, and obtain indirect lighting data of the hit voxel from a voxel cache; or,

[0183] Determine the voxel hit by the second ray within and outside the current screen space, and obtain indirect lighting data of the hit voxel from the voxel cache; determine a fourth ray among multiple second rays, where the fourth ray is a ray that does not hit the voxel; determine the primitive hit by the fourth ray within and outside the current screen space, and obtain indirect lighting data of the hit primitive from the primitive cache.

[0184] In some embodiments, the apparatus further comprises:

[0185] a discretization module, configured to discretize a scene into a plurality of primitives, perform ray tracing on the plurality of primitives, obtain indirect lighting data of the plurality of primitives, and store the indirect lighting data of the plurality of primitives in a first buffer of a primitive buffer library;

[0186] A partitioning module, for converting a scene into a three-dimensional grid, where the three-dimensional grid includes multiple grids;

[0187] A primitive storage module, configured to store, in each grid, a primitive index of a primitive corresponding to the grid, wherein the primitive index is used to obtain indirect lighting data of the primitive from the first buffer;

[0188] The graphic element storage module is further used to store the three-dimensional grid in the second buffer of the graphic element buffer library.

[0189] In some embodiments, the apparatus further comprises a screening module for:

[0190] Filter the grids in the second buffer that do not store the primitive index and remove the grids that do not store the primitive index.

[0191] In some embodiments, discrete modules are used to:

[0192] The scene is divided into multiple tiles, and the pixels in multiple tiles are covered by primitives;

[0193] For each tile, determine the target pixel with the minimum primitive coverage in the tile. If the primitive coverage of the target pixel does not meet the coverage threshold, generate the primitive at the target pixel until the primitive coverage of multiple pixels in the tile respectively meet the coverage threshold.

[0194] In some embodiments, the apparatus further comprises:

[0195] The voxelization module is used to voxelize the scene and obtain multiple voxels of each object in the scene;

[0196] An embedding module, used to embed the normal data, depth data and albedo data of the object into multiple voxels of the object to obtain a three-dimensional texture map of the object;

[0197] An adjustment module is configured to adjust the sizes of a plurality of voxels included in the three-dimensional texture map based on the distance between the voxels and the camera viewpoint, wherein the size of each voxel is positively correlated with the distance corresponding to the voxel;

[0198] The voxel storage module is used to perform ray tracing on the resized voxels to obtain indirect lighting data of the multiple voxels, and store the indirect lighting data of the multiple voxels in a voxel cache library.

[0199] In some embodiments, the voxel storage module is configured to:

[0200] The hemispherical space corresponding to the starting point of the ray is divided into multiple cones. Each cone has a cone angle and multiple levels of step distances. The multiple levels of step distances are used to indicate the different distances of the ray.

[0201] For each cone, target sizes corresponding to multiple levels of step distances are determined from multiple sizes of voxels, and indirect lighting data of voxels of the target sizes are obtained.

[0202] In some embodiments, the first indirect illumination data is obtained from a pixel cache, and the obtaining module 2401 is further configured to obtain scene data of a previous frame of the current frame of the scene, and determine a screen map of the current frame based on the scene data and light bounce data corresponding to the current frame, where the screen map includes depth data of a plurality of pixels.

[0203] The device also includes a pixel storage module, which is also used to perform ray tracing based on the picture map, obtain indirect lighting data of multiple pixels in the current frame picture, and store the indirect lighting data in a pixel cache library.

[0204] In some embodiments, the rendering module 2403 is configured to:

[0205] Based on the first indirect lighting data and the second indirect lighting data, lighting interpolation is performed on the pixels in the current screen space, and lighting rendering is performed on the scene based on the first indirect lighting data and the indirect lighting data obtained by interpolation.

[0206] In some embodiments, the rendering module 2403 is configured to:

[0207] De-noising is performed on the third indirect lighting data, and lighting rendering is performed on the scene based on the third indirect lighting data after the denoising, where the third indirect lighting data includes the first indirect lighting data and indirect lighting data obtained by interpolation.

[0208] In some embodiments, the rendering module 2403 is configured to:

[0209] Get the direct lighting data of the scene;

[0210] Determining global illumination data based on the first indirect illumination data, the second indirect illumination data, and the direct illumination data;

[0211] Render the scene based on global illumination data.

[0212] An embodiment of the present application provides a lighting rendering device, which uses lighting probes to determine the rays that hit the pixels and the rays that miss the pixels, and obtains the indirect lighting data of the hit pixels. For the rays that miss the pixels, the elements hit by these rays in the current screen space and outside the current screen space are obtained, and the indirect lighting data of the hit elements are obtained. In this way, the method performs lighting rendering through the indirect lighting data of the pixels and the indirect lighting data of the elements, which can achieve the effective supplementation of the indirect lighting data of the elements to the indirect lighting data of the pixels, and the indirect lighting data of the elements also includes the indirect lighting data outside the current screen space, which further improves the comprehensiveness and accuracy of the indirect lighting data, and then performs lighting rendering on the scene based on such indirect lighting data, which can improve the accuracy of lighting rendering.

[0213] FIG25 shows a block diagram of a terminal 2500 according to an exemplary embodiment of the present application. The terminal 2500 includes a processor 2501 and a memory 2502 .

[0214] Processor 2501 may include one or more processing cores, such as a quad-core processor or an octa-core processor. In some embodiments, processor 2501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, processor 2501 may also include an AI (Artificial Intelligence) processor, which is used to handle computing operations related to machine learning.

[0215] Memory 2502 may include one or more computer-readable storage media, which may be non-transitory. Memory 2502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 2502 is used to store at least one program code, which is executed by processor 2501 to implement the lighting rendering method provided in the method embodiment of the present application.

[0216] In some embodiments, terminal 2500 may optionally include a peripheral device interface 2503 and at least one peripheral device. The processor 2501, memory 2502, and peripheral device interface 2503 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 2503 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 2504, a display screen 2505, a camera assembly 2506, an audio circuit 2507, and a power supply 2508.

[0217] In some embodiments, the terminal 2500 further includes one or more sensors 2509 . The one or more sensors 2509 include, but are not limited to, an acceleration sensor 2510 , a gyroscope sensor 2511 , a pressure sensor 2512 , an optical sensor 2513 , and a proximity sensor 2514 .

[0218] Those skilled in the art will understand that the structure shown in FIG25 does not constitute a limitation on the terminal 2500 , and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0219] An embodiment of the present application further provides a computer-readable storage medium, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the lighting rendering method of any of the above implementation methods.

[0220] An embodiment of the present application also provides a computer program product, which includes computer program code, the computer program code is stored in a computer-readable storage medium, the processor of the terminal reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the terminal performs the lighting rendering method of any of the above-mentioned implementation methods.

[0221] In some embodiments, the computer program product involved in the embodiments of the present application can be deployed and executed on one terminal, or on multiple terminals located at one location, or on multiple terminals distributed at multiple locations and interconnected through a communication network. Multiple terminals distributed at multiple locations and interconnected through a communication network can constitute a blockchain system.

[0222] The above are merely optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A lighting rendering method, applied to a terminal, comprising: Obtaining light data of multiple light probes in the scene, wherein the light data includes starting points and directions of multiple rays on the light probes; Based on the light data, determine a first ray, a second ray, and first indirect illumination data among the multiple rays, wherein the first ray is a ray that hits a pixel in a current screen space of the scene, the second ray is a ray that does not hit a pixel in the current screen space, and the first indirect illumination data is indirect illumination data of the hit pixel; Obtaining elements hit by the second ray in the current screen space and outside the current screen space, and obtaining second indirect lighting data from a lighting cache library of the element, where the second indirect lighting data is indirect lighting data of the element; Perform lighting rendering on the scene based on the first indirect lighting data and the second indirect lighting data.

2. The method according to claim 1, wherein: The step of obtaining the elements hit by the second ray in the current screen space and outside the current screen space, and obtaining second indirect lighting data from a lighting cache of the elements, includes at least one of the following: Determine, within the current screen space and outside the current screen space, a primitive hit by the second ray, and obtain indirect lighting data of the hit primitive from a primitive cache library; The voxels hit by the second ray are determined in the current screen space and outside the current screen space, and the indirect lighting data of the hit voxels are obtained from a voxel cache library.

3. The method according to claim 1, wherein: There are multiple second rays, and obtaining elements hit by the second rays in the current screen space and outside the current screen space, and obtaining second indirect lighting data from a lighting cache library of the elements includes: Determine the primitive hit by the second ray in the current screen space and outside the current screen space, and obtain indirect lighting data of the hit primitive from a primitive cache library; determine a third ray among multiple second rays, wherein the third ray is a ray that does not hit a primitive; determine the voxel hit by the third ray in the current screen space and outside the current screen space, and obtain indirect lighting data of the hit voxel from a voxel cache library; or, Determine the voxel hit by the second ray in the current screen space and outside the current screen space, and obtain indirect lighting data of the hit voxel from the voxel cache library; determine a fourth ray among the multiple second rays, and the fourth ray is a ray that does not hit the voxel; determine the primitive hit by the fourth ray in the current screen space and outside the current screen space, and obtain indirect lighting data of the hit primitive from the primitive cache library.

4. The method according to claim 2 or 3, wherein: The acquisition process of the primitive cache library includes: Discretize the scene into a plurality of primitives, perform ray tracing on the plurality of primitives to obtain indirect lighting data of the plurality of primitives, and store the indirect lighting data of the plurality of primitives in a first buffer of the primitive buffer library; Converting the scene into a three-dimensional grid, wherein the three-dimensional grid includes a plurality of grids; storing in each grid a primitive index of a primitive corresponding to the grid, the primitive index being used to obtain indirect lighting data of the primitive from the first buffer; The three-dimensional mesh is stored in a second buffer of the primitive buffer library.

5. The method according to claim 4, wherein: The method further comprises: Filter the grids without storing the primitive index from the multiple grids in the second buffer, and remove the grids without storing the primitive index.

6. The method according to claim 4, wherein: The step of discretizing the scene into a plurality of primitives comprises: Dividing the scene into a plurality of tiles, pixels in the plurality of tiles being covered by primitives; For each tile, determine the target pixel with the minimum primitive coverage in the tile, and if the primitive coverage of the target pixel does not meet the coverage threshold, generate a primitive at the target pixel until the primitive coverage of multiple pixels in the tile respectively meet the coverage threshold.

7. The method according to claim 2 or 3, wherein: The process of obtaining the voxel cache library includes: voxelize the scene to obtain a plurality of voxels of each object in the scene; Embedding normal data, depth data and albedo data of the object into a plurality of voxels of the object to obtain a three-dimensional texture map of the object; For a plurality of voxels included in the three-dimensional texture map, based on the distance between the voxels and a camera viewpoint, adjusting the sizes of the plurality of voxels, wherein the size of each voxel is positively correlated with the distance corresponding to the voxel; Ray tracing is performed on the resized voxels to obtain indirect lighting data of the plurality of voxels, and the indirect lighting data of the plurality of voxels is stored in the voxel cache.

8. The method according to claim 7, wherein: The performing ray tracing on the resized voxels to obtain indirect illumination data of the plurality of voxels includes: The hemispherical space corresponding to the starting point of the ray is divided into multiple cones, each cone corresponds to a cone angle and multiple levels of step distances, and the multiple levels of step distances are used to indicate different distances of the ray emission; For each cone, target sizes corresponding to the multiple levels of step distances are determined from multiple sizes of voxels, and indirect illumination data of voxels of the target sizes are obtained.

9. The method according to claim 1, wherein: The first indirect illumination data is obtained from a pixel cache library, and the process of obtaining the pixel cache library includes: Acquire scene data of a previous frame of the current frame of the scene, and determine a picture map of the current frame based on the scene data and light bounce data corresponding to the current frame, wherein the picture map includes depth data of a plurality of pixels; Ray tracing is performed based on the screen map to obtain indirect lighting data of multiple pixels in the current frame screen, and the indirect lighting data of the multiple pixels are stored in the pixel cache library.

10. The method according to claim 1, wherein: The performing illumination rendering on the scene based on the first indirect illumination data and the second indirect illumination data includes: Based on the first indirect lighting data and the second indirect lighting data, lighting interpolation is performed on the pixels in the current screen space, and based on the first indirect lighting data and the interpolated indirect lighting data, lighting rendering is performed on the scene.

11. The method according to claim 10, wherein: The performing illumination rendering on the scene based on the first indirect illumination data and the interpolated indirect illumination data includes: The third indirect illumination data is denoised, and illumination rendering is performed on the scene based on the denoised third indirect illumination data, wherein the third indirect illumination data includes the first indirect illumination data and indirect illumination data obtained by interpolation.

12. The method according to claim 1, wherein: The performing illumination rendering on the scene based on the first indirect illumination data and the second indirect illumination data includes: Obtaining direct lighting data of the scene; determining global illumination data based on the first indirect illumination data, the second indirect illumination data, and the direct illumination data; Based on the global illumination data, the scene is illuminated and rendered.

13. A lighting rendering device, comprising: An acquisition module, used to acquire light data of multiple lighting probes in the scene, wherein the light data includes starting points and directions of multiple rays on the lighting probes; a detection module, configured to determine, based on the light data, a first ray, a second ray, and first indirect illumination data among the multiple rays, wherein the first ray is a ray that hits a pixel in a current screen space of the scene, the second ray is a ray that does not hit a pixel in the current screen space, and the first indirect illumination data is indirect illumination data of the hit pixel; The acquisition module is further used to acquire elements hit by the second ray in the current screen space and outside the current screen space, and acquire second indirect illumination data from the illumination cache library of the element, where the second indirect illumination data is the indirect illumination data of the element; A rendering module is used to perform lighting rendering on the scene based on the first indirect lighting data and the second indirect lighting data.

14. A terminal comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor to execute the illumination rendering method according to any one of claims 1 to 12.

15. A computer-readable storage medium, wherein the computer-readable storage medium is used to store at least one program, wherein the at least one program is used to execute the lighting rendering method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program code, wherein the computer program code is stored in a computer-readable storage medium, a processor of a terminal reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the terminal executes the lighting rendering method according to any one of claims 1 to 12.

Images