Cloud baking global illumination method and system

Abstract

The invention discloses a cloud baking global illumination method and system, and relates to the field of computer graphics. The method comprises the steps that a cloud system generates a directed distance field and a material attribute map of a full scene for each 3D model; performing ray tracing by using a directed distance field of a full scene, and generating a visibility cache of each block; sampling the illumination map of each light source according to the visibility cache, and generating hemispherical indirect illumination of each block; encoding the hemispherical indirect illumination to obtain a spherical harmonic function coefficient; placing illumination probes of world coordinates around the blocks of the 3D model; filtering the indirect illumination of the hemisphere by using an illumination probe to obtain a filtered spherical harmonic function coefficient, and determining an illumination probe parameter of each 3D model; and sending the illumination probe parameters of each 3D model to a respective web front end to render global illumination of the 3D scene. According to the invention, real-time and high-quality global illumination can be provided for a large number of users through cloud baking.

Description

Technical Field

[0001] The present application relates to the field of computer graphics, and in particular to a cloud-based global illumination baking method and system. Background Art

[0002] With the development of Web3D technology, the demand for creating and displaying 3D content on the web is growing. When displaying 3D content, real-time high-quality lighting performance is very important, and in many cases it is even the core factor that determines the display effect. However, real-time high-quality lighting performance is also a hot research field that computer graphics has been working hard to overcome and improve. How to provide high-quality lighting effects while quickly completing calculations to meet real-time requirements based on limited functions has always been a dilemma. Summary of the invention

[0003] The purpose of this application is to provide a cloud-based global illumination baking method and system product, which can provide real-time, high-quality global illumination to a large number of users through cloud-based baking.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a cloud-based global illumination baking method, comprising: receiving 3D models in a 3D scene uploaded by multiple web front-ends through a cloud system; generating a directed distance field of the entire scene for each 3D model; dividing the 3D model into blocks, and merging material attribute data of all blocks into a material attribute map; calculating a shadow map of each light source in the 3D scene; calculating a lighting map of each light source based on the shadow map of each light source and the material attribute map; performing ray tracing on each block using the directed distance field of the entire scene, and generating a light map for each block. The method comprises the steps of: collecting a visibility cache of each block; sampling a light map of each light source according to the visibility cache of each block to generate hemispherical indirect lighting of each block; encoding the hemispherical indirect lighting of each block to obtain spherical harmonic coefficients; placing world coordinate light probes around the blocks of the 3D model; filtering the hemispherical indirect lighting of each block using the light probes to obtain filtered spherical harmonic coefficients, and determining them as light probe parameters for each 3D model; and sending the light probe parameters of each 3D model to respective web front ends to render global illumination of the 3D scene.

[0006] In a second aspect, the present application provides a cloud-based global illumination baking system, including: a cloud-based system that adopts the above-mentioned cloud-based global illumination baking method.

[0007] According to the specific embodiments provided in this application, this application has the following technical effects:

[0008] The present application provides a cloud-based global illumination baking method and system. By baking a 3D scene in the cloud, densely distributed lighting probes of the scene part in world coordinates are obtained. Compared with the traditional light map calculation results, the light probes occupy less capacity and can well support the real-time computing needs of a large number of users, providing the data required for real-time global illumination rendering for multiple web front ends; further, the light map is sampled based on the visibility cache, the sampled calculation amount can be controlled, and time filtering can be performed to reduce noise and calculation amount. At the same time, the light probe filters the hemispherical indirect lighting of each block. After the above two layers of data filtering, the quality of global illumination is improved, so that the performance effect and transmission pressure are simultaneously achieved. Better results. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 This is a flow chart of a method for baking global illumination in the cloud in one embodiment of the present application;

[0011] Figure 2 A schematic diagram of the working principle of a cloud-based global illumination baking method provided in one embodiment of the present application;

[0012] Figure 3 A schematic diagram of the global illumination baking process provided in one embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0015] In an exemplary embodiment, Figure 1 As shown, the present application provides a cloud-based global illumination baking method, including the following steps 101 to 111. Among them:

[0016] Step 101: receiving 3D models of 3D scenes uploaded by multiple web front ends through a cloud system.

[0017] Step 102: For each 3D model, generate a signed distance field of the entire scene.

[0018] Step 103: Divide the 3D model into blocks, and merge the material attribute data of all the blocks into a material attribute map.

[0019] Step 104: Calculate the shadow map for each light source in the 3D scene.

[0020] Step 105: Calculate the light map of each light source according to the shadow map of each light source and the material attribute map.

[0021] Step 106: Use the signed distance field of the entire scene to perform ray tracing for each block, and generate a visibility cache for each block.

[0022] Step 107: Based on the visibility cache of each tile, sample the light map of each light source to generate hemispherical indirect lighting for each tile.

[0023] Step 108: Encode the hemispherical indirect illumination of each block to obtain spherical harmonic function coefficients.

[0024] Step 109: Place world-spaced light probes around the chunks of the 3D model.

[0025] Step 110: Filter the hemispherical indirect lighting of each block using the lighting probe to obtain filtered spherical harmonic function coefficients, and determine them as lighting probe parameters for each 3D model.

[0026] Step 111: Send the lighting probe parameters of each 3D model to the respective web front end.

[0027] The core idea of ​​implementing the above steps 101 to 111 is to bake the 3D scene in the cloud to obtain densely distributed lighting probes of the scene part in the world coordinates, and gradually send the probes of the area seen by the linked client to the client. After the client gets the lighting probe data, it can render various indirect lighting and merge it with the relatively easy-to-calculate direct lighting to obtain the final global illumination.

[0028] The cloud system (cloud host) can calculate the dense lighting probes of the 3D scene in the world coordinates, which consists of several main steps: first, the 3D model needs to be preprocessed to generate data for subsequent calculations, among which the more important is the signed distance field processing and patch processing of the 3D model; secondly, the 3D scene is calculated in blocks for direct lighting; then each block is projected on the hemisphere, the projected light is integrated to obtain the effect of light bounce, and it is continuously updated and accumulated, so that the result of infinite light bounce on each block can be calculated. After that, light probes are placed on the world coordinates of these blocks. These light probes are evenly placed according to the size of the model in the world coordinates, which can ensure smooth transition and stable effect of the final result.

[0029] The method of the present application works in the cloud. On the one hand, it can work independently on a traditional graphics card without requiring the hardware device to have the ability to implement ray tracing in hardware. On the other hand, it can also take into account the special needs of the cloud system, such as meeting the rendering needs of multiple users at the same time and sending the results to multiple users in real time.

[0030] Figure 2 The working principle diagram of the method of the present application is shown. Steps 101 to 110 in the method of the present application constitute a global illumination baking process. The global illumination baking process is as follows: Figure 3 shown.

[0031] In another exemplary embodiment of the present application, based on the 3D model in the 3D scene, the signed distance field (SDF) of the model is generated by preprocessing, and then the distance fields of multiple instances in the scene are integrated into the SDF of the whole scene. Then the above step 102 can be replaced by the following steps 201 to 204:

[0032] Step 201: voxelize the 3D model and calculate the signed distance field value and sign of each voxel of the 3D model.

[0033] The algorithm for calculating the signed distance field value and the sign of each voxel of the 3D model may be a Brute-Force algorithm or a Fast Marching Method (FMM) and a Fast Sweeping Method (FSM) algorithm.

[0034] Step 202: voxelize the 3D scene.

[0035] Step 203: According to the signed distance field value and sign of each voxel in the 3D model, a minimum distance merging algorithm is used to calculate the signed distance field value and sign of each voxel in the 3D scene.

[0036] Step 204: The signed distance field values ​​and signs of all voxels in the 3D scene are combined to form a signed distance field of the entire scene.

[0037] In another exemplary embodiment of the present application, the process of obtaining the material attribute map in step 103 above can be replaced by the following steps 301 to 302:

[0038] Step 301: using a mipmap method to generate blocks of different precisions for the 3D model.

[0039] Step 302: According to the material attribute data of each block, a texture stitching algorithm is used to obtain a material attribute map.

[0040] Block material attribute data such as albedo, metalness, smoothness, etc. Texture stitching algorithms such as MaxRectsBinPack algorithm, etc.

[0041] In another exemplary embodiment of the present application, before the above step 104, the method may further include the following steps 401 to 402:

[0042] Step 401: Modify the 3D scene on the web front end and send the modification operation to the cloud system.

[0043] Step 402: Synchronize modification operations on the 3D scene in the cloud system.

[0044] In another exemplary embodiment of the present application, the calculation of the shadow map of each light source in the 3D scene in step 104 may be: traversing all light sources in the 3D scene, and calculating the shadow map of each light source through a rasterization pipeline. The rasterization pipeline mentioned herein is a traditional rasterization pipeline.

[0045] In another exemplary embodiment of the present application, the specific operation of calculating the light map of each light source in the above step 105 can be: calling the shadow map of each light source and the material attribute map as parameters, calculating the direct lighting through the lighting model, and obtaining the corresponding light map; the lighting model includes the Phone lighting model or the Lambert lighting model.

[0046] In another exemplary embodiment of the present application, the above step 106 traverses all blocks in the 3D scene and uses the generated directed distance field of the entire scene to perform ray tracing, but what is traced is not the lighting situation, but the visibility cache.

[0047] In another exemplary embodiment of the present application, the sampling involved in the above step 107 refers to Monte Carlo importance sampling.

[0048] In another exemplary embodiment of the present application, the encoding involved in the above step 108 refers to Fourier transform.

[0049] In another exemplary embodiment of the present application, generally speaking, the spherical harmonics of the illumination on the block obtained in step 108 can calculate the indirect illumination to a certain extent, but it is easy to encounter various problems caused by the limited downsampling in step 107, such as noise, too bright spots, etc., so it is necessary to place a higher density of light probes around the model patch to filter the block-based illumination results to obtain a better final result.

[0050] In another exemplary embodiment of the present application, the specific operation of rendering the global illumination of the 3D scene in the above step 111 may be: on the web front end, using the illumination probe parameters to render the indirect illumination of the 3D model, and merging it with the direct illumination to obtain the final global illumination.

[0051] In order to better transmit the calculation results, this application selects the spherical harmonic coefficients of the lighting probe as the output result. Unlike other cloud baking methods, which select the lighting map as the output result, this method selects the lighting probe scheme of transmitting world coordinates, which can achieve better results between performance and transmission pressure at the same time. It is not only novel, but also provides good results and has good practicality.

[0052] After selecting the transmission light probe, the coupling between cloud computing and web front-end data is reduced. The update frequency of the cloud system can be inconsistent with the update frequency of the web front-end, and the result synchronization frequency can be adjusted according to the communication situation of the web front-end. When multiple web front-ends in different situations are connected to the cloud-based global illumination system, they each implement different update frequencies without interfering with each other.

[0053] In another exemplary embodiment of the present application, optimization is performed based on network traffic and the number of linked web front ends and sent to the front end user. The calculated results are sent to the web front end based on a variety of operating conditions: for example, for a better user experience, a progressive algorithm can be connected, and the screen data to be sent can be evaluated through the progressive algorithm. Different data are calculated with different weights, and the results with greater weights are sent first.

[0054] The method of this application is based on cloud services and supports providing the data required for real-time global illumination rendering for multiple web front ends, mainly including preprocessing, baking, and synchronization. Existing global illumination technologies are mostly implemented on the client side, while this application implements cloud-based baked global illumination. Therefore, special optimizations are made for cloud services in terms of technical solution implementation, especially the elastic controllable computing pressure, the form of computing results, and the control of synchronization pressure. These optimizations are highly novel, original, and practical.

[0055] In order to adapt to the large fluctuation range of cloud business pressure, the entire global illumination calculation scheme uses software-controllable implementation to facilitate the allocation of calculation pressure. The calculation scheme specifically includes:

[0056] In order to provide better rendering effects, when calculating global illumination data, higher precision calculations are provided for areas close to the camera. This method of dividing the area for calculation is not suitable for hardware-based ray tracing. Therefore, a solution of signed distance field plus ray marching is used to implement software-controlled ray tracing. When users upload 3D models to the cloud system, pre-processing is performed to generate the signed distance field of the model.

[0057] In order to control the amount of calculation, the 3D model uploaded by the user is preprocessed, and a method similar to mipmap is used to generate patch data of different degrees of precision. Different degrees of precision have different numbers of triangles. The higher the degree of precision of the patch, the higher the number of triangles it contains. The high-precision patch is reduced through a face reduction algorithm to reduce the number of triangles contained, and becomes a low-precision patch. It is used as the smallest unit for calculating lighting. By selecting patches of different precisions, the system can achieve a balance and control between the amount of calculation and the performance effect in 3D scenes.

[0058] In order to control the amount of calculation and reduce the noise calculation commonly seen in ray tracing technical solutions, this method implements multi-layer data filtering and selects a technical solution suitable for the rasterization pipeline.

[0059] In order to achieve a noise reduction scheme with controllable computational complexity, a multi-layer data filtering is designed. The number of filtered layers and the layers selected during the implementation process can be controlled. Using multi-layer data for global illumination noise reduction is creative. This method uses two layers of data filtering: the first layer of illumination data is based on indirect illumination with blocks as the smallest unit, and the second layer of illumination data is the illumination probes placed in the 3D scene. The specific process of the calculation of the two layers of data is as follows:

[0060] The calculation process of the first layer of data:

[0061] According to the model block data in the 3D scene, the material attribute data of the blocks are merged into a material map.

[0062] According to the material map in the previous step, direct lighting calculation is performed to generate the direct lighting map of the 3D scene.

[0063] Perform ray tracing based on block data and signed distance field data to generate a visibility buffer.

[0064] Based on the visibility cache, the direct light map generated in the previous step is sampled to generate the indirect lighting for the upper hemisphere. The amount of sampling calculation can be controlled, and temporal filtering can be performed to reduce noise, further reducing the amount of calculation.

[0065] The calculation process of the second layer of data:

[0066] Place world coordinate light probes around the 3D model in the 3D scene. The light probes filter the indirect lighting of the surrounding blocks and process them into coefficients of spherical harmonics.

[0067] The present application proposes a method for baking global illumination in the cloud, and the main process includes the following steps: the web front end uploads the 3D model, and the cloud server preprocesses the 3D model after receiving it, and generates preprocessed data such as the signed distance field and patch segmentation of the 3D model for use in subsequent calculations.

[0068] The web front end creates and modifies the 3D scene, puts in the uploaded 3D model, and sends the relevant operations to the cloud system, which maintains the 3D scene and synchronizes the modification operations. Based on a variety of situations, including the number of modifications, the number of web front end users connected, the position and orientation of each camera, etc., the global illumination is baked.

[0069] Compared with the prior art, the present application has the following advantages: 1. The implementation scheme does not rely on the graphics card hardware to have ray tracing capabilities and has wide adaptability; 2. The sub-processes of the entire calculation process are highly flexible. The high flexibility is reflected in: on the one hand, most of them can control the amount of calculation, and on the other hand, the priority of the calculation can be controlled in a fine-grained manner, which can cooperate well with cloud computing in multi-user situations where the calculation pressure fluctuates greatly; 3. The calculation result is a light probe, which occupies a lot less capacity than the traditional calculation result light map, and is suitable for cloud transmission. Combined with various progressive transmission schemes, it can well support the real-time computing needs of a large number of users.

[0070] Based on the same inventive concept, the embodiment of the present application also provides a cloud-baked global illumination system for implementing the cloud-baked global illumination method involved above. The implementation solution provided by the system to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more cloud-baked global illumination system embodiments provided below can refer to the limitations of the cloud-baked global illumination method above, and will not be repeated here.

[0071] In an exemplary embodiment, a cloud-based global illumination baking system is provided, including: a cloud-based system that adopts the above-mentioned cloud-based global illumination baking method.

[0072] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A cloud-based global illumination baking method, characterized in that: include: Receive 3D models of 3D scenes uploaded by multiple web front ends through the cloud system; For each 3D model, generate a signed distance field for the entire scene; Divide the 3D model into blocks, and merge the material attribute data of all the blocks into a material attribute map; Calculate the shadow map for each light source in the 3D scene; Calculate a light map for each light source based on the shadow map of each light source and the material attribute map; Perform ray tracing of each block using the directed distance field of the entire scene to generate a visibility cache for each block; According to the visibility cache of each tile, the light map of each light source is sampled to generate the hemispherical indirect lighting of each tile; Encode the hemispherical indirect illumination of each block to obtain the spherical harmonic function coefficients; Place world-coordinate light probes around chunks of your 3D model. Filtering the hemispherical indirect illumination of each block using the illumination probe to obtain filtered spherical harmonic function coefficients, and determining them as illumination probe parameters for each 3D model; The light probe parameters for each 3D model are sent to their respective web frontends to render global illumination of the 3D scene.

2. The cloud-based global illumination baking method according to claim 1, characterized in that: For each 3D model, generate a signed distance field for the entire scene, including: Voxelize the 3D model and calculate the signed distance field value and sign of each voxel of the 3D model; Voxelize the 3D scene; According to the signed distance field value and sign of each voxel of the 3D model, the minimum distance merging algorithm is used to calculate the signed distance field value and sign of each voxel of the 3D scene; The signed distance field values ​​and signs of all voxels in the 3D scene are combined to form the signed distance field of the entire scene.

3. The cloud-based global illumination baking method according to claim 1, characterized in that: The 3D model is divided into blocks, and the material attribute data of all the blocks are merged into a material attribute map, specifically including: Using a mipmap method, generating blocks of different precisions for the 3D model; According to the material attribute data of each block, the material attribute map is obtained using the map stitching algorithm.

4. The cloud-based global illumination baking method according to claim 1, characterized in that: Calculate the shadow map for each light source in the 3D scene, including: Iterates over all light sources in the 3D scene and calculates the shadow map for each light source through the rasterization pipeline.

5. The cloud-based global illumination baking method according to claim 1, characterized in that: According to the shadow map of each light source and the material attribute map, the light map of each light source is calculated, specifically including: The shadow map of each light source and the material attribute map are called as parameters, and the direct lighting is calculated through the lighting model to obtain the corresponding lighting map; the lighting model includes the Phone lighting model or the Lambert lighting model.

6. The cloud-based global illumination baking method according to claim 1, characterized in that: The sampling is Monte Carlo importance sampling.

7. The cloud-based global illumination baking method according to claim 1, characterized in that: The encoding is a Fourier transform.

8. The cloud-based global illumination baking method according to claim 1, characterized in that: Calculate the shadow map for each light source in the 3D scene, which previously included: Modify the 3D scene on the web front end and send the modified operation to the cloud system; Synchronize modification operations on 3D scenes in the cloud system.

9. The cloud-based global illumination baking method according to claim 1, characterized in that: Rendering global illumination of a 3D scene, including: On the web front end, the indirect lighting of the 3D model is rendered using the light probe parameters and combined with the direct lighting to obtain the final global illumination.

10. A cloud-based global illumination baking system, characterized in that: The cloud-based global illumination baking system includes: a cloud-based system that adopts the cloud-based global illumination baking method described in any one of claims 1-9.

Images