A cloud baking global illumination method and system

Abstract

The application discloses a cloud baking global illumination method and system, and relates to the field of computer graphics. The method comprises the following steps: a cloud system generates a directional distance field and a material attribute map of a whole scene for each 3D model; ray tracing is performed by using the directional distance field of the whole scene to generate a visibility cache of each block; the illumination map of each light source is sampled according to the visibility cache to generate hemispherical indirect illumination of each block; the hemispherical indirect illumination is encoded to obtain spherical harmonic function coefficients; a world coordinate illumination probe is placed around the block of the 3D model; the hemispherical indirect illumination is filtered by using the illumination probe to obtain filtered spherical harmonic function coefficients, and illumination probe parameters of each 3D model are determined; and the illumination probe parameters of each 3D model are sent to a respective web front end to render global illumination of a 3D scene. The application can provide real-time and high-quality global illumination for a large number of users through cloud baking.

Description

Technical Field

[0001] This application relates to the field of computer graphics, and in particular to a cloud-based baking global illumination method and system. Background Technology

[0002] With the development of Web3D technology, the demand for creating and displaying 3D content on the web is constantly growing. Real-time, high-quality lighting is crucial for displaying 3D content, and in many cases, it's even the core element determining the display effect. However, real-time, high-quality lighting is also a hot research area that computer graphics is constantly striving to improve. How to provide high-quality lighting effects while simultaneously achieving real-time computation within limited hardware capabilities has always been a dilemma. Summary of the Invention

[0003] The purpose of this application is to provide a cloud-based global illumination baking method and system that can provide real-time, high-quality global illumination to a large number of users through cloud-based baking.

[0004] To achieve the above objectives, this application provides the following solution:

[0005] In a first aspect, this application provides a cloud-based global illumination baking method, comprising: receiving 3D models of a 3D scene uploaded by multiple web front-ends through a cloud system; generating a directed distance field of the entire scene for each 3D model; dividing the 3D model into blocks and merging the material attribute data of all blocks into a material attribute map; calculating the shadow map of each light source in the 3D scene; calculating the light map of each light source based on the shadow map and the material attribute map; and performing ray tracing on each block using the directed distance field of the entire scene to generate a light map for each block. The visibility cache of each block is used; based on the visibility cache of each block, the lightmap of each light source is sampled to generate hemispherical indirect lighting for each block; the hemispherical indirect lighting of each block is encoded to obtain spherical harmonic function coefficients; world coordinate lighting probes are placed around the blocks of the 3D model; the hemispherical indirect lighting of each block is filtered using the lighting probes to obtain the filtered spherical harmonic function coefficients, and these are determined as lighting probe parameters for each 3D model; the lighting probe parameters of each 3D model are sent to their respective web front-ends to render the global illumination of the 3D scene.

[0006] Secondly, this application provides a cloud-based baking global illumination system, comprising: a cloud system employing the aforementioned cloud-based baking global illumination method.

[0007] According to the specific embodiments provided in this application, this application has the following technical effects:

[0008] This application provides a cloud-based global illumination baking method and system. By baking a 3D scene in the cloud, a densely distributed lighting probe is obtained for parts of the scene in world coordinates. Compared with traditional lightmap calculation results, the lighting probe occupies less space and can better support the real-time computing needs of a large number of users, providing the data required for real-time global illumination rendering for multiple web front-ends. Furthermore, the lightmap is sampled based on visibility caching. The computational load of sampling can be controlled, and time filtering can be performed to reduce noise and computational load. At the same time, the lighting probe filters the hemispherical indirect lighting of each block. Through the above two layers of data filtering, the quality of global illumination is improved, so that the performance effect and transmission pressure are both well achieved. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating a cloud-based global illumination method for baking according to an embodiment of this application.

[0011] Figure 2 This is a schematic diagram illustrating the working principle of a cloud-based global illumination method for baking, provided in one embodiment of this application.

[0012] Figure 3 This is a schematic diagram of the global illumination process for baking, provided as an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] In one exemplary embodiment, such as Figure 1 As shown, this application provides a cloud-based global illumination method for baking, including steps 101 to 111. Wherein:

[0016] Step 101: Receive 3D models of 3D scenes uploaded by multiple web front-ends through the cloud system.

[0017] Step 102: For each 3D model, generate a directed distance field for the entire scene.

[0018] Step 103: Divide the 3D model into blocks and merge the material attribute data of all blocks into a material attribute map.

[0019] Step 104: Calculate the shadow map for each light source in the 3D scene.

[0020] Step 105: Calculate the light map for each light source based on the shadow map and the material property map for each light source.

[0021] Step 106: Perform ray tracing for each block using the directed distance field of the entire scene to generate a visibility cache for each block.

[0022] Step 107: Based on the visibility cache of each block, sample the light map of each light source to generate hemispherical indirect lighting for each block.

[0023] Step 108: Encode the hemispherical indirect illumination of each block to obtain the spherical harmonic function coefficients.

[0024] Step 109: Place world coordinate lighting probes around the blocks of the 3D model.

[0025] Step 110: Use the lighting probe to filter the hemispherical indirect lighting of each block, obtain the filtered spherical harmonic function coefficients, and determine the lighting probe parameters for each 3D model.

[0026] Step 111: Send the lighting probe parameters of each 3D model to its respective web frontend.

[0027] The core idea of ​​implementing steps 101 to 111 above is to bake the 3D scene in the cloud to obtain densely distributed lighting probes of the scene in world coordinates. The probes of the area seen by the linked client are progressively sent to the client. After the client receives the lighting probe data, it can render various indirect lighting and merge it with the relatively easy-to-calculate direct lighting to obtain the final global illumination.

[0028] The cloud system (cloud host) is able to calculate dense lighting probes for a 3D scene in world coordinates through several main steps: First, the 3D model needs to be preprocessed to generate data for subsequent calculations. Crucially, this involves directional distance field processing and patch processing of the 3D model. Second, direct lighting calculations are performed on the 3D scene as a unit of patch. Then, light rays are projected onto each patch on a hemisphere, and the projected light is integrated to obtain the light bounce effect. This process is continuously updated and accumulated, allowing the calculation of infinite light bounces on each patch. Finally, ray probes are placed in world coordinates on these patches. These probes are evenly placed according to the model's size in world coordinates, ensuring a smooth transition and stable effect in the final result.

[0029] The method in this application works in the cloud. On the one hand, it can work independently on traditional graphics cards without requiring the hardware to have the ability to implement ray tracing. On the other hand, it can also take into account the special needs of cloud systems, such as meeting the rendering needs of multiple users at the same time and sending the results to multiple users in real time.

[0030] Figure 2 This diagram illustrates the working principle of the method described in this application. Steps 101 to 110 in the method constitute the global illumination stage for baking, and the process of the global illumination stage is as follows: Figure 3 As shown.

[0031] In another exemplary embodiment of this application, based on the 3D model in the 3D scene, a directed distance field (SDF) of the model is generated through preprocessing, and then the distance fields of multiple instances in the scene are integrated into the SDF of the entire scene. Therefore, step 102 can be replaced by steps 201 to 204:

[0032] Step 201: Voxelize the 3D model and calculate the directed distance field value and sign of each voxel in the 3D model.

[0033] The algorithm for calculating the directed distance field value and sign of each voxel in the 3D model can be the Brute-Force algorithm or the Fast Marching Method (FMM) and Fast Sweeping Method (FSM) algorithms.

[0034] Step 202: Voxelize the 3D scene.

[0035] Step 203: Based on the directed distance field value and sign of each voxel in the 3D model, use the minimum distance merging algorithm to calculate the directed distance field value and sign of each voxel in the 3D scene.

[0036] Step 204: Combine the directed distance field values ​​of all voxels in the 3D scene with their positive and negative signs to form the directed distance field of the entire scene.

[0037] In another exemplary embodiment of this application, the process of obtaining the material property map in step 103 above can be replaced by the following steps 301 to 302:

[0038] Step 301: Use the mipmap method to generate blocks of different precision for the 3D model.

[0039] Step 302: Based on the material attribute data of each block, use a texture stitching algorithm to obtain the material attribute texture.

[0040] Material properties data for each block, such as albedo, metallicity, and smoothness. Texture stitching algorithms, such as the MaxRectsBinPack algorithm.

[0041] In another exemplary embodiment of this application, before step 104 above, the method may further include the following steps 401 to 402:

[0042] Step 401: Modify the 3D scene in the web front-end and send the modification to the cloud system.

[0043] Step 402: Synchronize and modify the 3D scene in the cloud system.

[0044] In another exemplary embodiment of this application, the calculation of the shadow map of each light source in the 3D scene in step 104 can be as follows: traverse all light sources in the 3D scene and calculate the shadow map of each light source through the rasterization pipeline. The rasterization pipeline mentioned here is a traditional rasterization pipeline.

[0045] In another exemplary embodiment of this application, the specific operation of calculating the light map of each light source in step 105 above can be: calling the shadow map of each light source and the material attribute map as parameters, and performing direct lighting calculation through the lighting model to obtain the corresponding light map; the lighting model includes the Phone lighting model or the Lambert lighting model.

[0046] In another exemplary embodiment of this application, step 106 traverses all blocks in the 3D scene and performs ray tracing using the generated directed distance field of the entire scene, but the tracing is not of the lighting conditions, but of the visibility cache.

[0047] In another exemplary embodiment of this application, the sampling involved in step 107 above refers to Monte Carlo importance sampling.

[0048] In another exemplary embodiment of this application, the encoding involved in step 108 above refers to Fourier transform.

[0049] In another exemplary embodiment of this application, the spherical harmonic function of the illumination on the block obtained in step 108 can generally be used to calculate indirect illumination to a certain extent, but it is prone to various problems caused by the limited downsampling in step 107, such as noise, overly bright spots, etc. Therefore, it is necessary to place high-density light probes around the model patch to filter the block-based illumination results and obtain better final results.

[0050] In another exemplary embodiment of this application, the specific operation of rendering the global illumination of the 3D scene in step 111 above can be as follows: on the web front end, the indirect illumination of the 3D model is rendered using the lighting probe parameters, and then merged with the direct illumination to obtain the final global illumination.

[0051] To better transmit the computational results, this application selects the spherical harmonic function coefficients of the illumination probe as the output. Unlike other cloud baking methods that select a lighting map as the output, this method uses an illumination probe scheme that transmits world coordinates. This achieves a good balance between performance and transmission complexity, demonstrating novelty and providing excellent results with strong practicality.

[0052] By selecting the transmission illumination probe, the coupling between cloud computing and web front-end data is reduced. The update frequency of the cloud system and the update frequency of the web front-end can be different, and the result synchronization frequency can be adjusted according to the communication status of the web front-end. When multiple web front-ends in different situations connect to the cloud baking global illumination system, they each implement different update frequencies without interfering with each other.

[0053] In another exemplary embodiment of this application, optimization is performed based on network traffic and the number of linked web front-ends before being sent to the front-end user. The calculation results are sent to the web front-end based on various operating conditions: for example, for a better user experience, a progressive algorithm can be integrated to evaluate the screen data to be sent, calculate different weights for different data, and prioritize sending results with higher weights.

[0054] This application's method is based on cloud services and supports providing the data required for real-time global illumination rendering to multiple web front-ends, mainly including preprocessing, baking, and synchronization. Existing global illumination technologies are mostly implemented on the client side, while this application implements cloud-based baking global illumination. Therefore, the technical solution has been specifically optimized for cloud services, especially in terms of elastically controllable computational pressure and the format of computational results and synchronization pressure control. These optimizations possess strong novelty, originality, and practicality.

[0055] To adapt to the large fluctuations in cloud-based business load, the entire global illumination calculation scheme is implemented using software control, facilitating the allocation of computational load. The calculation scheme specifically includes:

[0056] To achieve better rendering results, higher precision calculations are performed on the area near the camera when calculating global illumination data. This method of dividing the area for calculation is not suitable for hardware-based ray tracing. Therefore, a scheme of directional distance field plus ray marching is used to achieve software-controlled ray tracing. When the user uploads the 3D model to the cloud system, the directional distance field of the model is preprocessed to generate the model.

[0057] To control computational load, the user-uploaded 3D models are preprocessed using a mipmap-like method to generate patches of varying fineness, each containing a different number of triangles. Higher-fineness patches contain more triangles. These high-fineness patches are then reduced in number using a facet reduction algorithm, becoming lower-fineness patches. These lower-fineness patches serve as the smallest unit for calculating lighting. By selecting patches of different precision, the system can achieve a balance and control between computational load and visual quality in 3D scenes.

[0058] To control computational load and reduce noise computation common in ray tracing solutions, this method implements multi-layer data filtering and selects a technology suitable for the rasterization pipeline.

[0059] To achieve a computationally manageable noise reduction scheme, a multi-layer data filtering approach was designed. The number of filtering layers and the layers selected during implementation can be controlled. Utilizing multi-layer data for global illumination noise reduction is innovative. This method uses two layers of data filtering: the first layer of illumination data is based on indirect illumination divided into blocks as the smallest unit, and the second layer of illumination data is based on illumination probes placed in the 3D scene. The specific calculation process for the two layers of data is as follows:

[0060] The calculation process for the first layer of data:

[0061] Based on the model block data in the 3D scene, the material attribute data of the blocks are merged into a material map.

[0062] Based on the material texture from the previous step, direct lighting calculations are performed to generate a direct lighting map for the 3D scene.

[0063] Ray tracing is performed based on block data and directed distance field data to generate a visibility buffer.

[0064] Based on the visibility cache, the direct lighting map generated in the previous step is sampled to generate block-based indirect lighting for the upper hemisphere. The computational cost of sampling can be controlled, and temporal filtering can be used to reduce noise, further reducing the computational cost.

[0065] The calculation process for the second layer of data:

[0066] Place world coordinate lighting probes around the 3D model in the 3D scene. The lighting probes filter the indirect lighting of the surrounding blocks and process it into coefficients of spherical harmonic functions.

[0067] This application proposes a cloud-based global illumination baking method, the main process of which includes the following steps: the web front-end uploads a 3D model, the cloud server receives the 3D model and performs preprocessing to generate preprocessed data such as the directional distance field and patch segmentation of the 3D model, which are used in subsequent calculations.

[0068] The web frontend creates and modifies 3D scenes, imports uploaded 3D models, and sends relevant operations to the cloud system. The cloud system maintains the 3D scene and synchronizes the modifications. Global illumination baking is performed based on various factors, including the number of modifications, the number of web frontend users, and the location and orientation of their respective cameras.

[0069] Compared with existing technologies, this application has the following advantages: 1. The implementation scheme does not rely on the graphics card hardware to have ray tracing capabilities, and has wide adaptability; 2. The sub-processes of the entire calculation process are highly flexible. This high flexibility is reflected in the fact that, on the one hand, most of the computational load can be controlled, and on the other hand, the priority of the calculation can be controlled in a fine-grained manner, which can well cooperate with cloud computing in situations where the computational pressure fluctuates greatly under multiple users; 3. The calculation result is a lighting probe, which occupies a much smaller capacity than the traditional calculation result lighting map, making it suitable for cloud transmission. Combined with various progressive transmission schemes, it can well support the real-time computing needs of a large number of users.

[0070] Based on the same inventive concept, this application also provides a cloud baking global illumination system for implementing the cloud baking global illumination method described above. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations of the one or more cloud baking global illumination system embodiments provided below can be found in the limitations of the cloud baking global illumination method described above, and will not be repeated here.

[0071] In one exemplary embodiment, a cloud-based baking global illumination system is provided, comprising: a cloud system employing the cloud-based baking global illumination method described above.

[0072] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A cloud-based global illumination method for baking, characterized in that, The method operates in the cloud and includes: Receive 3D models of 3D scenes uploaded from multiple web front-ends through the cloud system; For each 3D model, generate a directed distance field for the entire scene; The 3D model is divided into blocks, and the material attribute data of all blocks are merged into a material attribute map; Calculate the shadow map for each light source in the 3D scene; Based on the shadow map and the material property map of each light source, the light map of each light source is calculated directly using lighting. Ray tracing is performed on each block using the directed distance field of the entire scene, generating a visibility cache for each block; however, the tracing does not focus on the lighting conditions. Based on the visibility cache of each block, the lightmap of each light source is sampled to generate hemispherical indirect lighting for each block; Encode the hemispherical indirect illumination of each block to obtain the spherical harmonic function coefficients; Place world coordinate lighting probes around the blocks of the 3D model; When encountering noise and overly bright spots caused by downsampling, the illumination probe is used to filter the hemispherical indirect illumination of each block to obtain the filtered spherical harmonic function coefficients and determine the illumination probe parameters for each 3D model. The lighting probe parameters of each 3D model are sent to its respective web frontend to render the global illumination of the 3D scene.

2. The cloud-based global illumination method for baking according to claim 1, characterized in that, For each 3D model, generate a directed distance field for the entire scene, specifically including: Voxelize the 3D model and calculate the directed distance field value and sign for each voxel in the 3D model; Voxelize the 3D scene; Based on the directed distance field value and sign of each voxel in the 3D model, the minimum distance merging algorithm is used to calculate the directed distance field value and sign of each voxel in the 3D scene. The directed distance field of the entire scene is constructed by combining the directed distance field values ​​of all voxels in the 3D scene with their positive and negative signs.

3. The cloud-based global illumination method for baking according to claim 1, characterized in that, The 3D model is divided into blocks, and the material attribute data of all blocks are merged into a material attribute map, specifically including: The mipmap method is used to generate blocks of different precision for the 3D model; Based on the material attribute data of each block, a texture stitching algorithm is used to obtain the material attribute texture.

4. The cloud-based global illumination method for baking according to claim 1, characterized in that, Calculate the shadow map for each light source in the 3D scene, specifically including: Iterate through all light sources in the 3D scene and calculate the shadow map for each light source using the rasterization pipeline.

5. The cloud-based global illumination method for baking according to claim 1, characterized in that, Based on the shadow map and material property map of each light source, calculate the light map of each light source, specifically including: The shadow map of each light source and the material property map are used as parameters to perform direct lighting calculations through the lighting model to obtain the corresponding lighting map; the lighting model includes the Phone lighting model or the Lambert lighting model.

6. The cloud-based global illumination method for baking according to claim 1, characterized in that, The sampling is Monte Carlo importance sampling.

7. The cloud-based global illumination method for baking according to claim 1, characterized in that, The encoding is a Fourier transform.

8. The cloud-based global illumination method for baking according to claim 1, characterized in that, Calculating shadow maps for each light source in a 3D scene, which previously included: Modify the 3D scene on the web front end and send the changes to the cloud system; Synchronous modification of 3D scenes in the cloud system.

9. The cloud-based global illumination method for baking according to claim 1, characterized in that, Global illumination for rendering 3D scenes specifically includes: In the web front-end, indirect lighting of the 3D model is rendered using lighting probe parameters and then merged with direct lighting to obtain the final global illumination.

10. A cloud-based global illumination system for baking, characterized in that, The cloud-based baking global illumination system includes a cloud system employing the cloud-based baking global illumination method according to any one of claims 1-9.

Images