A screen space reflection technology implementation method and system based on CesiumJS

Abstract

This invention proposes a method and system for implementing screen-space reflection technology based on CesiumJS, relating to the field of deferred rendering technology. The method extracts and modifies the rendering model fragment shader in CesiumJS; then extracts and inputs the color data and roughness data from the initial model into the new shader to obtain a rendering shader; then uses the rendering shader to perform off-screen rendering on the initial model to obtain a rendered roughtexture; then extracts the roughness information; then uses screen-space reflection technology to perform reflection processing; and finally, synthesizes the reflected image with the initial model. During the rendering of the scene model, the glossiness of the relevant model material is rendered into an offline buffer using the rendering model drawcall, allowing the glossiness texture map to be used in the post-processing shader, thereby achieving advanced special effects related to the model material in deferred rendering.

Description

Technical Field

[0001] The present invention relates to the field of delayed rendering technology, and in particular to a method and system for implementing screen space reflection technology based on CesiumJS. Background Art

[0002] CesiumJS is an open source JavaScript library for creating world-class 3D globes and maps with the best performance, accuracy, visual quality, and ease of use. Developers across industries, from aerospace to smart cities to drones, use CesiumJS to create interactive web applications that share dynamic geospatial data.

[0003] Currently, the official website of the CesiumJS open source library provides many examples, including post-processing effects in deferred rendering technology to improve rendering quality, such as rain, snow, fog and daily effects. However, these effects are aimed at improving the visualization of the 3D scene environment, and the rendering effect of the 3D model itself is not ideal.

[0004] CesiumJS provides users with an Application Programming Interface (API) for post-processing. However, this API only retrieves the scene texture and depth map of the real-time rendering, not the model's original material information. This prevents advanced material-related effects in deferred rendering, such as screen-space reflections. These effects are one of the best ways to improve rendering quality without sacrificing efficiency. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for implementing screen space reflection technology based on CesiumJS, so as to improve the problem in the prior art that the original material related information of the model cannot be obtained in the CesiumJS API, resulting in the inability to realize advanced special effects related to the model material in deferred rendering.

[0006] In a first aspect, an embodiment of the present application provides a method for implementing screen space reflection technology based on CesiumJS, comprising the following steps:

[0007] Get the initial model;

[0008] Extract and modify the rendering model fragment shader in CesiumJS to obtain a new shader;

[0009] Extract and input the color data and roughness data in the initial model into the new shader to obtain a rendering shader;

[0010] Use the rendering shader to render the initial model off-screen and obtain the rendered roughTexture;

[0011] Extract the roughness information from the rendered roughTexture;

[0012] According to the roughness information in the rendered roughTexture, the initial model is reflected using the screen space reflection technology to obtain a reflected image;

[0013] The reflected image is synthesized with the initial model to obtain a new image.

[0014] In the above implementation process, the initial model is obtained; then the rendering model fragment shader in CesiumJS is extracted and modified to obtain a new shader. When using the CesiumJS three-dimensional geographic information library to load massive models, not only the performance is guaranteed, but also the visual effect can be improved; then the color data and roughness data in the initial model are extracted and input into the new shader to obtain a rendering shader, thereby enabling the GL_EXT_draw_buffers extension in the 3dtiles format outer shader in CesiumJS, and writing the model PBR material information of the fragment into the color attachment point in the GlobeDepth offline frame buffer in the shader; then the rendering shader is used to render the initial model off-screen to obtain the rendered roughTexture; then the rendered roughTexture is extracted e, thereby providing the roughness of the model and other related material information in the deferred rendering stage, which can fully utilize the advantages of screen space rendering technology and reduce the performance problems caused by the real-time rendering stage; then the initial model is reflected by the screen space reflection technology according to the roughness information in the rendered roughTexture to obtain a reflected image; finally, the reflected image is synthesized with the initial model to obtain a new image, so that without increasing the drawcall, the glossiness of the relevant model material is rendered into the offline buffer with the help of the rendering model drawcall during the rendering process of the scene model, and then the material gloss texture that has been rendered into the offline buffer is seamlessly connected with the post-processing stage API of CesiumJS, so that the glossiness texture map can be used in the post-processing shader, thereby realizing advanced special effects related to the model material in deferred rendering.

[0015] Based on the first aspect, in some embodiments of the present invention, extracting and modifying the rendering model fragment shader in CesiumJS to obtain a new shader includes the following steps:

[0016] Extract the code in the tile_main function of the rendering model fragment shader in CesiumJS into a Glsl file;

[0017] Enable the GLTF format extension KHR_techniques_webgl and add the URL of the GLSL file in the extension field to get the format external shader as a new shader.

[0018] Based on the first aspect, in some embodiments of the present invention, the step of extracting and inputting color data and roughness data from the initial model into a new shader to obtain a rendering shader includes the following steps:

[0019] Extracting the color data in the initial model and the roughness data of the initial model represented by the current slice;

[0020] Enable the GL_EXT_draw_buffers extension in the new shader code to allow writing data to multiple color attachment points of the frame buffer in the new shader;

[0021] Write the color data of the initial model to unit 1 in the frame buffer and put it into the first color attachment point as the colorTexture of the deferred rendering stage;

[0022] Write the roughness data of the model represented by the current piece to the second unit in the frame buffer, and put it into the second color attachment point as the roughTexture of the deferred rendering stage to obtain the rendering shader.

[0023] Based on the first aspect, in some embodiments of the present invention, the step of performing off-screen rendering on the initial model using a rendering shader to obtain a rendered roughTexture includes the following steps:

[0024] Use the rendering shader to render the initial model off-screen to obtain the original rendering scene;

[0025] Add a second color attachment point to the off-screen frame buffer of the original rendered scene to hold the rendered roughTexture.

[0026] Based on the first aspect, in some embodiments of the present invention, the step of extracting roughness information from the rendered roughTexture includes the following steps:

[0027] Extract the material information from the rendered roughTexture;

[0028] The material information in the rendered roughTexture is sampled to obtain the roughness information of each fragment.

[0029] Based on the first aspect, in some embodiments of the present invention, the step of performing reflection processing on the initial model using screen space reflection technology according to the roughness information in the rendered roughTexture to obtain the reflected image includes the following steps:

[0030] Use screen space ray tracing technology to calculate the reflection of the initial model and obtain the reflection color intensity;

[0031] The reflection color intensity is mixed with the roughness information in the rendered roughTexture to obtain the reflection image.

[0032] Based on the first aspect, in some embodiments of the present invention, the following steps are further included:

[0033] Get each pixel in screen space;

[0034] Calculate the difference vector between each pixel as the center pixel and its four horizontal and vertical neighboring pixels in three-dimensional coordinates;

[0035] Select two differential vectors with the smallest vector modulus among the vertical and horizontal vectors from among the multiple differential vectors;

[0036] Perform a cross product calculation on the two difference vectors with the smallest vector modulus among the vertical and horizontal vectors to obtain a new normal.

[0037] In a second aspect, an embodiment of the present application provides a system for implementing screen space reflection technology based on CesiumJS, including:

[0038] An initial model acquisition module is used to obtain an initial model;

[0039] The shader modification module is used to extract and modify the rendering model fragment shader in CesiumJS to obtain a new shader;

[0040] A rendering shader module is used to extract and input color data and roughness data in the initial model into a new shader to obtain a rendering shader;

[0041] The off-screen rendering module is used to perform off-screen rendering on the initial model using a rendering shader to obtain a roughTexture after rendering;

[0042] Roughness information extraction module, used to extract roughness information from the rendered roughTexture;

[0043] The screen space reflection module is used to reflect the initial model using the screen space reflection technology according to the roughness information in the rendered roughTexture to obtain a reflected image;

[0044] The image synthesis module is used to synthesize the reflected image with the initial model to obtain a new image.

[0045] In the above implementation process, the initial model is obtained through the initial model acquisition module; then the shader modification module extracts and modifies the rendering model fragment shader in CesiumJS to obtain a new shader. When using the CesiumJS three-dimensional geographic information library to load massive models, not only is performance guaranteed, but also visual effects can be improved; then the rendering shader module extracts and inputs the color data and roughness data in the initial model into the new shader to obtain a rendering shader, thereby enabling the GL_EXT_draw_buffers extension in the 3dtiles format outer band Shader in CesiumJS, and writing the model PBR material information of the fragment into the color attachment point in the GlobeDepth offline frame buffer in the Shader; then the off-screen rendering module uses the rendering shader to perform off-screen rendering on the initial model to obtain the rendered roughTexture; then the roughness information extraction module extracts the rendered The roughness information in the roughTexture is used to provide the roughness and other related material information of the model in the deferred rendering stage, which can fully utilize the advantages of screen space rendering technology and reduce the performance problems caused by the real-time rendering stage; then the screen space reflection module uses the screen space reflection technology to reflect the initial model according to the roughness information in the rendered roughTexture to obtain a reflected image; finally, the image synthesis module synthesizes the reflected image with the initial model to obtain a new image, so that without increasing the drawcall, the glossiness of the relevant model material is rendered into the offline buffer with the help of the rendering model drawcall during the rendering process of the scene model, and then the material gloss texture that has been rendered into the offline buffer is seamlessly connected with the post-processing stage API of CesiumJS, so that the glossiness texture map can be used in the post-processing shader, thereby realizing advanced special effects related to the model material in deferred rendering.

[0046] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory for storing one or more programs and a processor. When the one or more programs are executed by the processor, the method according to any one of the first aspects is implemented.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects above.

[0048] The embodiments of the present invention have at least the following advantages or beneficial effects:

[0049] The embodiment of the present invention provides a method and system for implementing screen space reflection technology based on CesiumJS, which obtains an initial model; then extracts and modifies the rendering model fragment shader in CesiumJS, and uses the CesiumJS three-dimensional geographic information library to load massive models, while not only ensuring performance but also improving visual effects; then extracts and inputs the color data and roughness data in the initial model into a new shader, thereby enabling the GL_EXT_draw_buffers extension in the 3dtiles format outer band Shader in CesiumJS, and writing the model PBR material information of the fragment into the color attachment point in the GlobeDepth offline frame buffer in the Shader; then uses the rendering shader to perform off-screen rendering on the initial model to obtain the rendered roughTexture; then extracts the rendered roughTexture The roughness information in the Texture is used to provide the model's roughness and other related material information during the deferred rendering phase, fully leveraging the advantages of screen-space rendering technology while reducing performance issues caused by the real-time rendering phase. The initial model is then reflected using screen-space reflection technology based on the roughness information in the rendered roughTexture to obtain a reflected image. Finally, the reflected image is synthesized with the initial model to obtain a new image. This allows the glossiness of the relevant model material to be rendered into the offline buffer with the help of the rendering model drawcall during the rendering of the scene model without increasing the drawcall. The material gloss texture already rendered into the offline buffer is then seamlessly connected to the CesiumJS post-processing API, allowing the glossiness texture map to be used in the post-processing shader, thus enabling the implementation of advanced special effects related to the model material in deferred rendering. By reconstructing normals in screen space, the large amount of memory space occupied by normals in the Gbuffer is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A flowchart of a method for implementing screen space reflection technology based on CesiumJS provided in an embodiment of the present invention;

[0052] Figure 2 A structural block diagram of a system for implementing screen space reflection technology based on CesiumJS provided in an embodiment of the present invention;

[0053] Figure 3 This is a structural block diagram of an electronic device provided by an embodiment of the present invention.

[0054] Icons: 110 - initial model acquisition module; 120 - shader modification module; 130 - rendering shader module; 140 - off-screen rendering module; 150 - roughness information extraction module; 160 - screen space reflection module; 170 - image synthesis module; 101 - memory; 102 - processor; 103 - communication interface. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0057] Example

[0058] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features thereof may be combined with each other.

[0059] Please check Figure 1 , Figure 1 A flowchart of a method for implementing screen space reflection technology based on CesiumJS provided in an embodiment of the present invention. The method for implementing screen space reflection technology based on CesiumJS includes the following steps:

[0060] Step S110: obtaining an initial model; the initial model refers to a scene model to be rendered, including scene information, roughness and other related material information, and the like.

[0061] Step S120: extract and modify the rendering model fragment shader in CesiumJS to obtain a new shader; the new shader obtained by the above modification can be an external shader in the format, or the shader can be modified in the source code, wherein the external shader can better customize the rendering.

[0062] The above steps of using the formatted shader as a new shader include the following:

[0063] First, extract the code in the tile_main function of the rendering model fragment shader in CesiumJS into an OpenGL Shading Language (Glsl) file;

[0064] Next, enable the GLTF format extension KHR_techniques_webgl and add the URL of the GLSL file to the extension field to obtain the format's external shader as a new shader. See the source code for the generateTechnique function in the processPbrMaterials.js file. The GLTF format is essentially a JSON file that describes the contents of the entire 3D scene. It includes a scene graph that describes the scene structure.

[0065] Step S130: extracting the color data and roughness data in the initial model and inputting them into a new shader to obtain a rendering shader. The above-mentioned obtaining of the rendering shader specifically includes the following steps:

[0066] First, extract the color data in the initial model and the roughness data of the initial model represented by the current slice;

[0067] Then, enable the GL_EXT_draw_buffers extension in the new shader code to allow writing data to multiple color attachment points of the frame buffer in the new shader;

[0068] Then, the color data in the initial model is written to unit 1 in the frame buffer and placed into the first color attachment point as the colorTexture of the deferred rendering stage; the above colorTexture refers to the color texture map.

[0069] Finally, the roughness data for the model represented by the current slice is written to unit 2 in the frame buffer and placed into the second color attachment point, which serves as the roughTexture for the deferred rendering phase to obtain the rendering shader. The roughTexture in the deferred rendering phase is used to calculate reflectance. In the Cesium3DTileBatchTable.js source code file, all gl_FragColor variables are modified to gl_FragData[0] to prevent the two built-in variables from coexisting when blending colors.

[0070] Step S140: Use a rendering shader to perform off-screen rendering on the initial model to obtain a rendered roughTexture; the above-mentioned off-screen rendering to obtain the rendered roughTexture specifically includes the following steps:

[0071] First, the initial model is rendered off-screen using a rendering shader to obtain the original rendering scene;

[0072] Then, add a second color attachment point to the off-screen frame buffer of the original rendered scene to save the rendered roughTexture. In the source code GlobeDepth.js file, create a roughTexture material object and add a second color attachment point to the off-screen frame buffer of the original rendered scene to save the rendered roughTexture.

[0073] Step S150: extract the roughness information from the rendered roughTexture; add a roughTexture material code segment to the source code of the updated rendering resource in the deferred rendering phase, so that the roughness of each pixel can be obtained inside the deferred rendering shader. The above step of extracting roughness information includes the following steps:

[0074] First, extract the material information from the rendered roughTexture; get the rendered roughTexture in the usePostProcess conditional statement of the resolveFramebuffers function in the Scene.js file, pass the material parameters to the subsequent postProcess.execute function, and modify the stage._uniformMap function of the createUniformMap in the PostProcessStage.js file to add the roughnessTexture variable. This makes it possible to obtain the roughnessTexture material inside the deferred rendering shader.

[0075] Then, the material information in the rendered roughTexture is sampled to obtain the roughness information of each fragment. The roughness value of each fragment is obtained by sampling. The above sampling can be implemented using existing technology and will not be described again.

[0076] Step S160: Based on the roughness information in the roughTexture after rendering, the initial model is reflected using the screen space reflection technology to obtain a reflected image; the screen space reflection technology can be a ray tracing algorithm using deferred rendering technology to obtain reflected light, or a ray tracing algorithm using real-time rendering to obtain reflected light. Among them, the efficiency of the ray tracing algorithm using deferred rendering technology is not affected by the size of the scene, and the performance is better. The process of using the ray tracing algorithm using deferred rendering technology for emission processing can be to write the createScreenSpaceReflectionStage function in the file PostProcessStageLibrary.js, and then perform the following steps:

[0077] First, screen-space ray tracing technology is used to calculate the reflection of the initial model to obtain the reflection color intensity; the reflection effect can be obtained by the reflection color intensity. The above-mentioned screen-space ray tracing technology is existing technology and will not be described in detail here.

[0078] The reflected color intensity is then mixed with the roughness information in the rendered roughTexture to produce the reflected image. To reflect the reflection intensity of different objects, the calculated reflected color intensity is mixed with the roughness sampled from the roughTexture, so that the rougher the object, the lighter the color of the surrounding objects will be reflected, and vice versa.

[0079] Step S170: The reflected image is synthesized with the initial model to obtain a new image. This synthesis can be achieved using the techniques described above and will not be further described here. To improve the quality of the synthesis, the reflected image can also be Gaussian blurred; then synthesized with the initial model. This Gaussian blurring process is intended to reduce noise in the reflected image, thereby improving synthesis quality. Gaussian blurring is a state of the art and will not be further described here.

[0080] In the above implementation process, the initial model is obtained; then the rendering model fragment shader in CesiumJS is extracted and modified to obtain a new shader. When using the CesiumJS three-dimensional geographic information library to load massive models, not only the performance is guaranteed, but also the visual effect can be improved; then the color data and roughness data in the initial model are extracted and input into the new shader to obtain a rendering shader, thereby enabling the GL_EXT_draw_buffers extension in the 3dtiles format outer shader in CesiumJS, and writing the model PBR material information of the fragment into the color attachment point in the GlobeDepth offline frame buffer in the shader; then the rendering shader is used to render the initial model off-screen to obtain the rendered roughTexture; then the rendered roughTexture is extracted e, thereby providing the roughness of the model and other related material information in the deferred rendering stage, which can fully utilize the advantages of screen space rendering technology and reduce the performance problems caused by the real-time rendering stage; then the initial model is reflected by the screen space reflection technology according to the roughness information in the rendered roughTexture to obtain a reflected image; finally, the reflected image is synthesized with the initial model to obtain a new image, so that without increasing the drawcall, the glossiness of the relevant model material is rendered into the offline buffer with the help of the rendering model drawcall during the rendering process of the scene model, and then the material gloss texture that has been rendered into the offline buffer is seamlessly connected with the post-processing stage API of CesiumJS, so that the glossiness texture map can be used in the post-processing shader, thereby realizing advanced special effects related to the model material in deferred rendering.

[0081] The surface normal vector can also be reconstructed in the post-rendering processing stage, which specifically includes the following steps:

[0082] First, each pixel in the screen space is obtained; the above-mentioned each pixel can be directly obtained from the system.

[0083] Then, the difference vectors between the central pixel and the four horizontal and vertical neighboring pixels in the three-dimensional coordinates are calculated for each pixel;

[0084] Then, two differential vectors with the smallest vector modulus among the vertical and horizontal vectors are selected from the multiple differential vectors. The above screening first requires respectively calculating the vector modulus of the vertical and horizontal vectors, and then selecting the two differential vectors with the smallest vector modulus.

[0085] Finally, perform a cross product of the two difference vectors with the smallest vector modulus between the vertical and horizontal vectors to obtain the new normal. The vector obtained by the cross product is used as the new normal.

[0086] In the above implementation process, each pixel in the screen space is obtained, and then the differential vectors between the center pixel point and the four horizontal and vertical neighboring pixel points in the three-dimensional coordinates of each pixel are calculated respectively; then the two differential vectors with the smallest vector modulus in the vertical and horizontal vectors are screened out from the multiple differential vectors; finally, the two differential vectors with the smallest vector modulus in the vertical and horizontal vectors are cross-producted to obtain a new normal, thereby realizing normal reconstruction, reducing the large amount of memory space occupied by normals in the Gbuffer, and improving the utilization of memory space.

[0087] Based on the same inventive concept, the present invention also proposes a screen space reflection technology implementation system based on CesiumJS, please see Figure 2 , Figure 2 This is a block diagram of a system structure for implementing screen space reflection technology based on CesiumJS provided in an embodiment of the present invention. The system for implementing screen space reflection technology based on CesiumJS includes:

[0088] An initial model acquisition module 110 is used to acquire an initial model;

[0089] The shader modification module 120 is used to extract and modify the rendering model fragment shader in CesiumJS to obtain a new shader;

[0090] A rendering shader module 130 is used to extract and input the color data and roughness data in the initial model into a new shader to obtain a rendering shader;

[0091] The off-screen rendering module 140 is used to perform off-screen rendering on the initial model using a rendering shader to obtain a rendered roughTexture;

[0092] The roughness information extraction module 150 is used to extract the roughness information in the rendered roughTexture;

[0093] A screen space reflection module 160 is configured to perform reflection processing on the initial model using a screen space reflection technique according to the roughness information in the rendered roughTexture to obtain a reflected image;

[0094] The image synthesis module 170 is used to synthesize the reflected image with the initial model to obtain a new image.

[0095] In the above implementation process, the initial model is obtained through the initial model acquisition module 110; then the shader modification module 120 extracts and modifies the rendering model fragment shader in CesiumJS to obtain a new shader. When using the CesiumJS three-dimensional geographic information library to load massive models, not only is the performance guaranteed, but also the visual effect can be improved; then the rendering shader module 130 extracts and inputs the color data and roughness data in the initial model into the new shader to obtain a rendering shader, thereby enabling the GL_EXT_draw_buffers extension in the 3dtiles format out-of-band Shader in CesiumJS, and writing the model PBR material information of the fragment into the color attachment point in the GlobeDepth offline frame buffer in the Shader; then the off-screen rendering module 140 uses the rendering shader to perform off-screen rendering on the initial model to obtain a rendered roughTexture; then the roughness information extraction module 150 extracts the roughness information in the rendered roughTexture , thereby providing the roughness and other related material information of the model in the deferred rendering stage, which can fully utilize the advantages of screen space rendering technology and reduce the performance problems caused by the real-time rendering stage; then the screen space reflection module 160 uses screen space reflection technology to reflect the initial model according to the roughness information in the rendered roughTexture to obtain a reflected image; finally, the image synthesis module 170 synthesizes the reflected image with the initial model to obtain a new image, so that without increasing the drawcall, the glossiness of the relevant model material is rendered into the offline buffer with the help of the rendering model drawcall during the rendering process of the scene model, and then the material gloss texture that has been rendered into the offline buffer is seamlessly connected with the post-processing stage API of CesiumJS, so that the glossiness texture map can be used in the post-processing shader, thereby realizing advanced special effects related to the model material in deferred rendering.

[0096] The shader modification module 120 includes:

[0097] Function extraction unit, used to extract the code in the tile_main function of the rendering model fragment shader in CesiumJS into the Glsl file;

[0098] The format extension unit is used to enable the GLTF format extension KHR_techniques_webgl, add the URL address of the Glsl file in the extension field, and obtain the format-external shader as a new shader.

[0099] The rendering shader module 130 includes:

[0100] A data extraction unit, configured to extract color data from the initial model and roughness data of the initial model represented by the current slice;

[0101] Extension enable unit, used to enable GL_EXT_draw_buffers extension in the new shader code, allowing the new shader to write data to multiple color attachment points of the frame buffer;

[0102] The first color attachment point unit is used to write the color data of the initial model to unit 1 in the frame buffer and put it into the first color attachment point as the colorTexture of the deferred rendering stage;

[0103] The second color attachment point unit is used to write the roughness data of the model represented by the current piece into the second unit in the frame buffer, and put it into the second color attachment point as the roughTexture of the deferred rendering stage to obtain the rendering shader.

[0104] The off-screen rendering module 140 includes:

[0105] The original rendering unit is used to perform off-screen rendering on the initial model using a rendering shader to obtain an original rendering scene;

[0106] The color attachment point adding unit is used to add a second color attachment point in the off-screen frame buffer of the original rendered scene to save the rendered roughTexture.

[0107] The roughness information extraction module 150 includes:

[0108] Material extraction unit, used to extract material information from the rendered roughTexture;

[0109] The sampling unit is used to sample the material information in the rendered roughTexture to obtain the roughness information of each fragment.

[0110] The screen space reflection module 160 includes:

[0111] A reflection calculation unit is used to perform reflection calculation on the initial model using screen space ray tracing technology to obtain the reflection color intensity;

[0112] The mixing unit is used to mix the reflection color intensity with the roughness information in the rendered roughTexture to obtain the reflection image.

[0113] Among them, also include:

[0114] Pixel acquisition module, used to obtain each pixel in the screen space;

[0115] A differential vector obtaining module is used to obtain the differential vectors between each pixel as the center pixel and its four horizontal and vertical neighboring pixels in three-dimensional coordinates;

[0116] A vector screening module is used to screen out two differential vectors with the smallest vector modulus among the vertical and horizontal vectors from multiple differential vectors;

[0117] The cross product calculation module is used to perform a cross product calculation on two differential vectors with the smallest vector modulus in the vertical and horizontal vectors to obtain a new normal.

[0118] See also Figure 3 , Figure 3 A schematic structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device includes a memory 101, a processor 102 and a communication interface 103, and the memory 101, the processor 102 and the communication interface 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, such as a program instruction / module corresponding to a screen space reflection technology implementation system based on CesiumJS provided in an embodiment of the present application, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.

[0119] Among them, the memory 101 can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0120] The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0121] I understand. Figure 3 The structure shown is only for illustration, and the electronic device may also include Figure 3 More or fewer components than shown, or with Figure 3 Different configurations shown. Figure 3 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0122] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0123] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0124] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0125] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0126] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A method for implementing screen space reflection technology based on CesiumJS, characterized in that: The following steps are involved: Get the initial model; Extract and modify the rendering model fragment shader in CesiumJS to obtain a new shader; Extracting and inputting color data and roughness data from the initial model into a new shader to obtain a rendering shader, including: extracting color data from the initial model and roughness data of the initial model represented by the current slice; enabling the GL_EXT_draw_buffers extension in the code of the new shader to allow writing data to multiple color attachment points of the frame buffer in the new shader; writing the color data from the initial model to unit No. 1 in the frame buffer and placing it into the first color attachment point as a colorTexture for a deferred rendering stage; writing the roughness data of the model represented by the current slice to unit No. 2 in the frame buffer and placing it into the second color attachment point as a roughTexture for a deferred rendering stage to obtain a rendering shader; Performing off-screen rendering on the initial model using a rendering shader to obtain a rendered roughTexture, including: performing off-screen rendering on the initial model using a rendering shader to obtain an original rendered scene; adding a second color attachment point in an off-screen frame buffer of the original rendered scene to store the rendered roughTexture; Extract the roughness information from the rendered roughTexture; According to the roughness information in the rendered roughTexture, the initial model is reflected using the screen space reflection technology to obtain a reflected image; The reflected image is synthesized with the initial model to obtain a new image.

2. The method for implementing screen space reflection technology based on CesiumJS according to claim 1, characterized in that: The step of extracting and modifying the rendering model fragment shader in CesiumJS to obtain a new shader includes the following steps: Extract the code in the tile_main function of the rendering model fragment shader in CesiumJS into a Glsl file; Enable the GLTF format extension KHR_techniques_webgl and add the URL of the GLSL file in the extension field to get the format external shader as a new shader.

3. The method for implementing screen space reflection technology based on CesiumJS according to claim 1, characterized in that: The step of extracting roughness information from the rendered roughTexture comprises the following steps: Extract the material information from the rendered roughTexture; The material information in the rendered roughTexture is sampled to obtain the roughness information of each fragment.

4. The method for implementing screen space reflection technology based on CesiumJS according to claim 1, characterized in that: The step of performing reflection processing on the initial model using screen space reflection technology according to the roughness information in the rendered roughTexture to obtain a reflected image comprises the following steps: Use screen space ray tracing technology to calculate the reflection of the initial model and obtain the reflection color intensity; The reflection color intensity is mixed with the roughness information in the rendered roughTexture to obtain the reflection image.

5. The method for implementing screen space reflection technology based on CesiumJS according to claim 1, characterized in that: The following steps are also included: Get each pixel in screen space; Calculate the difference vector between each pixel as the center pixel and its four horizontal and vertical neighboring pixels in three-dimensional coordinates; Select two differential vectors with the smallest vector modulus among the vertical and horizontal vectors from among the multiple differential vectors; Perform a cross product calculation on the two difference vectors with the smallest vector modulus among the vertical and horizontal vectors to obtain a new normal.

6. A screen space reflection technology implementation system based on CesiumJS, characterized in that: include: An initial model acquisition module is used to obtain an initial model; The shader modification module is used to extract and modify the rendering model fragment shader in CesiumJS to obtain a new shader; A rendering shader module is used to extract and input color data and roughness data in the initial model into a new shader to obtain a rendering shader, including: extracting color data in the initial model and roughness data of the initial model represented by the current slice; enabling the GL_EXT_draw_buffers extension in the code of the new shader to allow the new shader to write data to multiple color attachment points of the frame buffer; writing the color data of the initial model to unit No. 1 in the frame buffer and placing it in the first color attachment point as the colorTexture of the deferred rendering stage; writing the roughness data of the model represented by the current slice to unit No. 2 in the frame buffer and placing it in the second color attachment point as the roughTexture of the deferred rendering stage to obtain a rendering shader; The off-screen rendering module is used to perform off-screen rendering on the initial model using a rendering shader to obtain a rendered roughTexture, including: performing off-screen rendering on the initial model using a rendering shader to obtain an original rendered scene; adding a second color attachment point in an off-screen frame buffer of the original rendered scene to save the rendered roughTexture; Roughness information extraction module, used to extract roughness information from the rendered roughTexture; The screen space reflection module is used to reflect the initial model using the screen space reflection technology according to the roughness information in the rendered roughTexture to obtain a reflected image; The image synthesis module is used to synthesize the reflected image with the initial model to obtain a new image.

7. An electronic device, characterized in that: include: a memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images