Method and device for rendering high dynamic range content

Abstract

The invention relates to the technical field of image rendering, and discloses a method and device for rendering high dynamic range content, and the method comprises the steps: obtaining a triangular mesh representing a terrain, a triangular mesh representing a three-dimensional object, a corresponding terrain texture, and a corresponding three-dimensional object texture; rendering the scene model; rendering the celestial body and the sky atmosphere to obtain scene textures; performing downsampling operation on the scene texture, and calculating the average brightness of the current frame; extracting pixel points of which the brightness is higher than a preset threshold value from the downsampled scene texture, and generating an initial floodlight texture; performing down-sampling operation, Gaussian blur operation and up-sampling operation on the initial floodlight texture in sequence to obtain a target floodlight texture; and according to the scene texture and the target floodlight texture, combining the average brightness of the current frame, synthesizing an exposure result of high-dynamic rendering, performing tone mapping on the exposure result, and displaying the exposure result on a target display device. The dynamic range of the color gamut and the brightness of three-dimensional scene rendering is expanded, and the rendering effect is improved.

Description

Technical Field

[0001] The present application relates to the field of image rendering technology, and in particular to a method and device for rendering high dynamic range content. Background Art

[0002] Currently, on the Web, most 3D scene renderings are rendered in SDR (Standard Dynamic Range), and then the rendered SDR content is directly output to the SDR display for display. However, since the color gamut and brightness dynamic range that the normal human eye can accept are much larger than the dynamic range presented by the SDR content, the SDR content rendered by the existing technology cannot highly simulate the real images seen by the human eye, and it is difficult to meet the requirements of high-realistic rendering.

[0003] With respect to the above-mentioned related technologies, the inventors have found that the existing three-dimensional scene rendering methods have the problems of small dynamic range of color gamut and brightness, and cannot highly simulate the real images seen by the human eye, resulting in poor rendering effect. Summary of the invention

[0004] In order to expand the color gamut and dynamic range of brightness of three-dimensional scene rendering and improve the rendering effect, the present application provides a method and device for rendering high dynamic range content.

[0005] In a first aspect, the present application provides a method for rendering high dynamic range content.

[0006] This application is achieved through the following technical solutions:

[0007] A method for rendering high dynamic range content, comprising the following steps,

[0008] Acquire a triangular mesh representing the terrain and a triangular mesh representing the three-dimensional object, as well as corresponding terrain textures and three-dimensional object textures, to obtain a scene model;

[0009] Rendering the scene model, and saving the scene model rendering result in a half-floating point data type into a two-dimensional texture of the scene model;

[0010] Rendering the celestial body and the sky atmosphere, and superimposing the rendering results of the celestial body and the sky atmosphere on the two-dimensional texture of the scene model with half-floating point numbers to obtain a scene texture;

[0011] Performing a downsampling operation on the scene texture, calculating an expected average brightness, and calculating a target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness according to the expected average brightness, the time interval of the current frame, and the average brightness of the previous frame, to obtain an average brightness of the current frame;

[0012] Extracting pixel points with brightness higher than a preset threshold from the downsampled scene texture to generate an initial flood texture;

[0013] Performing a downsampling operation and a Gaussian blur operation on the initial floodlight texture in sequence to obtain an optimized texture, performing an upsampling operation on the optimized texture in sequence, and then superimposing the current upsampling result on the previous floodlight texture of the optimized texture to obtain a target floodlight texture;

[0014] An exposure result of high dynamic rendering is synthesized according to the scene texture and the target floodlight texture in combination with the average brightness of the current frame, and the exposure result is tone mapped and displayed on a target display device.

[0015] In a preferred example, the present application may be further configured as follows: performing a downsampling operation on the scene texture, the step of calculating the expected average brightness includes:

[0016] Performing a downsampling operation on the scene texture to generate textures with widths of 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original width and height respectively;

[0017] Convert the RGB value corresponding to the 1 / 32 texture to the pixel brightness value;

[0018] Calculate the average brightness of all pixel brightness values ​​in the 1 / 32 texture to get the expected average brightness.

[0019] In a preferred example, the present application can be further configured as follows: according to the desired average brightness, the time interval of the current frame and the average brightness of the previous frame, the step of calculating the target average brightness for smooth transition from the average brightness of the previous frame to the desired average brightness includes:

[0020] Calculate the difference between the expected average brightness and the average brightness of the previous frame;

[0021] The difference is divided by the time interval of the current frame to obtain a target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness.

[0022] In a preferred example, the present application may be further configured as follows: the step of synthesizing the exposure result of high dynamic rendering according to the scene texture and the target floodlight texture in combination with the average brightness of the current frame includes:

[0023] Adding the scene texture and the target flood texture to obtain a superposition result;

[0024] The superposition result is divided by the target average brightness to obtain the exposure result of high dynamic rendering.

[0025] In a preferred example, the present application can be further configured as follows: performing a downsampling operation and a Gaussian blur operation on the initial floodlight texture in sequence to obtain an optimized texture, performing an upsampling operation on the optimized texture in sequence, and then superimposing the current upsampling result on the previous floodlight texture of the optimized texture. The step of obtaining the target floodlight texture includes:

[0026] Downsampling the initial flood texture to generate flood input textures with widths of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original width and height, respectively;

[0027] Gaussian blur each floodlight input texture separately;

[0028] Upsample the Gaussian blurred 1 / 32 bloom input texture, add the upsampled result to the 1 / 16 bloom input texture, and update the Gaussian blurred 1 / 16 bloom input texture.

[0029] Upsample the Gaussian blurred 1 / 16 bloom input texture, add the upsampled result to the 1 / 8 bloom input texture, and update the Gaussian blurred 1 / 8 bloom input texture.

[0030] Upsample the Gaussian blurred 1 / 8 bloom input texture, add the upsampled result to the 1 / 4 bloom input texture, and update the Gaussian blurred 1 / 4 bloom input texture.

[0031] The Gaussian blurred 1 / 4 floodlight input texture is upsampled, and the upsampled result is superimposed on the initial floodlight texture to obtain the target floodlight texture.

[0032] In a preferred example, the present application may be further configured as follows: the step of rendering the triangular mesh of the three-dimensional object includes:

[0033] Calling a graphics API interface to rasterize the triangular mesh of the three-dimensional object into two-dimensional pixel points of the three-dimensional object;

[0034] By using physically based rendering technology, based on the two-dimensional pixel points of the three-dimensional object, the corresponding high dynamic range RGB color value of the three-dimensional object is calculated to obtain the rendering result of the three-dimensional object.

[0035] In a second aspect, the present application provides a device for rendering high dynamic range content.

[0036] This application is achieved through the following technical solutions:

[0037] A device for rendering high dynamic range content, comprising:

[0038] A data module is used to obtain a triangular mesh representing a terrain and a triangular mesh representing a three-dimensional object, as well as corresponding terrain textures and three-dimensional object textures, to obtain a scene model;

[0039] A two-dimensional texture optimization module, used for rendering the scene model, and saving the scene model rendering result into the two-dimensional texture of the scene model in a half-floating point data type;

[0040] A scene texture module is used to render celestial bodies and sky atmosphere, and superimpose the rendering results of the celestial bodies and sky atmosphere on the two-dimensional texture of the scene model in the form of half floating point numbers to obtain a scene texture;

[0041] an exposure optimization module, configured to perform a downsampling operation on the scene texture, calculate an expected average brightness, and calculate a target average brightness for a smooth transition from the average brightness of the previous frame to the expected average brightness based on the expected average brightness, a time interval of a current frame, and an average brightness of a previous frame, to obtain an average brightness of a current frame;

[0042] A flood initialization module is used to extract pixels with brightness higher than a preset threshold from the downsampled scene texture to generate an initial flood texture;

[0043] A flood light texture module is used to sequentially perform a downsampling operation and a Gaussian blur operation on the initial flood light texture to obtain an optimized texture, sequentially perform an upsampling operation on the optimized texture, and then superimpose the current upsampling result on the previous flood light texture of the optimized texture to obtain a target flood light texture;

[0044] The display module is used to synthesize the exposure result of high dynamic rendering according to the scene texture and the target floodlight texture in combination with the average brightness of the current frame, and display the exposure result on the target display device after tone mapping.

[0045] In a third aspect, the present application provides a computer device.

[0046] This application is achieved through the following technical solutions:

[0047] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned methods for rendering high dynamic range content when executing the computer program.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium.

[0049] This application is achieved through the following technical solutions:

[0050] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the above methods for rendering high dynamic range content.

[0051] In a fifth aspect, the present application provides a computer program product.

[0052] This application is achieved through the following technical solutions:

[0053] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the above methods for rendering high dynamic range content.

[0054] In summary, compared with the prior art, the technical solution provided by this application has at least the following beneficial effects:

[0055] Acquire a triangular mesh representing terrain and a triangular mesh representing three-dimensional objects, and the corresponding terrain texture and three-dimensional object texture, so as to obtain a scene model for large-scale geographical environment rendering and fine local rendering; render the scene model, save the scene model rendering result in the two-dimensional texture of the scene model as a half-floating point data type, and then render the celestial body and the sky atmosphere, and superimpose the rendering result of the celestial body and the sky atmosphere on the two-dimensional texture of the scene model as a half-floating point number to obtain the scene texture. The 10-bit quantization bit number of the half-floating point number is more than the 8-bit quantization bit number in the prior art, and the numerical representation range is wider, which is conducive to obtaining more image detail data and is conducive to subsequent high-realistic rendering; perform a downsampling operation on the scene texture, calculate the expected average brightness, and calculate the target average brightness for a smooth transition from the average brightness of the previous frame to the expected average brightness based on the expected average brightness, the time interval of the current frame and the average brightness of the previous frame, so as to obtain the average brightness of the current frame. Compared with the prior art that directly regards the expected average brightness as the average brightness of the current frame, a smooth transition is calculated by calculating a The average brightness of the scene is used to highly simulate the reaction of the human eye to changes in brightness, so as to achieve adaptive exposure of the human eye, which is conducive to meeting the needs of high-realistic rendering; pixels with brightness higher than the preset threshold are extracted from the downsampled scene texture to generate an initial flood texture, and the initial flood texture is sequentially downsampled and Gaussian blurred to obtain an optimized texture, which reduces the amount of calculation while ensuring the quality of image rendering, blurs the image edges and image details, and improves the rendering effect; the optimized texture is then upsampled in turn, and the current upsampling result is superimposed on the previous flood texture of the optimized texture, and the image is enlarged to the original resolution to obtain the target flood texture; finally, according to the scene texture and the target flood texture, combined with the average brightness of the current frame, the exposure result of high dynamic rendering is synthesized, and the exposure result is tone mapped and displayed on the target display device to achieve image rendering with a high dynamic range; thereby, the color gamut and dynamic range of brightness of the three-dimensional scene rendering are expanded, which can highly simulate the real image seen by the human eye, achieve high-realistic rendering, and improve the rendering effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 An overall flow chart of a method for rendering high dynamic range content provided by an exemplary embodiment of the present application.

[0057] Figure 2 A flowchart of human eye adaptive exposure optimization for a method of rendering high dynamic range content is provided as an exemplary embodiment of the present application.

[0058] Figure 3 A flowchart of generating a floodlight texture in a method for rendering high dynamic range content is provided as an exemplary embodiment of the present application.

[0059] Figure 4 A structural block diagram of a device for rendering high dynamic range content is provided as yet another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0060] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0063] In order to highly simulate the real images seen by the human eye, achieve high-realism rendering, and improve rendering effects, this application uses the half-floating-point data type as the rendering basis, combined with the human eye's adaptive automatic exposure, to simulate a wider range of brightness and color in the real world, showing scene details from the darkest to the brightest, providing more realistic lighting effects, and making colors more vivid and vivid. It expands the color gamut and brightness dynamic range of three-dimensional scene rendering, can highly simulate the real images seen by the human eye, meet the needs of high-realism rendering, achieve high dynamic range rendering, and improve image rendering effects.

[0064] The present application uses a half-floating point data type to store intermediate rendering results. Compared with the normalized unsigned integer data type used in SDR rendering in the prior art, the 10-bit quantization bit number of the half-floating point number is larger than the 8-bit quantization bit number of the normalized unsigned integer, and the half-floating point number can represent floating point numbers from 1 to 65504, which is impossible using a normalized unsigned integer.

[0065] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0066] Reference Figure 1, an embodiment of the present application provides a method for rendering high dynamic range content, and the main steps of the method are described as follows.

[0067] S1: Obtain a triangular mesh representing the terrain and a triangular mesh representing the three-dimensional object, as well as the corresponding terrain texture and three-dimensional object texture, to obtain a scene model;

[0068] S2: Rendering the scene model, and saving the scene model rendering result in a half-floating point data type into a two-dimensional texture of the scene model;

[0069] S3: Rendering the celestial body and the sky atmosphere, and superimposing the rendering results of the celestial body and the sky atmosphere on the two-dimensional texture of the scene model with half floating point numbers to obtain a scene texture;

[0070] S4: performing a downsampling operation on the scene texture, calculating an expected average brightness, and calculating a target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness according to the expected average brightness, the time interval of the current frame, and the average brightness of the previous frame, to obtain the average brightness of the current frame;

[0071] S5: extracting pixels whose brightness is higher than a preset threshold from the downsampled scene texture to generate an initial floodlight texture;

[0072] S6: performing a downsampling operation and a Gaussian blur operation on the initial floodlight texture in sequence to obtain an optimized texture, performing an upsampling operation on the optimized texture in sequence, and then superimposing the current upsampling result on the previous floodlight texture of the optimized texture to obtain a target floodlight texture;

[0073] S7: synthesizing an exposure result of high dynamic rendering according to the scene texture and the target floodlight texture in combination with the average brightness of the current frame, and displaying the exposure result on a target display device after tone mapping.

[0074] Specifically, a triangular mesh representing the terrain and a triangular mesh representing the three-dimensional object, and their corresponding textures are obtained.

[0075] Among them, the triangular mesh of the terrain includes mountains, valleys, plains, etc. The triangular mesh of the terrain is generated according to the height field data or laser scanning data for large-scale geographical environment rendering.

[0076] The triangular mesh of three-dimensional objects includes complex models such as people, buildings, and vehicles. Based on modeling tools, combined with polygonal modeling, surface subdivision and other technologies, the triangular mesh of three-dimensional objects is generated for fine local rendering.

[0077] Terrain textures and three-dimensional object textures can be in various forms such as one-dimensional, two-dimensional, and three-dimensional.

[0078] A scene model is obtained based on the triangular mesh of the terrain, the triangular mesh of the three-dimensional object and the corresponding texture.

[0079] Next, use PBR (Physically Based Rendering) technology to render the scene model, simulating the interaction between light and the surface of objects through the laws of physics, so that the rendered image is more in line with the lighting, shadows, reflections and other characteristics of the real world, thereby achieving a more realistic visual effect.

[0080] In one embodiment, the step of rendering the scene model includes:

[0081] Call the graphics API interface to rasterize the terrain triangular mesh into terrain two-dimensional pixels, and rasterize the three-dimensional object triangular mesh into three-dimensional pixel points;

[0082] By using physically based rendering technology, based on the two-dimensional pixel points of the terrain and the two-dimensional pixel points of the three-dimensional object, the corresponding high dynamic range RGB color values ​​of the terrain and the high dynamic range RGB color values ​​of the three-dimensional object are calculated to obtain the rendering result of the scene model.

[0083] The graphics API interface may be WebGL, OpenGL, etc. The graphics API interface is called to rasterize the input terrain and three-dimensional objects into two-dimensional pixels, and then the two-dimensional pixels are calculated using the PBR method to obtain the corresponding high dynamic range RGB color values, and then the RGB color values ​​are saved in a two-dimensional texture of a half-floating point data type, which is conducive to obtaining more image detail data and subsequent high-realistic rendering.

[0084] Further, render celestial bodies and sky atmosphere.

[0085] Specifically, the coordinates and brightness information of the celestial body are obtained. The coordinates of the celestial body are expressed in right ascension and declination of the equatorial coordinate system, and the brightness information of the celestial body is expressed in apparent magnitude. The smaller the apparent magnitude, the greater the brightness.

[0086] The coordinates and brightness information of celestial bodies come from the 5th edition of the Yale University Bright Star Catalog, which is a star catalog containing stars with brightness exceeding 6.5 apparent magnitude, covering almost all celestial bodies visible to the naked eye on Earth. There are a total of 9,110 celestial bodies, such as the famous North Star and Sirius.

[0087] The coordinates of the celestial body are mapped to the two-dimensional view plane of the camera using projection transformation, and the two-dimensional coordinates of the celestial body within the visible range of the target are calculated using the field of view (FOV) angle to obtain the celestial body rendering result.

[0088] Convert the brightness information of celestial bodies into linear brightness values. The specific formula is as follows:

[0089] L=10 -0.4×m

[0090] Where L represents the linear brightness value, and m represents the brightness information of the celestial body.

[0091] The luminous intensity and color of the celestial body in the rendering are set according to the linear brightness value to obtain the sky atmosphere rendering result.

[0092] Then, the PBR method is used to calculate the celestial body rendering results and the sky atmosphere rendering results to obtain the corresponding high dynamic range RGB color values. Finally, the RGB color values ​​calculated in the high dynamic range are superimposed on the two-dimensional texture of the scene model, that is, half floating point numbers are superimposed on the two-dimensional texture of the scene model to obtain the scene texture, which is conducive to obtaining more image detail data and subsequent high-realistic rendering. At this point, the three-dimensional rendering is ended.

[0093] Furthermore, a downsampling operation is performed on the scene texture to calculate the expected average brightness, and based on the expected average brightness, the time interval of the current frame and the average brightness of the previous frame, a target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness is calculated to obtain the average brightness of the current frame.

[0094] By performing human eye adaptive exposure on the rendered High Dynamic Range (HDR) texture, a smoothly transitioning average brightness is calculated to simulate the human eye's reaction to changes in brightness. When the brightness changes too much, the rendering effect will be similar to overexposure or underexposure. For example, after staying in a dim room for a long time, if you suddenly open the curtains, the human eye will see a white area, and then it will slowly return to normal; or after staying in a bright room for a long time, if you suddenly turn off the lights, the human eye will see darkness, and it will take a long time before a hint of brightness can be seen.

[0095] Reference Figure 2 In one embodiment, the step of performing a downsampling operation on the scene texture and calculating the expected average brightness includes:

[0096] Performing a downsampling operation on the scene texture to generate textures with widths of 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original width and height respectively;

[0097] Convert the RGB value corresponding to the 1 / 32 texture to the pixel brightness value;

[0098] Calculate the average brightness of all pixel brightness values ​​in the 1 / 32 texture to get the expected average brightness.

[0099] In one embodiment, the step of calculating a target average brightness for a smooth transition from the average brightness of the previous frame to the expected average brightness according to the expected average brightness, the time interval of the current frame and the average brightness of the previous frame includes:

[0100] Calculate the difference between the expected average brightness and the average brightness of the previous frame;

[0101] The difference is divided by the time interval of the current frame to obtain a target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness.

[0102] It is worth noting that if it is the first frame, that is, there is no previous frame, then the expected average brightness can be subtracted from the preset value, such as the preset value is set to 0.18, and an average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness is calculated as the average brightness of the current frame. In the next frame, the average brightness of the current frame calculated in the previous step can be used as the average brightness of the previous frame.

[0103] Next, pixels with brightness higher than a preset threshold are extracted from the downsampled scene texture to generate an initial flood light texture. In this embodiment, after selecting 1 / 2 of the texture from the downsampled scene texture, pixels with brightness higher than a preset threshold are extracted from the texture to generate an initial flood light texture. The preset threshold is set by the management personnel according to the actual situation.

[0104] Then, downsampling operations and Gaussian blur operations are sequentially performed on the generated initial floodlight texture to obtain an optimized texture, and upsampling operations are sequentially performed on the optimized texture, and then the current upsampling result is superimposed on the previous floodlight texture of the optimized texture to obtain a target floodlight texture.

[0105] Reference Figure 3 In one embodiment, the steps of performing a downsampling operation and a Gaussian blur operation on the initial flood light texture in sequence to obtain an optimized texture, performing an upsampling operation on the optimized texture in sequence, and then superimposing the current upsampling result on the previous flood light texture of the optimized texture to obtain a target flood light texture include:

[0106] Downsampling the initial flood texture to generate flood input textures with widths of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original width and height, respectively;

[0107] Gaussian blur each floodlight input texture separately;

[0108] Upsample the Gaussian blurred 1 / 32 bloom input texture, add the upsampled result to the 1 / 16 bloom input texture, and update the Gaussian blurred 1 / 16 bloom input texture.

[0109] Upsample the Gaussian blurred 1 / 16 bloom input texture, add the upsampled result to the 1 / 8 bloom input texture, and update the Gaussian blurred 1 / 8 bloom input texture.

[0110] Upsample the Gaussian blurred 1 / 8 bloom input texture, add the upsampled result to the 1 / 4 bloom input texture, and update the Gaussian blurred 1 / 4 bloom input texture.

[0111] The Gaussian blurred 1 / 4 floodlight input texture is upsampled, and the upsampled result is superimposed on the initial floodlight texture to finally obtain the target floodlight texture, wherein the initial floodlight texture is the 1 / 2 scene texture whose downsampled brightness is higher than the preset threshold.

[0112] Finally, according to the scene texture and the target floodlight texture, combined with the average brightness of the current frame, an exposure result of high dynamic rendering is synthesized, and the exposure result is tone mapped and displayed on a target display device.

[0113] In one embodiment, the step of synthesizing the exposure result of high dynamic rendering according to the scene texture and the target bloom texture in combination with the average brightness of the current frame includes:

[0114] Adding the scene texture and the target flood texture to obtain a superposition result;

[0115] The superposition result is divided by the target average brightness to obtain the exposure result of high dynamic rendering.

[0116] The HDR exposure result is obtained by superimposing the scene texture into the target flood texture and then dividing it by the target average brightness, that is, the average brightness value of the current frame. The exposure result is then subjected to ACES tone mapping and displayed on the target display device, so that the HDR content can be approximately displayed on the SDR display device.

[0117] In summary, a method for rendering high dynamic range content obtains a triangular mesh representing a terrain and a triangular mesh representing a three-dimensional object, and the corresponding terrain texture and three-dimensional object texture to obtain a scene model for large-scale geographical environment rendering and fine local rendering; renders the scene model, saves the rendering result of the scene model in a half-floating point data type into the two-dimensional texture of the scene model, and then renders the celestial body and the sky atmosphere, superimposes the rendering result of the celestial body and the sky atmosphere into the two-dimensional texture of the scene model in a half-floating point number to obtain the scene texture, the 10-bit quantization bit number of the half-floating point number is more than the 8-bit quantization bit number in the prior art, and the numerical representation range is wider, which is conducive to obtaining more image detail data and subsequent high-realistic rendering; performs a downsampling operation on the scene texture, calculates the expected average brightness, and calculates the target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness based on the expected average brightness, the time interval of the current frame and the average brightness of the previous frame, to obtain the average brightness of the current frame, compared with the prior art that directly regards the expected average brightness as the average brightness of the current frame. , by calculating a smooth transition average brightness, the human eye's reaction to brightness changes is highly simulated, and the human eye's adaptive exposure is achieved, which is conducive to meeting the needs of high-realistic rendering; pixels with brightness higher than a preset threshold are extracted from the downsampled scene texture to generate an initial floodlight texture, and the initial floodlight texture is sequentially downsampled and Gaussian blurred to obtain an optimized texture, which reduces the amount of calculation while ensuring the quality of image rendering, blurs the image edge and image details, and improves the rendering effect; then the optimized texture is sequentially upsampled, and the current upsampled result is superimposed on the previous floodlight texture of the optimized texture, and the image is enlarged to the original resolution to obtain the target floodlight texture; finally, according to the scene texture and the target floodlight texture, combined with the average brightness of the current frame, the exposure result of high dynamic rendering is synthesized, and the exposure result is tone mapped and displayed on the target display device to achieve high dynamic range image rendering; thereby, the color gamut and dynamic range of brightness of the three-dimensional scene rendering are expanded, which can highly simulate the real image seen by the human eye, achieve high-realistic rendering, and improve the rendering effect.

[0118] Compared with traditional SDR rendering, this application adopts the HDR rendering pipeline to develop a complete solution for rendering large outdoor scenes, which can meet the needs of industries such as digital twins, geographic information systems, and gaming and entertainment for high-realistic rendering.

[0119] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0120] Reference Figure 4The embodiment of the present application further provides a device for rendering high dynamic range content, and the device for rendering high dynamic range content corresponds one-to-one with a method for rendering high dynamic range content in the above embodiment. The device for rendering high dynamic range content includes:

[0121] A data module is used to obtain a triangular mesh representing a terrain and a triangular mesh representing a three-dimensional object, as well as corresponding terrain textures and three-dimensional object textures, to obtain a scene model;

[0122] A two-dimensional texture optimization module, used for rendering the scene model, and saving the scene model rendering result into the two-dimensional texture of the scene model in a half-floating point data type;

[0123] A scene texture module is used to render celestial bodies and sky atmosphere, and superimpose the rendering results of the celestial bodies and sky atmosphere on the two-dimensional texture of the scene model in the form of half floating point numbers to obtain a scene texture;

[0124] an exposure optimization module, configured to perform a downsampling operation on the scene texture, calculate an expected average brightness, and calculate a target average brightness for a smooth transition from the average brightness of the previous frame to the expected average brightness based on the expected average brightness, a time interval of a current frame, and an average brightness of a previous frame, to obtain an average brightness of a current frame;

[0125] A flood initialization module is used to extract pixels with brightness higher than a preset threshold from the downsampled scene texture to generate an initial flood texture;

[0126] A flood light texture module is used to sequentially perform a downsampling operation and a Gaussian blur operation on the initial flood light texture to obtain an optimized texture, sequentially perform an upsampling operation on the optimized texture, and then superimpose the current upsampling result on the previous flood light texture of the optimized texture to obtain a target flood light texture;

[0127] The display module is used to synthesize the exposure result of high dynamic rendering according to the scene texture and the target floodlight texture in combination with the average brightness of the current frame, and display the exposure result on the target display device after tone mapping.

[0128] For specific limitations on a device for rendering high dynamic range content, reference may be made to the above limitations on a method for rendering high dynamic range content, which will not be repeated here.

[0129] Each module in the above-mentioned device for rendering high dynamic range content can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0130] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, any of the above-mentioned methods for rendering high dynamic range content is implemented.

[0131] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, any one of the above-mentioned methods for rendering high dynamic range content is implemented.

[0132] In one embodiment, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, any one of the above methods for rendering high dynamic range content is implemented.

[0133] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0134] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A method for rendering high dynamic range content, characterized in that: The following steps are included: Acquire a triangular mesh representing the terrain and a triangular mesh representing the three-dimensional object, as well as corresponding terrain textures and three-dimensional object textures, to obtain a scene model; Rendering the scene model, and saving the scene model rendering result in a half-floating point data type into a two-dimensional texture of the scene model; Rendering the celestial body and the sky atmosphere, and superimposing the rendering results of the celestial body and the sky atmosphere on the two-dimensional texture of the scene model with half-floating point numbers to obtain a scene texture; Performing a downsampling operation on the scene texture, calculating an expected average brightness, and calculating a target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness according to the expected average brightness, the time interval of the current frame, and the average brightness of the previous frame, to obtain an average brightness of the current frame; Extracting pixel points with brightness higher than a preset threshold from the downsampled scene texture to generate an initial flood texture; Performing a downsampling operation and a Gaussian blur operation on the initial floodlight texture in sequence to obtain an optimized texture, performing an upsampling operation on the optimized texture in sequence, and then superimposing the current upsampling result on the previous floodlight texture of the optimized texture to obtain a target floodlight texture; An exposure result of high dynamic rendering is synthesized according to the scene texture and the target floodlight texture in combination with the average brightness of the current frame, and the exposure result is tone mapped and displayed on a target display device.

2. The method for rendering high dynamic range content according to claim 1, characterized in that: The step of performing a downsampling operation on the scene texture and calculating the expected average brightness includes: Performing a downsampling operation on the scene texture to generate textures with widths of 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original width and height respectively; Convert the RGB value corresponding to the 1 / 32 texture to the pixel brightness value; Calculate the average brightness of all pixel brightness values ​​in the 1 / 32 texture to get the expected average brightness.

3. The method for rendering high dynamic range content according to claim 1, characterized in that: The step of calculating a target average brightness for a smooth transition from the average brightness of the previous frame to the expected average brightness according to the expected average brightness, the time interval of the current frame and the average brightness of the previous frame comprises: Calculate the difference between the expected average brightness and the average brightness of the previous frame; The difference is divided by the time interval of the current frame to obtain a target average brightness that smoothly transitions from the average brightness of the previous frame to the expected average brightness.

4. The method for rendering high dynamic range content according to claim 1, characterized in that: The step of synthesizing the exposure result of high dynamic rendering according to the scene texture and the target flood light texture in combination with the average brightness of the current frame comprises: Adding the scene texture and the target flood texture to obtain a superposition result; The superposition result is divided by the target average brightness to obtain the exposure result of high dynamic rendering.

5. The method for rendering high dynamic range content according to claim 1, characterized in that: The steps of sequentially performing downsampling operations and Gaussian blur operations on the initial flood light texture to obtain an optimized texture, sequentially performing upsampling operations on the optimized texture, and then superimposing the current upsampling result onto the previous flood light texture of the optimized texture to obtain a target flood light texture include: Downsampling the initial flood texture to generate flood input textures with widths of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original width and height, respectively; Gaussian blur each floodlight input texture separately; Upsample the Gaussian blurred 1 / 32 bloom input texture, add the upsampled result to the 1 / 16 bloom input texture, and update the Gaussian blurred 1 / 16 bloom input texture. Upsample the Gaussian blurred 1 / 16 bloom input texture, add the upsampled result to the 1 / 8 bloom input texture, and update the Gaussian blurred 1 / 8 bloom input texture. Upsample the Gaussian blurred 1 / 8 bloom input texture, add the upsampled result to the 1 / 4 bloom input texture, and update the Gaussian blurred 1 / 4 bloom input texture. The Gaussian blurred 1 / 4 floodlight input texture is upsampled, and the upsampled result is superimposed on the initial floodlight texture to obtain the target floodlight texture.

6. The method for rendering high dynamic range content according to any one of claims 1 to 5, characterized in that: The step of rendering the scene model includes: Call the graphics API interface to rasterize the terrain triangular mesh into terrain two-dimensional pixels, and rasterize the three-dimensional object triangular mesh into three-dimensional pixel points; By using physically based rendering technology, based on the two-dimensional pixel points of the terrain and the two-dimensional pixel points of the three-dimensional object, the corresponding high dynamic range RGB color values ​​of the terrain and the high dynamic range RGB color values ​​of the three-dimensional object are calculated to obtain the rendering result of the scene model.

7. A device for rendering high dynamic range content, characterized in that include, A data module is used to obtain a triangular mesh representing a terrain and a triangular mesh representing a three-dimensional object, as well as corresponding terrain textures and three-dimensional object textures, to obtain a scene model; A two-dimensional texture optimization module, used for rendering the scene model, and saving the scene model rendering result into the two-dimensional texture of the scene model in a half-floating point data type; A scene texture module is used to render celestial bodies and sky atmosphere, and superimpose the rendering results of the celestial bodies and sky atmosphere on the two-dimensional texture of the scene model in the form of half floating point numbers to obtain a scene texture; an exposure optimization module, configured to perform a downsampling operation on the scene texture, calculate an expected average brightness, and calculate a target average brightness for a smooth transition from the average brightness of the previous frame to the expected average brightness based on the expected average brightness, a time interval of a current frame, and an average brightness of a previous frame, to obtain an average brightness of a current frame; A flood initialization module is used to extract pixels with brightness higher than a preset threshold from the downsampled scene texture to generate an initial flood texture; A flood light texture module is used to sequentially perform a downsampling operation and a Gaussian blur operation on the initial flood light texture to obtain an optimized texture, sequentially perform an upsampling operation on the optimized texture, and then superimpose the current upsampling result on the previous flood light texture of the optimized texture to obtain a target flood light texture; The display module is used to synthesize the exposure result of high dynamic rendering according to the scene texture and the target floodlight texture in combination with the average brightness of the current frame, and display the exposure result on the target display device after tone mapping.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 6 when being executed by a processor.

Images