A rendering method and device based on screen space

Abstract

The present application provides a screen-space rendering method and device for improving system performance. The method includes: obtaining a trigger instruction; obtaining image data of at least one frame of image; in response to the trigger instruction, rendering the image data in a specified mode to obtain at least one frame of rendered image, the specified mode including one of the following modes: a first mode, a second mode, and a third mode, the first mode being a screen-space reflection (SSR) mode, the second mode being a screen-space ambient occlusion (SSAO) mode, and the third mode being a combination of SSR and SSAO mode; and outputting the at least one frame of rendered image.

Description

Technical Field

[0001] The present application relates to the technical field of computer graphics, and in particular to a screen space-based rendering method and device. Background Art

[0002] Currently, there are some screen-space-based technologies in the field of real-time rendering. These technologies are designed to provide global illumination effects with lower precision but can be calculated in real time. For example, screen space reflection (SSR) is a rendering technology that uses screen space for reflected lighting; screen space ambient occlusion (SSAO) is a rendering technology that uses screen space for ambient occlusion.

[0003] As electronic device system performance continues to improve, in order to achieve more realistic image display effects, existing technologies will perform SSAO rendering and SSR rendering on images separately. For example, SSAO rendering is first performed on an image, and then SSR rendering is performed on the SSAO-rendered image. However, due to the need to render each frame in real time, if SSAO and SSR technologies are used to render the images in real time, there will be drawbacks such as consuming system performance and increasing system power consumption. Therefore, existing screen-space-based rendering technologies need to be further optimized. Summary of the Invention

[0004] The embodiments of the present application provide a screen space-based rendering method and apparatus for optimizing system performance in a scenario where SSAO and / or SSR technology are used to render displayed images in real time.

[0005] In a first aspect, an embodiment of the present application provides a screen space-based rendering method, the method comprising: obtaining a trigger instruction; obtaining image data of at least one frame of image; in response to the trigger instruction, rendering the image data in a specified mode to obtain at least one frame of rendered image, the specified mode comprising one of the following modes: a first mode, a second mode, and a third mode, the first mode being a screen space reflection SSR mode, the second mode being a screen space ambient occlusion SSAO mode, and the third mode being a combination of SSR and SSAO mode; and outputting the at least one frame of rendered image.

[0006] Compared to the conventional method of performing SSR rendering and SSAO rendering separately for the image data to be rendered, the method provided by the embodiments of the present application can render the image data to be rendered in a combined SSR and SSAO mode; and can also determine whether to render the image data in SSR and / or SSAO mode in response to configuration instructions. Thus, the method provided by the embodiments of the present application improves flexibility and optimizes system performance compared to the conventional technology.

[0007] In one possible design, the specified mode is the third mode, the image data includes multiple pixel points in the at least one frame of image; rendering the image data in the specified mode to obtain at least one frame of rendered image includes: selecting a first pixel point set corresponding to each pixel point in the multiple pixel points for each pixel point in the multiple pixel points; performing SSR calculation based on the first pixel point set to obtain a color value; performing SSAO calculation based on the first pixel point set to obtain an occlusion value; rendering each pixel point according to the color value and the occlusion value to obtain at least one frame of rendered image.

[0008] In the above design, during the SSAO calculation, the first pixel point set selected for each pixel point obtained in the SSR calculation process is reused, so that the first pixel point set of each pixel point obtained in the SSR calculation process can be used not only to continue the SSR calculation, but also to perform SSAO calculation. Therefore, this design reduces the calculation process of selecting the first pixel point set for each pixel point in the SSAO calculation process, thereby reducing the amount of calculation and reducing power consumption.

[0009] In one possible design, the image data also includes the sight vector and normal of each pixel point; and the first set of pixel points corresponding to each pixel point is selected for each pixel point in the multiple pixel points, including: determining the direction of reflected light based on the sight vector and normal of each pixel point; and selecting the first set of pixel points from the multiple pixel points in the direction of the reflected light.

[0010] In the above design, an implementation method of selecting a first set of pixel points for each pixel point is given. By determining the direction of reflected light for each pixel point, and further selecting a first set of pixel points in the direction of reflected light, the color value of the pixel point intersecting with the reflected light direction can be obtained based on the first set of pixel points selected in the direction of reflected light, and then SSR rendering is achieved through the pixel point intersecting with the reflected light direction corresponding to each pixel point.

[0011] In one possible design, the SSAO calculation based on the first pixel point set includes: selecting at least one other ambient light direction based on the reflected light direction; selecting at least one second pixel point set from the multiple pixel points in the other ambient light direction; and performing SSAO calculation based on the first pixel point set and the at least one second pixel point set.

[0012] In the above design, based on the fact that the first pixel point set selected for each pixel point obtained in the SSR calculation process can be reused during the SSAO calculation process, and at least one ambient light direction may be selected for the SSAO calculation when performing the SSAO calculation, it is possible to consider the reflected light direction used in the SSR calculation process as one of the ambient light directions, and use this as a benchmark to select other ambient light directions, thereby achieving a reduction in the amount of calculation for selecting a sampling point set in one of the ambient light directions compared to the prior art that requires selecting a second pixel point set according to each ambient light direction during the SSAO calculation process. This can reduce system power consumption when the present application is implemented.

[0013] In one possible design, before rendering each pixel according to the color value and the occlusion value to obtain at least one frame of rendered image, the method further includes: performing time domain filtering on the color value and the occlusion value.

[0014] In the above design, compared with the prior art, the obtained color value needs to be time-domain filtered before SSR rendering of the image data, and the obtained occlusion value needs to be time-domain filtered before SSAO rendering. In the embodiment of the present application, the obtained color value and occlusion value can be filtered together, thereby reducing the processing process of time-domain filtering and reducing system consumption.

[0015] In a possible design, the image data belongs to a material with reflective capability, and the designated mode is the third mode.

[0016] In the above design, since SSR rendering is only necessary for materials with reflective capabilities in the scene to be rendered, before rendering in the SSR and SSAO combined mode, it is determined that the image data to be rendered is a material with reflective capabilities, and then rendering in the SSR and SSAO combined mode is performed based on this part of the image data, avoiding unnecessary calculations, thereby reducing system power consumption and improving system performance.

[0017] In the second aspect, an embodiment of the present application also provides a screen space-based rendering device, which includes an acquisition unit, a processing unit, and an output unit; wherein the acquisition unit is used to acquire a trigger instruction; acquire image data of at least one frame of image; the processing unit is used to render the image data in a specified mode in response to the trigger instruction to obtain at least one frame of rendered image, and the specified mode includes one of the following modes: a first mode, a second mode, and a third mode, the first mode is a screen space reflection SSR mode, the second mode is a screen space ambient occlusion SSAO mode, and the third mode is a combination of SSR and SSAO mode; the output unit is used to output the at least one frame of rendered image.

[0018] In one possible design, the designated mode is the third mode, and the image data includes multiple pixel points in the at least one frame of image; the processing unit is specifically used to: select a first pixel point set corresponding to each pixel point in the multiple pixel points for each pixel point in the multiple pixel points; perform SSR calculation based on the first pixel point set to obtain a color value; perform SSAO calculation based on the first pixel point set to obtain an occlusion value; render each pixel point according to the color value and the occlusion value to obtain at least one frame of rendered image.

[0019] In one possible design, the image data also includes the line of sight vector and normal of each pixel point; the processing unit is specifically used to: determine the direction of reflected light based on the line of sight vector and normal of each pixel point; and select the first set of pixel points from the multiple pixel points in the direction of the reflected light.

[0020] In one possible design, the processing unit is specifically used to: select at least one other ambient light direction based on the reflected light direction; select at least one second pixel point set from the multiple pixel points in the other ambient light direction; and perform SSAO calculation based on the first pixel point set and the at least one second pixel point set.

[0021] In one possible design, the processing unit is further used to perform time-domain filtering on the color value and the occlusion value before rendering each pixel point according to the color value and the occlusion value to obtain at least one frame of rendered image.

[0022] In a possible design, the image data belongs to a material with reflective capability, and the designated mode is the third mode.

[0023] In a third aspect, embodiments of the present application further provide a computing device comprising a processor and a memory, the memory being coupled to the processor, and optionally further comprising a display screen. The processor executes program instructions in the memory to perform the method provided in the first aspect or any possible design of the first aspect. The display screen is configured to display information to a user when triggered by the processor.

[0024] In a fourth aspect, the present application provides a computing device cluster, which includes at least one computing device provided in the third aspect.

[0025] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed on a computer, the computer executes the method described in the first aspect or any possible design.

[0026] In a sixth aspect, an embodiment of the present application provides a computer program product, which is used to store computer instructions. When the computer instructions are run on a computer, the computer executes the method described in the first aspect or any possible design.

[0027] In a seventh aspect, an embodiment of the present application provides a chip system, which includes a processor and may also include a memory, for implementing the method described in the first aspect or any possible design. The chip system can be composed of a chip or can include a chip and other discrete devices.

[0028] The beneficial effects of the second to seventh aspects and their implementation methods can refer to the description of the beneficial effects of the method of the first aspect and its implementation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the structure of a computer system provided in an embodiment of the present application;

[0030] Figure 2a A schematic diagram of a flow chart of a screen-space-based rendering method provided in an embodiment of the present application;

[0031] Figure 2b Another schematic flow chart of a screen-space-based rendering method provided in an embodiment of the present application;

[0032] Figure 3a A schematic diagram for describing the reflection principle provided in an embodiment of the present application;

[0033] Figure 3b A schematic diagram of calculating a reflection intersection point provided in an embodiment of the present application;

[0034] Figure 4a A schematic diagram of selecting the direction of ambient light provided in an embodiment of the present application;

[0035] Figure 4b A schematic diagram of calculating a shielding value provided in an embodiment of the present application;

[0036] Figure 5a This is an example diagram after SSR rendering provided in an embodiment of the present application;

[0037] Figure 5b This is an example image after SSAO rendering provided in an embodiment of the present application;

[0038] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application;

[0039] Figure 7 A schematic structural diagram of a screen space-based rendering device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0041] A possible application scenario of this application is introduced below.

[0042] See Figure 1 , is a structural diagram of a computer system provided in an embodiment of the present application, which is applicable to the field of computer graphics. The computer system includes at least a central processing unit (CPU), a graphics processing unit (GPU), and a display device. Through the collaborative work between the CPU, GPU, and display device, images are drawn by the computer and displayed on the display device. Among them,

[0043] The CPU is used to perform three-dimensional (3D) modeling to obtain a 3D scene. For example, common 3D scenes include game scenes.

[0044] The GPU is used to render the 3D scene generated by the CPU, thereby obtaining image data that can be output on a display device, which can also be understood as image data in screen space. GPU rendering can also achieve a more realistic display effect of the 3D scene. For example, the SSR rendering involved in the background technology is used to increase the reflection effect in the 3D scene, improving the color perception of the 3D scene; SSAO rendering is used to increase the ambient light occlusion effect in the 3D scene, improving the layering of the 3D scene.

[0045] The display device is used to display the 3D scene after being rendered by the GPU.

[0046] Based on the description in the background technology, the current SSR rendering and SSAO rendering of the image data of the scene to be rendered are generally based on multi-channel rendering technology, that is, SSAO calculation is performed in one rendering channel (PASS), and SSR calculation is performed in another PASS; then, after the calculation results of the two PASSes are respectively subjected to time-domain filtering, the image of the scene to be rendered is rendered twice; finally, the rendering effect is obtained. In addition, since SSAO rendering and SSR rendering are post-processing effects based on screen space, in order to realize SSAO rendering and SSR rendering, each PASS will be used to convert the sampling points in the scene to be rendered into pixels in screen space, obtain the depth value of the pixel in screen space according to the depth map of the scene to be rendered, and obtain the normal information of the pixel. However, performing SSAO rendering and SSR rendering on the image of the scene to be rendered separately has the disadvantage of large amount of calculation, which will lead to consumption of system performance, increase of system power consumption, and further lead to the problem of poor user experience.

[0047] In view of this, the present application provides a rendering method based on screen space, which provides the possibility of selecting a rendering mode from SSAO mode rendering, SSR mode rendering, or considering merging the SSAO rendering and SSR rendering processes to implement SSAO and SSR combined mode rendering in one PASS, so that rendering technology has more optional solutions, improves flexibility and optimizes system performance compared to existing technologies. The following introduces the implementation process of a rendering method based on screen space provided by the present application through multiple embodiments.

[0048] In the following embodiments of the present application, "multiple" refers to two or more, and in the embodiments of the present application, "multiple" can also be understood as "at least two". "At least one" can be understood as one or more, for example, one, two or more. For example, including at least one means including one, two or more, and does not limit which ones are included. For example, including at least one of A, B and C, then the included ones may be A, B, C, A and B, A and C, B and C, or A and B and C. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the previous and next associated objects are in an "or" relationship. Unless otherwise specified, the ordinal numbers "first", "second" and so on mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects.

[0049] In order to more clearly understand the contents of the following embodiments, the terms and nouns that may be involved in the following are explained, including:

[0050] 1) Screen space, typically a two-dimensional space, is a two-dimensional representation of a 3D scene created in a three-dimensional object space, obtained by performing multiple coordinate transformations on the 3D scene. This representation is then displayed as a two-dimensional image on a display device. The implementation of multiple coordinate transformations from object space to screen space image data for the created 3D scene can be found in the prior art. The method provided in this application is based on post-processing of the resulting screen space image data, and therefore this application does not further elaborate on the coordinate transformations.

[0051] The image data in screen space is composed of multiple pixels, and the frame buffer (G-buffer) stores information such as the line of sight vector, color value, normal, and depth value corresponding to each pixel. In the process of rendering based on screen space, the image data of at least one frame of the scene to be rendered in screen space is first obtained. When rendering multiple pixels contained in one frame of image data, it can be combined with other pixels around the pixel in screen space to facilitate rendering. For example, SSR rendering and SSAO rendering are both technologies based on screen space rendering.

[0052] See Figure 2a As shown, a flowchart of a rendering method in screen space provided by an embodiment of the present application includes: Step S201: Obtain a trigger instruction. The trigger instruction is used to determine the rendering mode. In an optional example, the trigger instruction is generated by the electronic device based on the processing capability of the chip. For example, if the processing capability of the chip is strong, the generated trigger instruction is used to determine whether the image data is rendered in the third mode, that is, rendering is performed in a combination of SSR and SSAO; if the processing capability of the chip is weak, the generated trigger instruction is used to determine whether the image data is rendered in SSR or SSAO. Alternatively, in another optional example, the trigger instruction can also be generated based on the user's configuration. For example, if the user wants to obtain a more realistic game screen and selects a high rendering mode, the corresponding generated trigger instruction is used to determine whether the image data is rendered in the third mode; if the user has a general demand for the rendering mode but is more concerned about system power consumption and selects the first or second mode of rendering, the corresponding generated trigger instruction is used to determine whether the image data is rendered in SSR or SSAO.

[0053] Step S202: Obtain image data of at least one frame of the scene to be rendered in the screen space. Exemplarily, assuming that the scene to be rendered is a game scene, during the game, it is necessary to perform real-time rendering on multiple frames of images that make up the dynamic screen of the game, and based on the introduction of the aforementioned content, it can be obtained that SSR rendering and SSAO rendering are post-processing effects based on the screen space. Therefore, in order to realize the rendering of the scene to be rendered, obtain image data of at least one frame of the scene to be rendered in the screen space. The specific implementation is to read the required image data from the G-buffer, and the image data obtained is image data that has not been rendered by SSR and / or SSAO.

[0054] Step S203: In response to the trigger instruction, the image data is rendered in a specified mode to obtain at least one frame of rendered image, the specified mode including one of the following modes: a first mode, a second mode, and a third mode, the first mode being a screen space reflection (SSR) mode, the second mode being a screen space ambient occlusion (SSAO) mode, and the third mode being a combination of SSR and SSAO mode. Exemplarily, each frame of image is rendered based on the image data of each frame of image in the specified mode to obtain each rendered frame of image.

[0055] Step S204: Output the at least one frame of rendered image. Figure 1 The display device included in the computer system shown reads the rendered image data for display, and illustratively, dynamically plays at least one frame of the rendered image.

[0056] In response to a trigger instruction, the image data is rendered in one of the first, second, and third modes according to the trigger instruction, thereby outputting a rendered image. In this way, based on the judgment of chip performance or business needs, the appropriate rendering mode can be selected to render the image data to be rendered, thereby optimizing system performance and improving user experience.

[0057] To better understand the implementation process of rendering in the SSR and SSAO combined mode provided by the embodiment of the present application, refer to Figure 2b , is another flowchart of a screen space-based rendering method provided by an embodiment of the present application. When the present application is implemented, rendering is performed based on the design idea of ​​merging SSR rendering and SSAO rendering into PASS, for example, by Figure 2b The SSAO_SSR PASS in the implementation implements the rendering of the SSR and SSAO combined mode. The implementation process includes the following steps:

[0058] Step S1: Obtain multiple pixel points in the image data as well as the depth values, normals, etc. of the pixel points. Exemplarily, the SSR and / or SSAO rendering performed in the present application is implemented based on the screen space, so the image data of the scene to be rendered in the screen space is first obtained. Specifically, after determining the sampling points to be converted to the screen space in the scene to be rendered, the determined sampling points are converted to pixel points in the screen space, and the display position of the pixel points is represented by two-dimensional coordinates in the screen space, for example, it can be represented by (x, y); and, in order to represent the exact position of the sampling points in the scene to be rendered, it can also be reflected in the screen space by a depth value, for example, it can be represented by z.

[0059] Step S2: For each of the plurality of pixel points included in the image data, a first pixel point set corresponding to each pixel point is selected. For example, the first pixel point set corresponding to each pixel point is obtained during the SSR calculation process. In order to better understand the SSR calculation, first combine Figure 3a The content shown introduces the reflection principle involved in the SSR calculation, based on the reverse ray tracing method (i.e. ray tracing with the eye as the starting point, the forward ray tracing is opposite to the reverse ray tracing direction, and is ray tracing starting from the object), after the eye looks at the reflection plane from any line of sight, it can see the pixel point (referred to as the reflection intersection in the following embodiments) that intersects with the object in the direction of the reflected light of the line of sight at point B on the reflection plane, for example, pixel point A on the object. Through the introduced reflection principle, it can be obtained that some reflection effects caused by the existence of reflected light can be seen on the reflection plane, such as imaging effects in a mirror, reflections in water, etc. Optionally, for each pixel point contained in the image data, a first set of pixel points is selected by the following steps, including: Step S21: Determine the direction of the reflected light based on the line of sight vector and the normal of each pixel point. Step S22: Select the first set of pixel points from the multiple pixel points in the direction of the reflected light.

[0060] Exemplarily, the method involves obtaining reflected light data converted from reflected light to screen space, wherein the reflected light data includes the two-dimensional coordinates of multiple sampling points selected when stepping on the reflected light and the depth values ​​of the multiple sampling points; starting from the position of the pixel on the screen, sampling points are sequentially selected in the direction of the reflected light; a pixel in the image data having the same two-dimensional coordinates as the sampling point selected on the reflected light is determined; the depth value of the sampling point with the same two-dimensional coordinates is compared with the depth value of the pixel; if the depth value of the pixel is less than the depth value of the sampling point on the reflected light, the method returns to the step of sequentially selecting sampling points in the direction of the reflected light until the depth value of the sampling point on the reflected light is less than or equal to the depth value of the pixel, thereby obtaining a reflection intersection point. Thus, the pixel points involved in determining the reflection intersection point are the first set of pixel points selected for the pixel point.

[0061] See Figure 3b For example, there is a pixel point L0 to be rendered in the image data. The direction of the reflected light of L0 is determined. Sampling points are selected in sequence from L0 along the direction of the reflected light. Then, the pixel points with the same position as the sampling point in the image data are selected. Figure 3b For L1 to L7 in the image, the depth values ​​of the sampling points of the reflected light at the same position are compared with the depth values ​​of the pixels in the image. If the depth value of pixel L1 is less than the depth value of the sampling point of the reflected light at the same position, the depth values ​​of pixels L2, L3, and so on are compared until the depth value of pixel L7 is determined to be equal to the depth value of the sampling point of the reflected light at the same position. Pixel L7 is then used as the reflection intersection point. Pixels L1 to L7 are the first set of pixels selected for pixel L0.

[0062] Step S2': perform SSAO calculation based on the first pixel set, and determine the pixel with the largest depth value in the first pixel set. Exemplarily, SSAO calculation is similar to SSR calculation, and is also processed in units of pixels contained in the image data, and in the process of performing SSAO calculation on each pixel contained in the image data, it is necessary to rely on multiple other pixels around each pixel as sampling points for processing. Taking any pixel in the image data as an example, the brightness and darkness effect of the pixel on the display screen is determined by the occlusion of the pixel to be rendered by other pixels with higher depth values ​​around the pixel. It can be understood that the brightness value of the concave area is darker and the brightness value of the convex area is brighter. Therefore, in the process of SSAO calculation, a pixel set is selected for each pixel, and the occlusion value of the pixel is calculated by the pixel with the largest depth value in the pixel set, and then the illumination intensity of the ambient light of the pixel is determined by the occlusion value.

[0063] Specifically, under normal circumstances, a possible implementation method for selecting a pixel set for each pixel is to first select multiple ambient light directions based on each pixel, wherein the multiple ambient light directions are evenly distributed around the pixel to be rendered. For example, two ambient light directions with a certain angle (generally 180°), four ambient light directions perpendicular to each other (that is, the angle between each two ambient light directions is 90°), and eight ambient light directions (the angle between each two ambient light directions is 45°) can be selected. Among them, the more ambient light directions are selected, the more the ambient light occlusion effect obtained after the SSAO calculation will be in line with the real scene. However, based on system performance and power consumption considerations, the number of selected ambient light directions can be determined according to the computing power of the system chip. Secondly, for each ambient light direction selected for each pixel, multiple other pixels are selected starting from the pixel as the pixel set in each ambient light direction. Among them, the number of pixels selected in the pixel point set can also be determined according to the computing power of the system chip. For example, 4, 8 or more pixels can be selected in each ambient light direction to obtain a pixel point set; similarly, the more pixels are selected in each ambient light direction, the more the ambient light occlusion effect obtained after the SSAO calculation will be in line with the real scene.

[0064] Through the above introduction, it can be concluded that if the image data is rendered in a combined mode of SSR and SSAO, a first set of pixels is selected in the direction of the reflected light of each pixel in the SSR calculation process in step S2, and the ambient light has a scattering characteristic. Therefore, when performing SSAO calculation on each pixel, the conditions of the selected multiple ambient lights are satisfied and evenly distributed around each pixel. In an optional example, the first set of pixels selected in the SSR process can be reused in the SSAO calculation process. When the present application is implemented, the reflected light direction of each pixel is used as the first ambient light direction, and the first set of pixels is used as the set of pixels selected for each pixel in the first ambient light direction. For example, step S2' continues to perform SSAO calculation directly based on the pixel set obtained in step S2, so there is no need to select a pixel set once in the SSR calculation process and select a pixel set again in the SSAO calculation process, as in the prior art. Therefore, the implementation of the present application reduces the amount of calculation in the SSR and SSAO calculation processes, thereby reducing system power consumption.

[0065] In addition, since multiple ambient light directions can be selected for processing during the SSAO calculation process, when the present application is implemented, after the first ambient light direction of the reflected light direction is obtained based on the SSR calculation, an optional implementation method can continue the SSAO calculation through the following steps, including: Step S2'1: Based on the reflected light direction, select at least one other ambient light direction.

[0066] For example, see Figure 4a , continue to use Figure 3a As described in the introduction, it is assumed that four ambient light directions are selected for pixel B for SSAO calculation. During implementation, the reflected light direction is used as the first ambient light direction of pixel B. Then, the second, third, and fourth ambient light directions are selected based on the first ambient light direction. The four selected ambient light directions can be perpendicular to each other in the screen space.

[0067] Step S2'2: Select at least one second pixel set from the plurality of pixels in the other ambient light direction. Exemplarily, selecting the second pixel set in the other ambient light direction may be performed by sequentially selecting, starting with each pixel, a specified number of pixels in each other ambient light direction as the second pixel set in each other ambient light direction. The specified number may be pre-set, such as selecting 4 or 8 pixels.

[0068] Step S2'3: Perform SSAO calculation based on the first pixel point set and the at least one second pixel point set. Exemplarily, after obtaining the first pixel point set and the at least one second pixel point set, determine the pixel point with the maximum depth value in each pixel point set. For example, refer to Figure 4b , assuming that another ambient light direction is selected as the second ambient light direction, for the pixel point A to be rendered, the first pixel point set in the first ambient light direction includes B1, B2, B3, and B4, where B4 has the maximum depth value; the second pixel point set in the second ambient light direction includes C1, C2, C3, and C4, where C4 has the maximum depth value.

[0069] The present application provides an implementation method for performing SSR calculation based on the first pixel point set and performing SSAO calculation based on the first pixel point set, thereby realizing the sharing of pixel point sets and pixel point information in the SSR calculation and SSAO calculation processes, thereby reducing some calculation amounts in scenarios that require rendering in a combined SSR and SSAO mode. Therefore, there is no need to select a pixel point set for each pixel point in the SSR calculation process, obtain the pixel point information stored in the G-buffer, and perform a calculation once, as in the prior art; and in the SSAO calculation process, select a pixel point set for each pixel point one by one in each selected ambient light direction, obtain the pixel point information stored in the G-buffer, and perform multiple calculations. For example, assuming that two ambient light directions are selected for SSAO calculation, in the prior art, it is necessary to obtain pixel sets twice when performing SSAO calculation, and it is also necessary to obtain pixel sets once when performing SSR calculation, so a total of three pixel set acquisition processes are required; while through the method provided in the present application, a pixel set is acquired once when performing SSR calculation, and when performing SSAO calculation, processing can be performed based on the pixel set acquired in the SSR calculation, so it is only necessary to acquire pixel sets once more. Therefore, when the present application is implemented, SSAO calculation and SSR calculation can be realized by acquiring pixel sets twice, thereby reducing the amount of calculation. It should be noted that since the image data contains multiple pixels and the scene to be rendered contains image data of multiple frames of images, if one pixel can reduce a part of the amount of calculation, then in the process of rendering the multiple frames of images contained in the scene to be rendered, a large amount of calculation will be reduced, thereby reducing system power consumption and improving user experience.

[0070] In addition, by reusing the first pixel point set selected in the SSR calculation process through SSAO calculation, a more realistic and accurate rendering effect can be achieved. The reason is that if the roughness of the object surface is greater, since in the SSR calculation process, pixel points are selected in sequence starting from each pixel point in the direction of the reflected light until the pixel point that intersects with the reflected light is calculated, the number of pixel points included in the pixel point set in the SSR calculation process is determined according to the roughness of the object where the pixel point is located in the scene to be rendered. The greater the roughness, the more pixel points are selected; on this basis, when the present application is implemented, the pixel point set selected in the SSR calculation process will continue to be used for SSAO calculation, which may be more than the number of pixel points included in the pixel point set selected when implementing SSAO calculation separately in the prior art. For example, during the SSR calculation process, 7 pixel points may be obtained in the direction of the reflected light before the reflection intersection is obtained. The original SSAO calculation requires selecting 4 pixel points in each ambient light direction as the pixel point set. Therefore, when the first pixel point set selected in the reflected light direction is used as the pixel point set in the first ambient light direction in the SSAO calculation process, the pixel point set for the SSAO calculation includes 7 pixels, thereby increasing the number of selected pixel points, thereby obtaining a more realistic occlusion effect, which is more in line with the real scene.

[0071] Step S3: Obtain the color value of the reflection intersection obtained in step S2. Step S3': Calculate the occlusion value used for SSAO based on the pixel with the maximum depth value obtained in step S2'. The order of executing steps S3 and S3' is not limited; they can be processed in parallel by different threads or in a random order.

[0072] For example, after obtaining multiple pixels with maximum depth values ​​in multiple ambient light directions selected for each pixel, assuming they are represented by h1, h2, ..., hn, the occlusion value of each pixel can be calculated for each pixel based on information such as h1, h2, ..., hn values ​​and normals using the existing occlusion value formula for SSAO rendering. The occlusion value is used to adjust the intensity of the ambient light displayed by each pixel. The larger the occlusion value, the stronger the occlusion ability of other pixels around the pixel, and the smaller the intensity of the ambient light displayed at the pixel.

[0073] Step S4: Merge the occlusion value and the color value and output them. An optional example is based on the color value obtained after the SSR calculation, which can generally be represented by three variables, such as R (red), G (green), and B (blue), and the general color value information can be carried by the four-channel color space of RGBA (alpha, the alpha channel is generally used as an opacity parameter), so it can be obtained that the utilization rate of the A channel is low when the image data is rendered by SSR based on the color value. Therefore, when the present application is implemented, the occlusion value obtained after the SSAO calculation can be merged into the A channel of the four-channel color space used to carry the color value after the SSR calculation. That is, the output result of SSAO_SSR PASS is a four-dimensional variable containing the color value for SSR rendering and the occlusion value for SSAO rendering. For example, the four-dimensional variable can be expressed in the form of (R, G, B, O), wherein the R, G, and B represent the color value respectively, and the O represents the occlusion value.

[0074] By merging the occlusion values ​​used for SSAO rendering and the color values ​​used for SSR rendering, it is possible to obtain a scene rendering image rendered in a combined SSR and SSAO mode through one rendering when displaying on a display device, without the need to perform SSAO rendering and SSR rendering separately on the scene to be rendered as in the prior art, thereby obtaining the scene rendering image after two rendering processes.

[0075] Step S5: Use the occlusion value and color value outputted by step S4 as inputs and perform time domain filtering. In order to make the final rendered scene effect diagram smoother, in the prior art, after obtaining the occlusion value and color value, it is necessary to perform time domain filtering on the occlusion value and color value respectively, so as to optimize the rendering results obtained by SSAO rendering and SSR rendering. When the present application is implemented, the color value and the occlusion value are combined and outputted as a four-dimensional variable, so the four-dimensional variable can be used as input and inputted into the filter for time domain filtering to achieve smoothing operation. Therefore, the implementation method provided by the present application can realize filtering operation on the color value used for SSR rendering and the occlusion value used for SSAO rendering through the operation of inputting the filter once, thereby reducing the number of operations for time domain filtering, thereby improving system performance, reducing system consumption, and thus improving user experience.

[0076] Step S6: Outputting the scene rendering after SSAO and SSR rendering. For example, the SSR and SSAO calculation results after temporal filtering of each frame of image data are obtained in step S5, and the calculation results are superimposed on the original image data, thereby outputting the scene rendering after SSAO and SSR rendering.

[0077] In addition, since SSR rendering is performed for multiple pixels contained in objects with reflective capabilities, a possible implementation method is that before performing SSR and SSAO calculations through SSAO_SSR PASS, the pixel points corresponding to the object after being converted to the screen space can be determined based on the materials of each object associated with the scene to be rendered (wherein, the materials of some objects have reflective capabilities, such as mirrors, water surfaces, etc.), and the image data of the material with emissive capabilities is used as input image data to perform rendering in a combined SSR and SSAO mode.

[0078] In order to better understand the scene renderings displayed on the display device after SSR rendering and SSAO rendering, the following Figure 5a 、 5b To illustrate the interface display effects after SSR rendering and SSAO rendering respectively.

[0079] Combine Figure 5a To illustrate the effect after SSR rendering, take a mirror as an example. Before SSR rendering, you can only see the basic shape and color information of the mirror from the image displayed on the display device screen, such as Figure 5a As shown in 1, no reflection effect can be seen in the mirror, and there is a lack of color. After SSR rendering, based on the SSR calculation of multiple pixels contained in the mirror, the color values ​​of the pixels intersecting in the direction of the reflected light are obtained, and the color values ​​of the reflection intersections are displayed on the corresponding pixels, so that the objects intersecting with the reflected light can be seen in the mirror, thereby obtaining a more realistic mirror perception and improving the color sense of the image, such as Figure 5a As shown in 2.

[0080] Combine Figure 5b To illustrate the effect after SSAO rendering, take curtains as an example. Before SSAO rendering, only the basic shape and color information of the curtains can be seen from the image displayed on the display device screen, such as Figure 5b As shown in Figure 1, only the basic shape of the curtains, curtain rods, curtain edges, etc. can be seen. After SSAO rendering, different occlusion values ​​are calculated based on different pixels. After adjusting the ambient light intensity of the pixel points by the occlusion value, the sense of hierarchy can be reflected by having different ambient light intensities at different pixels. That is, the folds of the curtains can be seen, thus obtaining a more realistic curtain look, such as Figure 5b As shown in 2.

[0081] Through the above introduction, the implementation of this application mainly achieves the purpose of reducing the occupation of computing resources, reducing the amount of calculation, thereby reducing system power consumption and improving system performance through three aspects. Among them, the three aspects include: First aspect: SSAO calculation reuses the first pixel point set selected by SSR calculation. Second aspect: The occlusion value rendered by SSAO and the color value rendered by SSR are merged and output. Third aspect: The occlusion value rendered by SSAO and the color value rendered by SSR are simultaneously subjected to time domain filtering.

[0082] The following test results are combined to more accurately illustrate the improvement of system performance by using a screen space-based rendering method provided by this application, as shown in Table 1:

[0083] Table 1

[0084] Number of sampling points Existing technology First aspect Second aspect The third aspect Total revenue 32 18.7 21.2(+13.3％) 22.4(+5.6％) 23.7(+5.8％) +23.1％ 16 21.2 23.7(+11.7％) 24.6(+4.1％) 25.7(+4.4％) +20.2％ 8 24.5 27.1(+10.6％) 27.2(+0.1％) 28.3(+4％) +14.7％ 2 26.3 27.8(+5.7％) 26.1(-6.1％) 27.5(+5.2％) +4.8％

[0085] According to Table 1 above, compared with the technical implementation scheme adopted in the prior art, the present application can mainly achieve the improvement of system performance through three aspects. The implementation method of at least one of the aspects can improve the system performance. If the implementation methods of the three aspects are adopted at the same time, the maximum benefit of improving the system performance can be achieved. For example, when the number of sampling points in Table 1 is 32, the system performance can be improved by 13.3% through the first aspect (i.e., SSAO calculation reuses the first pixel point set selected by SSR calculation), the system performance can be improved by 5.6% through the second aspect (i.e., merging the occlusion value rendered by SSAO and the color value rendered by SSR and outputting it), and the system performance can be improved by 5.8% through the third aspect (i.e., performing time-domain filtering on the occlusion value rendered by SSAO and the color value rendered by SSR at the same time). If the implementation methods of the three aspects are adopted for rendering at the same time, the total benefit of the system performance can be improved by 23.1%.

[0086] The screen-space-based rendering method provided in the embodiments of the present application can be executed by various computing devices, which can be electronic devices. The electronic devices can include but are not limited to personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, mobile phones, tablet computers, personal digital assistants, media players, etc.), consumer electronic devices, minicomputers, mainframe computers, mobile robots, drones, etc.

[0087] In the following embodiments, the computing device is an electronic device as an example to introduce the screen space-based rendering method provided in the embodiments of the present application. The screen space-based rendering method provided in the embodiments of the present application is suitable for Figure 6 The electronic device shown in the figure, the specific structure of the electronic device is briefly introduced below. Figure 6As shown in FIG, a hardware structure diagram of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 600 may include a processor 610 and a collection device 620. The processor 610 processes the data acquired by the collection device 620. In the embodiment of the present application, the data acquired by the collection device 620 is, for example, data of a scene to be rendered.

[0088] The processor 610 is the control center of the electronic device 600. It connects the various components of the entire electronic device using various interfaces and circuits. It executes the various functions of the electronic device 600 and processes data by running or executing software programs and / or data stored in the memory. The processor 610 may include one or more processing units. For example, the processing units included in the processor 610 may be the CPU and GPU mentioned in the above embodiments, or one or more of an application processor (AP), a modem processor, an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, a neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors. The NPU is a neural network (NN) computing processor that draws on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, to quickly process input information and can also continuously self-learn. The NPU can realize intelligent cognitive applications such as image recognition, face recognition, speech recognition, and text understanding in the electronic device 600.

[0089] The acquisition device 620 may include a camera 621 for capturing images or videos. The camera 621 may be an ordinary camera or a focus camera. Furthermore, the camera 621 may be used to capture RGB images. The acquisition device 620 may also include one or more sensors 622, such as an image sensor, an infrared sensor, a laser sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a speed sensor, a distance sensor, a proximity light sensor, an ambient light sensor, a fingerprint sensor, a touch sensor, a temperature sensor, or a bone conduction sensor. Among them, the image sensor is, for example, a time of flight (TOF) sensor or a structured light sensor. The acceleration sensor and the speed sensor may form an inertial measurement unit (IMU), which may measure the three-axis attitude angle (or angular rate) and acceleration of an object.

[0090] The electronic device may further include a memory 630. The memory 630 may be used to store software programs and data, and the processor 610 may execute various functional applications and data processing of the electronic device 600 by running the software programs and data stored in the memory 630. The memory 630 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an image acquisition function, an image recognition function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 600 (such as audio data, text information, image data, semantic maps, etc.), etc. In addition, the memory 630 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0091] The electronic device may further include a display device 640, and the display device 640 includes a display panel 641 for displaying information input by the user, information provided to the user, or one or more of the various menu interfaces of the electronic device 600. In the embodiment of the present application, the display device 640 is mainly used to display the display interface effect after real-time rendering based on the embodiment of the present application. Optionally, the display panel 641 may include a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0092] The electronic device 600 may further include an input device 650 for receiving input digital information, character information, or contact touch operations / contactless gestures, and generating signal inputs related to user settings and function control of the electronic device 600. In the embodiment of the present application, the input device 650 can receive user operations on the display interface displayed by the display device 640, thereby rendering the rendering scene in combination with the user operations. For example, based on the user's operations on the game interface, the game scene is rendered and special effects corresponding to the user operations are output.

[0093] In some embodiments, the processor 610 may include one or more interfaces. The interfaces may include a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0094] The MIPI interface can be used to connect the processor 610 to peripheral devices such as the display device 640 and the camera 621. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI) of the camera 621. In some embodiments, the processor 610 and the camera 621 communicate via the CSI interface to implement the camera function of the electronic device 600. The processor 610 and the display device 640 communicate via the DSI interface to implement the display function of the electronic device 600.

[0095] The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 610 with the camera 621, the display device 640, the sensor 622, etc.

[0096] A USB interface is an interface that complies with USB standards and specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. A USB interface can be used to connect a charger to charge the electronic device 600, or to transfer data between the electronic device 600 and peripheral devices. This interface can also be used to connect other electronic devices, such as augmented reality (AR) devices.

[0097] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 600. In other embodiments of the present application, the electronic device 600 may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0098] Although not in Figure 6 As shown in FIG, the electronic device 600 may further include a radio frequency (RF) circuit, a power supply, a flash, an external interface, a button, a motor and other possible functional modules, which are not described in detail here.

[0099] Based on the above introduction, the present invention provides a screen space-based rendering method and device, which can be applied to Figure 6 In the electronic device architecture shown. The method can provide a screen-space-based rendering solution that optimizes system functions. In the embodiments of the present application, the method and the device are based on the same technical concept. Since the principles of the method and the device for solving the problem are similar, the embodiments of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0100] In the embodiment of the present application, the computing device is described as an electronic device 600, but the present application embodiment is not limited to being executed by other types of computing devices. The screen space-based rendering method provided in the embodiment of the present application can be performed by Figure 6 The electronic device 600 shown is executed, for example, by the processor 610 in the electronic device 600 .

[0101] In view of the above method flow, the embodiment of the present application also provides a rendering device based on screen space, and the specific implementation of the device can refer to the above method flow. Based on the same inventive concept, the embodiment of the present application also provides a rendering device based on screen space, which can be Figure 6 The processor 610 shown or the software device applied to the processor 610 can be used to perform Figure 2a-Figure 4b See the process shown. Figure 7The device 700 includes an acquisition unit 701, a processing unit 702, and an output unit 703; wherein the acquisition unit 701 is used to acquire a trigger instruction; acquire image data of at least one frame of image; the processing unit 702 is used to render the image data in a specified mode in response to the trigger instruction to obtain at least one frame of rendered image, and the specified mode includes one of the following modes: a first mode, a second mode, and a third mode, the first mode is a screen space reflection SSR mode, the second mode is a screen space ambient occlusion SSAO mode, and the third mode is a combination of SSR and SSAO mode; the output unit 703 is used to output the at least one frame of rendered image.

[0102] In one possible design, the designated mode is the third mode, and the image data includes multiple pixel points in the at least one frame of image; the processing unit 702 is specifically used to: select a first pixel point set corresponding to each pixel point in the multiple pixel points for each pixel point in the multiple pixel points; perform SSR calculation based on the first pixel point set to obtain a color value; perform SSAO calculation based on the first pixel point set to obtain an occlusion value; render each pixel point according to the color value and the occlusion value to obtain at least one frame of rendered image.

[0103] In one possible design, the image data also includes the line of sight vector and normal of each pixel point; the processing unit 702 is specifically used to: determine the direction of reflected light based on the line of sight vector and normal of each pixel point; and select the first set of pixel points from the multiple pixel points in the direction of the reflected light.

[0104] In one possible design, the processing unit 702 is specifically used to: select at least one other ambient light direction based on the reflected light direction; select at least one second pixel point set from the multiple pixel points in the other ambient light direction; and perform SSAO calculation based on the first pixel point set and the at least one second pixel point set.

[0105] In one possible design, the processing unit 702 is further used to perform time-domain filtering on the color value and the occlusion value before rendering each pixel point according to the color value and the occlusion value to obtain at least one frame of rendered image.

[0106] In a possible design, the image data belongs to a material with reflective capability, and the designated mode is the third mode.

[0107] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical functional division. In actual implementation, there may be other division methods. The functional units in the embodiments of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of software functional units, or in the form of a combination of software and hardware.

[0108] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figures 2a to 4b A method according to any one of the embodiments shown.

[0109] According to the method provided in the embodiment of the present application, the present application also provides a system, which includes the aforementioned one or more terminal display devices and one or more network devices.

[0110] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0111] The electronic devices in the above-mentioned apparatus embodiments correspond to the electronic devices in the method embodiments, and the corresponding modules or units perform the corresponding steps. For example, the communication unit (transceiver) performs the receiving or sending steps in the method embodiments, and other steps except sending and receiving can be performed by the processing unit (processor). The functions of the specific units can be referred to the corresponding method embodiments. Among them, there can be one or more processors.

[0112] As used in this specification, the terms "component," "module," "system," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on one computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component across a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0113] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0117] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0119] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A screen space-based rendering method, characterized in that: include: Get the trigger instruction; Acquire image data of at least one frame of image, wherein the image data includes a plurality of pixels in the at least one frame of image; In response to the trigger instruction, performing real-time rendering on the image data in a specified mode to obtain at least one frame of real-time rendered image, the specified mode comprising one of the following modes: a first mode, a second mode, and a third mode, the first mode being a screen space reflection (SSR) mode, the second mode being a screen space ambient occlusion (SSAO) mode, and the third mode being a combination of SSR and SSAO mode; Outputting the at least one frame of real-time rendered image; Wherein, the designated mode is the third mode; The real-time rendering of the image data in a specified mode to obtain at least one frame of real-time rendered image includes: Selecting a first pixel point set corresponding to each pixel point from the plurality of pixel points for each pixel point; Performing SSR calculation based on the first pixel point set to obtain a color value; Performing SSAO calculation based on the first pixel point set to obtain an occlusion value; Each pixel is rendered in real time according to the color value and the shielding value to obtain at least one frame of real-time rendered image.

2. The method according to claim 1, characterized in that The image data further includes a sight vector and a normal line of each pixel; The selecting, from the plurality of pixel points, a first pixel point set corresponding to each pixel point in the plurality of pixel points, respectively, includes: Determine the direction of the reflected light based on the sight vector and normal of each pixel point; The first pixel point set is selected from the plurality of pixel points in the direction of the reflected light.

3. The method according to claim 2, characterized in that The performing SSAO calculation based on the first pixel point set includes: Based on the reflected light direction, select at least one other ambient light direction; Selecting at least one second pixel point set from the plurality of pixel points in the other ambient light direction; SSAO calculation is performed based on the first pixel point set and the at least one second pixel point set.

4. The method according to any one of claims 1 to 3, characterized in that: Before performing real-time rendering on each pixel according to the color value and the masking value to obtain at least one frame of real-time rendered image, the method further includes: Temporal filtering is performed on the color value and the occlusion value.

5. The method according to any one of claims 1 to 3, characterized in that: The image data belongs to a material with reflective capability, and the designated mode is the third mode.

6. A rendering device based on screen space, characterized in that: The device includes an acquisition unit, a processing unit, and an output unit; Wherein, the acquisition unit is used to acquire the trigger instruction; Acquire image data of at least one frame of image, wherein the image data includes a plurality of pixels in the at least one frame of image; The processing unit is configured to, in response to the trigger instruction, perform real-time rendering on the image data in a specified mode to obtain at least one frame of real-time rendered image, wherein the specified mode includes one of the following modes: a first mode, a second mode, and a third mode, wherein the first mode is a screen space reflection (SSR) mode, the second mode is a screen space ambient occlusion (SSAO) mode, and the third mode is a combination of SSR and SSAO mode; The output unit is configured to output the at least one frame of real-time rendered image; Wherein, the designated mode is the third mode; The processing unit is specifically configured to: Selecting a first pixel point set corresponding to each pixel point from the plurality of pixel points for each pixel point; Performing SSR calculation based on the first pixel point set to obtain a color value; Performing SSAO calculation based on the first pixel point set to obtain an occlusion value; Each pixel is rendered in real time according to the color value and the shielding value to obtain at least one frame of real-time rendered image.

7. The device according to claim 6, characterized in that The image data further includes a sight vector and a normal line of each pixel; The processing unit is specifically configured to: Determine the direction of the reflected light based on the sight vector and normal of each pixel point; The first pixel point set is selected from the plurality of pixel points in the direction of the reflected light.

8. The device according to claim 7, characterized in that The processing unit is specifically configured to: Based on the reflected light direction, select at least one other ambient light direction; Selecting at least one second pixel point set from the plurality of pixel points in the other ambient light direction; SSAO calculation is performed based on the first pixel point set and the at least one second pixel point set.

9. The device according to any one of claims 6 to 8, characterized in that The processing unit is further configured to perform time-domain filtering on the color value and the occlusion value before performing real-time rendering on each pixel point according to the color value and the occlusion value to obtain at least one frame of real-time rendered image.

10. The device according to any one of claims 6 to 8, characterized in that The image data belongs to a material with reflective capability, and the designated mode is the third mode.

11. A computing device, characterized in that The computing device includes a processor and a memory; The memory stores computer program instructions; The processor calls the computer program instructions in the memory to perform the method according to any one of claims 1 to 5.

12. A computing device cluster, characterized in that: A plurality of computing devices according to claim 11 are included.

13. A computer program product, characterized in that The method comprises computer instructions, which, when executed on a computing device, cause the computer to perform the method according to any one of claims 1 to 5.

Images