Image rendering method and device, computer device and storage medium

Abstract

The application relates to an image rendering method and device, computer equipment and a storage medium. The application relates to the field of games and comprises the following steps: creating a plurality of rendering threads corresponding to a mirror reflection rendering pipeline; the mirror reflection rendering pipeline is a rendering pipeline for rendering an image with a mirror reflection effect; a rendering stage in the mirror reflection rendering pipeline is used to implement a plurality of rendering tasks related to the mirror reflection, each rendering thread corresponds to at least one rendering task; for each rendering thread, recording rendering instructions of the rendering task corresponding to the rendering thread by using the rendering thread; the rendering instructions of the rendering task are instructions required to be executed to implement the rendering task; and the rendering instructions of each rendering task recorded are executed, so that a target image with a mirror reflection effect is rendered. The method can improve rendering efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image rendering method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of computer technology, rendering technology emerged. Rendering technology is the process of projecting a model in a 3D scene into a 2D digital image according to predefined environment, lighting, materials, and rendering parameters. A model is a description of a 3D object, including geometry, viewpoint, texture, and lighting information.

[0003] Since rendering is a complex process, the rendering time will increase when the scene to be rendered is more complex. Taking the complex scene of mirror reflection rendering as an example, the rendering process is more complicated because the mirror reflection effect needs to be rendered. Using traditional rendering methods will result in long rendering time and low rendering efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide an image rendering method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve rendering efficiency in response to the above-mentioned technical problems.

[0005] On one hand, this application provides an image rendering method. The method includes: creating multiple rendering threads corresponding to a specular reflection rendering pipeline; the specular reflection rendering pipeline is a rendering pipeline used to render images with specular reflection effects; the rendering stage in the specular reflection rendering pipeline is used to implement multiple rendering tasks related to specular reflection, and each rendering thread corresponds to at least one rendering task; for each rendering thread, using the rendering thread to record the rendering instructions of the rendering task corresponding to the rendering thread; the rendering instructions of the rendering task are the instructions required to implement the rendering task; executing the recorded rendering instructions of each rendering task to render a target image with a specular reflection effect.

[0006] On the other hand, this application also provides an image rendering apparatus. The apparatus includes: a rendering thread creation module, used to create multiple rendering threads corresponding to a specular reflection rendering pipeline; the specular reflection rendering pipeline is a rendering pipeline used to render images with specular reflection effects; the rendering stage in the specular reflection rendering pipeline is used to implement multiple rendering tasks related to specular reflection, each rendering thread corresponding to at least one rendering task; a rendering instruction recording module, used to record, for each rendering thread, rendering instructions for the rendering task corresponding to the rendering thread; the rendering instructions for the rendering task are instructions required to execute to implement the rendering task; and a rendering instruction execution module, used to execute the recorded rendering instructions for each rendering task to render a target image with a specular reflection effect.

[0007] In some embodiments, the rendering instruction execution module is further configured to: sort the rendering instructions of each rendering task according to a preset stage in the specular reflection rendering pipeline according to the rendering stage corresponding to each rendering task, arrange the rendering instructions of each rendering task to obtain a target rendering instruction sequence; and execute the rendering instructions of each rendering task in the target rendering instruction sequence in sequence.

[0008] In some embodiments, the rendering thread belongs to the central processing unit (CPU), and the rendering instruction execution module is further configured to: sort the rendering instructions of each rendering task in the specular reflection rendering task according to the preset stage of the rendering stage corresponding to each rendering task through the CPU, arrange the rendering instructions of each rendering task to obtain a target rendering instruction sequence belonging to the CPU, add the target rendering instruction sequence to the rendering instruction library of the graphics processor, and use the graphics processor to execute the rendering instructions of each rendering task in the rendering instruction library in sequence.

[0009] In some embodiments, the rendering task includes a plurality of first scene rendering tasks for rendering a first scene image and a second scene rendering task for rendering a second scene image; the first scene image is a scene image observed from the viewing perspective of a first camera; the second scene image is a scene image observed from the viewing perspective of a second camera; the rendering instruction recording module is further configured to: record the rendering instructions of each of the first scene rendering tasks using at least two of the plurality of rendering threads; record the rendering instructions corresponding to the second scene rendering task using at least one of the plurality of rendering threads; the rendering instruction execution module is further configured to: execute the rendering instructions of each of the first scene rendering tasks to generate the first scene image, and execute the rendering instructions of the second scene rendering task to generate the second scene image; and perform merging processing based on the first scene image and the second scene image to generate a target image with a mirror reflection effect.

[0010] In some embodiments, the plurality of first scene rendering tasks include an attribute map generation task and a lighting calculation task; the rendering instruction execution module is further configured to: execute the rendering instruction of the attribute map generation task to project scene elements within the field of view of the first camera into the screen space to generate a target pixel attribute map; the target pixel attribute map includes the pixel attribute values ​​of each pixel in the screen space; and execute the rendering instruction of the lighting calculation task to perform lighting calculations based on the target pixel attribute map to generate the first scene map.

[0011] In some embodiments, the scene elements correspond to a three-dimensional mesh structure; the rendering instruction execution module is further configured to: determine a current primitive from the three-dimensional mesh structure of the scene elements within the field of view of the first camera; the primitive is a basic geometric shape constituting the three-dimensional mesh structure; determine the pixels covered by the projection area of ​​the current primitive in the screen space, and obtain the covered pixels of the current primitive; calculate the depth value of the current primitive at the covered pixels, obtain a first depth value, and compare the first depth value with the current depth value of the covered pixels stored in the current depth map; when it is determined that the first depth value is less than the current depth of the covered pixels... When the value is obtained, the pixel attribute value of the current primitive at the covered pixel point is obtained to obtain the first pixel attribute value. The pixel attribute value of the covered pixel point in the current pixel attribute map is updated to the first pixel attribute value to obtain a new current pixel attribute map. The current depth value of the covered pixel point in the current depth map is updated to the first depth value to obtain a new current depth map. The step of determining the current primitive from the three-dimensional mesh structure of scene elements in the field of view of the first camera is returned until the primitives in the three-dimensional mesh structure of the scene elements are traversed. The target pixel attribute map is generated based on the current pixel attribute map when traversing the primitives in the three-dimensional mesh structure of the scene elements.

[0012] In some embodiments, there are multiple attribute map generation tasks, and at least two of the attribute map generation tasks correspond to different rendering threads. The rendering instruction recording module is further configured to: record the rendering instructions of each attribute map generation task using at least two of the multiple rendering threads. The rendering instruction execution module is further configured to: execute the rendering instructions of each attribute map generation task to project scene elements located within the field of view of the first camera in the observation space into the screen space to generate target pixel attribute maps.

[0013] In some embodiments, the rendering task further includes a scene graph merging task, and the rendering instruction execution module is further configured to: execute the rendering instructions of the scene graph merging task to determine the mirror pixels in the first scene graph based on the reflection plane and the first scene graph, and determine the reflection color value of the mirror pixels based on the mirror pixels and the second scene graph, update the color value of the mirror pixels in the first scene graph using the reflection color value of the mirror pixels, and generate a mirror reflection scene graph with a mirror reflection effect; the mirror pixels are pixels that undergo reflection; and generate a target image with a mirror reflection effect based on the mirror reflection scene graph.

[0014] In some embodiments, the rendering task further includes a user interface rendering task and a post-processing task; the rendering instruction execution module is further configured to: execute the rendering instructions of the user interface rendering task to generate a user interface image; execute the rendering instructions of the post-processing task to merge the user interface image with the specular reflection scene image to generate a target image with a specular reflection effect.

[0015] In some embodiments, the user interface rendering task and the second scene rendering task correspond to the same rendering thread, while the post-processing task corresponds to a different rendering thread than the user interface rendering task and the second scene rendering task.

[0016] In some embodiments, the rendering instruction execution module is further configured to: for each of the mirror pixels, obtain the coordinates of the mirror pixel on the reflection plane to obtain mirror coordinates; transform the mirror coordinates based on the observation matrix and projection matrix of the second camera to obtain the sampled pixel corresponding to the mirror coordinates in the screen space; determine the color value of the sampled pixel from the second scene image to obtain the reflection color value of the mirror pixel.

[0017] In some embodiments, the rendering instruction execution module is further configured to: obtain a target depth map of the first scene map; the pixel value of a pixel in the target depth map is the depth value of the pixel; determine the reflective screen area projected by the reflective plane into the screen space, and for each pixel in the reflective screen area, when the depth value of the reflective plane at the pixel is less than the depth value of the pixel in the target depth map, the pixel is determined as a mirror pixel.

[0018] On the other hand, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above-described image rendering method.

[0019] On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in the above-described image rendering method.

[0020] On the other hand, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the above-described image rendering method.

[0021] The aforementioned image rendering method, apparatus, computer equipment, storage medium, and computer program products create multiple rendering threads corresponding to a specular reflection rendering pipeline. The specular reflection rendering pipeline is used to render images with specular reflection effects. The rendering stage within the specular reflection rendering pipeline implements multiple rendering tasks related to specular reflection. Each rendering thread corresponds to at least one rendering task. For each rendering thread, the rendering instructions for the corresponding rendering task are recorded. The rendering instructions for the rendering task are the instructions required to execute the rendering task. By executing the recorded rendering instructions for each rendering task, a target image with a specular reflection effect is rendered. Because multiple rendering threads record the rendering instructions, the time consumed by the step of recording rendering instructions during the rendering process is reduced, improving the efficiency of recording rendering instructions. Furthermore, since recording rendering instructions is part of the rendering process, using multiple threads to record rendering instructions reduces the overall rendering process time, thereby improving rendering efficiency. Attached Figure Description

[0022] Figure 1 This is a diagram illustrating the application environment of the image rendering method in some embodiments;

[0023] Figure 2 This is a flowchart illustrating the image rendering method in some embodiments;

[0024] Figure 3A This is a schematic diagram illustrating the principle of planar reflection in some embodiments;

[0025] Figure 3B This is a schematic diagram illustrating the principle of screen space reflection in some embodiments;

[0026] Figure 4 These are illustrations of the effect of specular reflection in some embodiments;

[0027] Figure 5 These are illustrations of the effect of specular reflection in some embodiments;

[0028] Figure 6 These are schematic diagrams of the first scene diagram and the second scene diagram in some embodiments;

[0029] Figure 7This is a schematic diagram of a specular reflection scene in some embodiments;

[0030] Figure 8 This is a structural block diagram of the image rendering apparatus in some embodiments;

[0031] Figure 9 These are internal structural diagrams of the computer device in some embodiments;

[0032] Figure 10 This is a diagram showing the internal structure of a computer device in some embodiments. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] The image rendering method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. The data that terminal 102 or server 104 needs to process can be stored in a data storage system. The data storage system can be integrated on server 104, or it can be located in the cloud or on other servers.

[0035] Specifically, terminal 102 can create multiple rendering threads corresponding to the specular reflection rendering pipeline. For each rendering thread, the rendering instructions for the corresponding rendering task are recorded, and the recorded rendering instructions for each rendering task are executed to render a target image with a specular reflection effect. The rendering instructions for each rendering task are the instructions required to execute the rendering task. The rendering stage in the specular reflection rendering pipeline is used to implement multiple rendering tasks related to specular reflection, and each rendering thread corresponds to at least one rendering task. After obtaining the target image, terminal 102 can transmit the target image to server 104. Server 104 can store the target image or transmit it to other devices. Terminal 102 can also display the generated target image, for example, by displaying it on a screen. Alternatively, the process of generating the target image can also be executed by server 104. For example, server 104 can create multiple rendering threads corresponding to the specular reflection rendering pipeline, and for each rendering thread, the rendering instructions for the corresponding rendering task are recorded, and the recorded rendering instructions for each rendering task are executed to render a target image with a specular reflection effect. Server 104 can send the generated target image to terminal 102, and terminal 102 can display the target image.

[0036] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers. The server 104 can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0037] In some embodiments, such as Figure 2 As shown, an image rendering method is provided. This method can be executed by a server or a terminal, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0038] Step 202: Create multiple rendering threads for the specular reflection rendering pipeline; the specular reflection rendering pipeline is a rendering pipeline used to render images with specular reflection effects; the rendering stage in the specular reflection rendering pipeline is used to implement multiple rendering tasks related to specular reflection, and each rendering thread corresponds to at least one rendering task.

[0039] The rendering pipeline is the process required to render a 2D image. It can also be called a rendering pipeline and can be divided into multiple rendering stages arranged in an orderly manner. The specular rendering pipeline is used to render 2D images with a specular reflection effect. Similarly, the specular rendering pipeline can be divided into multiple rendering stages arranged in an orderly manner. The order of the rendering stages in the specular rendering pipeline can be preset. The rendering stages in the specular rendering pipeline are used to perform the rendering task. The rendering task is the task that needs to be completed to render a 2D image with a specular reflection effect.

[0040] Methods for achieving specular reflection include, but are not limited to, at least one of Reflection Probe, SSR (Screen Space Reflection), planar reflection, and RTXreflect (Ray Tracing Texel eXtreme). Reflection Probe generates an environment map by pre-baking the scene and then samples this map to achieve a specular reflection effect. SSR reflects objects existing in screen space by emitting rays pixel by pixel. Planar reflection achieves the reflection effect by setting up a reflection camera and redrawing the entire scene. RTXreflect achieves the reflection effect through ray tracing. The principle of planar reflection is to use the reflection plane as a reference, flip the coordinates according to the current camera height, and then redraw the scene with the flipped coordinates. Figure 3A The image shows the effect of planar reflection. The upper half of the water surface represents the actual scene, while the lower half represents the scene reflected from the water surface. For example, point B is the mirror image of point A in the actual scene. Figure 3B The diagram shown illustrates the principle of screen space reflection.

[0041] A rendering stage is used to implement at least one rendering task. Different rendering stages may implement different numbers of rendering tasks; for example, one rendering stage may implement one rendering task, or one rendering stage may implement two rendering tasks. Each rendering thread is matched with at least one rendering stage, and each rendering thread corresponds to at least one rendering task. The rendering task corresponding to a rendering thread is the rendering task to be implemented by the rendering stage matched with that rendering thread.

[0042] The rendering task may include a plurality of first scene rendering tasks for rendering a first scene graph and a second scene rendering task for rendering a second scene graph. The plurality of first scene rendering tasks may include at least one of shadow calculation tasks, attribute graph generation tasks, or lighting calculation tasks. The rendering task may also include at least one of user interface generation tasks, scene graph merging tasks, and post-processing tasks.

[0043] The first scene image is a view of the scene observed from the perspective of the first camera. The second scene image is a view of the scene observed from the perspective of the second camera. The second camera and the first camera are symmetrical with respect to the reflecting plane. The positions of the first camera and the second camera are symmetrical with respect to the reflecting plane, and the viewing angles of the first camera and the second camera are symmetrical with respect to the reflecting plane. One of the first camera and the second camera is a virtual camera, and the other is a real camera. For example, the first camera is a real camera, and its symmetrical position with respect to the reflecting plane is determined. The camera at that position is then designated as the location of the second camera. The real camera is located above the reflecting plane, while the virtual camera is located below it. For example, when the reflecting plane is a water surface, the first camera is located above the water surface, and the second camera is located below it. The reflecting plane is a plane where specular reflection occurs. The position of the reflecting plane in three-dimensional space can be preset or set as needed. The three-dimensional space includes, but is not limited to, at least one of world space or observation space.

[0044] The specular reflection rendering pipeline can use either a forward rendering method or a deferred rendering method. Taking the deferred rendering method as an example, the specular reflection rendering pipeline could be: UI → Shadow → GBuffer → Gbuffer Vegetation → Gbuffer terrain → lighting → Planar reflection → postprocess. Here, UI, Shadow, GBuffer, Gbuffer Vegetation, Gbuffer terrain, lighting, Planar reflection, and postprocess are each rendering stage in the specular reflection rendering pipeline. In GBuffer, G stands for Geometry. UI (User Interface) is the user interface rendering stage, corresponding to the user interface generation task, which generates the user interface graph. Shadow is the shadow calculation stage, corresponding to the shadow generation task, which generates shadows. GBuffer is the first attribute map generation stage, Gbuffer Vegetation is the second attribute map generation stage, and Gbuffer terrain is the third attribute map generation stage. The first attribute map generation stage performs the first attribute map generation task, the second attribute map generation stage performs the second attribute map generation task, and the third attribute map generation stage performs the third attribute map generation task. These three attribute map generation tasks are used to generate pixel attribute maps. Lighting is the lighting merging stage, which performs lighting calculations and scene map merging tasks. The scene map merging task merges different scene maps. The lighting calculation task performs lighting calculations to generate the scene map observed from the first camera's viewpoint. Planar reflection is the rendering stage corresponding to the second scene rendering task, which generates the scene map observed from the second camera's viewpoint. The positions of the first and second cameras are symmetrical with respect to the reflection plane, and the viewpoints of the first and second cameras are also symmetrical with respect to the reflection plane.

[0045] The reflecting plane is the plane where specular reflection occurs. The position of the reflecting plane in three-dimensional space can be preset or set as needed. Three-dimensional space includes, but is not limited to, at least one of world space or observation space. Postprocessing is the stage used to perform post-processing tasks. Observation space can also be called eye space. Observation space is the result of converting world space coordinates into coordinates in front of the user's field of vision; that is, observation space is the space observed from the camera's perspective.

[0046] Each rendering task corresponds to a rendering instruction, which is the instruction executed to implement the rendering task. Each rendering task can have one or more rendering instructions; multiple tasks refer to at least two. Rendering threads have the function of recording the rendering instructions of rendering tasks. Each rendering thread can correspond to at least one rendering task, and the rendering thread can record the rendering instructions for its corresponding rendering task. When a rendering thread corresponds to multiple rendering tasks, it can record the rendering instructions for all multiple tasks in the same sequence, or it can record the rendering instructions for each rendering task separately. For example, a rendering command sequence can be set for each rendering task, and the rendering instructions for the same rendering task can be stored in that rendering command sequence. The sequence can be represented by an array or a list. When the sequence is a list, the rendering command sequence can also be called a rendering command list.

[0047] Specifically, in response to an instruction to render an image with a specular reflection effect, the terminal can create multiple rendering threads corresponding to the specular emission rendering pipeline, and use the rendering threads to record the rendering instructions of their corresponding rendering tasks, execute the recorded rendering instructions, and generate a target image with a specular reflection effect.

[0048] In some embodiments, the terminal can utilize the image rendering method provided in this application to render multiple target images in real time. For example, when the image to be rendered with a mirror reflection effect is a scene in a game scene, the terminal can utilize the image rendering method provided in this application to generate the scene in the game scene in real time. Because the image rendering method provided in this application employs multiple rendering threads, it improves the concurrency characteristics of the image rendering process, reduces the time required to render the image, and improves rendering efficiency. This increases the number of frames rendered per unit time, for example, 1 second, thereby increasing the refresh rate of the scene in the game and improving the smoothness of the game screen. The game includes, but is not limited to, MMO games, where MMO stands for Massive Multiplayer Online Role-Playing Game.

[0049] Step 204: For each rendering thread, use the rendering thread to record the rendering instructions of the rendering task corresponding to the rendering thread; the rendering instructions of the rendering task are the instructions that need to be executed to implement the rendering task.

[0050] Rendering instructions are the commands executed during the image rendering process. Each rendering task corresponds to one or more rendering instructions. Different rendering instructions can have different functions. A rendering task corresponding to a rendering thread is the rendering task to be implemented by the rendering stage matched with that rendering thread. For example, if rendering stage 1 matches rendering thread 1 and is used to implement the user interface generation task, then the rendering task corresponding to rendering thread 1 is the user interface generation task. Each rendering thread matches at least one rendering stage, therefore each rendering thread can correspond to at least one rendering task.

[0051] Specifically, each rendering task can correspond to a rendering instruction sequence. For each rendering task, the terminal can use the rendering thread corresponding to the rendering task to record the rendering instructions of that rendering task into the rendering instruction sequence corresponding to that rendering task. Each rendering thread can record at least one rendering instruction sequence. For example, rendering thread 1 corresponds to two rendering tasks, namely rendering task 1 and rendering task 2. The rendering instruction sequence of rendering task 1 is sequence L1, and the rendering instruction sequence of rendering task 2 is sequence L2. Then, rendering thread 1 can record each rendering instruction of rendering task 1 into sequence L1 and each rendering instruction of rendering task 2 into sequence L2, thereby obtaining two rendering instruction sequences.

[0052] Step 206: Execute the rendering instructions for each recorded rendering task to render the target image with a specular reflection effect.

[0053] The target image is a rendered image with a specular reflection effect. The rendering instructions for each rendering task can be executed in parallel or sequentially. For example, they can be executed sequentially according to the preset stages in the specular reflection rendering pipeline, based on the rendering stage to which the rendering task belongs.

[0054] Specifically, the terminal can arrange the rendering instructions of each rendering task to obtain a target rendering instruction sequence, which includes the rendering instructions of each rendering task. For example, if there are two rendering tasks, rendering task 1 and rendering task 2, the rendering instructions of rendering task 1 and rendering task 2 are arranged. If the resulting sequence is [rendering instructions of rendering task 1, rendering instructions of rendering task 2], then [rendering instructions of rendering task 1, rendering instructions of rendering task 2] can be determined as the target rendering instruction sequence. The terminal can sort the rendering instructions according to the target rendering instruction sequence, and sequentially obtain and execute the obtained rendering instructions. By executing the rendering instructions of each rendering task, a target image with a specular reflection effect can be rendered. For example, if the target rendering instruction sequence is [rendering instructions of rendering task 1, rendering instructions of rendering task 2], then the rendering instructions of rendering task 1 are executed first, followed by the rendering instructions of rendering task 2, thereby rendering a target image with a specular reflection effect.

[0055] In some embodiments, the terminal can arrange the rendering instructions of each rendering task according to the preset stage order of the rendering stage in which the rendering task is located, to obtain a target rendering instruction sequence. The earlier the preset stage order of the rendering stage in which the rendering task is located, the earlier the rendering instructions of the rendering task are in the target rendering instruction sequence. The preset stage order of the rendering stage refers to the order of the rendering stage in the specular reflection rendering pipeline. For example, the specular reflection rendering pipeline is "UI→Shadow→GBuffer→Gbuffer Vegetation→Gbuffer terrain→lighting→Planarreflection→postprocess". The specular reflection rendering pipeline is divided into 8 rendering stages. The preset stage order of UI is the earliest (i.e., 1), and the preset stage order of postprocess is the latest (i.e., 8). For example, if there are 2 rendering tasks, namely rendering task 1 and rendering task 2, and the preset stage order of the rendering stage in which rendering task 1 is located is 2, then the target rendering instruction sequence is [rendering instructions of rendering task 2, rendering instructions of rendering task 1].

[0056] In some embodiments, each rendering task corresponds to multiple rendering instructions. For each rendering task, the terminal can arrange the multiple rendering instructions of the rendering task into a sub-rendering instruction sequence, resulting in one sub-rendering instruction sequence for each rendering task. The terminal can arrange the sub-rendering instruction sequences of each rendering task according to a preset stage of the rendering phase in which the rendering task is located, to obtain the target rendering instruction sequence. The earlier the preset stage of the rendering phase in which the rendering task is located is in the order, the earlier the sub-rendering instruction sequence of the rendering task is in the target rendering instruction sequence.

[0057] In some embodiments, the terminal can execute the sub-rendering instruction sequences sequentially according to their order in the target rendering instruction sequence. For example, the sub-rendering instruction sequences that appear earlier in the sequence can be executed first. For instance, if the target rendering instruction sequence is [sub-rendering instruction sequence 1, sub-rendering instruction sequence 2], then sub-rendering instruction sequence 1 will be executed first, followed by sub-rendering instruction sequence 2, thereby rendering a target image with a mirror reflection effect.

[0058] In the image rendering method described above, multiple rendering threads are created for the specular reflection rendering pipeline. The specular reflection rendering pipeline is used to render images with specular reflection effects. The rendering stages within the specular reflection rendering pipeline implement multiple rendering tasks related to specular reflection. Each rendering thread corresponds to at least one rendering task. For each rendering thread, the rendering instructions for the corresponding rendering task are recorded. These instructions are the commands required to execute the rendering task. The rendering task corresponding to the rendering thread is the rendering task to be implemented by the rendering stage matched to that thread. By executing the recorded rendering instructions for each rendering task, a target image with a specular reflection effect is rendered. Because multiple rendering threads record the rendering instructions, the time consumed by recording these instructions during the rendering process is reduced, improving the efficiency of recording rendering instructions. Furthermore, since recording rendering instructions is part of the rendering process, using multiple threads to record them reduces the overall rendering time, thereby improving rendering efficiency.

[0059] The image rendering method provided in this application improves the efficiency of achieving specular reflection effects without compromising the reflection quality. This method can be applied to the gaming industry, achieving specular reflection with lower performance costs, allowing players to smoothly experience the visual enjoyment brought by specular reflection effects in games. Figure 4 and Figure 5 As shown, this displays a scene in the game with a mirror-like reflection effect. Figure 4 Taking character 402 as an example, character 404 is obtained by reflecting character 402 through a mirror.

[0060] In some embodiments, executing the rendering instructions of each recorded rendering task to render a target image with a specular reflection effect includes: sorting the rendering instructions of each rendering task according to the preset stages in the specular reflection rendering pipeline, arranging the rendering instructions of each rendering task to obtain a target rendering instruction sequence; and sequentially executing the rendering instructions of each rendering task in the target rendering instruction sequence.

[0061] Specifically, the terminal can sort the rendering instructions of each rendering task according to the preset stages in the specular rendering pipeline, and arrange the rendering instructions of each rendering task to obtain the target rendering instruction sequence. The earlier the preset stage is sorted, the earlier the rendering instructions of the rendering task are sorted in the target rendering instruction sequence.

[0062] In some embodiments, the terminal retrieves rendering instructions from the target rendering instruction sequence according to the order of the rendering instructions of the rendering tasks in the target rendering instruction sequence, and executes the retrieved rendering instructions, prioritizing the rendering instructions of the rendering tasks that appear earlier in the sequence. For example, if the target rendering instruction sequence is [rendering instructions for rendering task 1, rendering instructions for rendering task 2, rendering instructions for rendering task 3], then the rendering instructions for rendering task 1 are executed first, followed by the rendering instructions for rendering task 2, and finally the rendering instructions for rendering task 3. It should be noted that the rendering instructions for the later-executed rendering tasks can be executed before the rendering instructions for the earlier-executed rendering tasks have been completed. For example, if the rendering instructions for rendering task 1 are the earlier-executed rendering instructions, the rendering instructions for rendering task 2 can be executed before the rendering instructions for rendering task 1 have been completed.

[0063] In this embodiment, the rendering instructions of each rendering task are arranged according to the preset stages in the specular reflection rendering pipeline corresponding to each rendering task, to obtain the target rendering instruction sequence. The rendering instructions of each rendering task in the target rendering instruction sequence are executed in sequence, thereby enabling the rendering tasks to be executed in an orderly manner, improving the rationality of the execution of rendering tasks, reducing the time consumed by the execution of rendering tasks, and improving the rendering efficiency.

[0064] In some embodiments, the rendering thread belongs to the central processing unit (CPU). Arranging the rendering instructions of each rendering task according to the preset stages in the specular rendering pipeline for each rendering task, and obtaining the target rendering instruction sequence, includes: using the CPU to arrange the rendering instructions of each rendering task according to the preset stages in the specular rendering task for each rendering task, obtaining the target rendering instruction sequence belonging to the CPU; adding the target rendering instruction sequence to the graphics processing unit's (GPU) rendering instruction library; and sequentially executing the rendering instructions of each rendering task in the target rendering instruction sequence, including: using the GPU to sequentially execute the rendering instructions of each rendering task in the rendering instruction library.

[0065] In this context, the central processing unit (CPU) refers to the CPU, and the graphics processing unit (GPU) refers to the GPU. The rendering instruction library stores rendering instructions to be executed; it belongs to the GPU. The rendering instruction library can be a storage unit of a predefined data structure, including but not limited to arrays or queues. When a queue is used, the rendering instruction library can also be called a rendering instruction queue.

[0066] Specifically, the terminal can create multiple rendering threads in the CPU for the specular rendering pipeline. These threads record the rendering instructions for each task and transmit them to the GPU, which then executes them. This allows the CPU to store the recorded instructions in the rendering instruction library and perform other tasks, such as recording the rendering instructions for the next image, without waiting for the GPU to complete its execution. In other words, the CPU can execute ahead of the GPU. While the GPU is executing its rendering tasks, the CPU can proceed directly to the next frame's logic without waiting for the GPU: for example, if the GPU is in frame n, the CPU might be in frame n+1 or n+2. This asynchronous characteristic reduces the time spent on synchronization between the CPU and GPU, thereby improving the data processing efficiency of both the CPU and GPU, saving rendering time, and increasing overall rendering efficiency.

[0067] In some embodiments, the terminal can arrange the rendering instructions of each rendering task through the central processing unit (CPU) to obtain a target rendering instruction sequence, which belongs to the CPU. For example, the CPU can arrange the rendering instructions of each rendering task according to a preset stage in the specular reflection rendering task, based on the rendering stage corresponding to the rendering task, to obtain a target rendering instruction sequence belonging to the CPU.

[0068] In some embodiments, each rendering task corresponds to multiple rendering instructions, where multiple means two or more. Each rendering task also corresponds to a preset sub-rendering instruction sequence. The preset sub-rendering instruction sequence can be an empty sequence, i.e., a sequence that does not contain any data. For each rendering task, the terminal can control the rendering thread corresponding to that rendering task to add each rendering instruction of that rendering task to the sub-rendering instruction sequence, thereby recording each rendering instruction of the rendering task in the sub-rendering instruction sequence. The terminal can arrange the sub-rendering instruction sequences of each rendering task that have recorded the rendering instructions, and determine the result of the arrangement as the target rendering instruction sequence. The target rendering instruction sequence includes the sub-rendering instruction sequences corresponding to each rendering task. The preset sub-rendering instruction sequence can be in list form; for example, the terminal can create an empty list for each rendering task and determine this list as the preset sub-rendering instruction sequence.

[0069] In some embodiments, the terminal can sequentially add the rendering instructions of the rendering tasks in the target rendering instruction sequence to the rendering instruction library. Specifically, the terminal obtains the rendering instructions of the current rendering task from the target rendering instruction sequence, adds the rendering instructions of the current rendering task to the rendering instruction library, determines the subsequent rendering task of the current rendering task as the new current rendering task, and returns to the step of obtaining the rendering instructions of the current rendering task from the target rendering instruction sequence, until the rendering instructions of each rendering task in the target rendering instruction sequence are added to the rendering instruction library. The preset stage order of the rendering stage to which the subsequent rendering task of the current rendering task is located is after the preset stage order of the rendering stage to which the current rendering task is located, and the preset stage order of the rendering stage to which the subsequent rendering task of the current rendering task is located is adjacent to the preset stage order of the rendering stage to which the current rendering task is located. For example, if the preset stage order of the rendering stage to which the current rendering task is located is 1, then the preset stage order of the rendering stage to which the subsequent rendering task of the current rendering task is located is 2. Since the rendering instructions in the target rendering instruction sequence are arranged according to the rendering stage of the rendering task, the rendering instructions in the rendering instruction library are also arranged according to the rendering stage of the rendering task. Rendering commands in a rendering task have execution priorities, which are determined by the preset stage order of the rendering phases. For example, the earlier a stage appears in the preset stage order, the higher the execution priority of its rendering commands. The terminal can execute the rendering commands of each rendering task sequentially according to their execution priorities.

[0070] In this embodiment, the rendering thread belongs to the central processing unit (CPU). The CPU sorts the rendering instructions of each rendering task according to a preset stage within the specular reflection rendering task, resulting in a target rendering instruction sequence belonging to the CPU. This target rendering instruction sequence is added to the graphics processing unit's (GPU) rendering instruction library. The GPU then executes the rendering instructions of each rendering task in the rendering instruction library sequentially. Multiple rendering threads within the CPU record the rendering instructions and store them in the GPU's rendering instruction library. By executing the rendering instructions in the library, the CPU can perform other tasks after storing the recorded instructions, such as recording the rendering instructions for the next image to be rendered, without waiting until the GPU has finished executing all its instructions. For example, the CPU can execute before the GPU. In other words, when the GPU has not finished executing all its rendering tasks, the CPU can directly enter the logical operation of the next frame without waiting for the GPU: for example, when the GPU is in frame n, the CPU might be in frame n+1 or n+2. This asynchronous characteristic also reduces the time spent by the CPU and GPU on synchronization, thereby improving the data processing efficiency of the central processing unit and graphics processing unit, saving the time required for rendering, and improving rendering efficiency.

[0071] In some embodiments, the rendering task includes a plurality of first scene rendering tasks for rendering a first scene image and a second scene rendering task for rendering a second scene image; the first scene image is a scene image observed from the viewing perspective of a first camera; the second scene image is a scene image observed from the viewing perspective of a second camera; for each rendering thread, recording the rendering instructions of the rendering task corresponding to the rendering thread includes: using at least two of the plurality of rendering threads to record the rendering instructions of each of the first scene rendering tasks; using at least one of the plurality of rendering threads to record the rendering instructions corresponding to the second scene rendering task; executing the recorded rendering instructions of each rendering task to render a target image with a specular reflection effect includes: executing the rendering instructions of each of the first scene rendering tasks to generate a first scene image; and executing the rendering instructions of the second scene rendering task to generate a second scene image; and merging the first scene image and the second scene image to generate a target image with a specular reflection effect.

[0072] The first and second cameras can be cameras in three-dimensional space (e.g., world space), which can include various scene elements, including but not limited to at least one of a game scene or a virtual reality scene. The viewing angles of the first and second cameras are symmetrical with respect to the reflecting plane.

[0073] The specular reflection rendering pipeline includes multiple first-scene rendering stages and second-scene rendering stages. The first-scene rendering stages implement the first scene rendering task, and the second-scene rendering stages implement the second scene rendering task. There are multiple first-scene rendering stages, and the rendering thread matched by the first-scene rendering stage and the rendering thread matched by the second-scene rendering stage can be the same or different. For example, the second-scene rendering stage may match the same rendering thread as at least one of the first-scene rendering stages, or the rendering thread matched by the second-scene rendering stage may be different from the rendering threads matched by each of the first-scene rendering stages. Taking the specular reflection rendering pipeline as: UI→Shadow→GBuffer→Gbuffer Vegetation→Gbuffer terrain→lighting→Planar reflection→postprocess as an example, the first-scene rendering stages may include Shadow, GBuffer, Gbuffer Vegetation, and Gbuffer terrain stages, and the second-scene rendering stage is Planar reflection. The specular reflection rendering pipeline may also include a composite rendering stage, which is used to implement other tasks besides the first-scene rendering tasks, such as scene merging. For example, the lighting stage belongs to the composite rendering stage and is used to perform lighting calculation tasks (belonging to the first scene rendering task, used to perform lighting calculations to generate the first scene graph) and scene graph merging tasks (used to merge the first scene graph with the second scene graph).

[0074] The first scene rendering task is used to render the scene image observed from the perspective of the first camera; the second scene rendering task is used to render the scene image observed from the perspective of the second camera. The first scene image is the rendered scene image observed by the first camera, that is, the first scene image represents the scene image observed in three-dimensional space from the perspective of the first camera as the observer; the second scene image is the rendered scene image observed by the second camera, that is, the second scene image represents the scene image observed in three-dimensional space from the perspective of the second camera as the observer.

[0075] The execution order of the rendering instructions of each first scene rendering task and the rendering instructions of the second scene rendering task can be arbitrary. For example, the rendering instructions of at least one first scene rendering task can be executed first, and then the rendering instructions of the second scene rendering task can be executed. Alternatively, the rendering instructions of each first scene rendering task and the rendering instructions of the second scene rendering task can be executed in parallel. Of course, the rendering instructions of the second scene rendering task can also be executed first, and then the rendering instructions of each first scene rendering task can be executed.

[0076] Specifically, the terminal can acquire scene elements within the field of view of the first camera and generate a first scene image based on these elements. Similarly, the terminal can acquire scene elements within the field of view of the second camera and generate a second scene image based on these elements. The method for generating the second scene image is the same as that for generating the first scene image. The method for generating the first scene image will be described in detail below, and the method for generating the second scene image can be found in the description of generating the first scene image. The resolution of both the first and second scene images is a preset resolution, which can be pre-set as needed, for example, 1600*1024.

[0077] Scene elements are things within a scene, including but not limited to at least one of living or inanimate things. Living things include, but are not limited to, at least one of plants, animals, or people. Inanimate things include, but are not limited to, at least one of buildings or the ground. The scene can be a virtual scene, including but not limited to game scenes or virtual reality scenes. When the scene is a game scene, the scene elements are things within the game scene.

[0078] In some embodiments, the terminal may utilize at least two of the multiple rendering threads to record the rendering instructions for each first scene rendering task, or it may utilize two or three rendering threads to record the rendering instructions for each first scene rendering task. The terminal may determine the rendering instructions that the rendering thread needs to record based on the rendering stage it is matched with. For example, if the rendering stage matched with the rendering thread includes the attribute graph generation stage, then the rendering thread is used to record the rendering instructions for the attribute graph generation task corresponding to the attribute graph generation stage.

[0079] In some embodiments, each first scene rendering stage includes multiple attribute graph generation stages, each attribute graph generation stage corresponding to an attribute graph generation task. At least two of the multiple attribute graph generation stages are matched with different rendering threads. For example, there are three attribute graph generation stages: a first attribute graph generation stage, a second attribute graph generation stage, and a third attribute graph generation stage. There are also three rendering threads: rendering thread 1, rendering thread 2, and rendering thread 3. The first and second attribute graph generation stages are matched with rendering thread 1, and the third attribute graph generation stage is matched with rendering thread 2. Rendering thread 1 is used to record the rendering instructions of the attribute graph generation task corresponding to the first attribute graph generation stage, rendering thread 1 is used to record the rendering instructions of the attribute graph generation task corresponding to the second attribute graph generation stage, and rendering thread 2 is used to record the rendering instructions of the attribute graph generation task corresponding to the third attribute graph generation stage.

[0080] In some embodiments, the terminal may utilize at least one of multiple rendering threads to record the rendering instructions corresponding to the second scene rendering task. For example, a single rendering thread may be used to record the rendering instructions corresponding to the second scene rendering task. The rendering thread that records the rendering instructions for the second scene rendering task and the rendering thread that records the rendering instructions for the first scene rendering task may be the same thread or different threads. For example, rendering thread 1 and rendering thread 2 may be used to record the rendering instructions for each of the first scene rendering tasks, and rendering thread 3 may be used to record the rendering instructions for the second scene rendering task, or either rendering thread 1 or rendering thread 2 may be used to record the rendering instructions for the second scene rendering task.

[0081] In some embodiments, each first scene rendering stage includes multiple attribute map generation stages, which are used to implement attribute map generation tasks. These multiple first scene rendering tasks include at least one attribute map generation task, which generates target pixel attribute maps. The attribute map generation tasks can generate one or more target pixel attribute maps. "Multiple" means at least two, for example, three target pixel attribute maps can be generated. The resolution of the target pixel attribute maps is a preset resolution. A lighting calculation stage is used to implement lighting calculation tasks. The terminal can execute rendering instructions for the attribute map generation tasks to generate the respective target pixel attribute maps. Each first scene rendering stage may also include a lighting calculation stage, which implements lighting calculation tasks and executes rendering instructions for the lighting calculation tasks to perform lighting calculations based on the respective target pixel attribute maps to generate a first scene map. Alternatively, each first scene rendering stage may not include a lighting calculation stage. The lighting calculation task may correspond to a composite rendering stage, which implements the lighting calculation task and at least one other rendering task (e.g., a scene map merging task).

[0082] In some embodiments, the rendering stage may include a scene graph merging stage, which is used to implement the scene graph merging task. After the terminal obtains the first scene graph and the second scene graph, it can execute the rendering instructions of the scene graph merging task to perform merging processing based on the first scene graph and the second scene graph to generate a target image with a mirror reflection effect.

[0083] In some embodiments, each first scene rendering stage does not include a lighting calculation stage, the rendering stage does not include a scene graph merging stage, and the rendering stage includes a composite rendering stage, which is used to implement the lighting calculation task and the scene graph merging task.

[0084] In some embodiments, the merging process includes: determining a mirror pixel from a first scene image, determining a sampling pixel corresponding to the mirror pixel from a second scene image, obtaining the color value of the sampling pixel from the second scene image, and updating the color value of the mirror pixel in the first scene image using the color value of the sampling pixel, that is, updating the color value of the mirror pixel in the first scene image to the color value of the sampling pixel in the second scene image. Here, a mirror pixel is a pixel that presents the phenomenon of mirror reflection, that is, a pixel used to present the virtual image formed by mirror reflection. A sampling pixel is a pixel used to sample the reflection color value corresponding to the mirror pixel.

[0085] In this embodiment, at least two of the multiple rendering threads are used to record the rendering instructions for each first scene rendering task; at least one of the multiple rendering threads is used to record the rendering instructions for the second scene rendering task, thereby reducing the time consumed by the step of recording rendering instructions during the rendering process and improving the efficiency of generating images with mirror reflection effects.

[0086] In some embodiments, the plurality of first scene tasks include an attribute map generation task and a lighting calculation task; executing the rendering instructions of the first scene rendering task to generate the first scene map includes: executing the rendering instructions of the attribute map generation task to project scene elements within the field of view of the first camera into the screen space to generate a target pixel attribute map; the target pixel attribute map includes the pixel attribute values ​​of each pixel in the screen space; executing the rendering instructions of the lighting calculation task to perform lighting calculations based on the target pixel attribute map to generate the first scene map.

[0087] The rendering phase can include a lighting calculation phase, which performs lighting calculations, and an attribute graph generation phase, which performs attribute graph generation. In short, the specular rendering pipeline includes both the attribute graph generation phase and the lighting calculation phase, which are responsible for generating the first scene graph.

[0088] Pixel attribute values ​​are the values ​​of pixel attributes, which include at least one of color, normal, or specular values. There can be multiple target pixel attribute maps, including but not limited to at least one of a target color map, a target normal map, and a target specular map. A color map uses color values ​​as pixel values, a normal map uses normal values ​​as pixel values, and a specular map uses specular values ​​as pixel values. The pixel attribute value in the i-th row and j-th column of the target pixel attribute map is the attribute value of the pixel in the i-th row and j-th column of the screen corresponding to the screen space. i and j are positive integers. The color map includes the color values ​​of each pixel in the screen space, the normal map includes the normal values ​​of each pixel in the screen space, and the specular map includes the specular values ​​of each pixel in the screen space. The depth value refers to the distance of a pixel from the camera in three-dimensional space; the larger the depth value, the farther the pixel is from the camera. Depth testing is used to determine the pixel attribute map based on the depth value. Each screen space corresponds to one screen, and the screen resolution is the preset resolution mentioned above, meaning the screen resolution is the same as the resolution of the first scene image. Pixel values ​​can be represented using either RGB or RGBA.

[0089] Specifically, the terminal can utilize at least two of the multiple rendering threads to record the rendering instructions for the attribute graph generation task and the lighting calculation task. The rendering thread recording the rendering instructions for the lighting calculation task is different from the rendering thread recording the rendering instructions for the attribute graph generation task. For example, rendering thread 1 can be used to record the rendering instructions for the attribute graph generation task, and rendering thread 3 can be used to record the rendering instructions for the lighting calculation task. There can be one or more attribute graph generation tasks. When there are multiple attribute graph generation tasks, each attribute graph generation task corresponds to an attribute graph generation stage. Each attribute graph generation stage can be matched with the same rendering thread or with different rendering threads. For example, at least two of the attribute generation stages can be matched with different rendering threads, thereby recording the rendering instructions for each attribute generation stage through at least two rendering threads.

[0090] In some embodiments, the terminal can generate an initial pixel attribute map with a resolution of the aforementioned preset resolution. The terminal can determine the initial pixel attribute map as the current pixel attribute map. The terminal can determine the current scene element from scene elements within the field of view of the first camera. The current scene element can be any scene element within the field of view of the first camera. The terminal projects the current scene element onto the screen space and performs a depth test. Based on the depth test result, the terminal updates the current pixel attribute map and determines the updated current pixel attribute map as the new current pixel attribute map. The terminal returns to the step of determining the current scene element from scene elements within the field of view of the first camera, until all scene elements within the field of view of the first camera are traversed. The updated current pixel attribute map obtained while traversing all scene elements within the field of view of the first camera is determined as the target pixel attribute map. The resolution of the initial pixel attribute map is the aforementioned preset resolution. The initial pixel attribute map can be any one of an initial color map, an initial normal map, and an initial specular map. The pixel attribute values ​​in the initial pixel attribute map can be preset. For example, the color value in the initial color map is the color value corresponding to the background color, such as black.

[0091] Depth testing is used to determine the unoccluded portion of a 3D object, which is the part ultimately displayed on the screen. The depth testing process is as follows: the depth value of a pixel corresponding to an object is compared with the depth value of that pixel stored in the depth map. If the depth value is less than the depth value stored in the depth buffer, the depth map and pixel attribute map are updated using the pixel's corresponding depth value and attribute value on the object. If the depth value is greater than the currently stored depth value, the depth map and pixel attribute map are not updated.

[0092] When the initial pixel attribute map is the initial color map, the obtained target pixel attribute map is the target color map. When the initial pixel attribute map is the initial normal map, the obtained target pixel attribute map is the target normal map. When the initial pixel attribute map is the initial specular map, the obtained target pixel attribute map is the target specular map. Taking the initial pixel attribute map as the initial color map as an example, the terminal can determine the initial color map as the current color map. The terminal can determine the current scene element from the scene elements within the field of view of the first camera. The current scene element can be any scene element within the field of view of the first camera. The current scene element is projected onto the screen space and a depth test is performed. Based on the depth test result, the current color map is updated, and the updated current color map is determined as the new current color map. The process of determining the current scene element from the scene elements within the field of view of the first camera is repeated until all scene elements within the field of view of the first camera are traversed. The updated current color map obtained when traversing all scene elements within the field of view of the first camera is determined as the target color map.

[0093] In some embodiments, scene elements correspond to a 3D mesh structure. The terminal can generate an initial depth map with a resolution of the aforementioned preset resolution. The depth values ​​in the initial depth map are initial depth values, which are greater than a depth threshold. The depth threshold is set as needed and can be greater than the largest distance between the pixel and the camera in 3D space; that is, the depth threshold should be as large as possible. The terminal can determine the initial depth map as the current depth map. The pixel values ​​in the depth map represent the distance (i.e., depth) between the pixel and the camera. The steps described above—projecting the current scene element into the screen space and performing a depth test, updating the current pixel attribute map based on the depth test results, and determining the updated current pixel attribute map as the new current pixel attribute map—include: determining the current primitive from the 3D mesh structure of the current scene element; determining the pixels covered by the projection area of ​​the current primitive in the screen space; obtaining the covered pixels of the current primitive; determining the depth value of the current primitive at the covered pixels as the first depth value; comparing the first depth value with the current depth value of the covered pixels stored in the current depth map; when the first depth value is determined to be less than the current depth value, obtaining the pixel attribute value of the current primitive at the covered pixels; obtaining the first pixel attribute value; and then... The pixel attribute value of the covered pixel in the previous pixel attribute map is updated to the first pixel attribute value, resulting in an updated current pixel attribute map. Similarly, the current depth value of the covered pixel in the current depth map is updated to this first depth value, resulting in an updated current depth map. This updated current pixel attribute map is then designated as the new current pixel attribute map. The process of determining the current primitive from the 3D mesh structure of scene elements within the field of view of the first camera is repeated until all primitives in the 3D mesh structure of the scene elements are traversed. The current pixel attribute map obtained when traversing the primitives in the 3D mesh structure of the scene elements is then designated as the updated current pixel attribute map obtained from the current scene elements. Here, a primitive is a basic geometric shape that constitutes the 3D mesh structure; for example, a triangle. The terminal can determine the updated current depth map obtained when traversing each scene element within the field of view of the first camera as the target depth map of the first scene map.

[0094] In some embodiments, the terminal can execute rendering instructions for a lighting calculation task. For each pixel in the screen space, the terminal can obtain the pixel attribute value of the pixel from the target pixel attribute map, such as the color value of the pixel from the color map, the normal value of the pixel from the normal map, and the specular value of the pixel from the specular map. Lighting calculation is performed based on the color value, normal value, and specular value of the pixel, and the result of the lighting calculation is determined as the lighting color value of the pixel. Thus, the lighting color value of each pixel in the screen space is obtained. A first scene map is generated based on the lighting color values ​​of each pixel. The first scene map includes each pixel in the screen space, and the color value of the pixel in the first scene map is the corresponding lighting color value. The position of the pixel in the screen space is consistent with the position of the pixel in the first scene map. For example, the pixel in the first row and first column in the screen space is also located in the first row and first column in the first scene map.

[0095] In some embodiments, the same method can be used to generate a second scene map. Specifically, in the process of generating the second scene map, the terminal can generate an initial pixel attribute map, determine the initial pixel attribute map as the current pixel attribute map, determine the current scene element from the scene elements within the field of view of the second camera (the current scene element can be any scene element within the field of view of the second camera), project the current scene element into the screen space and perform a depth test, update the current pixel attribute map based on the depth test result, determine the updated current pixel attribute map as the new current pixel attribute map, return to the step of determining the current scene element from the scene elements within the field of view of the second camera, until all scene elements within the field of view of the second camera have been traversed, determine the updated current pixel attribute map when traversing all scene elements within the field of view of the second camera as the target pixel attribute map, perform illumination calculations based on each target pixel attribute map, and generate the second scene map.

[0096] In some embodiments, the rendering phase does not include the scene graph merging phase and the lighting calculation phase, but instead includes a lighting merging phase. The lighting calculation task and the scene graph merging task are tasks within the lighting merging phase. The lighting merging phase belongs to the composite rendering phase.

[0097] In this embodiment, lighting calculations are performed based on the target pixel attribute map to generate a first scene map. Since the target pixel attribute map includes the pixel attribute values ​​of each pixel in the screen space, the complexity of the lighting calculation is determined by the number of pixels in the screen space. For parts of the scene that are occluded and will not be displayed in the end, lighting calculations are not required, which reduces the complexity of the lighting calculations and improves the efficiency of the lighting calculations, thereby further improving the rendering efficiency.

[0098] In some embodiments, scene elements correspond to a three-dimensional mesh structure; projecting scene elements within the field of view of the first camera onto the screen space to generate a target pixel attribute map includes: determining a current primitive from the three-dimensional mesh structure of the scene elements within the field of view of the first camera; the primitive is the basic geometric shape constituting the three-dimensional mesh structure; determining the pixels covered by the projection area of ​​the current primitive in the screen space to obtain the covered pixels of the current primitive; calculating the depth value of the current primitive at the covered pixels to obtain a first depth value, comparing the first depth value with the current depth value of the covered pixels stored in the current depth map; when it is determined that the first depth value is less than the covered pixels... When determining the current depth value of a pixel, obtain the pixel attribute value of the current primitive at the covered pixel to get the first pixel attribute value. Update the pixel attribute value of the covered pixel in the current pixel attribute map to the first pixel attribute value to obtain a new current pixel attribute map. Update the current depth value of the covered pixel in the current depth map to the first depth value to obtain a new current depth map. Return to the step of determining the current primitive from the 3D mesh structure of scene elements within the field of view of the first camera, until the primitives in the 3D mesh structure of scene elements are traversed. Generate the target pixel attribute map based on the current pixel attribute map when traversing the primitives in the 3D mesh structure of scene elements.

[0099] A 3D mesh is a polygonal grid composed of a series of geometric shapes used to simulate the surface of complex objects. For example, in game development, the surfaces of various virtual objects are simulated by one or more 3D mesh structures. The geometric shapes include, but are not limited to, at least one of triangles or quadrilaterals.

[0100] In this embodiment, the three-dimensional mesh structure of the scene element is a polygonal mesh simulating the surface of the scene element. The three-dimensional mesh structure includes multiple primitives, which are the basic geometric shapes constituting the three-dimensional mesh structure. A basic geometric shape refers to the smallest geometric shape, such as a triangle. Each primitive includes multiple vertices; for example, when the primitive is a triangle, it includes three vertices. Vertex data refers to the data of the vertices, including but not limited to at least one of the vertex's color value, normal value, or specular value. Specifically, when there are multiple scene elements within the field of view of the first camera, the terminal can obtain any scene element from the scene elements within the field of view of the first camera as the current scene element. It then determines any primitive from the 3D mesh structure of the current scene element as the current primitive. The current primitive is projected from 3D space (e.g., viewing space) to screen space. The pixels covered by the projection area of ​​the current primitive in screen space are determined, and these pixels are identified as the covered pixels of the current primitive. The plane equation z = f(x,y) of the current primitive is determined. The coordinates (x,y) of the covered pixels are substituted into the plane equation to calculate the value of z, which is then determined as the first depth value. The value of z represents the distance between a point in space and the first camera. The terminal can determine the first pixel attribute value corresponding to the covered pixels based on the vertex data of the current primitive. After traversing all the current scene elements, the terminal returns to the step of obtaining any scene element from the scene elements within the field of view of the first camera as the current scene element, continuing until all scene elements within the field of view of the first camera are traversed, resulting in the target pixel attribute map.

[0101] In some embodiments, the target pixel attribute map is stored in a geometry buffer (Gbuffer). For example, three geometry buffers can be set to store the target color map, the target normal map, and the target specular map, respectively.

[0102] In this embodiment, the target pixel attribute map is generated quickly using the depth map, which improves the efficiency of generating the target pixel attribute map and thus improves the rendering efficiency.

[0103] In some embodiments, there are multiple attribute map generation tasks, and at least two of the attribute map generation tasks correspond to different rendering threads. Recording the rendering instructions of each first scene rendering task using at least two of the multiple rendering threads includes: recording the rendering instructions of each attribute map generation task using at least two of the multiple rendering threads. Executing the rendering instructions of the attribute map generation tasks to project scene elements located within the field of view of the first camera in the observation space onto the screen space to generate a target pixel attribute map includes: executing the rendering instructions of each attribute map generation task to project scene elements located within the field of view of the first camera in the observation space onto the screen space to generate a target pixel attribute map.

[0104] The rendering stage can include multiple property graph generation stages, each of which is used to implement a property graph generation task. For example, the rendering stage includes a first property graph generation stage, a second property graph generation stage, and a third property graph generation stage, and the property graph generation tasks include a first property graph generation task, a second property graph generation task, and a third property graph generation task. The first property graph generation stage is used to implement the first property graph generation task, the second property graph generation stage is used to implement the second property graph generation task, and the third property graph generation stage is used to implement the third property graph generation task.

[0105] At least two property graph generation tasks within each property graph generation task correspond to different rendering threads. The execution order of rendering instructions for each property graph generation task can be arbitrary; they can be executed in parallel or sequentially, and the order can be set as needed.

[0106] Specifically, different attribute map generation tasks are used to project different scene elements into the screen space and perform depth testing. For example, there are three attribute map generation tasks: the first attribute map generation task is used to project scene elements such as people and buildings into the screen space and perform depth testing; the second attribute map generation task is used to project plants into the screen space and perform depth testing; and the third attribute map generation task is used to project the ground into the screen space and perform depth testing. By executing the rendering instructions of the three attribute map generation tasks, the target pixel attribute map is generated.

[0107] In some embodiments, the terminal may utilize at least two of the multiple rendering threads to record the rendering instructions for each attribute graph generation task. For example, the rendering stage includes a first attribute graph generation stage, a second attribute graph generation stage, and a third attribute graph generation stage. The first attribute graph generation stage is used to implement the first attribute graph generation task, the second attribute graph generation stage is used to implement the second attribute graph generation task, and the third attribute graph generation stage is used to implement the third attribute graph generation task. The first attribute graph generation stage is matched with rendering thread 1, and the second and third attribute graph generation stages are matched with rendering thread 2. In this case, the terminal uses rendering thread 1 to record the rendering instructions for the first attribute graph generation task, and uses rendering thread 2 to record the rendering instructions for the second and third attribute graph generation stages.

[0108] In this embodiment, at least two of the multiple rendering threads are used to record the rendering instructions of each attribute map generation task. Since each attribute map generation task is used to generate the target pixel attribute map, the efficiency of recording the rendering instructions used to generate the target pixel attribute map is accelerated, the time for generating the target pixel attribute map is reduced, and the efficiency of generating the target pixel attribute map is improved, thereby improving the rendering efficiency.

[0109] In some embodiments, the rendering task further includes a scene graph merging task, which merges a first scene graph and a second scene graph to generate a target image with a mirror reflection effect. This includes: executing rendering instructions for the scene graph merging task to determine mirror pixels in the first scene graph based on the reflection plane and the first scene graph, determining the reflection color value of the mirror pixels based on the mirror pixels and the second scene graph, updating the color value of the mirror pixels in the first scene graph using the reflection color value of the mirror pixels, and generating a mirror reflection scene graph with a mirror reflection effect; the mirror pixels are pixels that undergo reflection; and generating a target image with a mirror reflection effect based on the mirror reflection scene graph.

[0110] Mirror pixels are pixels that represent the phenomenon of mirror reflection; that is, pixels used to represent the virtual image formed by mirror reflection. There can be multiple mirror pixels. When the reflecting plane is the water surface, the mirror is the water surface, and the mirror pixels can also be called water surface pixels. The reflection color value refers to the color value of the reflected image.

[0111] Specifically, the terminal can project the reflective plane onto the screen space and compare the depth values ​​of the pixels covered in the screen space with the depth values ​​of those pixels in the target depth map, thereby determining each mirror pixel from the first scene map. For each mirror pixel, the terminal can determine the corresponding reflection color value from the second scene map, and update the color values ​​of the mirror pixels in the first scene map based on the reflection color values ​​of each mirror pixel, for example, using the reflection color values ​​as the color values ​​of the mirror pixels, to obtain a mirror reflection scene map with a mirror reflection effect. Figure 6 As shown, Figure 6 (a) shows the first scene. Figure 6 The second scene shown in (b) is as follows. Figure 7 The image shows a mirror-reflection scene with a mirror effect, generated based on the first scene image and the second scene image.

[0112] In some embodiments, the terminal can determine the specular reflection scene image as the target image with a specular reflection effect, or the terminal can further process the specular reflection scene image and determine the result of the processing as the target image. For example, the terminal can acquire a user interface image, which can be rendered by the terminal or pre-generated. The user interface image and the specular reflection scene image are then merged to obtain the target image. Of course, after merging the user interface image and the specular reflection scene image, the terminal can further process the merged image to obtain the target image.

[0113] In this embodiment, the reflection color value of the mirror pixel is determined based on the mirror pixel and the second scene map. The color value of the mirror pixel in the first scene map is updated using the reflection color value of the mirror pixel to generate a mirror reflection scene map. This method can quickly and accurately determine the reflection color value, thus improving rendering efficiency.

[0114] In some embodiments, the rendering task further includes a user interface rendering task and a post-processing task; generating a target image with a mirror reflection effect based on the mirror reflection scene map includes: executing the rendering instructions of the user interface rendering task to generate a user interface map; executing the rendering instructions of the post-processing task to merge the user interface map and the mirror reflection scene map to generate a target image with a mirror reflection effect.

[0115] The rendering stage includes a user interface rendering stage and a post-processing stage. The user interface rendering stage is used to implement the user interface rendering task, which renders the user interface image. The post-processing stage is used to implement post-processing tasks, which generate a target image with a mirror reflection effect based on the mirror reflection scene image. The post-processing task can include rendering instructions corresponding to various post-processing methods. Merging the user interface image and the mirror reflection scene image is one such post-processing method. The rendering instructions for the post-processing task can include an interface merging rendering instruction, which, by executing the interface merging rendering instruction, can merge the user interface image and the mirror reflection scene image.

[0116] Specifically, the terminal can generate a user interface image by executing the rendering instructions of the user interface rendering task, and can merge the user interface image with the mirror reflection scene image by executing the interface compositing rendering instructions in the rendering instructions of the post-processing task, and determine the merged image as the target image. Alternatively, when the rendering instructions of the post-processing task also include rendering instructions of other post-processing methods, the terminal can also execute rendering instructions of other post-processing methods to further process the merged image and generate the target image.

[0117] In this embodiment, the user interface diagram and the mirror reflection scene diagram are merged to generate a target image with a mirror reflection effect, thereby enabling the rapid rendering of images with a mirror reflection effect for different users.

[0118] In some embodiments, the user interface rendering task and the second scene rendering task correspond to the same rendering thread, while the post-processing task corresponds to a different rendering thread than the user interface rendering task and the second scene rendering task.

[0119] Specifically, the user interface rendering task and the second scene rendering task correspond to the same rendering thread, while the post-processing task corresponds to a different rendering thread than the user interface rendering task and the second scene rendering task. For example, the specular reflection rendering pipeline has three rendering threads, namely the first rendering thread, the second rendering thread, and the third rendering thread. The user interface rendering task and the second scene rendering task correspond to the third rendering thread, while the post-processing task corresponds to the first rendering thread.

[0120] In some embodiments, the user interface rendering task, the lighting calculation task, and the second scene rendering task correspond to the same rendering thread, such as the third rendering thread.

[0121] It should be noted that the matching relationship between the rendering stage and the rendering thread in this application was determined through the inventor's experiments. In other words, the correspondence between the rendering thread and the task in this application was determined through the inventor's experiments and was not arbitrarily matched. That is, the correspondence between the task and the rendering thread in this application was determined through creative effort.

[0122] In this embodiment, the user interface rendering task and the second scene rendering task correspond to the same rendering thread, while the post-processing task corresponds to a different rendering thread than the user interface rendering task and the second scene rendering task. Thus, through the reasonable correspondence between tasks and rendering threads, the rendering threads achieve time balance, thereby reducing the time to render the image and improving rendering efficiency.

[0123] In some embodiments, determining the reflection color value of a mirror pixel based on the mirror pixel and the second scene image includes: for each mirror pixel, obtaining the coordinates of the mirror pixel on the reflection plane to obtain mirror coordinates; transforming the mirror coordinates based on the observation matrix and projection matrix of the second camera to obtain the sampled pixel corresponding to the mirror coordinates in the screen space; and determining the color value of the sampled pixel from the second scene image to obtain the reflection color value of the mirror pixel.

[0124] Here, the sampled pixel refers to the pixel used to sample the reflected color value. The observation matrix is ​​used to map three-dimensional coordinates in world space to three-dimensional coordinates in observation space, and the projection matrix is ​​used to map three-dimensional coordinates in observation space to two-dimensional coordinates.

[0125] Specifically, the terminal can perform coordinate transformation on the mirror coordinates based on the observation matrix and projection matrix of the second camera to obtain the pixel points corresponding to the mirror coordinates in the screen space. The pixel points corresponding to the mirror coordinates in the screen space are determined as the sampled pixel points of the mirror pixel points. The color value of the sampled pixel points is determined from the second scene image to obtain the reflection color value of the mirror pixel points.

[0126] In this embodiment, the target pixel to be merged is determined based on the position information of the reflected pixels, thereby quickly identifying the pixel that needs to be processed, saving the efficiency of generating a mirror reflection scene map, and thus improving rendering efficiency.

[0127] In some embodiments, determining the mirror pixel in the first scene map based on the reflective plane and the first scene map includes: obtaining a target depth map of the first scene map; the pixel value of the pixel in the target depth map is the depth value of the pixel; determining the reflective screen area projected by the reflective plane into the screen space, and for each pixel in the reflective screen area, when the depth value of the reflective plane at the pixel is less than the depth value of the pixel in the target depth map, the pixel is determined as a mirror pixel.

[0128] The reflective screen area is the projection area of ​​the reflective plane onto the screen space. A specular pixel is a pixel that undergoes specular reflection, meaning that specular reflection occurs at the location of that pixel.

[0129] Specifically, for each pixel in the reflective screen area, assuming the pixel's coordinates are (x, y), the terminal can calculate the Z value of the reflective plane Z = F(X, Y) at (x, y), and determine the calculated Z value as the depth value of the reflective plane at that pixel. When the depth value of the reflective plane at the pixel is less than the depth value of that pixel in the target depth map, the pixel is determined as a mirror pixel.

[0130] In this embodiment, the screen area projected onto the screen space by the reflecting plane is determined. For each pixel in the screen area, when the depth value of the reflecting plane at the pixel is less than the depth value of the pixel in the target depth map, the pixel is determined as a mirror pixel. This accurately determines the position of each pixel that undergoes mirror reflection, improves the efficiency of determining mirror pixels, and further improves the rendering efficiency.

[0131] This application also provides an embodiment of an image rendering method, specifically, the image rendering method includes:

[0132] The central processing unit creates three rendering threads for the specular reflection rendering pipeline: the first rendering thread, the second rendering thread, and the third rendering thread. The specular reflection rendering pipeline includes the user interface rendering stage, the shadow calculation stage, the first attribute map generation stage, the second attribute map generation stage, the third attribute map generation stage, the lighting merging stage, the second scene rendering stage, and the post-processing stage.

[0133] The first rendering thread matches the first attribute graph generation stage and the post-processing stage; the second rendering thread matches the shadow calculation stage, the second attribute graph generation stage, and the third attribute graph generation stage; and the third rendering thread matches the user interface rendering stage, the lighting merging stage, and the second scene rendering stage.

[0134] The user interface rendering stage is used to implement the user interface generation task; the shadow calculation stage is used to implement the shadow calculation task; the first attribute map generation stage is used to implement the first attribute map generation task; the second attribute map generation stage is used to implement the second attribute map generation task; the third attribute map generation stage is used to implement the third attribute map generation task; the lighting merging stage is used to implement the lighting calculation task and the scene map merging task; the second scene rendering stage is used to implement the second scene rendering task; and the post-processing stage is used to implement the post-processing task.

[0135] The rendering instructions for the first attribute graph generation task are recorded using the first rendering thread to obtain the first attribute graph rendering instructions, and the rendering instructions for the post-processing stage are recorded using the first rendering thread to obtain the post-processing rendering instructions.

[0136] The second rendering thread records the rendering instructions for the shadow calculation task to obtain the shadow rendering instructions. The second rendering thread also records the rendering instructions for the second attribute graph generation task to obtain the second attribute graph rendering instructions. Finally, the second rendering thread records the rendering instructions for the third attribute graph generation stage to obtain the third attribute graph rendering instructions.

[0137] The third rendering thread records the rendering instructions for the user interface generation task to obtain the user interface rendering instructions. The third rendering thread also records the rendering instructions for the lighting calculation task and the scene graph merging task to obtain the lighting merging rendering instructions. Finally, the third rendering thread records the rendering instructions for the second scene rendering task to obtain the second scene rendering instructions.

[0138] The graphics processor executes the following instructions in sequence: user interface rendering, shadow rendering, first attribute graph rendering, second attribute graph rendering, third attribute graph rendering, lighting merging rendering, second scene rendering, and post-processing rendering, to render a target image with a mirror reflection effect.

[0139] The first, second, and third rendering threads can record rendering instructions in parallel. These threads are all threads within the central processing unit (CPU). The logic threads within the CPU are separate from the rendering threads in the graphics processing unit (GPU). Therefore, even though specular reflections in the scene may increase the GPU's execution time (e.g., by 5-10 milliseconds), the CPU can share some of this time, reducing the time consumed by the logic threads and thus reducing overall image rendering time and improving rendering efficiency. Recording can also be referred to as recording.

[0140] In this embodiment, the rendering task is distributed across different threads, thereby improving the efficiency of specular reflection.

[0141] The image rendering method provided in this application can be applied to any scene that requires rendering an image with a mirror reflection effect, including but not limited to at least one of the following: game scenes, animation scenes, or virtual reality scenes. When applied to game, animation, or virtual reality (VR) scenes, the image rendering method provided in this application can improve the refresh rate of the scene image and enhance the user experience.

[0142] In a game scene, the image rendering method provided in this application can be used to render game screens. Specifically, the terminal can determine the screen to be rendered. When it is determined that the screen to be rendered is a screen with a mirror reflection effect, for example, when the screen to be rendered includes a water surface, multiple rendering threads created by the mirror reflection rendering pipeline can be used to record the rendering instructions of multiple rendering tasks in parallel, thereby improving the rendering efficiency. By executing the recorded rendering instructions, the game screen is rendered, and the rendered screen presents a mirror reflection effect formed by the water surface as a reflection plane, which improves the realism of the game screen.

[0143] In virtual reality scenarios, the image rendering method provided in this application can be used to render scene images in virtual reality. Water surfaces, smooth glass tabletops, or glass curtain walls of buildings in virtual reality scenes can all exhibit mirror reflection effects. When the scene image to be rendered in virtual reality includes water surfaces or glass surfaces, the image rendering method provided in this application can be used to render the scene image through multi-threaded rendering, improving rendering efficiency and enabling the rendered scene image to exhibit mirror reflection effects. For example, when the scene image to be rendered includes a glass curtain wall of a building, the generated scene image can display the mirror reflection effect produced on the glass curtain wall, improving the realism of the image in the virtual reality scene.

[0144] In some embodiments, experiments were conducted to demonstrate the actual effect of the image rendering method proposed in this application. In the same scene, the game running efficiency on a 1060 3G graphics card was compared with and without specular reflection as follows: With specular reflection enabled: one frame takes between 20 and 23 milliseconds; with specular reflection disabled: one frame takes between 15 and 18 milliseconds. The overall frame rate is above 40 frames per second, improving game smoothness, for example, in MMO games. Enabling specular reflection refers to using planar reflection to achieve the specular reflection effect, while disabling specular reflection refers to using screen space reflection instead of planar reflection. 40 frames per second refers to rendering 40 frames per second.

[0145] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0146] Based on the same inventive concept, this application also provides an image rendering apparatus for implementing the image rendering method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image rendering apparatus embodiments provided below can be found in the limitations of the image rendering method described above, and will not be repeated here.

[0147] In some embodiments, such as Figure 8 As shown, an image rendering apparatus is provided, including: a rendering thread creation module 802, a rendering instruction recording module 804, and a rendering instruction execution module 806, wherein:

[0148] The rendering thread creation module 802 is used to create multiple rendering threads corresponding to the specular reflection rendering pipeline. The specular reflection rendering pipeline is a rendering pipeline used to render images with specular reflection effects. The rendering stage in the specular reflection rendering pipeline is used to implement multiple rendering tasks related to specular reflection, and each rendering thread corresponds to at least one rendering task.

[0149] The rendering instruction recording module 804 is used to record the rendering instructions of the rendering task corresponding to each rendering thread; the rendering instructions of the rendering task are the instructions that need to be executed to implement the rendering task.

[0150] The rendering instruction execution module 806 is used to execute the rendering instructions for each recorded rendering task and render a target image with a specular reflection effect.

[0151] In some embodiments, the rendering instruction execution module is further configured to: sort the rendering instructions of each rendering task according to the preset stages in the specular reflection rendering pipeline corresponding to each rendering task, arrange the rendering instructions of each rendering task to obtain a target rendering instruction sequence; and execute the rendering instructions of each rendering task in the target rendering instruction sequence in sequence.

[0152] In some embodiments, the rendering thread belongs to the central processing unit (CPU), and the rendering instruction execution module is further configured to: sort the rendering instructions of each rendering task according to the preset stage in the specular reflection rendering task by the CPU, arrange the rendering instructions of each rendering task to obtain the target rendering instruction sequence belonging to the CPU, add the target rendering instruction sequence to the rendering instruction library of the graphics processor, and use the graphics processor to execute the rendering instructions of each rendering task in the rendering instruction library in sequence.

[0153] In some embodiments, the rendering task includes multiple first scene rendering tasks for rendering a first scene image and a second scene rendering task for rendering a second scene image. The first scene image is a scene image observed from the viewing perspective of a first camera; the second scene image is a scene image observed from the viewing perspective of a second camera. The rendering instruction recording module is further configured to: record the rendering instructions of each first scene rendering task using at least two of the multiple rendering threads; and record the rendering instructions corresponding to the second scene rendering task using at least one of the multiple rendering threads. The rendering instruction execution module is further configured to: execute the rendering instructions of each first scene rendering task to generate a first scene image, and execute the rendering instructions of the second scene rendering task to generate a second scene image; and perform a merging process based on the first scene image and the second scene image to generate a target image with a mirror reflection effect.

[0154] In some embodiments, the plurality of first scene rendering tasks include an attribute map generation task and a lighting calculation task; the rendering instruction execution module is further configured to: execute the rendering instructions of the attribute map generation task to project scene elements within the field of view of the first camera into the screen space to generate a target pixel attribute map; the target pixel attribute map includes the pixel attribute values ​​of each pixel in the screen space; and execute the rendering instructions of the lighting calculation task to perform lighting calculations based on the target pixel attribute map to generate a first scene map.

[0155] In some embodiments, scene elements correspond to a three-dimensional mesh structure; the rendering instruction execution module is further configured to: determine the current primitive from the three-dimensional mesh structure of scene elements within the field of view of the first camera; the primitive is the basic geometric shape constituting the three-dimensional mesh structure; determine the pixels covered by the projection area of ​​the current primitive in the screen space, and obtain the covered pixels of the current primitive; calculate the depth value of the current primitive at the covered pixels, obtain a first depth value, and compare the first depth value with the current depth value of the covered pixels stored in the current depth map; when it is determined that the first depth value is less than the current depth value of the covered pixels, obtain the pixel attribute value of the current primitive at the covered pixels, obtain the first pixel attribute value, update the pixel attribute value of the covered pixels in the current pixel attribute map to the first pixel attribute value, obtain a new current pixel attribute map, update the current depth value of the covered pixels in the current depth map to the first depth value, obtain a new current depth map, return to the step of determining the current primitive from the three-dimensional mesh structure of scene elements within the field of view of the first camera, until the primitives in the three-dimensional mesh structure of scene elements are traversed; generate a target pixel attribute map based on the current pixel attribute map when traversing the primitives in the three-dimensional mesh structure of scene elements.

[0156] In some embodiments, there are multiple attribute graph generation tasks, and at least two of the attribute graph generation tasks correspond to different rendering threads. The rendering instruction recording module is further configured to: use at least two of the multiple rendering threads to record the rendering instructions of each attribute graph generation task; the rendering instruction execution module is further configured to: execute the rendering instructions of each attribute graph generation task to project scene elements located in the field of view of the first camera in the observation space into the screen space and perform depth testing to generate target pixel attribute graphs.

[0157] In some embodiments, the rendering task further includes a scene graph merging task, and the rendering instruction execution module is further configured to: execute the rendering instructions of the scene graph merging task to determine the mirror pixels in the first scene graph based on the reflection plane and the first scene graph, and determine the reflection color value of the mirror pixels based on the mirror pixels and the second scene graph, update the color value of the mirror pixels in the first scene graph using the reflection color value of the mirror pixels, and generate a mirror reflection scene graph with a mirror reflection effect; the mirror pixels are the pixels that undergo reflection; and generate a target image with a mirror reflection effect based on the mirror reflection scene graph.

[0158] In some embodiments, the rendering task further includes a user interface rendering task and a post-processing task; the rendering instruction execution module is further configured to: execute the rendering instructions of the user interface rendering task to generate a user interface graph; execute the rendering instructions of the post-processing task to merge the user interface graph with the specular reflection scene graph to generate a target image with a specular reflection effect.

[0159] In some embodiments, the user interface rendering task and the second scene rendering task correspond to the same rendering thread, while the post-processing task corresponds to a different rendering thread than the user interface rendering task and the second scene rendering task.

[0160] In some embodiments, the rendering instruction execution module is further configured to: for each mirror pixel, obtain the coordinates of the mirror pixel on the reflection plane to obtain the mirror coordinates; transform the mirror coordinates based on the observation matrix and projection matrix of the second camera to obtain the sampled pixel corresponding to the mirror coordinates in the screen space; determine the color value of the sampled pixel from the second scene image to obtain the reflection color value of the mirror pixel.

[0161] In some embodiments, the rendering instruction execution module is further configured to: obtain a target depth map of the first scene map; the pixel value of a pixel in the target depth map is the depth value of the pixel; determine the reflective screen area projected onto the screen space by the reflective plane; and for each pixel in the reflective screen area, if the depth value of the reflective plane at the pixel is less than the depth value of the pixel in the target depth map, the pixel is determined as a mirror pixel.

[0162] Each module in the aforementioned image rendering device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0163] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data involved in the image rendering method. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an image rendering method.

[0164] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image rendering method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0165] Those skilled in the art will understand that Figure 9 and Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0166] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the image rendering method described above.

[0167] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the image rendering method described above.

[0168] In some embodiments, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the image rendering method described above.

[0169] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image rendering method, characterized in that, The method includes: Multiple rendering threads are created for the specular reflection rendering pipeline. The specular reflection rendering pipeline is used to render images with specular reflection effects. The rendering stage in the specular reflection rendering pipeline is used to implement multiple rendering tasks related to specular reflection. Each rendering thread corresponds to at least one rendering task. The rendering thread belongs to the central processing unit. For each rendering thread, the rendering instructions of the rendering task corresponding to the rendering thread are recorded; the rendering instructions of the rendering task are the instructions required to execute the rendering task. Execute the rendering instructions for each of the recorded rendering tasks to render a target image with a specular reflection effect, including: The central processing unit (CPU) sorts the rendering instructions of each rendering task according to the preset stage in the specular reflection rendering task, and arranges the rendering instructions of each rendering task to obtain a target rendering instruction sequence belonging to the CPU. The target rendering instruction sequence is then added to the rendering instruction library of the graphics processor. Using the graphics processor, the rendering instructions for each rendering task in the rendering instruction library are executed sequentially.

2. The method according to claim 1, characterized in that, The rendering task includes multiple first scene rendering tasks for rendering a first scene image and a second scene rendering task for rendering a second scene image; the first scene image and the second scene image are scene images observed from two different viewing angles that are symmetrical with respect to the reflection plane. The step of recording the rendering instructions for the rendering task corresponding to each rendering thread includes: At least two of the plurality of rendering threads are used to record the rendering instructions for each of the first scene rendering tasks; at least one of the plurality of rendering threads is used to record the rendering instructions for the second scene rendering task. The rendering instructions for each rendering task in the execution record, which render a target image with a specular reflection effect, include: Execute the rendering instructions of each of the first scene rendering tasks to generate the first scene graph, and execute the rendering instructions of the second scene rendering task to generate the second scene graph; The first scene image and the second scene image are merged to generate a target image with a mirror reflection effect.

3. The method according to claim 2, characterized in that, The multiple first scene rendering tasks include attribute graph generation tasks and lighting calculation tasks; The step of executing the rendering instructions for each of the first scene rendering tasks to generate the first scene map includes: executing the rendering instructions for the attribute map generation task to project scene elements within the field of view of the first camera into the screen space to generate a target pixel attribute map; the target pixel attribute map includes the pixel attribute values ​​of each pixel in the screen space; The rendering instructions for the lighting calculation task are executed to perform lighting calculations based on the target pixel attribute map and generate the first scene map.

4. The method according to claim 3, characterized in that, The scene elements correspond to a three-dimensional mesh structure; the step of projecting the scene elements within the field of view of the first camera onto the screen space to generate a target pixel attribute map includes: The current primitive is determined from the three-dimensional mesh structure of scene elements within the field of view of the first camera; the primitive is the basic geometric shape that constitutes the three-dimensional mesh structure. Determine the number of pixels covered by the projection area of ​​the current graphic element in the screen space to obtain the number of pixels covered by the current graphic element; Calculate the depth value of the current primitive at the covered pixel to obtain a first depth value, and compare the first depth value with the current depth value of the covered pixel stored in the current depth map; When it is determined that the first depth value is less than the current depth value of the covered pixel, the pixel attribute value of the current primitive at the covered pixel is obtained to obtain the first pixel attribute value. The pixel attribute value of the covered pixel in the current pixel attribute map is updated to the first pixel attribute value to obtain a new current pixel attribute map. The current depth value of the covered pixel in the current depth map is updated to the first depth value to obtain a new current depth map. The step of determining the current primitive from the three-dimensional mesh structure of scene elements in the field of view of the first camera is returned until the primitives in the three-dimensional mesh structure of the scene elements are traversed. The target pixel attribute map is generated based on the current pixel attribute map of primitives in the 3D mesh structure of the scene elements.

5. The method according to claim 3, characterized in that, There are multiple attribute graph generation tasks, and at least two attribute graph generation tasks in each attribute graph generation task correspond to different rendering threads. The step of using at least two of the plurality of rendering threads to record the rendering instructions for each of the first scene rendering tasks includes: Using at least two of the plurality of rendering threads, the rendering instructions for each of the attribute graph generation tasks are recorded; The rendering instructions for executing the attribute map generation task, to project scene elements within the field of view of the first camera onto the screen space, and to generate a target pixel attribute map, include: Execute the rendering instructions for each of the attribute map generation tasks to project scene elements located within the field of view of the first camera in the observation space into the screen space to generate target pixel attribute maps.

6. The method according to claim 2, characterized in that, The rendering task also includes a scene graph merging task, wherein the merging process based on the first scene graph and the second scene graph to generate a target image with a mirror reflection effect includes: The rendering instructions for the scene graph merging task are executed to determine the mirror pixels in the first scene graph based on the reflection plane and the first scene graph, and to determine the reflection color value of the mirror pixels based on the mirror pixels and the second scene graph. The color value of the mirror pixels in the first scene graph is updated using the reflection color value of the mirror pixels to generate a mirror reflection scene graph with a mirror reflection effect; the mirror pixels are the pixels that undergo reflection. Generate a target image with a mirror reflection effect based on the mirror reflection scene map.

7. The method according to claim 6, characterized in that, The rendering task also includes a user interface rendering task and a post-processing task; generating a target image with a mirror reflection effect based on the mirror reflection scene map includes: Execute the rendering instructions of the user interface rendering task to generate a user interface graph; The rendering instructions of the post-processing task are executed to merge the user interface image with the mirror reflection scene image to generate a target image with a mirror reflection effect.

8. The method according to claim 7, characterized in that, The user interface rendering task and the second scene rendering task correspond to the same rendering thread, while the post-processing task corresponds to a different rendering thread than the user interface rendering task and the second scene rendering task.

9. The method according to claim 6, characterized in that, Determining the reflection color value of the mirror pixel based on the mirror pixel and the second scene image includes: For each of the mirror pixels, obtain the coordinates of the mirror pixel on the reflection plane to get the mirror coordinates; The mirror coordinates are transformed based on the observation matrix and projection matrix of the second camera to obtain the sampled pixel points corresponding to the mirror coordinates in the screen space; The color value of the sampled pixel is determined from the second scene image to obtain the reflection color value of the mirror pixel.

10. The method according to claim 6, characterized in that, The step of determining the mirror pixels in the first scene image based on the reflecting plane and the first scene image includes: Obtain the target depth map of the first scene image; the pixel value of each pixel in the target depth map is the depth value of that pixel. The reflective screen area projected onto the screen space by the reflective plane is determined. For each pixel in the reflective screen area, if the depth value of the reflective plane at the pixel is less than the depth value of the pixel in the target depth map, the pixel is determined as a mirror pixel.

11. An image rendering apparatus, characterized in that, The device includes: A rendering thread creation module is used to create multiple rendering threads corresponding to the specular reflection rendering pipeline; the specular reflection rendering pipeline is a rendering pipeline used to render images with specular reflection effects; the rendering stage in the specular reflection rendering pipeline is used to implement multiple rendering tasks related to specular reflection, and each rendering thread corresponds to at least one rendering task; the rendering thread belongs to the central processing unit. The rendering instruction recording module is used to record the rendering instructions of the rendering task corresponding to each rendering thread; the rendering instructions of the rendering task are the instructions required to execute the rendering task. The rendering instruction execution module is used to execute the rendering instructions of each recorded rendering task to render a target image with a specular reflection effect. This includes: using the central processing unit (CPU), sorting the rendering instructions of each rendering task according to a preset stage in the specular reflection rendering task, obtaining a target rendering instruction sequence belonging to the CPU, and adding the target rendering instruction sequence to the graphics processor's rendering instruction library; and using the graphics processor to sequentially execute the rendering instructions of each rendering task in the rendering instruction library.

12. The image rendering apparatus according to claim 11, characterized in that, The rendering task includes multiple first scene rendering tasks for rendering a first scene image and a second scene rendering task for rendering a second scene image; the first scene image and the second scene image are scene images observed from two different viewing angles that are symmetrical with respect to the reflection plane. The rendering instruction recording module is further configured to use at least two of the plurality of rendering threads to record the rendering instructions for each of the first scene rendering tasks; and to use at least one of the plurality of rendering threads to record the rendering instructions corresponding to the second scene rendering task. The rendering instruction execution module is further configured to execute rendering instructions for each of the first scene rendering tasks to generate the first scene image, and execute rendering instructions for the second scene rendering task to generate the second scene image; and perform merging processing based on the first scene image and the second scene image to generate a target image with a mirror reflection effect.

13. The image rendering apparatus according to claim 12, characterized in that, The multiple first scene rendering tasks include attribute graph generation tasks and lighting calculation tasks; The rendering instruction execution module is also used to execute the rendering instructions of the attribute map generation task to project scene elements within the field of view of the first camera into the screen space and generate a target pixel attribute map; the target pixel attribute map includes the pixel attribute values ​​of each pixel in the screen space; The rendering instructions for the lighting calculation task are executed to perform lighting calculations based on the target pixel attribute map and generate the first scene map.

14. The image rendering apparatus according to claim 13, characterized in that, The scene elements correspond to a three-dimensional mesh structure; The rendering instruction execution module is further configured to determine the current primitive from the three-dimensional mesh structure of scene elements within the field of view of the first camera; the primitive is the basic geometric shape constituting the three-dimensional mesh structure; Determine the number of pixels covered by the projection area of ​​the current graphic element in the screen space to obtain the number of pixels covered by the current graphic element; Calculate the depth value of the current primitive at the covered pixel to obtain a first depth value, and compare the first depth value with the current depth value of the covered pixel stored in the current depth map; When it is determined that the first depth value is less than the current depth value of the covered pixel, the pixel attribute value of the current primitive at the covered pixel is obtained to obtain the first pixel attribute value. The pixel attribute value of the covered pixel in the current pixel attribute map is updated to the first pixel attribute value to obtain a new current pixel attribute map. The current depth value of the covered pixel in the current depth map is updated to the first depth value to obtain a new current depth map. The step of determining the current primitive from the three-dimensional mesh structure of scene elements in the field of view of the first camera is returned until the primitives in the three-dimensional mesh structure of the scene elements are traversed. The target pixel attribute map is generated based on the current pixel attribute map of primitives in the 3D mesh structure of the scene elements.

15. The image rendering apparatus according to claim 13, characterized in that, There are multiple attribute graph generation tasks, and at least two attribute graph generation tasks in each attribute graph generation task correspond to different rendering threads. The rendering instruction recording module is also used to record the rendering instructions of each of the multiple rendering threads using at least two of the multiple rendering threads. The rendering instruction execution module is also used to execute the rendering instructions of each of the attribute map generation tasks, so as to project the scene elements located in the field of view of the first camera in the observation space into the screen space to generate target pixel attribute maps.

16. The image rendering apparatus according to claim 12, characterized in that, The rendering task also includes a scene graph merging task; The rendering instruction execution module is further configured to execute the rendering instructions for the scene graph merging task, to determine the mirror pixels in the first scene graph based on the reflection plane and the first scene graph, and to determine the reflection color value of the mirror pixels based on the mirror pixels and the second scene graph, and to update the color value of the mirror pixels in the first scene graph using the reflection color value of the mirror pixels, thereby generating a mirror reflection scene graph with a mirror reflection effect; the mirror pixels are pixels that undergo reflection; and a target image with a mirror reflection effect is generated based on the mirror reflection scene graph.

17. The image rendering apparatus according to claim 16, characterized in that, The rendering task also includes user interface rendering task and post-processing task; The rendering instruction execution module is also used to execute the rendering instructions of the user interface rendering task to generate a user interface image; and to execute the rendering instructions of the post-processing task to merge the user interface image with the mirror reflection scene image to generate a target image with a mirror reflection effect.

18. The image rendering apparatus according to claim 17, characterized in that, The user interface rendering task and the second scene rendering task correspond to the same rendering thread, while the post-processing task corresponds to a different rendering thread than the user interface rendering task and the second scene rendering task.

19. The image rendering apparatus according to claim 16, characterized in that, The rendering instruction execution module is further configured to, for each of the mirror pixels, obtain the coordinates of the mirror pixel on the reflection plane to obtain the mirror coordinates; and transform the mirror coordinates based on the observation matrix and projection matrix of the second camera to obtain the sampled pixel point corresponding to the mirror coordinates in the screen space. The color value of the sampled pixel is determined from the second scene image to obtain the reflection color value of the mirror pixel.

20. The image rendering apparatus according to claim 16, characterized in that, The rendering instruction execution module is further configured to obtain a target depth map of the first scene map; the pixel value of a pixel in the target depth map is the depth value of the pixel; determine the reflective screen area projected by the reflective plane into the screen space, and for each pixel in the reflective screen area, when the depth value of the reflective plane at the pixel is less than the depth value of the pixel in the target depth map, the pixel is determined as a mirror pixel.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images