Image rendering method and electronic device

Abstract

This application discloses an image rendering method and an electronic device, relating to the field of image processing. It enables the reasonable reuse of semi-transparent particle rendering results, thereby effectively reducing the repetitive overhead of rendering semi-transparent particles during multi-frame image rendering, and thus reducing the resulting waste of power consumption and computing power. Specifically, the application sends a first instruction stream to instruct the electronic device to perform a rendering operation on a first frame image, which includes a first main scene and first semi-transparent particles. The electronic device synthesizes a first rendering result and a second rendering result to obtain the first frame image. The first rendering result is the rendering result of the first main scene, and the second rendering result is the rendering result of the first semi-transparent particles. The second rendering result is stored in the first frame buffer of the electronic device.

Description

[0001] This application is a divisional application. The original application has the application number 202210159851.5 and the original application date is February 22, 2022. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing, and more particularly to an image rendering method and an electronic device. Background Technology

[0003] With the development of electronic devices, the content of displayed images is becoming increasingly rich. Some images may include semi-transparent particles. Rendering semi-transparent particles across multiple consecutive frames can incur significant rendering overhead for electronic devices, resulting in high computational power consumption, overheating, and negatively impacting user experience. Summary of the Invention

[0004] This application provides an image rendering method and an electronic device that can reasonably reuse the rendering results of semi-transparent particles, thereby effectively reducing the repetitive overhead of rendering semi-transparent particles during multi-frame image rendering, and thus reducing the waste of power consumption and computing power.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, an image rendering method is provided, applied to an electronic device having an application installed thereon. The method includes: the application sending a first instruction stream to instruct the electronic device to perform a rendering operation on a first frame image, the first frame image including a first main scene and first semi-transparent particles; the electronic device synthesizing a first rendering result and a second rendering result to obtain the first frame image; the first rendering is the rendering result of the first main scene, the second rendering result is the rendering result of the first semi-transparent particles, and the second rendering result is stored in a first frame buffer of the electronic device.

[0007] Based on this scheme, the electronic device can avoid rendering the semi-transparent particles in the first frame of the image. Instead, the electronic device can read the rendering result of the semi-transparent particles from the corresponding storage space, such as the first frame buffer. In this way, the rendering overhead of the semi-transparent particles in the current frame of the image can be saved, while the display of the semi-transparent particles in the first frame of the image can be maintained.

[0008] In one possible design, the first instruction stream includes a first instruction segment and a second instruction segment. The first instruction segment instructs the electronic device to render the first main scene to obtain the first rendering result, and the second instruction segment instructs the electronic device to render the first semi-transparent particle. Before the electronic device synthesizes the first and second rendering results, the method further includes: the electronic device rendering according to the first instruction segment to obtain the first rendering result; and the electronic device obtaining the second rendering result from the first frame buffer. Based on this scheme, a method for obtaining the main scene rendering result and the semi-transparent particle rendering result in the first frame image is provided. For example, the electronic device can render based on the corresponding instruction stream issued in the current frame image to obtain the corresponding main scene rendering result. Alternatively, the electronic device can directly read the rendering result corresponding to the semi-transparent particle from the first frame buffer. In this example, the first frame image can be a frame image for reusing the semi-transparent particle rendering result.

[0009] In one possible design, the second rendering result is stored in the first frame buffer after the electronic device renders the second frame image, and the rendering of the second frame image precedes the rendering of the first frame image. Based on this scheme, the electronic device can store the completed rendering result in the first frame buffer when rendering before the first frame image, such as when rendering the second frame image, thereby enabling the reuse of subsequent frame images.

[0010] In one possible design, before the application issues the first instruction stream, the method further includes: the application issuing a second instruction stream, which instructs the electronic device to perform a rendering operation on a second frame image, the second frame image including a second main scene and the first semi-transparent particle. Based on this scheme, the application can issue an instruction to render the second frame image before reusing the semi-transparent particle. The second frame image may include corresponding main scene rendering instructions and semi-transparent particle rendering instructions. In the display of multiple consecutive frame images, each frame image can include semi-transparent particles, so the semi-transparent particle rendering results can be reused through the scheme provided in this application embodiment.

[0011] In one possible design, the second instruction stream includes a third instruction segment and a fourth instruction segment. The third instruction segment instructs the electronic device to render the second main scene to obtain a third rendering result, and the fourth instruction segment instructs the electronic device to render the first semi-transparent particle in the second frame image. Before the application issues the second instruction stream, the method further includes: the electronic device rendering according to the third instruction segment to obtain the third rendering result; and the electronic device obtaining the fourth rendering result according to the fourth instruction segment. Similar to the instruction stream corresponding to the first frame image, the instruction streams of other frame images may also include rendering instructions for the main scene and the semi-transparent particle. In this example, the second frame image may be a frame image that does not reuse the semi-transparent particle. The second frame image may be the first frame image after the application starts running. Since it has not undergone rendering of other frame images, there is no reusable semi-transparent particle rendering result. Alternatively, the second frame image may be a frame image that does not meet the requirements of preset rules. For example, the position of the semi-transparent particle differs significantly from the already stored semi-transparent particle rendering result frame image, making good reuse impossible. Therefore, the semi-transparent particles can be re-rendered in the second frame image to obtain new rendering results for reuse in subsequent frames.

[0012] In one possible design, the method further includes: the electronic device creating the first frame buffer. The electronic device obtains the fourth rendering result according to the fourth instruction segment, including: the electronic device replacing the frame buffer indicated by the fourth instruction segment with the first frame buffer to obtain a fifth instruction segment. The electronic device executes the rendering operation of the fifth instruction segment to obtain a second rendering result of the first semi-transparent particle, and stores the second rendering result in the first frame buffer. Based on this scheme, a specific scheme for backing up and storing semi-transparent particle rendering results is provided. In the second frame image, the semi-transparent particle rendering result can be stored in the corresponding first frame buffer by replacing the frame buffer ID. The first frame buffer can be newly created based on this scheme, so in subsequent rendering processes, the electronic device can reuse data through the newly created frame buffer.

[0013] In one possible design, the electronic device determines the second instruction segment based on preset start and end instructions in the first instruction stream. Based on this approach, a specific scheme for determining the rendering instruction stream of semi-transparent particles is provided. For example, the start instruction can be a glEnable() or glEnablei() instruction. Similarly, the end instruction can be a glDisable() or glDisablei() instruction. By identifying this second instruction segment, the electronic device can clearly determine the rendering instruction stream of the semi-transparent particles. In other implementations of this application, the end instruction can also be glDiscardFramebufferEXT().

[0014] In one possible design, the electronic device determines the fourth instruction segment based on preset start and end instructions in the second instruction stream. Based on this approach, a scheme is provided for determining the rendering instruction stream for semi-transparent particles in the second instruction stream. For example, the start instruction can be a glEnable() or glEnablei() instruction. Similarly, the end instruction can be a glDisable() or glDisablei() instruction. In other implementations of this application, the end instruction can also be glDiscardFramebufferEXT().

[0015] In one possible design, the electronic device includes an interception module, a creation module, and a replacement module. The method includes: the interception module intercepting the fourth instruction segment; the creation module creating the first frame buffer; and the replacement module replacing the frame buffer ID in the fourth instruction segment according to the identifier (ID) of the first frame buffer and the intercepted fourth instruction segment to obtain a fifth instruction segment pointing to the first frame buffer. The graphics processing unit (GPU) of the electronic device executes the rendering of the first semi-transparent particle according to the fifth instruction segment and stores the obtained second rendering result in the first frame buffer. Based on this scheme, a specific software partitioning in the electronic device is provided, realizing the semi-transparent particle reuse scheme provided in this application through the interaction of various modules.

[0016] In one possible design, the electronic device also includes a merging module. The method further includes: the merging module instructing the GPU to merge the second rendering result and the third rendering result to obtain the rendering result of the second frame image. Based on this scheme, in this application, the rendering result of the semi-transparent particles and the rendering result of the main scene can be rendered separately. Therefore, the GPU can be instructed to merge the two rendering results through the instructions issued by the merging module to obtain a complete rendering result.

[0017] In one possible design, the method further includes: determining the framebuffer ID of the main scene based on the process of the third frame image, wherein the framebuffer of the main scene is the framebuffer with the largest number of draw calls during the processing of the third frame image. Based on this scheme, a method for determining the main scene is provided. After determining the framebuffer ID of the main scene, subsequent framebuffers can also use this framebuffer ID to render the main scene. This allows the electronic device to clearly define the instruction flow for rendering the main scene in subsequent frame images, and can also merge the data in the corresponding framebuffer of the main scene with the rendering results of the semi-transparent particles to obtain the complete rendering result of the corresponding frame image.

[0018] In one possible design, a counter is provided in the electronic device, which increments by 1 each time the electronic device renders a frame image. Before the electronic device synthesizes the first rendering result and the second rendering result and obtains the first frame image, the method further includes: the electronic device determining that the value of the counter conforms to a preset rule when the first frame image is rendered. Based on this scheme, a reuse scheme for semi-transparent particles is provided. During the rendering process of multiple frame images, semi-transparent particles can be reused for several of the frame images. For example, they can be reused once every other frame. In this way, the rendering overhead of semi-transparent particles for some frame images can be saved, and the timely updating of semi-transparent particles can be achieved by reusing them once every other frame, thereby making the reuse effect accurate and reasonable.

[0019] In one possible design, if the counter value does not conform to a preset rule when the electronic device determines that the first frame image is being rendered, the method further includes: the electronic device creating the first frame buffer, replacing the frame buffer pointed to by the instruction segment in the first instruction stream that indicates the rendering of the first semi-transparent particle with the first frame buffer, the electronic device executing the rendering of the first semi-transparent particle, and storing the rendering in the first frame buffer. Based on this scheme, for frame images that do not need to be reused, the electronic device can perform normal rendering according to the native logic instructions. This achieves the rendering of the corresponding frame image and the updating of the rendering results of the semi-transparent particle.

[0020] In one possible design, the preset rule is that the counter value is even. Based on this scheme, a preset rule scheme is provided. This achieves the effect of reusing every other frame. For example, reuse starts from the second frame. In addition, the scheme setting of not reusing odd-numbered frames allows the first frame image to not be reused, thereby avoiding reuse failures due to the lack of rendering of semi-transparent particles beforehand.

[0021] In one possible design, before the electronic device synthesizes the first and second rendering results and acquires the first frame image, the method further includes: the electronic device determining that the viewpoint change during the rendering of the first frame image is less than a preset viewpoint threshold. Based on this scheme, other judgment mechanisms can be used to make the reuse effect more accurate before performing reuse. For example, when the viewpoint change is small, it indicates that the positions of the semi-transparent particles in the two frame images are relatively close, thereby ensuring the accuracy of the subsequent reuse effect.

[0022] In one possible design, the electronic device determines the viewpoint change based on the MVP matrix of the first frame image and the MVP matrix of the second frame image, wherein the rendering of the second frame image precedes the rendering of the first frame image. Based on this approach, a specific scheme for determining the viewpoint change is provided. The MVP matrix of the current frame image (such as the first frame image) can be determined through a command stream issued by the application. The MVP matrix of the second frame image can be cached in the electronic device during the rendering of the second frame image. In this application, when updating the rendering of semi-transparent particles, the electronic device can simultaneously update the MVP matrix of the corresponding frame image.

[0023] In one possible design, if the viewpoint change during the rendering of the first frame exceeds a preset viewpoint threshold, the method further includes: the electronic device creating the first frame buffer, replacing the frame buffer pointed to by the instruction segment in the first instruction stream that indicates the rendering of the first semi-transparent particle with the first frame buffer, the electronic device executing the rendering of the first semi-transparent particle, and storing it in the first frame buffer. Based on this scheme, when the viewpoint change is large, the reuse of the semi-transparent particle can be skipped, and the rendering of the semi-transparent particle can be performed directly.

[0024] In one possible design, the method further includes: the electronic device merging the first rendering result and the rendering result in the first frame buffer to obtain the rendering result of the first frame image. Based on this scheme, the rendering result of the corresponding frame image can be obtained through a merging instruction.

[0025] In a second aspect, an electronic device is provided, comprising one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; when the one or more processors execute the computer instructions, the electronic device performs an image rendering method as described in the first aspect above and any of the various possible designs.

[0026] Thirdly, a chip system is provided, the chip system including an interface circuit and a processor; the interface circuit and the processor are interconnected by a line; the interface circuit is used to receive signals from a memory and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the chip system executes an image rendering method as described in the first aspect above and any of the various possible designs.

[0027] Fourthly, a computer-readable storage medium is provided, comprising computer instructions that, when executed, perform an image rendering method as described in the first aspect above and any of the various possible designs.

[0028] Fifthly, a computer program product is provided, which includes instructions that, when the computer program product is run on a computer, enable the computer to execute an image rendering method as described in the first aspect above and any of the various possible designs.

[0029] It should be understood that the technical features of the technical solutions provided in the second, third, fourth and fifth aspects mentioned above can all correspond to the image rendering methods provided in the first aspect and its possible designs, so the beneficial effects that can be achieved are similar, and will not be repeated here. Attached Figure Description

[0030] Figure 1 A schematic diagram of a semi-transparent particle;

[0031] Figure 2 This is a schematic diagram of a rendering process;

[0032] Figure 3 A schematic diagram of semi-transparent particle rendering for multiple frames;

[0033] Figure 4 A schematic diagram of semi-transparent particle rendering of multiple frames of images provided in an embodiment of this application;

[0034] Figure 5 A schematic diagram of the software partitioning of an electronic device provided in an embodiment of this application;

[0035] Figure 6 This is a schematic diagram of the module interaction for image rendering provided in an embodiment of this application;

[0036] Figure 7 A schematic diagram illustrating the module interaction for image rendering provided in an embodiment of this application;

[0037] Figure 8 A schematic diagram illustrating the module interaction for image rendering provided in an embodiment of this application;

[0038] Figure 9A flowchart illustrating an image rendering method provided in an embodiment of this application;

[0039] Figure 10 A schematic diagram illustrating the module interaction for image rendering provided in an embodiment of this application;

[0040] Figure 11 A schematic diagram illustrating the module interaction for image rendering provided in an embodiment of this application;

[0041] Figure 12 A flowchart illustrating another image rendering method provided in an embodiment of this application;

[0042] Figure 13 A flowchart illustrating another image rendering method provided in an embodiment of this application;

[0043] Figure 14 A flowchart illustrating another image rendering method provided in an embodiment of this application;

[0044] Figure 15 This is a schematic diagram of a coordinate transformation;

[0045] Figure 16 A schematic diagram of a reference line of sight provided for an embodiment of this application;

[0046] Figure 17 A flowchart illustrating another image rendering method provided in an embodiment of this application;

[0047] Figure 18 A schematic diagram illustrating the composition of an electronic device provided in an embodiment of this application;

[0048] Figure 19 This is a schematic diagram of the composition of a chip system provided in an embodiment of this application. Detailed Implementation

[0049] Electronic devices can display images to users through their onboard displays. In some scenarios, these images may include semi-transparent particles. These particles can appear semi-transparent within the image. For example, consider an image displayed on an electronic device within a game scene. In shooting games, the image might include semi-transparent particles such as smoke, gunfire, and splashes. Electronic devices can enhance the realism of the scene and improve the user experience by adding rendering effects of semi-transparent particles to the image. For instance, ... Figure 1As shown, region A can include rendering effects of semi-transparent particles corresponding to waves. It can be seen that the waves in region A can be rendered as semi-transparent. Therefore, in this frame of image, the user can see the semi-transparent waves and also see the scenery behind them, thus achieving a simulated realistic visual experience. Similarly, region B can include rendering effects of semi-transparent particles corresponding to smoke. It can be seen that the smoke in region B can be rendered as semi-transparent. Compared to the waves in region A, the transparency of this smoke can be lower. Therefore, by displaying this frame of image, the user obtains a simulated realistic visual experience.

[0050] In order to obtain image data for display, electronic devices can render images according to the rendering instruction stream issued by applications (such as game applications) before displaying images, thereby obtaining image data for display.

[0051] Combination Figure 2 When rendering a frame of an image, a game application can issue a rendering command stream. The Central Processing Unit (CPU) can then call interfaces in the graphics library based on this rendering command stream, instructing the Graphics Processing Unit (GPU) to perform the corresponding rendering operation. The rendering result executed by the GPU can be stored in the electronic device, and after subsequent rendering commands, the data can be retrieved for display. The electronic device can then display that frame of the image on the screen based on this display data.

[0052] In some scenarios, if the current frame image includes semi-transparent particles, then the command stream issued by the game application can include command stream A instructing the rendering of semi-transparent particles. Correspondingly, electronic devices can also use methods such as... Figure 2 The process shown implements the rendering of the corresponding semi-transparent particles.

[0053] It should be understood that translucent particles do not exist in isolation within a single frame; rather, the same or similar translucent particles appear in multiple adjacent frames, thus achieving continuity in their display. In other words, during the rendering of multiple adjacent frames, the game application issues command streams similar to command stream A to render the same or similar translucent particles. For example, as... Figure 3 As shown, during the rendering of the first frame, the game application sends command stream A to instruct the electronic device to generate semi-transparent particles (such as...). Figure 1 The rendering of the waves in area A shown. Correspondingly, the CPU, graphics library, and GPU will render according to... Figure 2 The process shown executes the rendering of the waves in region A to obtain, as... Figure 1The rendering result corresponds to the wave shown in region A of the image. The second frame will also include waves similar to those in the first frame. Therefore, the rendering instruction stream for the second frame will include instruction stream A (or an instruction stream similar to instruction stream A) to instruct the electronic device to render the waves. Correspondingly, the CPU, graphics library, and GPU will still render the waves according to... Figure 2 The process shown executes the rendering of the wave.

[0054] As can be seen, the rendering process of the semi-transparent particles (such as waves) corresponding to instruction stream A is repeatedly executed multiple times by the CPU, graphics library, and GPU across multiple frames, with essentially the same results. Furthermore, the rendering process for semi-transparent particles is quite complex. Therefore, this results in repetitive rendering overhead for semi-transparent particles across multiple frames. This leads to a waste of power and computing power in the image rendering process of electronic devices, and also causes problems such as overheating, stuttering, and dropped frames.

[0055] To address the aforementioned issues, this application provides a rendering method for semi-transparent particles in an image, enabling the reasonable reuse of semi-transparent particle rendering results. This effectively reduces the repetitive overhead of rendering semi-transparent particles during multi-frame image rendering, thereby reducing the resulting waste of power consumption and computing power.

[0056] For example, in combination Figure 4 Based on the solution provided in this application, the rendering result of the semi-transparent particles can be stored in a preset location during the rendering process of the previous frame (e.g., the first frame). Thus, during the rendering process of the next frame (e.g., the second frame), after receiving the instruction stream A from the game application to render the semi-transparent particles, the CPU can directly return to that instruction stream. In other words, during the rendering process of the second frame, it is not necessary to render the semi-transparent particles again; the electronic device can reuse the rendering result of the semi-transparent particles from the first frame when needed. For example, the rendering result of the semi-transparent particles can be retrieved from the preset location. This avoids the repeated execution of the rendering process for the same or similar semi-transparent particles in multiple frames, thereby reducing the rendering overhead for semi-transparent particles.

[0057] The solutions provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0058] It should be noted that the image rendering method provided in this application embodiment can be applied to a user's electronic device. For example, the electronic device can be a mobile phone, tablet computer, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, media player, or other portable mobile device. It can also be a wearable electronic device capable of providing display capabilities, such as a smartwatch. This application embodiment does not impose any special limitations on the specific form of the device.

[0059] For example, in some embodiments, from a hardware composition perspective, the electronic device involved in this application may include a processor, an external memory interface, internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, antenna 1, antenna 2, a mobile communication module, a wireless communication module, an audio module, a speaker, a receiver, a microphone, a headphone jack, a sensor module, buttons, a motor, an indicator, a camera, a display screen, and a subscriber identification module (SIM) card interface, etc. The sensor module may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0060] It should be noted that the above hardware components do not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components, or combine some components, or split some components, or have different component arrangements.

[0061] In other embodiments, the electronic device described in this application may also have software partitioning. For example, an electronic device running an Android operating system may have layered software partitioning.

[0062] For example, Figure 5 This is a schematic diagram illustrating the software composition of an electronic device provided in an embodiment of this application. For example... Figure 5 As shown, the electronic device may include an application (APP) layer, a framework layer, a system library, and a hardware layer.

[0063] The application layer can also be called the application layer. In some implementations, the application layer may include a series of application packages. Application packages may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS. In this embodiment, the application package may also include applications that need to display images or videos to the user by rendering images. Here, video can be understood as the continuous playback of multiple frames of images. It may include frame images with semi-transparent particles. For example, the application may include game applications, such as Peacekeeper Elite®, Honor of Kings®, etc.

[0064] The framework layer, also known as the application framework layer, provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes predefined functions. For example, it may include a window manager, content providers, a view system, a resource manager, a notification manager, an activity manager, and an input manager. The window manager provides a Window Manager Service (WMS), which can be used for window management, window animation management, surface management, and as an intermediary for the input system. The content provider stores and retrieves data, making it accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon may include a view for displaying text and a view for displaying images. The resource manager provides various resources to the application, such as localized strings, icons, images, layout files, video files, etc. The Notification Manager allows applications to display notifications in the status bar. These notifications can be used to convey informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the Notification Manager can be used to notify users of download completion or message alerts. The Notification Manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog windows on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights. The Activity Manager provides the Activity Manager Service (AMS), which is used for starting, switching, and scheduling system components (such as activities, services, content providers, and broadcast receivers), as well as managing and scheduling application processes. The Input Manager provides the Input Manager Service (IMS), which manages system input, such as touchscreen input, keypad input, and sensor input. The IMS retrieves events from input device nodes and, through interaction with the WMS, distributes the events to the appropriate windows.

[0065] In this embodiment, one or more functional modules can be provided in the frame layer to implement the solution provided in this embodiment. For example, the frame layer can include an interception module, a creation module, a replacement module, and a compositing module. The interception module can be used to intercept relevant instructions. The creation module can be used to create a new framebuffer (FB), which corresponds to... Figure 4 The preset positions are shown. The Replace module can be used to replace the framebuffer bound to the original command stream with the newly created framebuffer, so that the rendering results of the translucent particles can be stored in the newly created framebuffer for later reuse. The Compositing module can be used to merge the translucent particles stored in the newly created framebuffer with the main scene to obtain a complete rendering result.

[0066] The system library may include a graphics library. In different implementations, the graphics library may include at least one of the following: Open Graphics Library (OpenGL), OpenGL for Embedded Systems (OpenGL ES), Vulkan, etc. In some embodiments, the system library may also include other modules, such as: surface manager, media framework, standard C library (libc), SQLite, Webkit, etc.

[0067] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Framework supports playback and recording of various common audio and video formats, as well as still image files. The Media Library supports multiple audio and video encoding formats, such as Moving Picture Experts Group (MPEG4), H.264, Moving Picture Experts Group Audio Layer 3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR), Joint Photographic Experts Group (JPEG, or JPG), and Portable Network Graphics (PNG). OpenGL ES and / or Vulkan provide drawing and manipulation of 2D and 3D graphics in applications. SQLite provides a lightweight relational database for applications on electronic devices.

[0068] In such Figure 5 In the example, the electronic device may also include a hardware layer. This hardware layer may include a CPU, a GPU, and a memory with storage capabilities. In some implementations, the CPU may be used to control the various modules in the framework layer to perform their respective functions, and the GPU may be used to perform corresponding rendering processes based on the APIs in the graphics library (such as OpenGL ES) called by the instructions processed by the various modules in the framework layer.

[0069] The following explanation will be combined with, for example Figure 5 The software partitioning shown below provides a detailed description of the solution provided in the embodiments of this application.

[0070] The rendering scheme provided in this application embodiment can pre-store semi-transparent particles in a newly created frame buffer during the rendering of the Nth frame image. During the rendering of subsequent frames, the electronic device can directly return the rendering command stream for the semi-transparent particles issued by the application (such as a game application), without repeating the execution, and directly reuse the rendering result of the semi-transparent particles in the newly created frame buffer.

[0071] In the following description, the example of reusing the semi-transparent particles rendered in the Nth frame image in the (N+1)th frame image is used. In this application, the N+1th frame image can also correspond to the first frame image, corresponding to the first instruction stream. The Nth frame image can correspond to the second frame image, corresponding to the second instruction stream.

[0072] For ease of understanding, the following will first take the application as a game application and briefly explain the composition of the instruction flow during the game operation process involved in the solution provided in this application embodiment, as well as related concepts (such as the main scene).

[0073] Understandably, after the game application runs, when the display of the Nth frame image is required, a command stream containing multiple instructions is sent to the electronic device. This may include command stream 12 instructing the electronic device to render semi-transparent particles; command stream 11 instructing the electronic device to render the main scene; and command stream 14 instructing the electronic device to merge the semi-transparent particles with the main scene of the Nth frame onto the same texture, etc. Command stream 12 can correspond to the fourth instruction segment of the first command stream. Command stream 11 can correspond to the third instruction segment of the first command stream.

[0074] Similar to the Nth frame image, the command stream issued by the game application can also include multiple streams in the rendering of other frames. Taking the N+1th frame image as an example, the command stream issued by the game application may include: command stream 22 for instructing the electronic device to render semi-transparent particles, command stream 21 for instructing the electronic device to render the main scene, and command stream 24 for instructing the electronic device to merge the semi-transparent particles with the main scene of the N+1th frame (such as the main scene 21) onto the same texture, etc. Among them, command stream 22 can correspond to the second command segment of the first command stream. Command stream 21 can correspond to the first command segment of the first command stream.

[0075] The main scene corresponds to the rendering pipeline (renderpass) with the most draw calls during the rendering process of the current frame. The rendering of a single frame can include multiple renderpasses. The rendering result of each renderpass can be stored in the framebuffer. Each renderpass can include multiple draw calls. The more draw calls executed, the richer the texture content obtained after the corresponding renderpass completes its execution.

[0076] Generally, the main scene of different frames can be different. For example, the main scene of the Nth frame can be main scene 11, and the main scene in the Nth frame can also be called the second main scene. The main scene of the (N+1)th frame can be main scene 21. The main scene in the (N+1)th frame can also be called the first main scene.

[0077] It should be noted that for a fixed game scene, the main scene's render pass (i.e., the frame buffer corresponding to the main scene) generally does not change. That is, for several consecutive frames, the frame buffer for rendering the main scene's render pass is the same. Therefore, in this embodiment, the main scene of subsequent frames can be determined based on the number of draw calls included in each render pass during the rendering of the (N-1)th frame. For example, after the (N-1)th frame is rendered, the electronic device can determine that the number of draw calls executed on frame buffer FB0 (i.e., the frame buffer with ID 0) is the highest. Therefore, the frame buffer corresponding to the main scene of subsequent frames can be determined to be FB0. In this application, FB0 can also be referred to as frame buffer 11.

[0078] In other implementations of this application, the confirmation of the main scene can also be performed in real time. For example, after executing all rendering instructions for the Nth frame image, the electronic device can use the frame buffer with the largest number of draw calls during the rendering of the Nth frame image as the frame buffer of the main scene. Similarly, for other frame images, such as the N+1th frame image, the electronic device can also perform the confirmation and updating of the main scene frame buffer in real time.

[0079] In this application, the process of determining the main scene can be completed before the rendering of the Nth frame image. In some embodiments, the electronic device can determine the main scene in the rendering process of subsequent frame images based on the rendering instruction of the first frame image issued by the game application after the game starts running and completes loading, or it can determine the main scene in the rendering process of subsequent frame images based on the preset rendering instruction of the Mth frame image. The Mth frame image can be a frame image preceding the Nth frame image. In some implementations, M can be greater than 1, which ensures that the main scene is determined after the game is running stably.

[0080] Furthermore, the step of determining the main scene can be executed only once during game execution, with subsequent frame images determined based on this result. In other embodiments, this step of determining the main scene can also be executed cyclically according to a preset period to update the frame buffer information (such as frame buffer ID) of the main scene. In still other embodiments, this step of determining the main scene can also be triggered based on the real-time load of the electronic device. For example, for game applications, the load of the electronic device may change significantly when switching main scenes. Therefore, when the load change of the electronic device exceeds a preset load threshold, the step of determining the main scene can be triggered to update the buffer information of the main scene. During subsequent frame image rendering, relevant operations can be performed based on the updated buffer information of the main scene.

[0081] In this embodiment of the application, taking the rendering process of the Nth frame image as an example, the electronic device can respond to the instruction stream 11 issued by the game application to render the main scene.

[0082] For example, in combination Figure 6 The game application can issue command stream 11 to instruct the electronic device to render the main scene. In this example, the interception module can be used to intercept command streams related to semi-transparent particles. For other command streams, the interception module can send these command streams back to the graphics library to instruct the GPU to perform corresponding rendering operations, thereby avoiding rendering errors. The scheme for the interception module to identify whether the command stream from the game application is related to semi-transparent particle rendering will be discussed later. Figure 7 The documentation details this process. For example, the command stream related to semi-transparent particle rendering can begin and end with a fixed command. The interception module can then intercept the semi-transparent particle command stream by recognizing this fixed opening and closing command.

[0083] like Figure 6 As shown, for instruction stream 11, the interception module can directly call back instruction stream 11 to the graphics library, thereby invoking the corresponding API to instruct the GPU to execute the rendering of the main scene. It can be understood that before the rendering of the Nth frame begins, the electronic device can determine the frame buffer corresponding to the main scene as frame buffer 11 (e.g., FB0) according to the aforementioned scheme. Therefore, after completing the rendering on FB0, the rendering of the main scene in the Nth frame is completed. Thus, the electronic device can obtain the rendering result of the main scene in the Nth frame, i.e., the main scene 11 stored in frame buffer 11.

[0084] Based on the aforementioned description of the instruction stream for the Nth frame image, the game application can also issue instruction stream 12 to perform semi-transparent particle rendering.

[0085] For example, Figure 7 The rendering process for semi-transparent particles in the Nth frame image is shown in an embodiment of this application.

[0086] like Figure 7 As shown, during the rendering of the Nth frame image, the game application can issue command stream 12 to instruct the electronic device to render semi-transparent particles. In this application, command stream 12 can be a segment of the command stream issued by the game application that begins with the glEnable command and ends with the glDisable command. The glEnable command corresponds to the opening command in the above description, and the glDisable command corresponds to the closing command in the above description.

[0087] It should be understood that, during the rendering of the Nth image, taking OpenGL as an example, since OpenGL is a state machine, the corresponding rendering state needs to be changed when rendering semi-transparent particles. Furthermore, to render different levels of transparency, the color blending state needs to be enabled. In this example, the game application can enable the color blending state using the glEnable instruction, that is, instruct the electronic device to start rendering semi-transparent particles.

[0088] For example, Table 1 below shows a schematic of the beginning portion of instruction stream 12 in this example.

[0089] Table 1

[0090] Instruction ID (EID) Command content (Event) >3245 glEnablei(GL_BLEND,0) >3246 glBlendFuncSeparate(GL_DST_COLOR,GL_NONE,GL_NONE,GL_LINES) >3247 glViewport(0,0,1480,720)

[0091] In the example in Table 1, the game application uses the instruction with ID 3245, `glEnablei(GL_BLEND, 0)`, to instruct the electronic device to enable color blending. Then, in subsequent instructions, the game application can use different instructions to instruct the electronic device to perform operations corresponding to the rendering of semi-transparent particles. For example, the game application can set the blending factor using the `glBlendFuncSeparate` instruction with ID 3246. The game application can set viewport parameters using the `glViewport` instruction with ID 3247.

[0092] It should be noted that the examples in Table 1 use the `glEnablei` instruction as an example for illustration. In different implementations of this application, the `glEnablei` instruction can perform a similar function to the `glEnable` instruction. The difference lies in that when the instructions issued by the game application use indexing for data addressing, an index parameter is added to the instruction (such as 0 carried in the `glEnablei` instruction in Table 1), and the `glEnablei` instruction is used for enabling. Conversely, when the instructions issued by the game application do not use indexing for data addressing, the index parameter is not needed, and the `glEnable` instruction is used for enabling. The following examples illustrate the use of the `glEnable` instruction for enabling.

[0093] Each instruction in Table 1 can include one or more specific instructions. For example, an instruction can include multiple drawing elements. As an example, Table 2 below shows a schematic of the specific drawing elements included in an instruction.

[0094] Table 2

[0095] 6101 glDrawElements(1020) 6142 glDrawElements(231) 6161 glDrawElements(231) 6162 glDiscardFramebufferEXT(Framebuffer 557)

[0096] As shown in Table 2, the game application can sequentially instruct the electronic device to perform the corresponding drawing operations via functions 6101, 6142, and 6161. Once all DrawElements have been executed, the rendering of the semi-transparent particles is complete.

[0097] In the example in Table 2, after issuing rendering commands for all semi-transparent particles, the game application can call the `glDiscardFramebufferEXT()` function. For instance, after issuing the last `Drawelements`, the game application can issue `glDiscardFramebufferEXT(Framebuffer 557)` with ID 6162, as shown in Table 2. This interface is mainly used in the underlying Tile-Based Deferred Rendering (TBRT) rendering framework on mobile devices to reduce bandwidth consumption caused by data synchronization between tiles and video memory. This interface informs the graphics card driver that the data in the current frame's FrameBuffer is not needed in the next frame, reducing the bandwidth consumption of synchronizing the current frame's tiles back to video memory and synchronizing the next frame's video memory back to tiles. In this example, the `glDiscardFramebufferEXT()` function can be used to indicate that the rendering commands for the semi-transparent particles have been issued.

[0098] After the game application completes the glDiscardFramebufferEXT() call, before it calls the next DrawElements call for other objects (i.e., other objects that are not semi-transparent particles), it can instruct the electronic device to completely disable the current color blending operation for semi-transparent particles by issuing the glDisable instruction.

[0099] For example, Table 3 below shows a schematic of the end portion of instruction stream 12 in this example.

[0100] Table 3

[0101] Instruction ID (EID) Command content (Event) >6167 glBindBuffer(GL_UNIFORM_BUFFER,Buffer 15245) >6168 glBufferSubData(Buffer 15254,(96 bytes)) >6169 glBindBuffer(GL_UNIFORM_BUFFER,Buffer 21484) >6170 glBufferSubData(Buffer 21484,(48 bytes)) >6171 glDisablei(GL_BLEND,0)

[0102] In the instruction stream example output in Table 3, the game application can bind the frame buffer with ID 15245 using the glBindBuffer() instruction with ID 6167, and pass data to Buffer 15254 using the glBufferSubData() instruction with ID 6168. The game application can also bind the frame buffer with ID 21484 using the glBindBuffer() instruction with ID 6169, and pass data to Buffer 21484 using the glBufferSubData() instruction with ID 6170. This completes the issuance of the rendering instructions for the semi-transparent particles in this frame. The game application can then issue the glDisablei() instruction with ID 6171 to instruct the electronic device to complete the rendering instructions for this semi-transparent particle and disable color blending.

[0103] Similar to the aforementioned explanation of the relationship between the glEnablei and glEnable instructions, when disabling color blending, electronic devices can use either the glDisablei() or glDisable() instructions to achieve this functionality, depending on the data addressing used. The following example demonstrates how an electronic device can disable color blending using the glDisable() instruction.

[0104] As can be seen from the above description, in the rendering instruction stream of the current frame image (such as the Nth frame image), the instruction stream 12, which starts with the glEnable() instruction and ends with the glDisable() instruction, can include all rendering instructions for semi-transparent particles.

[0105] It should be noted that the above analysis, which uses the glEnable() instruction as the beginning and the glDisable() instruction as the end to identify instruction stream 12, is merely an illustration. In other implementation environments (such as rendering environments different from OpenGL), the beginning and / or end instructions of instruction stream 12 may also be different.

[0106] Continue to refer to Figure 7 In this embodiment, the interception module can intercept the instruction stream 12 so that other modules can perform corresponding operations. For example, the interception module can begin interception when it detects the glEnable() instruction in the instruction stream of the Nth frame image, and continue interception until it detects the glDisable() instruction in the instruction stream. This achieves the interception of instruction stream 12.

[0107] For instructions outside instruction stream 12, the interception module can directly return to the graphics library to execute the corresponding operation. For example, ... Figure 6As shown in the main scene rendering process, the interception module can directly send the instruction stream 11 back to the graphics library in order to realize the rendering of the main scene.

[0108] In this example, the intercept module can transmit the intercepted instruction stream 12 to the creation module.

[0109] For example, after receiving instruction stream 12, the creation module can create a new frame buffer in the memory of the electronic device to store the rendering result of the semi-transparent particles in the Nth frame image. For instance, the creation module can create frame buffer 12 to store the rendering result of the semi-transparent particles. This frame buffer 12 can also be referred to as the first frame buffer.

[0110] It should be understood that in the prior art, the instruction stream 12 issued by the game application can instruct the GPU to render the semi-transparent particles and store them in the corresponding frame buffer (such as the original frame buffer) by calling the interface in the graphics library. The data stored in the original frame buffer is generally not visible to the CPU, so it is impossible to reuse the contents of the original frame buffer in the subsequent frame image rendering process. In the embodiments of this application, by creating a new frame buffer, such as frame buffer 12, the data subsequently stored in the frame buffer 12 (such as the rendering result of the semi-transparent particles) can continue to be called by the electronic device, thereby realizing data reuse.

[0111] In some embodiments of this application, the creation module may also create other frame buffers. For example, the creation module may create frame buffer 13. This frame buffer 13 may be used to perform other rendering operations. For example, after the GPU completes the rendering of the main scene and the semi-transparent particles, the compositing operation of the main scene and the semi-transparent particles may be performed on this frame buffer 13.

[0112] It should be noted that, in cases such as Figure 7 The example shown illustrates how the creation module creates frame buffer 12 and frame buffer 13 after receiving instruction stream 12 from the interception module. In other examples of this application, the timing of creating frame buffer 12 and / or frame buffer 13 may differ. For example, the creation module may pre-create frame buffer 12 and / or frame buffer 13 when rendering of the Nth frame begins. The creation module may record the IDs of the pre-created frame buffer 12 and / or frame buffer 13 for subsequent direct use.

[0113] As can be seen, the creation module creates frame buffers 12 and 13 for data storage during subsequent rendering. The purpose of not directly using the original frame buffer is to facilitate subsequent reuse. Therefore, in some other embodiments of this application, if a usable frame buffer exists that can be accessed by the CPU, the creation module may not create a new frame buffer but can directly use the already created one.

[0114] As an example, let's take the Nth frame as the first frame after the game has finished loading. Since the current frame is the first frame, there are generally no existing idle frame buffers that can be used by the CPU. Therefore, the creation module can be based on, for example... Figure 7 The mechanism shown creates frame buffer 12 and frame buffer 13 after receiving instruction stream 12 sent by the intercept module.

[0115] In this embodiment, to ensure the normal subsequent use of frame buffer 12 and frame buffer 13, the creation module can also perform related processing in addition to creating frame buffer 12 and frame buffer 13. For example, creating corresponding textures, binding textures, etc.

[0116] The following example illustrates the process of creating a new framebuffer in the creation module.

[0117] For example, taking frame buffer 12 with ID alpha_A as an example, frame buffer 12 can be identified as FB(alpha_A). The creation module can create FB(alpha_A) and perform related processing through the following process.

[0118] 1. Create a texture map Texture_A to store the semi-transparent particle rendering using glGenTextures(1,&(Texture_A)).

[0119] 2. Create the FB (alpha_A) using glGenFramebuffers(1, &(alpha_A)).

[0120] 3. Bind the frame buffer using glBindFramebuffer(GL_FRAMEBUFF,alpha_A).

[0121] 4. Bind Texture_A to FB(alpha_A) using glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, Texture_A, 0).

[0122] In this way, image rendering can be performed on the newly created FB (alpha_A). For example, the data to be rendered can be rendered onto the texture_A and stored in the storage space corresponding to FB (alpha_A).

[0123] Similarly, the creation module can also create and process frame buffer 13. Taking frame buffer 13 with the ID alpha_B as an example, frame buffer 13 can be identified as FB (alpha_B). The creation module can create and process FB (alpha_B) through the following process.

[0124] 1. Create a texture map Texture_B to store the semi-transparent particle rendering using glGenTextures(1,&(Texture_B)).

[0125] 2. Create the FB (alpha_B) using glGenFramebuffers(1, &(alpha_B)).

[0126] 3. Bind the frame buffer using glBindFramebuffer(GL_FRAMEBUFF,alpha_B).

[0127] 4. Bind Texture_B to FB(alpha_B) using glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, Texture_B, 0).

[0128] In this way, image rendering can be performed on the newly created FB (alpha_B). For example, the data to be rendered can be rendered onto the texture_B and stored in the storage space corresponding to the FB (alpha_B).

[0129] In this embodiment, the creation module can also be used to transmit the instruction stream 12 and the ID of the newly created frame buffer to the replacement module. For example, continuing with the example of newly created frame buffer 12 being FB(alpha_A) and frame buffer 13 being FB(alpha_B), the creation module can send the instruction stream 12, the ID of FB(alpha_A) (e.g., alpha_A), and the ID of FB(alpha_B) (e.g., alpha_B) to the replacement module.

[0130] The replacement module can be used to replace the ID of the original frame buffer in the instruction stream 12 issued by the game application with the ID of the newly created frame buffer, so that subsequent related rendering operations can be performed on the newly created frame buffer that the CPU can call.

[0131] For example, the replacement module can replace the framebuffer ID used for rendering semi-transparent particles in instruction stream 12 with alpha_A. This allows the rendering results to be stored on the texture_A of FB (alpha_A) during the subsequent rendering of semi-transparent particles.

[0132] Taking the framebuffer ID beta_A for rendering semi-transparent particles as indicated in instruction stream 12 as an example, the replacement module can replace the command bound to framebuffer ID beta_A in instruction stream 12 with the command bound to framebuffer ID alpha_A. The replacement module can also replace other commands bound to beta_A in instruction stream 12 with commands bound to alpha_A. This yields instruction stream 13 pointing to FB (alpha_A). Instruction stream 13 after this replacement can also be referred to as the fifth instruction segment.

[0133] like Figure 7 As shown, the replacement module can transmit the instruction stream 13 to the graphics library so that the graphics library can call the corresponding interface to instruct the GPU to render the semi-transparent particles in the Nth frame image. Since the frame buffer bound in the instruction stream 13 points to the frame buffer 12 (such as FB(alpha_A) in the example above), the rendering result of the semi-transparent particles in the Nth frame image by the GPU can be stored on the texture_A of FB(alpha_A).

[0134] This completes the rendering of the semi-transparent particles in the Nth frame image, and the rendering result can be stored in frame buffer 12.

[0135] It should be noted that in different implementations, such as Figure 6 The rendering process of the main scene shown, and as... Figure 7 The rendering process for the translucent particles shown can have two steps in different order. For example, in some implementations, the game application can first instruct the electronic device to execute, such as... Figure 6 The main scene rendering is shown. Afterwards, the game application can instruct the electronic device to perform actions such as... (The sentence is incomplete and requires more context to translate accurately). Figure 7 The image shows a semi-transparent particle rendering.

[0136] In this embodiment of the application, after the main scene rendering and the semi-transparent particle rendering are completed, since the two rendering results are stored on different textures, it is necessary to merge the main scene and the semi-transparent particles into one texture through a merging operation.

[0137] For example, the rendering result of the main scene of the Nth frame image (such as the main scene 11) is stored in frame buffer 11, and the rendering result of the semi-transparent particles is stored in frame buffer 12.

[0138] Therefore, the compositing module in the electronic device can instruct the GPU to perform the compositing action of the two rendering results after the main scene and the semi-transparent particles have been rendered.

[0139] As one possible implementation, electronic devices would first accomplish something like... Figure 6 The main scene rendering is shown, and then the following is completed: Figure 7 The example shown is a semi-transparent particle rendering. Combined with... Figure 7 After the replacement module sends instruction stream 13 to the graphics library, the GPU can then sequentially render the translucent particles. For example... Figure 8 As shown, the replacement module can send a semi-transparent particle rendering completion indication to the compositing module after issuing instruction stream 13 to the graphics library. This informs the compositing module that the semi-transparent particle rendering is complete. Next, the compositing module can send instruction stream 14 to the GPU, instructing the GPU to perform compositing of the main scene rendering result and the semi-transparent particle rendering result.

[0140] Understandably, in this example, when the compositing module sends instruction stream 14, even if the GPU has not yet fully completed the rendering operation of the semi-transparent particles, since instruction stream 14 arrives at the GPU later than instruction stream 13, the execution of instruction stream 14 will also be after instruction stream 13 in the GPU's instruction queue. Therefore, it can be guaranteed that when the compositing operation is performed, the main scene rendering result and the semi-transparent particle rendering result have been stored in the corresponding frame buffer.

[0141] In other implementations of this application, the triggering mechanism for the synthesis module to issue instruction stream 14 may also differ from that of other modules. Figure 8 For example, after the GPU completes the execution of instruction stream 13, it can return a semi-transparent particle rendering completion indication to the compositing module. Then, in response to this semi-transparent particle rendering completion indication, the compositing module can issue instruction stream 14 to instruct the GPU to perform compositing of the main scene rendering result and the semi-transparent particle rendering result.

[0142] In response to instruction stream 14 issued by the compositing module, the GPU can read stored data from frame buffer 11 and frame buffer 12. It can be understood that during the rendering of the Nth frame, the data in frame buffer 11 is the main scene rendering result (i.e., main scene 11) of the Nth frame, and the data in frame buffer 12 is the rendering result of the semi-transparent particles in the Nth frame. The GPU can perform this compositing operation on frame buffer 13 created by the creation module. For example, the GPU can copy the main scene 11 onto the texture of frame buffer 13, and then composite the rendering result of the semi-transparent particles in frame buffer 12 onto the texture of frame buffer 13, thereby completing the compositing operation.

[0143] As an example, the following is an example of an algorithm for a synthesis operation provided in this application.

[0144] "highp vec4 color1 = texture(FB0_main, TexCoord);\n"

[0145] "highp vec4 color2 = texture(Texture_A, TexCoord);\n"

[0146] if (color1.a < 0.001)\n"

[0147] "outColor.rgb = color1.rgb + color2.rgb;\n"

[0148] "else\n"

[0149] "outColor.rgb = color1.a * color1.rgb + (1.0f - color1.a) *color2.rgb;\n"

[0150] Thus, the rendering results, including the main scene and semi-transparent particles, can be obtained on frame buffer 13. Next, the electronic device can continue processing the data on frame buffer 13 based on other instructions in the Nth frame image issued by the game application. For example, user interface (UI) rendering can be performed on the textures of frame buffer 13, and then display data can be obtained. Based on this display data, the Nth frame image can be displayed on the screen.

[0151] In this way, the rendering process of the Nth frame image can be completed. As can be seen, in the embodiment of this application, during the rendering process of the Nth frame image, the rendering result of the semi-transparent particles can be stored on a newly created frame buffer, thereby realizing the reuse function of subsequent frame images.

[0152] To provide a clearer explanation of the solutions provided in the embodiments of this application, the following description is provided in conjunction with... Figure 9 The interactive flowchart shown below will continue to explain the rendering process of the Nth frame image. For example... Figure 9 As shown, the solution may include:

[0153] S901. After the game application sends instruction stream 11, the interception module will send instruction stream 11 back to the graphics library.

[0154] Instruction stream 11 is used to instruct the electronic device to render the main scene. As explained above, the interception module can intercept the corresponding instruction stream based on preset opening and closing instructions. Other instruction streams can be directly called back to the graphics library. For example, instruction stream 11 can be directly called back to the graphics library, i.e., executing S901.

[0155] S902, the graphics library instructs the GPU to perform the corresponding rendering operation.

[0156] For example, the graphics library calls the corresponding API according to instruction stream 11, instructing the GPU to perform rendering operations related to the main scene. The rendering result (such as the main scene 11) can be stored in the frame buffer 11 indicated by instruction stream 11.

[0157] S903, GPU rendering to obtain main scene 11 data.

[0158] S904 and GPU store the main scene 11 data in frame buffer 11.

[0159] This completes the rendering of the main scene, and the corresponding texture of the main scene is obtained and stored in frame buffer 11.

[0160] The specific execution process of S901-S904 mentioned above can be referred to as follows: Figure 6 The explanation will not be repeated here.

[0161] S905. After the game application sends out instruction stream 12, the interception module intercepts instruction stream 12.

[0162] In this context, instruction stream 12 may be an instruction that instructs an electronic device to render translucent particles.

[0163] For example, the interception module can monitor whether a preset opening instruction appears in the command stream issued by the game application. For instance, this opening instruction could be a `glEnable()` instruction. Upon detecting this `glEnable()` instruction, the interception module can begin intercepting the instruction. The interception module can also monitor whether a preset closing instruction appears in the command stream issued by the game application. For instance, this closing instruction could be a `glDisable()` instruction. Upon detecting this `glDisable()` instruction, the interception module can stop intercepting the instruction. The intercepted instructions would then constitute command stream 12.

[0164] S906, The interception module sends instruction stream 12 to the creation module.

[0165] S907, Create module to create frame buffer 12 and frame buffer 13.

[0166] S908, the creation module sends instruction stream 12 and the new frame buffer ID to the replacement module.

[0167] For example, the newly created frame buffer ID may include the IDs of frame buffer 12 and frame buffer 13. The ID of frame buffer 12 may be sent before the replacement module executes S909 below. The ID of frame buffer 13 may be sent before the compositing operation is performed. For example, the ID of frame buffer 13 may be sent before executing S914 below. In different implementations, the IDs of frame buffer 12 and frame buffer 13 may be sent together or separately.

[0168] S909, The replacement module replaces the frame buffer bound in instruction stream 12 with frame buffer 12.

[0169] In this example, instruction stream 12 is used to instruct the rendering of the semi-transparent particles, and the frame buffer bound to instruction stream 12 is used to store the rendering results of the semi-transparent particles. In this example, the replacement module can replace the frame buffer bound to instruction stream 12 with a newly created frame buffer 12 to store the rendering results of the semi-transparent particles in frame buffer 12.

[0170] After the replacement module completes the S909 operation, the corresponding instruction stream 13 can be obtained. This instruction stream 13, like instruction stream 12, is used to instruct the electronic device to render semi-transparent particles. The difference is that the frame buffer storing the semi-transparent particles is replaced by frame buffer 12.

[0171] S910, The replacement module sends instruction stream 13 to the graphics library.

[0172] S911, the graphics library instructs the GPU to perform the corresponding rendering operation.

[0173] For example, the graphics library calls the corresponding API according to instruction stream 13, instructing the GPU to perform rendering operations related to the translucent particles. The rendering results of the translucent particles can then be stored in the frame buffer 12 indicated by instruction stream 13.

[0174] S912, GPU rendering to obtain the rendering result of semi-transparent particles.

[0175] S913, the GPU stores the rendering results of the semi-transparent particles in frame buffer 12.

[0176] For the specific execution process of S905-S913 mentioned above, please refer to the following: Figure 7 The explanation will not be repeated here.

[0177] S914, The replacement module sends a semi-transparent particle rendering completion indication to the compositing module.

[0178] For example, the replacement module may execute S914 after sending instruction stream 13. In some embodiments, the replacement module may send the ID of frame buffer 13 to the synthesis module when executing S914, so as to facilitate subsequent synthesis operations on frame buffer 13.

[0179] S915, the compositing module sends instruction stream 14 to the GPU. This instruction stream 14 is used to instruct the GPU to perform compositing operations on the frame buffer 13.

[0180] S916, GPU reads main scene 11 data from frame buffer 11.

[0181] S917, GPU reads the rendering results of the semi-transparent particles from frame buffer 12.

[0182] In some embodiments, since the GPU has already completed the rendering of the main scene and the semi-transparent particles, it is aware of their storage locations. Therefore, the GPU can execute S914 and S915 after receiving instruction stream 14. In other embodiments, when executing S915, the instruction stream 14 sent by the compositing module to the GPU may also carry the IDs of the frame buffers to be composited. For example, the instruction stream 14 may carry the IDs of frame buffers 11 and 12, so that the GPU can execute subsequent S916 and S917 according to the frame buffers indicated by instruction stream 14. As a possible implementation, the frame buffer IDs to be composited carried by the compositing module in instruction stream 14 may be sent to the compositing module by the replacement module in S914.

[0183] S918, GPU synthesized the main scene data 11 and semi-transparent particle data.

[0184] S919 and the GPU store the composited result in frame buffer 13. For the specific execution process of S914-S919 described above, please refer to [reference needed]. Figure 8 The explanation will not be repeated here.

[0185] It should be noted that, in cases such as Figure 9 In the example, the textures that composite the rendering results of the main scene and the semi-transparent particles are stored in frame buffer 13. Therefore, the electronic device can also replace the frame buffer pointed to by other rendering instructions (such as instructions instructing the electronic device to perform UI rendering) based on the rendering results of the main scene and the semi-transparent particles during the current frame image rendering process with frame buffer 13. This allows the electronic device to continue rendering UI elements on the textures in frame buffer 13 according to subsequent rendering instructions, thereby obtaining the complete rendering result of the current frame image.

[0186] Thus, through the aforementioned Figures 6-9The rendering of the Nth frame can be completed by following the instructions. The rendering result of the semi-transparent particles of the Nth frame can be stored in frame buffer 12.

[0187] The following describes the scheme for reusing the rendering result of the semi-transparent particles in the Nth frame image in the (N+1)th frame image.

[0188] For example, please refer to Figure 10 During the rendering of the N+1th frame, the game application can issue instruction stream 21 to instruct the electronic device to render the main scene of the N+1th frame.

[0189] Similar to the interception mechanism in the Nth frame image, the interception module can directly call back instruction stream 21 to the graphics library. Correspondingly, the graphics library can call the API corresponding to instruction stream 21 to instruct the GPU to perform rendering operations. The GPU can then perform rendering operations on the main scene in the (N+1)th frame according to the instructions of instruction stream 21. With the main scene unchanged, instruction stream 21 instructs the GPU to store the rendering result of the main scene in the (N+1)th frame image (such as main scene 21) in frame buffer 11. Correspondingly, the GPU can then perform the rendering operation of main scene 21 on the texture of frame buffer 11 (such as FB0).

[0190] Thus, after completing such Figure 10 Following the rendering process shown, the rendering result of the main scene in frame buffer 11 can be updated to the N+1th frame image. For example, the data corresponding to the main scene 21 can be stored on the texture of FB0.

[0191] In this example, such as Figure 10 The main scene rendering process of the N+1th frame image shown is similar to that of... Figure 6 The rendering process of the main scene in the Nth frame image shown is similar, and their specific execution processes can be referenced from each other, so they will not be repeated here.

[0192] In this example, the rendering of the N+1th frame image may also include the rendering process of the semi-transparent particles.

[0193] For example, please refer to Figure 11 The game application can issue command stream 22 to instruct the electronic device to render the semi-transparent particles. Similar to command stream 12 for the Nth frame, the beginning and end commands of command stream 22 can be relatively fixed. For example, the beginning command of command stream 22 can be the glEnsable() command, and the end command can be the glDisable() command. The commands issued between the glEnsable() and glDisable() commands constitute the rendering commands for the semi-transparent particles that the game application instructs the electronic device to execute in the (N+1)th frame.

[0194] In this example, after recognizing instruction stream 22, the interception module no longer forwards instruction stream 22 to other modules. In other words, the interception module can monitor the instruction streams issued by the game application, and upon detecting the glEnsable() instruction, it returns all subsequent instructions up to the glDisable() instruction, without forwarding them to the GPU or other modules for processing.

[0195] Thus, during the rendering of the (N+1)th frame, although the game application issues rendering commands for the semi-transparent particles, the electronic device does not actually execute the rendering process for those particles. This saves rendering overhead during the rendering of the (N+1)th frame.

[0196] like Figure 11 As shown, the interception module can also send a synthesis trigger indication to the synthesis module after receiving instruction stream 22. Exemplarily, in some embodiments, the interception module can send a synthesis trigger indication to the synthesis module after detecting the glEnsable() instruction. In other embodiments, the interception module can send a synthesis trigger indication to the synthesis module after detecting the glDisable() instruction. This synthesis trigger indication can be used to instruct the synthesis module to trigger a synthesis instruction.

[0197] It is understandable that when the N+1th frame begins rendering, the rendering results of the semi-transparent particles saved during the rendering process of the Nth frame can be stored in frame buffer 12.

[0198] During the rendering of the (N+1)th frame image, based on, Figure 10 The main scene rendering diagram shown illustrates that the data stored in frame buffer 11 can be updated to the main scene 21 of the N+1th frame image. As for frame buffer 12, since the interception module returns the instruction stream 22, the electronic device does not execute the rendering of the corresponding semi-transparent particles. Therefore, the rendering result of the semi-transparent particles of the Nth frame image can still be stored in frame buffer 12.

[0199] In this example, the electronic device can synthesize the rendering results of the translucent particles in the Nth frame image and the rendering results of the main scene in the N+1th frame image (such as the main scene 21) to obtain the rendering results of the N+1th frame image.

[0200] For example, continue to refer to Figure 11The compositing module, upon receiving a compositing trigger instruction, can send instruction stream 23 to the GPU, instructing the GPU to perform compositing of the rendering results in frame buffer 11 and frame buffer 12. For example, in response to instruction stream 23, the GPU can read the main scene 21 from frame buffer 11, read the rendering results of the semi-transparent particles from frame buffer 12, and composite the main scene 21 and the rendering results of the semi-transparent particles onto frame buffer 13. Thus, the rendering result of the N+1th frame image can be obtained in frame buffer 13. It should be noted that, in this example, instruction stream 23 is related to... Figure 8 or Figure 9 The instruction stream 14 shown functions similarly, and the GPU responds to the synthesis operations performed by this instruction stream 23 as follows: Figure 8 or Figure 9 The GPU response to the synthesis operation executed by instruction stream 14 is similar to that shown, and their execution processes can be referenced from each other, so they will not be described in detail here.

[0201] In this way, during the rendering process of the (N+1)th frame, the rendering result of the semi-transparent particles in the Nth frame is reused, thereby reducing the rendering overhead of the (N+1)th frame.

[0202] To provide a clearer explanation of the solutions provided in the embodiments of this application, the following description is provided in conjunction with... Figure 12 The interactive flowchart shown continues to explain the rendering process of the (N+1)th frame image. Figure 12 As shown, the solution may include:

[0203] S1201 After the game application sends out instruction stream 21, the interception module sends instruction stream 21 back to the graphics library.

[0204] The instruction stream 21 can be used to instruct the electronic device to render the main scene of the N+1th frame image.

[0205] S1202, The graphics library instructs the GPU to perform the corresponding rendering operation.

[0206] S1203, GPU rendering acquires main scene 21 data.

[0207] S1204, the GPU stores the main scene 21 data in frame buffer 11.

[0208] In this example, the rendering process of the main scene in the (N+1)th frame is similar to the rendering process of the Nth frame. The execution process of S1201-S1204 can be mapped to, for example, ... Figure 10 The description is as follows. It should be understood that, in some embodiments, the execution process of S1201-S1204 can be referred to as follows. Figure 9The specific implementation processes of S901-S904 shown can be referenced interchangeably and will not be repeated here. Through S1201-S1204, the rendering result of the main scene of the N+1th frame image can be obtained in frame buffer 11, such as main scene 21.

[0209] S1205. After the game application sends out command stream 22, the interception module returns the command stream 22 and sends a compositing trigger instruction to the compositing module. Command stream 22 can be used to instruct the electronic device to render the semi-transparent particles of the N+1th frame image.

[0210] S1206, The merging module sends instruction stream 23 to the GPU. This instruction stream 23 can be used to instruct the GPU to perform a merging operation.

[0211] Through the operations of S1205-S1206, the electronic device can achieve the effect of returning the instruction stream 22 and instructing the GPU to reuse the semi-transparent particle rendering results. The specific execution process can be found in... Figure 11 The explanation in the document.

[0212] S1207, GPU reads main scene 21 data from frame buffer 11.

[0213] S1208, the GPU reads the rendering results of the semi-transparent particles from frame buffer 12.

[0214] S1209, GPU-synthesized main scene 21 data and semi-transparent particle data.

[0215] S1210, the GPU stores the synthesized results in frame buffer 13.

[0216] The merging process of S1207-S1210 can be referenced as follows: Figure 9 As shown in S914-S919, the rendering result of the (N+1)th frame image is obtained by reusing the rendering result of the semi-transparent particles in the Nth frame image. Similar to the aforementioned... Figure 9 As explained, the electronic device can replace the frame buffer pointed to by the instruction of the rendering result based on the main scene and semi-transparent particles in the subsequent rendering of other N+1 frames with frame buffer 13, thereby realizing the complete rendering process and obtaining the rendering result of the complete N+1 frame image.

[0217] Through the above, as shown Figures 6-12The explanation states that the electronic device can store the rendering result of the semi-transparent particles in a newly created frame buffer during the rendering of the Nth frame. This allows the semi-transparent particles to be rendered again in the (N+1)th frame; instead, the rendering result stored in the newly created frame buffer is reused and merged with the main scene of the (N+1)th frame to obtain the rendering result of the (N+1)th frame. This saves at least the rendering overhead of the semi-transparent particles in the (N+1)th frame.

[0218] The above examples illustrate the solutions provided in this application from the perspective of interaction between modules. The following will continue to describe the solutions provided in this application from the perspective of electronic devices.

[0219] For example, please refer to Figure 13 This is a schematic diagram illustrating an image rendering process provided in an embodiment of this application. Figure 13 Therefore, the solution may include:

[0220] S1301, The electronic device determines the frame buffer 11 corresponding to the main scene.

[0221] Based on the foregoing explanation, the electronic device can determine the frame buffer 11 corresponding to the main scene before processing the Nth frame image begins. For example, it can determine that the frame buffer of the rendering pipeline with the largest number of draw calls among the already rendered frame images is the frame buffer of the main scene.

[0222] S1302, The electronic device performs main scene rendering of the Nth frame image on the frame buffer 11 to obtain the main scene 11.

[0223] Starting from S1302, the electronic device can render the Nth frame image according to the instruction stream issued by the game application.

[0224] S1303, The electronic device stores the rendering result of the semi-transparent particles of the Nth frame image on the newly created frame buffer 12.

[0225] S1304. The electronic device determines the rendering result of the Nth frame image based on the rendering results of the main scene 11 and the semi-transparent particles.

[0226] The execution process of S1302-S1304 can be corresponding to, for example: Figures 6-9 The proposed solutions and their implementation details can be referenced from each other.

[0227] S1305, The electronic device performs main scene rendering of the N+1th frame image on frame buffer 11 to obtain main scene 21.

[0228] S1306, The electronic device determines the rendering result of the N+1th frame image based on the main scene 21 and the semi-transparent particle data.

[0229] The execution process of S1305-S1306 corresponds to the rendering process of the (N+1)th frame image by the electronic device. The specific implementation process can be found in the following example: Figures 10-12 Explanation.

[0230] It should be understood that the above-mentioned Figures 6-13 The explanations all use the example of reusing the semi-transparent particle rendering result of the Nth frame in the (N+1)th frame. The Nth frame can be any frame after the game starts running. For frames different from the N+1th frame after the Nth frame, in some embodiments, the semi-transparent particle rendering result of the Nth frame can be reused; in other embodiments, the semi-transparent particle rendering result of other frames can be reused, or the rendering of the semi-transparent particles in the current frame can be re-executed to update the semi-transparent particle rendering result and obtain a more accurate result.

[0231] In the embodiments of this application, the electronic device may have a preset strategy for determining the frame image that needs to be rendered as a semi-transparent particle, and for reusing the frame image of the semi-transparent particle.

[0232] For example, an electronic device can determine whether to reuse translucent particles based on the characteristics of the currently rendered frame image (such as which frame image is being rendered after the game has started running).

[0233] As an example, an electronic device can be equipped with a counter that increments by 1 when each frame of an image begins rendering. For instance, when the first frame of a game starts rendering, the counter increments by 1, resulting in a value of 1, which identifies the current frame as the first frame. Similarly, when the Nth frame of an image begins rendering, the counter increments by 1, resulting in a value of N, which identifies the current frame as the Nth frame. In this way, the electronic device can determine whether to render or reuse the semi-transparent particles for the current frame based on the counter's value and predefined rules.

[0234] It's important to note that when rendering the first frame after the game starts, the semi-transparent particles won't be reused since no rendering of these particles has been performed yet. However, this can be mitigated by default rules. For example, in electronic devices, a default rule could be: if the counter value is even, reuse of the semi-transparent particles is performed; otherwise (if the counter value is odd), rendering of the semi-transparent particles is performed. Therefore, when rendering the first frame, since the counter value is 1 (odd), rendering of the semi-transparent particles can be performed on the newly created framebuffer. Correspondingly, when rendering the second frame, the counter is incremented by 1, resulting in 2 (even). Therefore, reuse of the semi-transparent particles is possible.

[0235] For example, in combination Figure 13 Please refer to the flowchart. Figure 14 This is a schematic diagram illustrating another image rendering process provided in an embodiment of this application. Figure 14 As shown, the solution may include:

[0236] S1401. Determine the frame buffer 11 corresponding to the main scene.

[0237] Based on the foregoing explanation, the process of determining the main scene can be performed before the rendering of the current frame image begins.

[0238] S1402. When rendering of the current frame image begins, increment the counter by 1.

[0239] In this example, by incrementing a counter by 1, the counter's value can be used to identify the features of the current frame image. In this way, different frame images can have different features, and correspondingly, the different frame images can be distinguished by the counter's value.

[0240] S1403. Render the main scene of the current frame image on frame buffer 11 to obtain the main scene 11.

[0241] For example, this process can refer to the main scene rendering process for the Nth frame image or the N+1th frame image in the previous example.

[0242] S1404. Determine whether the value of the counter conforms to the preset rules.

[0243] In this example, the decision to render or reuse semi-transparent particles can be determined based on preset rules. For example, the preset rule is: whether the counter value is even. If the counter value is even, the semi-transparent particles are reused, i.e., S1407 is executed. Conversely, if the counter value is not even, i.e., odd, the semi-transparent particles are rendered, i.e., S1405-S1406 are executed.

[0244] S1405, Render the semi-transparent particles of the current frame image on the newly created frame buffer 12.

[0245] S1406. Determine the rendering result of the current frame image based on the rendering results of the main scene 11 and the semi-transparent particles.

[0246] This process can refer to the rendering process of the Nth frame image in the previous example. Therefore, while rendering the current frame image, the rendering result of the corresponding semi-transparent particles is saved on frame buffer 12 so that the rendering result of the semi-transparent particles can be reused in other frame images.

[0247] S1407. Determine the rendering result of the current frame image based on the main scene 11 data and the semi-transparent particle rendering result stored in the frame buffer 12.

[0248] This process can refer to the rendering process of the N+1th frame image in the previous example. This allows for the reuse of the semi-transparent particle rendering result from the current frame.

[0249] Understandable Figure 14 The flowchart shown is one possible implementation of this application and can be applied to the rendering process of images including the Nth frame and the (N+1)th frame, thereby supporting, for example... Figures 6-13 The scheme shown is implemented.

[0250] In the above example, during the rendering of the current frame image, a preset rule can be used to determine whether to trigger the reuse of the rendering results of existing semi-transparent particles. In other embodiments of this application, the electronic device can also combine other judgment conditions to determine whether to trigger the reuse of the rendering results of semi-transparent particles, thereby making the reuse of the rendering results of semi-transparent particles more stringent and thus obtaining more accurate rendering results.

[0251] For example, an electronic device can determine whether the rendering results of the semi-transparent particles in two frames can be reused by combining the changes in the position of the semi-transparent particles in the current frame image and the semi-transparent particles in the previous frame image.

[0252] It should be understood that translucent particles belong to high-frequency signals (i.e., corresponding to details in the image that change drastically). The human eye is more sensitive to high-frequency signals, so when the game character's perspective shakes significantly, the translucent particles need to be updated in real time.

[0253] In this example, the changes in the Model-View-Projection (MVP) matrix can be used to determine whether the character's viewpoint is shaking significantly.

[0254] The following combination Figure 15A brief explanation of the MVP matrix follows. During image rendering, an electronic device needs to determine the vertex positions of one or more objects included in the current frame image. For example, the rendering command issued by the application may include the vertex coordinates of the objects. In some implementations, the vertex coordinates included in the rendering command may be coordinates based on the object's own local coordinate system. In this application, the distribution space of the object based on the local coordinate system can be called the local space. To determine the coordinates of each vertex of the object on the display screen, the electronic device can perform a matrix transformation based on the object's coordinates in the local space. This yields the object's coordinates in the display-based space (e.g., screen space).

[0255] As an example, electronic devices can perform matrix transformations from local space to world space, then to view space, then to clip space, and finally to screen space to convert the local coordinates of an object's vertices in local space to coordinates in screen space.

[0256] For example, such as Figure 15 The diagram illustrates the logical process of a matrix transformation of coordinates from local space to world space to view space to clip space. In this example, the rendering commands issued by the game application may include rendering of object 1. Figure 15 As shown, in local space, the coordinate system can be based on object 1. For example, the origin of the coordinate system in local space can be set at the center of object 1, or at the location of a vertex, etc. The game application can include the coordinates of each vertex of object 1 in the local space coordinate system when issuing rendering commands for object 1; these are the local coordinates. The electronic device can convert the coordinates in local space to coordinates in world space using the M-matrix issued by the game application. World space can be a larger region than local space. For example, taking the rendering command issued by the game application for rendering game images as an example, local space can correspond to a small area that covers a specific object (such as object 1). World space can correspond to a map area in the game that includes object 1 and other objects (such as object 2). The electronic device can combine the local coordinates in local space with the M-matrix to perform an M-matrix transformation, thereby obtaining the coordinates of object 1 in world space. Similarly, when the game application issues a rendering command for object 2 in the current frame, the electronic device can also obtain the coordinates of object 2 in world space through the aforementioned M-matrix transformation.

[0257] After obtaining the coordinates of the vertices of each object in the current frame image in world space, the electronic device can convert these coordinates into coordinates in the view space based on the V matrix provided by the game application. It's understandable that the coordinates in world space can be three-dimensional. However, when the electronic device displays the frame image to the user, each object (such as object 1, object 2, etc.) is displayed on a two-dimensional screen. Different two-dimensional images will be seen when viewing the objects in world space from different viewing angles. This viewing angle can be related to the position of the camera (or observer) set in world space. In this example, the coordinate space corresponding to the camera position can be called the view space. For example, let's take the camera being set in the positive y-axis direction of world space. Then, based on the transformation of the V matrix, the coordinates of each vertex of object 1 and object 2 in the view space corresponding to that camera position can be obtained. Figure 15 As shown, since the camera is located in the positive y-axis direction and shoots downwards, objects 1 and 2 in the observation space can be presented as a top-down view.

[0258] After acquiring the coordinates of each object in the observation space, the electronic device can project them onto clipping coordinates. The coordinate space corresponding to these clipping coordinates can be called the clipping space. It's understandable that the V-matrix transformation can involve a large region of world space, resulting in a potentially large image range. However, due to the limited size of the electronic device's display screen, it may not be possible to display all objects in the observation space simultaneously. In this example, the electronic device can project the coordinates of each object in the observation space onto the clipping space. After projection into the clipping space, the coordinates of objects that can be displayed on the screen are within the range of -1.0 to 1.0. The coordinates of objects that cannot be displayed are outside this range. Thus, the electronic device can display the corresponding vertices based on their coordinates within the range of -1.0 to 1.0. For example, the electronic device can perform a P-matrix transformation on each coordinate in the observation space based on the P-matrix provided by the game application, thereby obtaining the clipping coordinates in the clipping space corresponding to each coordinate.

[0259] Therefore, a significant change in the MVP matrix indicates a significant shift in the character's perspective in the game. Thus, the previously used positions of the translucent particles are no longer applicable in the current frame. In this example, the electronic device can also determine whether to reuse the translucent particles based on whether the perspective change is within a preset perspective threshold. In some embodiments, it is possible to determine perspective changes in different frames, such as... Figure 16As shown, a reference camera based on the observation space can be constructed. Based on the MVP matrix of the current frame image, the reference viewing direction of the reference camera in world space can be calculated. Similarly, based on the MVP matrix of previous frame images, the reference viewing direction of the corresponding frame image can be calculated. The electronic device determines the viewpoint change by comparing the two reference time directions. In some embodiments, the viewpoint change can be the angle between the reference viewing directions of different frame images.

[0260] As one possible implementation, a reference camera viewing direction can be constructed in the electronic device, such as a matrix represented as (10,0,0,0), which can be a direction based on the viewing space. When rendering the current frame image, the electronic device can determine the MVP matrix of the current frame image based on instructions issued by the game application. For example, the electronic device can obtain the MVP matrix data based on the uniform matrix passed through instructions such as glBufferSubData() in the instructions issued by the game application to the CPU. In this way, the electronic device can obtain the P matrix (e.g., denoted as P_N), the VP inverse matrix (e.g., denoted as VP_INV_N), and the M matrix (e.g., denoted as M_N) of the Nth frame image. Similarly, the electronic device can obtain the MVP matrices of other frames. For example, during the rendering of the N+1th frame image, the electronic device can obtain the P matrix (e.g., denoted as P_N+1), the VP inverse matrix (e.g., denoted as VP_INV_N+1), and the M matrix (e.g., denoted as M_N+1) of the N+1th frame image.

[0261] In this way, during the rendering of the (N+1)th frame, the electronic device can determine whether the rendering result of the semi-transparent particles in the Nth frame can be reused based on the change in viewpoint between the (N+1)th frame and the Nth frame.

[0262] For example, an electronic device can determine the change in viewing angle based on the following calculation method:

[0263] cameraToWorld = (10,0,0,0) * P_N * VP_INV_N; / / The position of the camera in world coordinates in the Nth frame;

[0264] preCameraToWorld = (10,0,0,0) * P_N+1 * (VP_INV_N+1); The position of the camera in world coordinates in the (N+1)th frame;

[0265] alpha = cameraToWorld - M_N; / / Camera orientation matrix in the Nth frame;

[0266] beta = preCameraToWorld - (M_N+1); / / Camera orientation matrix in the (N+1)th frame;

[0267] aProductb = alpha[0] * beta[0] + alpha[1] * beta[1] + alpha[2] * beta[2];

[0268] aMode = std::sqrt(alpha[0] * alpha[0] + alpha[1] * alpha[1] + alpha[2] * alpha[2]);

[0269] bMode = std::sqrt(beta[0] * beta[0] + beta[1] * beta[1] + beta[2] *beta[2]);

[0270] cosRes = aProductb / (aMode *bMode);

[0271] turnTheta = (std::acos(cosRes)*180) / PI.

[0272] The final obtained turnTheta angle can be the change in viewpoint between the (N+1)th frame and the Nth frame.

[0273] The electronic device can determine whether the rendering result of the semi-transparent particles in the Nth frame image can be reused based on the relationship between turnTheta and a preset angle threshold. For example, if turnTheta is less than the preset angle threshold, it indicates that the difference between the two frames is small, and the rendering result of the semi-transparent particles can be reused. Conversely, if turnTheta is greater than the preset angle threshold, it indicates that the difference between the two frames is large, and the rendering result of the semi-transparent particles cannot be reused.

[0274] Based on this, please refer to Figure 17 This is a schematic flowchart illustrating another image rendering method provided in an embodiment of this application. This solution... Figure 14 Based on this, a step to judge changes in viewpoint has been added, thereby obtaining more accurate rendering results. For example... Figure 17 As shown, the solution in this example differs from... Figure 14The proposed solution, after executing S1404, if the judgment meets the preset rules, proceeds to S1701 for further judgment. This involves determining whether the viewpoint change is less than a viewpoint threshold. Before this S1701 judgment, S1702 can be executed, which determines the viewpoint change based on the MVP matrix of the current frame image and the backup MVP matrix. The specific execution method can be found in the viewpoint change determination scheme in the example above. In the S1702 judgment, if the viewpoint change is less than the viewpoint threshold, it indicates that the semi-transparent particles can be reused, and S1407 is executed. Conversely, if the viewpoint change is greater than the viewpoint threshold, it indicates that the semi-transparent particles cannot be reused, and the process returns to execute S1405.

[0275] In this way, before performing reuse, it can be further determined whether the semi-transparent particles can be reused based on the change of viewing angle, thereby improving the accuracy of the reuse of semi-transparent particles and improving the quality of the final image.

[0276] The above mainly describes the solutions provided by the embodiments of this application from the perspective of various service modules. To achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. It should be noted that the module division in the embodiments of this application is illustrative and only represents a logical functional division; other division methods may exist in actual implementation.

[0277] Figure 18 A schematic diagram of the composition of an electronic device 1800 is shown. (As shown) Figure 18 As shown, the electronic device 1800 may include a processor 1801 and a memory 1802. The memory 1802 is used to store computer execution instructions. Exemplarily, in some embodiments, when the processor 1801 executes the instructions stored in the memory 1802, the electronic device 1800 may perform any of the image rendering methods shown in the above embodiments.

[0278] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0279] Figure 19A schematic diagram of a chip system 1900 is shown. The chip system 1900 may include a processor 1901 and a communication interface 1902, used to support related devices in implementing the functions involved in the above embodiments. In one possible design, the chip system also includes a memory for storing necessary program instructions and data for the terminal. This chip system may be composed of chips or may include chips and other discrete devices. It should be noted that in some implementations of this application, the communication interface 1902 may also be referred to as an interface circuit.

[0280] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0281] The functions, actions, operations, or steps in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented using software programs, they can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or include one or more data storage devices such as servers and data centers that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0282] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. An image rendering method, characterized in that, Applied to an electronic device having an application installed, the method includes: The application sends a first instruction stream, which instructs the electronic device to perform a rendering operation of a first frame image, the first frame image including a first main scene and a first semi-transparent particle; The electronic device synthesizes a first rendering result and a second rendering result to obtain the first frame image; the first rendering result is the rendering result of the first main scene, the second rendering result is the rendering result of the first semi-transparent particle, and the second rendering result is stored in the first frame buffer of the electronic device; the second rendering result is the rendering result of the semi-transparent particle during the rendering process of the second frame image, and the second frame image is earlier than the first frame image.

2. The method according to claim 1, characterized in that, The first instruction stream includes a first instruction segment and a second instruction segment. The first instruction segment is used to instruct the electronic device to render the first main scene to obtain the first rendering result, and the second instruction segment is used to instruct the electronic device to render the first translucent particle. Before the electronic device synthesizes the first rendering result and the second rendering result, the method further includes: The electronic device performs rendering according to the first instruction segment to obtain the first rendering result; The electronic device obtains the second rendering result from the first frame buffer.

3. The method according to claim 1 or 2, characterized in that, The second rendering result is stored in the first frame buffer after the electronic device renders the second frame image, and the rendering of the second frame image is earlier than the rendering of the first frame image; The electronic device is equipped with a counter, which increments by 1 each time the electronic device renders a frame image; the conditions under which the second rendering result is used to synthesize the first frame image include: the value of the counter is an even number when the first frame image is rendered.

4. The method according to claim 1, characterized in that, Before the application issues the first instruction stream, the method further includes: The application sends a second instruction stream, which instructs the electronic device to perform a rendering operation of a second frame image, the second frame image including a second main scene and the first semi-transparent particle.

5. The method according to claim 4, characterized in that, The second instruction stream includes a third instruction segment and a fourth instruction segment. The third instruction segment is used to instruct the electronic device to render the second main scene to obtain a third rendering result, and the fourth instruction segment is used to instruct the electronic device to render the first semi-transparent particle in the second frame image. Before the application issues the second instruction stream, the method further includes: The electronic device performs rendering according to the third instruction segment to obtain the third rendering result; The electronic device obtains the fourth rendering result according to the fourth instruction segment.

6. The method according to claim 5, characterized in that, The method further includes: The electronic device creates the first frame buffer; The electronic device obtains the fourth rendering result according to the fourth instruction segment, including: The electronic device replaces the frame buffer indicated by the fourth instruction segment with the first frame buffer to obtain the fifth instruction segment; The electronic device executes the rendering operation of the fifth instruction segment to obtain the second rendering result of the first semi-transparent particle and stores the second rendering result in the first frame buffer.

7. The method according to claim 2, characterized in that, The electronic device determines the second instruction segment based on the preset start and end instructions in the first instruction stream.

8. The method according to claim 6, characterized in that, The electronic device determines the fourth instruction segment based on the preset start and end instructions in the second instruction stream.

9. The method according to any one of claims 5-8, characterized in that, The electronic device is equipped with an interception module, a creation module, and a replacement module; the method includes: The interception module is used to intercept the fourth instruction segment; The creation module is used to create the first frame buffer; The replacement module is used to replace the frame buffer ID in the fourth instruction segment according to the identifier ID of the first frame buffer and the intercepted fourth instruction segment, so as to obtain a fifth instruction segment pointing to the first frame buffer. The graphics processing unit (GPU) of the electronic device executes the rendering of the first semi-transparent particle according to the fifth instruction segment, and stores the obtained second rendering result in the first frame buffer.

10. The method according to claim 9, characterized in that, The electronic device is further provided with a merging module, and the method further includes: The merging module is used to instruct the GPU to merge the second rendering result and the third rendering result to obtain the rendering result of the second frame image.

11. The method according to claim 1, characterized in that, The method further includes: The frame buffer ID of the main scene is determined based on the process of the third frame image. The frame buffer of the main scene is the frame buffer with the largest number of draw calls during the processing of the third frame image.

12. The method according to any one of claims 1, 2, or 4-8, characterized in that, The electronic device is equipped with a counter, which increments by 1 each time the electronic device renders a frame of image. Before the electronic device synthesizes the first rendering result and the second rendering result, and obtains the first frame image, the method further includes: When the electronic device determines that the first frame image is being rendered, the value of the counter conforms to a preset rule.

13. The method according to claim 12, characterized in that, If the value of the counter does not conform to a preset rule when the electronic device determines that the first frame image is being rendered, the method further includes: The electronic device creates the first frame buffer and replaces the frame buffer pointed to by the instruction segment in the first instruction stream that indicates the rendering of the first semi-transparent particles with the first frame buffer. The electronic device renders the first semi-transparent particle and stores it in the first frame buffer.

14. The method according to claim 13, characterized in that, The preset rule is that the value of the counter is an even number.

15. The method according to claim 1, 13, or 14, characterized in that, Before the electronic device synthesizes the first rendering result and the second rendering result, and obtains the first frame image, the method further includes: The electronic device determines that the change in viewing angle during the rendering of the first frame image is less than a preset viewing angle threshold.

16. The method according to claim 15, characterized in that, The electronic device determines the viewpoint change based on the model-view-projection MVP matrix of the first frame image and the MVP matrix of the second frame image, wherein the rendering of the second frame image is earlier than the rendering of the first frame image.

17. The method according to claim 15, characterized in that, If the change in viewing angle during the rendering of the first frame image exceeds a preset viewing angle threshold, the method further includes: The electronic device creates the first frame buffer and replaces the frame buffer pointed to by the instruction segment in the first instruction stream that indicates the rendering of the first semi-transparent particles with the first frame buffer. The electronic device renders the first semi-transparent particle and stores it in the first frame buffer.

18. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; When the one or more processors execute the computer instructions, the electronic device performs the image rendering method as described in any one of claims 1-17.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed, perform the image rendering method as described in any one of claims 1-17.

Images