Rendering method and electronic equipment

Abstract

The invention provides a rendering method and electronic equipment, and aims to provide a rendering method capable of improving the rendering concurrency, so that the performance of the electronic equipment can be improved, and the power consumption can be reduced. In the method, a to-be-rendered task is divided into a plurality of to-be-rendered sub-tasks, then the to-be-rendered sub-tasks are processed through a plurality of threads, and a plurality of groups of drawing commands in one-to-one correspondence with the to-be-rendered sub-tasks are generated and stored in a plurality of command buffer areas; then, the multiple groups of drawing commands in the multiple command buffer areas are jointly submitted to the GPU in batches, so that splicing is not needed, and the drawing accuracy is guaranteed. Therefore, through parallel processing of multiple threads and batch submission of multiple groups of drawing commands obtained through parallel processing, graphic rendering with higher performance and lower power consumption can be realized.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, and more particularly to a rendering method and electronic device. Background Technology

[0002] As a crucial component of electronic devices, the user interface presents users with a wide variety of information. Therefore, the display quality of the user interface has a significant impact on the user experience.

[0003] Graphics rendering services are used to create user interfaces. The performance and power consumption of graphics rendering services have always been challenging technical issues that need to be addressed. Furthermore, as the frame rate requirements of user interfaces gradually increase, achieving high-performance, low-power graphics rendering is of great research significance. Summary of the Invention

[0004] This application provides a rendering method and an electronic device for achieving higher performance and lower power consumption graphics rendering.

[0005] Firstly, this application provides a rendering method applicable to electronic devices. In this method, the electronic device divides a task to be rendered into M subtasks, where M is a positive integer greater than or equal to 2. Using L threads, the electronic device generates M sets of drawing commands corresponding to the M subtasks, where L is less than or equal to M and is a positive integer greater than or equal to 2. These M sets of drawing commands are stored in M ​​command buffers. The electronic device then submits the M sets of drawing commands stored in the M command buffers in batches to a graphics processing unit (GPU), which performs rendering based on the M sets of drawing commands.

[0006] This method utilizes multiple threads to draw multiple subtasks in parallel, improving concurrency and rendering performance. Furthermore, by batch-submitting multiple sets of drawing commands, these commands can share the same screen buffer. Therefore, this method eliminates the need for concatenation, enhancing rendering accuracy. It also reduces CPU load and power consumption, shortening the time compared to multiple separate submissions and improving overall electronic device performance. Additionally, by setting a command buffer, earlier-generated drawing commands can be temporarily stored. Once all M sets of drawing commands are generated, the electronic device batch-submits all M sets to the GPU, allowing multiple sets of commands to share the same screen buffer. Therefore, this method eliminates the need for concatenation, further improving rendering accuracy.

[0007] In some implementations, the method further includes: the electronic device allocates N recycling buckets to L threads, where N is less than or equal to L. Each recycling bucket is used to cache resource release instructions from at least one thread. Optionally, each recycling bucket can collect resource release instructions from one thread. Alternatively, each recycling bucket can also collect resource release instructions from multiple threads.

[0008] In this implementation, resource release commands are collected through a recycling bin, enabling electronic devices to release resources in batches. This improves efficiency and reduces the frequency of locking and unlocking.

[0009] In some implementations, the method further includes: in response to a first set of drawing commands for a first subtask to be rendered, a first thread releases a resource release instruction for a first resource to a first recycle bin. Here, the first thread is any one of L threads, the first resource is the resource used by the first thread, and the first recycle bin is the recycle bin allocated to the first thread.

[0010] In this implementation, after the thread successfully generates drawing commands for the subtask to be rendered, the electronic device can temporarily store the resource release instructions instead of releasing the used resources. This allows for batch release of resources.

[0011] In some implementations, the method further includes executing resource release instructions included in the second recycling bucket via a second thread. Here, the second thread is the target thread among at least one thread corresponding to the second recycling bucket. The second thread can also be understood as the main thread among at least one thread.

[0012] In this implementation, when a single recycling bin is used to collect resource release commands from multiple threads, the main thread among these threads can execute the resource release commands. This improves the accuracy of the processing.

[0013] In some implementations, the method further includes: when the electronic device detects that the GPU has completed rendering the M sub-tasks to be rendered, executing resource release instructions included in the N recycling buckets at a specified time. The specified time is any time after the rendering of the M sub-tasks to be rendered is completed.

[0014] In this implementation, once all the sub-tasks included in the rendering task have been processed, the electronic device can release resources in batches by uniformly executing resource release commands. It can be understood that by releasing the resources used by the rendering task only after detecting that the rendering task has been completed, the efficiency of resource scheduling can be improved, and the additional overhead caused by repeatedly scheduling resources can be reduced.

[0015] In some implementations, for the third thread, the following steps are performed: In response to a request instruction for the first virtual resource used by the fourth thread, a second virtual resource is generated using a first token. Here, the third thread is any one of the L threads. The fourth thread is another thread among the L threads that is different from the third thread. The second virtual resource maps to the same physical resource as the first virtual resource. The first token is used to manage the virtual resource mapped to the physical resource.

[0016] This implementation, which uses multiple threads for parallel rendering, reduces the load caused by duplicate uploads due to resource duplication by sharing resources used by multiple threads. Furthermore, managing physical resources with tokens and copying virtual resources enhances resource management security.

[0017] In some implementations, the electronic device divides the task to be rendered into M sub-tasks, including dividing the node tree corresponding to the frame to be drawn into M node subtrees. The task to be rendered is obtained based on the node tree corresponding to the frame to be drawn, and the M sub-tasks are obtained based on the M node subtrees. Optionally, the number of nodes in the first node subtree is the same as or differs from the number of nodes in the second node subtree by a preset difference.

[0018] In this implementation, by splitting the node tree corresponding to the frame to be drawn, and by splitting based on the number of nodes as a reference, the processing balance of multiple threads can be achieved, thereby improving the efficiency and performance of parallel processing.

[0019] Secondly, this application also provides another rendering method that can be applied to electronic devices. The method provided in this second aspect may include: a graphics rendering service dividing a task to be rendered into a first sub-task group, a second sub-task group, and a third sub-task group. A first thread of the graphics rendering service generates a first set of drawing commands corresponding to the first sub-task group; a second thread of the graphics rendering service generates a second set of drawing commands corresponding to the second sub-task group; and a third thread of the graphics rendering service generates a third set of drawing commands corresponding to the third sub-task group. The graphics rendering service submits the first set of drawing commands, the second set of drawing commands, and the third set of drawing commands to the GPU, which performs rendering based on the first set of drawing commands, the second set of drawing commands, and the third set of drawing commands.

[0020] In some implementations, the first group of subtasks to be rendered includes at least two subtasks to be rendered.

[0021] In some implementations, the method further includes: allocating a first recycling bucket to the first thread and the second thread, and allocating a second recycling bucket to the third thread; or, allocating a first recycling bucket to the first thread, the second thread, and the third thread. The first recycling bucket or the second recycling bucket is used to cache resource release instructions for the corresponding thread.

[0022] In some implementations, the method further includes: allocating a first recycling bucket to a first thread and a second thread; in response to a first set of drawing commands, collecting resource release instructions from the first thread for a first resource, wherein the first resource is a resource used by the first thread, through the first recycling bucket; and in response to a second set of drawing commands, collecting resource release instructions from the second thread for a second resource, wherein the second resource is a resource used by the second thread, through the first recycling bucket.

[0023] In some implementations, the method further includes: executing resource release instructions for the first resource and the second resource included in the first recycling bucket through a first thread; wherein the first thread is the main thread serving the graphics rendering.

[0024] In some implementations, the method further includes: in response to a request instruction from a second thread for a first virtual resource used by a first thread, generating a second virtual resource using a first token; wherein the second virtual resource is mapped to the same physical resource as the first virtual resource, and the first token is used to manage the virtual resource mapped to the physical resource.

[0025] In some implementations, the graphics rendering service divides the tasks to be rendered into a first sub-task group, a second sub-task group, and a third sub-task group. This includes: the graphics rendering service dividing the node tree corresponding to the frame to be drawn into a first subtree, a second subtree, and a third subtree. The tasks to be rendered are obtained based on the node tree corresponding to the frame to be drawn. The first sub-task group is obtained based on the first subtree, the second sub-task group is obtained based on the second subtree, and so on.

[0026] In some implementations, any two subtrees of the first, second, and third node subtrees contain the same number of nodes or have a difference of less than a preset value.

[0027] In some implementations, the first, second, and third threads can be configured with corresponding command buffers, such as first, second, and third command buffers corresponding to their sequence numbers, to store the corresponding drawing commands. It should be understood that when the number of threads used to generate drawing commands in the graphics rendering service decreases or increases, the number of command buffers can also decrease or increase accordingly.

[0028] Thirdly, this application also provides an electronic device comprising: at least one memory and at least one processor; wherein the at least one memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the at least one processor, the electronic device performs the method in the first aspect and any implementation thereof, or performs the method in the second aspect and any implementation thereof.

[0029] Fourthly, this application also provides a rendering apparatus for long interfaces, which includes modules / units for executing the methods described in the first aspect and any implementation thereof. These modules / units can be implemented in hardware or by hardware executing corresponding software.

[0030] Fifthly, this application also provides a rendering apparatus, which includes modules / units for executing the methods described in the second aspect and any implementation thereof. These modules / units can be implemented in hardware or by hardware executing corresponding software.

[0031] In a sixth aspect, a computer-readable storage medium is provided, which stores a computer program (also referred to as code or instructions) that, when executed on a computer, causes the computer to perform the methods of the first aspect and any implementation thereof, or to perform the methods of the second aspect and any implementation thereof.

[0032] In a seventh aspect, a computer program product is provided, comprising: a computer program (also referred to as code or instructions), which, when executed, causes the methods in the first aspect and any implementation thereof to be executed, or causes the methods in the second aspect and any implementation thereof to be executed.

[0033] Eighthly, a graphical user interface is also provided for an electronic device having a display screen, one or more memories, and one or more processors, the one or more processors being used to execute one or more computer programs stored in one or more memories. The graphical user interface may include the graphical user interface displayed when the electronic device executes the first aspect of the present application and any implementation thereof, or may also include the graphical user interface displayed when the electronic device executes the second aspect of the present application and any implementation thereof.

[0034] It should be noted that the beneficial effects of the electronic devices provided in the second to eighth aspects of this application in various scenarios can be referred to the beneficial effects of any of the implementation methods in the first aspect above, and will not be repeated here. Attached Figure Description

[0035] Figure 1a A schematic diagram of an architecture suitable for graphics rendering services;

[0036] Figure 1b A schematic diagram of another architecture suitable for graphics rendering services;

[0037] Figure 2 A schematic diagram of the hardware structure of a possible electronic device is shown;

[0038] Figure 3 A software system architecture block diagram of an electronic device provided in this application embodiment;

[0039] Figure 4 This is a schematic diagram illustrating an application scenario for a rendering method provided in this application embodiment;

[0040] Figure 5 This is a schematic diagram illustrating a possible splitting result of the node tree provided in an embodiment of this application;

[0041] Figure 6a This is one of the schematic diagrams of an architecture applicable to a rendering method provided in an embodiment of this application;

[0042] Figure 6b A second schematic diagram of an architecture applicable to a rendering method provided in an embodiment of this application;

[0043] Figure 6c A third schematic diagram of an architecture applicable to a rendering method provided in an embodiment of this application;

[0044] Figure 7a Fourth schematic diagram of an architecture applicable to a rendering method provided in this application embodiment;

[0045] Figure 7b Fifth schematic diagram of an architecture applicable to a rendering method provided in this application embodiment;

[0046] Figure 8a This is one of the schematic diagrams illustrating a scenario where a rendering method provided in this application is applicable to resource migration and release;

[0047] Figure 8b A second schematic diagram illustrating a scenario for resource migration and release applicable to a rendering method provided in this application embodiment;

[0048] Figure 9a A schematic diagram illustrating a resource-sharing scenario to which a rendering method provided in this application is applicable;

[0049] Figure 9b A schematic diagram illustrating a resource-sharing scenario to which a rendering method provided in this application is applicable;

[0050] Figure 10 A flowchart illustrating a rendering method provided in an embodiment of this application;

[0051] Figure 11 This is another flowchart illustrating a rendering method provided in an embodiment of this application. Detailed Implementation

[0052] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0053] This application's embodiments can be applied to the field of computer graphics technology, specifically to scenarios involving graphics rendering on electronic devices. The operating system of an electronic device can use graphics rendering services to draw the user interface. For example, the graphics rendering service can render 2D graphics.

[0054] For example, Figure 1a This is a schematic diagram of an architecture suitable for graphics rendering services. Optionally, graphics rendering services can generally divide the rendering process into three main stages:

[0055] (1) The prepare phase is used to call the corresponding application programming interface (API) to prepare for graphics rendering. The prepare phase is usually triggered in response to user operations or application calls.

[0056] For example, the preparation work for graphics rendering includes, but is not limited to: checking permissions, loading resources, synchronizing the node tree, traversing and calculating dirty regions, and occlusion culling. The dirty region is used to indicate the area in the current frame that has changed compared to the previous adjacent frame.

[0057] (2) The process phase is used to execute graphics drawing operations and generate graphics drawing commands. For example, the process phase can convert elements on the user interface into bitmaps or other data formats that can be displayed by a computer. Furthermore, the process phase can also record graphics drawing operations as graphics drawing commands to be stored in the graphics context.

[0058] (3) The flush and submit phase. Among them,

[0059] (3-1) The flush phase is used to generate a sequence of drawing commands corresponding to different API standards based on the graphics drawing commands recorded in the process phase. API standards include, for example, OpenGL or Vulkan.

[0060] (3-2) The submit phase is used to submit the drawing command sequence to the GPU. In this way, graphics rendering operations can be performed on the surface of the graphics system to obtain the final image displayed on the screen.

[0061] Furthermore, the flush and submit phase also cleans up or synchronizes the resources related to the rendering of the previous frame, releases GPU memory, resets the state in the graphics context, and clears the GPU command cache related to the already drawn frames, so as to execute the next rendering cycle.

[0062] Figure 1a The diagram shows a single-threaded serial architecture, which can also be understood as using a single thread to execute the entire rendering process for each frame of graphics sequentially. For example... Figure 1a As shown, each frame of graphics needs to go through the prepare, process, and flush and submit stages in sequence. For example, in a serial architecture, the thread used for graphics rendering executes the entire rendering work of the current frame of graphics, including the prepare, process, and flush and submit stages, before executing the entire rendering work of the next frame of graphics, including the prepare, process, and flush and submit stages, and so on.

[0063] In some possible scenarios, when the screen frame rate of an electronic device is high, such as... Figure 1a The processing power of the serial architecture shown may no longer be sufficient to meet the requirements of high screen frame rates. For example, if 120 frames of graphics need to be refreshed per second, the rendering time of a single frame needs to be 8.33 milliseconds (ms) or less.

[0064] For example, Figure 1b A schematic diagram of another architecture suitable for graphics rendering services. (Compared to...) Figure 1a The architecture shown is the same; the graphics rendering service still divides the rendering process into three main stages: the prepare stage, the process stage, and the flush and submit stage.

[0065] and Figure 1a The architectures shown are different. Figure 1b The diagram illustrates a multi-threaded asynchronous pipeline architecture. It can be understood that, since the prepare, process, and flush / submit phases do not overlap, the graphics rendering service can further divide these three phases into two threads for asynchronous pipelined operations. For example, these two threads could include a main thread and a rendering thread.

[0066] (1) The main thread, which can be used to execute the prepare phase. And, as... Figure 1b As shown, while the process and flushand submit phases are processing the current frame, the prepare phase can already start running the next frame. This increases concurrency and shortens the rendering time for a single frame.

[0067] For example, the main thread can obtain pre-permutation parameters by executing the prepare phase. During synchronization, the pre-permutation parameters of the main thread are interchanged with the rendering parameters of the rendering thread, thereby achieving the purpose of cross-thread data synchronization or transmission.

[0068] (2) The rendering thread can be used to execute the process phase and the flush and submit phase. Corresponding to the main thread, the rendering thread can copy the pre-permutation parameters of the current frame obtained by the main thread as the rendering parameters of the process phase and the flush and submit phase, thereby enabling the execution of graphics rendering operations by executing the process phase and the flush and submit phase.

[0069] In this way, the rendering task can be split through an asynchronous pipeline architecture, enabling multi-core asynchronous pipelined rendering. This allows the rendering time of a single frame of graphics to meet the screen frame rate requirements, thereby improving performance. For example, when the rendering load is heavy, taking the example of needing to refresh 120 frames of graphics per second, by using... Figure 1b The asynchronous pipeline architecture shown can reduce the rendering time of a single frame to meet screen frame rate requirements of less than 8.33ms. For example, when the rendering load is light, the asynchronous pipeline architecture can split the rendering task, implementing multi-core asynchronous pipeline rendering, allowing the rendering task to run on multiple threads, which can improve load balancing. Additionally, as... Figure 1b The asynchronous pipeline architecture shown typically requires locks to ensure thread safety between the main thread and the rendering thread. However, multithreading can lead to frequent entry and exit from locks, impacting rendering efficiency.

[0070] Regardless of Figure 1a The serial architecture shown is still as Figure 1b The asynchronous pipelined architecture shown depicts the fact that the main load for graphics rendering services remains in the process and flush-and-submit phases. Experimental data reveals that the prepare phase of graphics rendering services accounts for only about 10% to 20% of the total load. Therefore, achieving high performance and low power consumption in the process and flush-and-submit phases remains a challenging problem to be solved in the industry.

[0071] In view of this, embodiments of this application provide a rendering method and an electronic device. In this method, the task to be rendered can be divided into multiple sub-tasks, which are then processed in parallel by multiple threads to obtain drawing commands corresponding to each sub-task. Then, multiple sets of drawing commands are submitted to the GPU in batches. For example, one task to be rendered can be used to render one frame of graphics.

[0072] In this way, parallel rendering can improve the efficiency of graphics rendering, thereby enhancing the performance of electronic devices. Therefore, the method provided in this application embodiment can distribute the load from large cores to various small and medium cores, improving performance while optimizing the energy consumption of graphics rendering. Furthermore, this method can also meet the rendering needs of more graphics rendering scenarios.

[0073] The technical solutions in this application can be applied to electronic devices, such as mobile phones, tablets, writing tablets, wearable devices (e.g., watches, bracelets), in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and smart home devices (e.g., smart TVs, smart speakers with displays), etc. It is understood that this application does not limit the specific type of electronic device.

[0074] The electronic devices to which this application embodiment can be applied, exemplary embodiments of which may include, but are not limited to, those equipped with... Or other electronic devices with different operating systems. The aforementioned electronic devices can also be other electronic devices, such as laptops with touch-sensitive surfaces (e.g., touch panels).

[0075] Figure 2 A schematic diagram of the hardware structure of a possible electronic device is shown. (For example...) Figure 2As shown, the electronic device 200 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0076] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the electronic device 200. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. Processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has recently used or is repeatedly used. When processor 110 needs to reuse an instruction or data, it can directly retrieve it from memory. This avoids repeated access, reduces the waiting time of processor 110, and thus improves system efficiency. For example, the controller included in processor 110 can be used to implement parallel rendering of multiple subtasks to be rendered, which are divided into multiple subtasks to be rendered by multiple threads. Furthermore, processor 110 can also batch submit multiple sets of rendering commands corresponding to the multiple subtasks to be rendered obtained by multiple threads to the GPU.

[0077] The display screen 194 is used to display the application's display interface, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 200 may include the display screen 194. This application does not limit the number of display screens 194; for example, it can be one or multiple. In the embodiments of this application, the display screen 194 can be used to display a graphic frame rendered based on the rendering method provided in the embodiments of this application.

[0078] Camera 193 is used to capture still images or videos. Camera 193 may include a front-facing camera and a rear-facing camera.

[0079] Internal memory 121 can be used to store computer executable program code, including instructions. Processor 110 executes various functional applications and data processing of electronic device 200 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and software code of at least one application. The data storage area may store data (e.g., images, videos, etc.) generated during the use of electronic device 200. In addition, internal memory 121 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. In this embodiment, internal memory 121 may, for example, store command buffers corresponding to multiple subtasks to be rendered. Internal memory 121 may also store resource buffers corresponding to multiple subtasks to be rendered. And internal memory 121 may also store an onscreen buffer; wherein the onscreen buffer is used to store a target image for display on the screen, drawn based on the drawing commands included in the multiple command buffers.

[0080] Touch sensor 180K and display screen 194 constitute a touchscreen, also known as a "touchscreen". Touch sensor 180K is used to detect touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be disposed on the surface of electronic device 200, in a different location than display screen 194. In the embodiments of this application, touch sensor 180K can collect user operations on the interface displayed by electronic device through display screen 194, for example, user operations used to trigger the display interface.

[0081] Understandable, Figure 2 The components shown do not constitute a specific limitation on the mobile phone. A mobile phone may include more or fewer components than shown, or combine some components, or split some components, or have different component arrangements. In the following embodiments, [the following text is incomplete and likely refers to a different embodiment]. Figure 2 The electronic device 200 shown is used as an example for introduction.

[0082] The operating system (OS) involved in this application embodiment is the most basic system software running on the electronic device 200. The software system of the electronic device 200 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes an operating system adopting a layered architecture as an example to illustrate the software system architecture of the electronic device 200.

[0083] Figure 3 This is a software system architecture block diagram of an electronic device provided as an embodiment of this application. For example... Figure 3 As shown, the software system architecture of an electronic device can be a layered architecture. For example, the software can be divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into five layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the system libraries and runtime, the kernel layer, and the hardware layer.

[0084] The application layer can include a series of application packages. For example... Figure 3As shown, the application layer can include native operating system applications and third-party applications. Native operating system applications can include settings, music, SMS, camera, and call functions. Third-party applications can include notes, etc. This application does not limit the specific implementation. In this embodiment, the applications included in the application layer can generally display a user interface. The method provided in this embodiment can be applied to any scenario where graphical rendering is required when displaying a user interface on an electronic device.

[0085] In some embodiments of this application, the application layer can be used to implement the presentation of the user interface. The user interface can be rendered using the methods provided in the embodiments of this application. For example, the user interface can be an interface displayed in scenarios with high screen frame rates, such as a game interface or a video interface. Alternatively, the user interface can be an interface displayed in scenarios with high-frequency user interaction, such as a desktop display interface, a multi-screen display interface, or a multi-window display interface.

[0086] In one possible implementation, the application can be developed using languages ​​such as Arkts, JavaScript, Objective-C, Swift, Kotlin, or Java, and completes its work by calling APIs provided by the application framework layer. Developers can use the application framework layer to interact with the underlying operating system (e.g., kernel layer, hardware layer) to develop their own applications. This application framework layer primarily consists of a series of services and management systems within the operating system. In this embodiment, the electronic device can detect and respond to events that trigger the display of the user interface within the application layer, calling the corresponding APIs provided by the application framework layer.

[0087] The application framework layer provides application programming interfaces and a programming framework for applications within the application layer. The application framework layer includes some predefined functions. For example... Figure 3 As shown, the application framework layer may include a graphics rendering service, such as a unified rendering service (RS); an activity manager; a window manager; a view system; a phone manager; a content provider; a notification manager; a resource manager, etc.

[0088] The Activity Manager manages the lifecycle of each application and provides commonly used navigation and back functions, offering an interactive interface for all program windows.

[0089] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0090] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0091] A view system includes visual components, such as components that display text, components that display images, etc. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon could include views that display text and views that display images.

[0092] In some embodiments of this application, the application framework layer is primarily responsible for calling the service interface for communication with the hardware abstraction layer to pass rendering requests from the graphics rendering service to the system library. In this way, by calling the system library, multiple subtasks to be rendered can be drawn in parallel using multithreading; these multiple subtasks are obtained by dividing the rendering task.

[0093] Furthermore, the application framework layer can receive parallel rendering progress from the system library. When it detects that each subtask to be rendered has been completed, it can batch submit multiple sets of rendering commands corresponding to multiple subtasks to the GPU. In this way, multiple sets of rendering commands can be rendered using the same screen buffer, eliminating the need for stitching and ensuring rendering accuracy.

[0094] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0095] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0096] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0097] The runtime includes the core libraries and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0098] The core library consists of two parts: one part contains callable functionalities, and the other part is the operating system's core library. The application layer and application framework layer run in a virtual machine. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0099] The system library can include multiple functional modules. For example: surface manager, media framework, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), etc.

[0100] The Surface Manager is used to manage the display subsystem and provides the blending of two-dimensional and three-dimensional layers for multiple applications.

[0101] The media framework supports playback and recording of various commonly used audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, and AMR; and multiple image file formats, such as JPG and PNG.

[0102] The 3D graphics processing library is used to implement 3D graphics drawing, graphics rendering, compositing, and layer processing.

[0103] A two-dimensional graphics engine is a graphics engine for two-dimensional drawing. In some embodiments of this application, the two-dimensional graphics engine may include M contexts. Each context may correspond to a command buffer, that is, M command buffers. Here, M is a positive integer greater than or equal to 2.

[0104] The kernel layer is the layer between hardware and software. The kernel layer contains many drivers related to electronic devices, including at least display drivers, camera drivers, audio drivers, and sensor drivers; alternatively, it may include keyboard drivers as input devices, Flash drivers for memory-based devices, camera drivers, Bluetooth drivers, Wi-Fi drivers, etc. This application embodiment does not impose any limitations on this. In this application embodiment, the kernel layer can receive drawing operations from the unified rendering service and display them via the display driver, i.e., display them on the display screen 194.

[0105] The hardware layer can include various types of sensors, such as accelerometers, gravity sensors, and touch sensors.

[0106] Electronic devices can typically run multiple applications simultaneously. In a simpler scenario, one application corresponds to one process; in a more complex scenario, one application corresponds to multiple processes. Each process has a unique process ID.

[0107] It should be understood that in the embodiments of this application, "at least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple. "Multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0108] In addition, it should be understood that in the description of this application, the words "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0109] It should be understood that the hardware structure of electronic devices can be as follows: Figure 2 As shown, the software system architecture can be as follows: Figure 3 As shown, the software programs and / or modules corresponding to the software system architecture in the electronic device can be stored in the internal memory 121, and the processor 110 can run the software programs and applications stored in the internal memory 121 to execute the flow of a rendering method provided in the embodiments of this application.

[0110] To facilitate understanding of the rendering method provided in this application, the following is combined with... Figures 4 to 9b The content shown describes the implementation process of the method provided in this application.

[0111] The method provided in this application can be applied to a variety of possible graphics rendering scenarios. For example, graphics rendering scenarios may include desktop display scenarios of electronic devices, game interface display scenarios, video interface display scenarios, multi-window display scenarios, etc. It should be understood that the application scenarios of the provided method are not limited in this application, and it can be used in any scenario requiring graphics rendering.

[0112] For example, Figure 4 This is a schematic diagram illustrating an application scenario for a rendering method provided in an embodiment of this application. For example... Figure 4 As shown in interface 401, this is a schematic diagram of the expanded interface of the large folders included in the desktop display. Figure 4 Interface 402 is shown as a schematic diagram of the node tree corresponding to interface 401. This node tree may include, but is not limited to:

[0113] (1) The root node is used to manage the entire rendering process and coordinate with each child node to achieve efficient rendering of the final image displayed on the screen.

[0114] For example, the root node can be a display render node. Render nodes can be used to set up the rendering environment, provide the rendering context, and complete the final rendering.

[0115] (2) Child nodes of the root node (wallpaper), used to draw desktop wallpaper.

[0116] (3) The root node's child nodes (desktop) are used to draw elements on the desktop. It can be understood that in applications where large folders are expanded, the node (desktop) can also act as a container control. For example, the node (desktop) can also include lower-level child nodes, namely, child nodes (lower-level desktop), child nodes (desktop blur), and child nodes (upper-level desktop).

[0117] (3-1) The child nodes (lower desktop) of the node (desktop) are used to draw the elements included in the lower desktop. The lower desktop can also be understood as the desktop interface displayed when the large folder is not expanded. For example, the node (lower desktop) can also act as a container control. For instance, the node (lower desktop) can also include child nodes at the next level, namely child nodes (dock bar) and child nodes (lower desktop icons).

[0118] The dock may include icons of commonly used applications, such as the camera icon, contacts icon, phone icon, and message icon shown in display interface 401. The application icons included in the dock can be set according to the user's needs. In this embodiment, the specific application icons included in the dock are not limited.

[0119] A node (lower-level desktop icon) can also serve as a container control. In the example of the New Year's Pavilion, the node (lower-level desktop icon) can also include multiple child nodes at the next lower level. These child nodes can be used to draw application icons and folders on the lower-level desktop, specifically for drawing the background color of application icons, the text within application icons, application icon names, folder names, etc. It is understood that the number of child nodes at the next lower level included in the node (lower-level desktop icon) is not limited in this embodiment and can be determined based on the user's desktop settings.

[0120] (3-2) The child node of the node (desktop) (desktop blur) can be used to draw the blur effect of the lower desktop.

[0121] (3-3) The child nodes (upper desktop) of the node (desktop) can be used to draw elements included in the upper desktop. The upper desktop can also be understood as the display interface of an expanded large folder. For example, the node (upper desktop) can also act as a container control. For instance, the node (upper desktop) can also include lower-level child nodes, namely child nodes (folder icons) and multiple child nodes (folder icons). Multiple child nodes (folder icons) can be used to draw application icons included in the folder; for example, they can be used to draw the background color of the application icon, the text in the application icon, the application icon name, etc.

[0122] (4) The child nodes of the root node (gesture operation bar) are used to draw the relevant elements corresponding to the gesture operation bar.

[0123] based on Figure 4 In the application scenario described, this embodiment of the application splits the node tree corresponding to the frame to be drawn into M node subtrees. It can be understood that each of the M node subtrees corresponds to one of the M subframes to be drawn.

[0124] For example, Figure 5 This is a schematic diagram illustrating a possible splitting result of the node tree provided in an embodiment of this application. Taking M as 3 as an example, the node tree is split as follows: Figure 4 The node tree shown in interface 402 is divided into three node subtrees. Each node subtree is represented by a context. The context is used to indicate the interface content included in the subframe to be drawn. For example, node subtree 1 corresponds to context 1, node subtree 2 corresponds to context 2, and node subtree 3 corresponds to context 3.

[0125] For example, the node tree can be split based on preset rules. It is understood that the preset rules are not limited in the embodiments of this application.

[0126] For example, the preset rule could be to split the node tree according to the number of nodes, resulting in multiple node subtrees. Optionally, each node subtree contains the same or similar number of nodes. Another option is that at least two of the three node subtrees have the same number of nodes, and the third node subtree has a similar number of nodes. In this way, load balancing of multiple threads can be achieved in some scenarios. The similar number of nodes in the node subtrees can be understood as a difference less than a preset difference, such as 10% of the average, or a preset value, such as 20, 30, or 50 nodes.

[0127] For example, the preset rules can also be used to split the node tree according to node type. This allows nodes with close relationships to be grouped into the same node subtree.

[0128] based on Figure 4 and Figure 5The method described in this application embodiment uses L threads to draw M node subtrees in parallel. The surfaces used by the L threads can share the same screen buffer. For example, the screen buffer can be the memory of the surface used by the main thread among the L threads. The main thread might be, for example, the following... Figure 6a Thread 1 in the context is used. Therefore, by batch submitting the drawing commands obtained by the L threads, which are used to process different contexts, to the GPU, the drawing content of different threads can be drawn directly on the screen together without stitching, thus achieving parallel rendering. Here, L is a positive integer greater than or equal to 2, which can also be understood as L threads including at least multiple threads.

[0129] Optionally, if L equals M, then each thread can process a subtree of nodes.

[0130] Alternatively, L can be less than M, in which case there are two or more node subtrees that are processed by a single thread.

[0131] In some possible implementations, the method provided in this application embodiment can be as described above. Figure 1a Based on the introduced serial architecture, parallel rendering is implemented using L threads, as shown below. Figures 6a-6c Let me introduce it.

[0132] In other possible implementations, the method provided in the embodiments of this application can also be as described above. Figure 1b Based on the asynchronous pipeline architecture introduced, parallel rendering is implemented using L threads, as shown below. Figures 7a-7b Let me introduce it.

[0133] For example, Figure 6a This is a schematic diagram of the architecture applicable to a rendering method provided in an embodiment of this application. Figure 6a Based on Figure 1a The illustration shows a parallel rendering implementation using a serial architecture. For example... Figure 6a As shown, the rendering process can include:

[0134] (1) The prepare phase is used to call the corresponding APIs, thereby preparing for graphics rendering. The prepare phase is typically triggered in response to user actions or application calls. For example, the preparation work for graphics rendering includes, but is not limited to: checking permissions, loading resources, synchronizing the node tree, traversing and calculating dirty regions, and occlusion culling. The dirty region indicates the area in the current frame that has changed compared to the previous adjacent frame.

[0135] In one optional implementation, the node tree can be split into M node subtrees during the prepare stage. Optionally, thread 1 can perform the prepare stage processing; then, the resulting M node subtrees are distributed to L threads. Thread 1 can also be understood as the main thread in parallel rendering. For example, ... Figure 3 The graphics rendering service in the application framework layer shown can send the obtained M node subtrees to the system library. This allows the 2D graphics engine in the system library to manage the M contexts representing the M node subtrees separately; and also allows the system library to call L threads from a thread pool to process the M contexts respectively.

[0136] In another optional implementation, the node tree can be split into M node subtrees before the graphics drawing operation is performed in the process phase. Optionally, thread 1 can split the node tree during the processing of the process phase, before the graphics drawing operation; then, the resulting M node subtrees can be distributed to L threads respectively. For example, as shown... Figure 3 The graphics rendering service in the application framework layer shown can generate a node tree splitting strategy and send it to thread 1 of the L threads included in the system library. Then, thread 1 can perform the node tree splitting process and distribute it to the other threads of the L threads included in the system library. Here, thread 1 can be understood as the main thread in parallel rendering, while the other threads can be understood as rendering threads in parallel rendering.

[0137] In this embodiment, the current frame can be rendered in parallel by L threads. For each thread, the following is executed:

[0138] (2) The process phase is used to perform graphics drawing operations and generate graphics drawing commands. For example, the process phase can convert elements on the user interface into bitmaps or other data formats that can be displayed by a computer, and record them as graphics drawing commands in the graphics context.

[0139] For example, the process stage can be achieved through, for instance, as shown in the example... Figure 3 The system library implementation shown uses one thread to process a subtree of nodes corresponding to a given context. The system library can arbitrarily schedule one thread from the thread pool to process each subtree of nodes corresponding to a given context. This can also be understood as a one-to-one correspondence between M subtrees of nodes corresponding to M contexts and L threads. Optionally, the process phase can be implemented, for example, through... Figure 3 The application does not limit which part of the implementation is included in the system library, including at least one of the two-dimensional graphics engine, resource manager, etc. shown.

[0140] It should be noted that the specific thread scheduling method is not limited in the embodiments of this application. Furthermore, the above examples describe the rendering process of the current frame; thread scheduling for different frames can employ different methods. For instance, parallel rendering of different frames can schedule different numbers of threads.

[0141] In this embodiment of the application, the flush and submit stage can be divided into two parts, including:

[0142] (3-1) The flush phase, also known as the flush-driver development kit (DDK), is used to generate a sequence of drawing commands corresponding to different graphics APIs based on the graphics drawing commands recorded in the process phase.

[0143] In some possible embodiments, such as Figure 3 The 2D graphics engine included in the system library shown can also create a corresponding command buffer for each context. The command buffer stores the drawing commands obtained by each thread after performing drawing operations based on the context. Alternatively, it can be understood that there is a one-to-one correspondence between M contexts and M command buffers.

[0144] (3-2) The submit phase is used by the graphics rendering service to monitor the generation of drawing commands in the M command buffers included in the refresh phase 1. When the graphics rendering service detects that the drawing commands in all M command buffers have been completed, it instructs the M contexts to be batch submitted to the GPU, so that the graphics rendering operation can be performed to obtain the final image displayed on the screen. It can be understood that by batch submission, the GPU can perform graphics rendering operations on the drawing commands of the M command buffers together in the same screen buffer, so that the final image displayed on the screen can be obtained without stitching, thus ensuring rendering accuracy.

[0145] pass Figure 6a The architecture shown allows the current frame to be divided into multiple subframes, enabling parallel rendering through multiple threads. Finally, after each thread completes its rendering commands for its corresponding subframe, the commands can be submitted to the GPU in batches, allowing multiple threads to use the same screen buffer and render together on the screen. This parallel rendering improves rendering concurrency and performance. Furthermore, batch submission allows multiple sets of rendering commands to share the same screen buffer, eliminating the need for stitching, improving rendering accuracy, and reducing CPU load. In addition, compared to multiple separate submissions, batch submission reduces processing time and improves the performance of electronic devices.

[0146] For example, Figure 6b This is a schematic diagram of another architecture applicable to a rendering method provided in an embodiment of this application. Figure 6a The difference is that the preparation stage can also be processed in parallel using multiple threads.

[0147] For example, before performing the preparatory work required in the prepare phase, thread 1 can split the node tree to obtain M node subtrees, and then distribute the M node subtrees to L threads for execution. Each thread can perform preparatory work such as traversing and calculating dirty regions in its corresponding node subtree. Then, the L threads can send the calculated dirty regions to thread 1 for dirty region merging and occlusion culling. Furthermore, after thread 1 completes dirty region merging and occlusion culling, it can distribute the split M node subtrees back to the L threads to execute the process, flush, and submit phases. Thread 1 can be understood as the main thread in parallel rendering, while the other threads can be understood as rendering threads in parallel rendering.

[0148] Understandable. Figure 6b For details on the implementation of the process, flush, and submit phases, please refer to [link / reference]. Figure 6a The content described in the previous section will not be repeated here.

[0149] Or for example, Figure 6c This is a schematic diagram of another architecture applicable to a rendering method provided in an embodiment of this application. This architecture may include a main thread and L threads.

[0150] The main thread can be used to check the workload of the tasks to be rendered based on the graphics rendering task requests. Optionally, when the workload of the tasks to be rendered is determined to be small, it can be processed through the serial architecture 601. For details on the serial architecture 601, please refer to [link to relevant documentation]. Figure 1a As shown. Alternatively, when the workload of the rendering task is determined to be large, it can be processed by threads 1 to L through a parallel architecture 602. The parallel architecture 602 can be as follows: Figure 6a The architecture shown may be... Figure 6b The architecture shown. In some possible scenarios, Figure 6c The main thread can also be thread 1 among threads 1 to L. In this case, thread 1 performs the inspection and processing, and when processing is performed through the parallel architecture 602, threads 1 to L perform parallel drawing.

[0151] For example, the workload of the task to be rendered can be determined based on the screen frame rate. For instance, when the screen frame rate is low, the workload is determined to be small; and when the screen frame rate is high, the workload is determined to be large.

[0152] It is understandable that the specific implementation process of the main thread and L threads can be found in the detailed introduction above, and will not be repeated here.

[0153] For example, Figure 7a This is a schematic diagram of an architecture applicable to a rendering method provided in an embodiment of this application. Figure 7a Based on Figure 1b The asynchronous pipeline architecture shown implements parallel rendering. Compared to Figure 1b , Figure 7a It can include L rendering threads, for example, rendering thread 1 to rendering thread L.

[0154] (1) The main thread, which can be used to execute the prepare phase. And, as... Figure 7a As shown, while the process and flush-submit phases are processing the current frame, the prepare phase can already run the next frame, thereby increasing concurrency and shortening the drawing time of a single frame.

[0155] For example, the main thread can obtain pre-permutation parameters by executing the prepare phase. During synchronization, the pre-permutation parameters of the main thread are interchanged with the rendering parameters of the rendering thread, thereby achieving the purpose of cross-thread data synchronization or transmission.

[0156] In one alternative implementation, the main thread can also perform node tree splitting during the prepare phase to obtain M node subtrees.

[0157] In another alternative implementation, the main thread may also generate a splitting strategy for the node tree and instruct the rendering thread via pre-permutation parameters.

[0158] (2) Rendering threads 1 to L can be used to execute the process phase, flush phase and submit phase through parallel drawing and batch submission.

[0159] For example, corresponding to the main thread, rendering thread 1 can copy the pre-permutation parameters of the current frame obtained by the main thread as rendering parameters for the process and flush-submit phases. Then, based on the node tree splitting result indicated by the rendering parameters, rendering thread 1 distributes M node subtrees to L rendering threads, thereby obtaining M sets of drawing commands through parallel drawing execution of the process and flush-DDK phases. In some possible embodiments, each rendering thread can also create a corresponding command buffer. The command buffer can be used to store the drawing commands obtained by each rendering thread after performing drawing operations according to the context. It can also be understood that there is a one-to-one correspondence between the M contexts and the M command buffers.

[0160] In another example, rendering thread 1 can also perform node tree splitting based on the splitting strategy indicated by the pre-permutation parameters. Then, the M node subtrees are distributed to L rendering threads, thereby obtaining M sets of drawing commands through parallel drawing execution of the process phase and flush-DDK.

[0161] Optionally, the L rendering threads can submit multiple drawing command sequences to the GPU in batches, enabling graphics rendering operations to produce the final image displayed on the screen. In essence, batch submission allows the GPU to perform graphics rendering operations on the same screen buffer containing drawing commands from M command buffers, eliminating the need for stitching and ensuring rendering accuracy.

[0162] pass Figure 7a The architecture shown not only shortens graphics rendering time through asynchronous pipelined processing but also enables parallel drawing through multiple threads by dividing the current frame into multiple subframes. Finally, after each thread completes its drawing commands for its corresponding subframe, the commands can be submitted to the GPU in batches. This combination of asynchronous pipelined architecture and parallel drawing improves rendering concurrency and performance. Furthermore, batch submission allows multiple sets of drawing commands to share the same screen buffer, eliminating the need for stitching, improving rendering accuracy, and reducing CPU load. In addition, compared to multiple separate submissions, batch submission reduces processing time and improves the performance of electronic devices.

[0163] For example, Figure 7b This is a schematic diagram of another architecture applicable to a rendering method provided in an embodiment of this application. This architecture may include a main thread, rendering thread 0, and rendering threads 1 to L.

[0164] The main thread and rendering thread 0 can be used to execute the prepare stage using an asynchronous pipelined processing method. For details on the implementation of the asynchronous pipelined processing method, please refer to [link to relevant documentation]. Figure 7a As shown.

[0165] and Figure 7a The difference lies in that rendering thread 0 can check the workload of the task to be rendered based on the rendering parameters. Optionally, when it is determined that the workload of the task to be rendered is small, it can be handled using the asynchronous pipeline architecture 701. For details on the asynchronous pipeline architecture 701, please refer to [link to relevant documentation]. Figure 1b As shown. Alternatively, when the workload of the task to be rendered is determined to be large, it can be processed by rendering threads 1 to L through parallel architecture 702. The parallel architecture 702 can be as follows: Figure 7a The architecture shown. In some possible scenarios, Figure 7b Rendering thread 0 can also be rendering thread 1. In this case, rendering thread 1 performs the inspection process, and when processing is performed through the parallel architecture 702, rendering threads 1 to L perform parallel drawing.

[0166] For example, the workload of the rendering task can be determined based on the screen frame rate. For instance, a low screen frame rate indicates a smaller workload, while a high screen frame rate indicates a larger workload.

[0167] It is understood that the specific implementation process of the main thread, rendering thread 0, and rendering threads 1 to L can be found in the detailed introduction above, and will not be repeated here.

[0168] In some possible embodiments, based on Figures 6a to 7b In the parallel rendering scenario shown, after each thread processes its corresponding context, additional overhead can be reduced through resource migration and release. For example, a recycling bucket can be allocated to each thread. Optionally, L threads can correspond to N recycling buckets, where N is less than or equal to L.

[0169] For example, Figure 8a This is a schematic diagram illustrating a scenario for resource migration and release applicable to a rendering method provided in this application embodiment. For example, N equals L, where L threads correspond one-to-one with N recycling buckets. Furthermore, from... Figure 8a This also shows that a single thread can be used to handle multiple contexts. Multiple contexts can also be understood as being used to process a group of subtasks to be rendered. For example, thread L is used to process contexts M-1 and M. In other words, contexts M-1 and M belong to the same group of subtasks to be rendered.

[0170] For example, Figure 8bThis is a schematic diagram illustrating a scenario for resource migration and release applicable to a rendering method provided in this application embodiment. In another example, N is less than L. In this case, multiple threads can be assigned the same recycling bucket, for example, recycling bucket 1 can be assigned to both thread 1 and thread 2.

[0171] In this embodiment of the application, resource migration and release can be achieved through the following steps:

[0172] Step A1: For each thread, execute the following: After submitting the resources required by the context to the GPU, the resource release instructions that require resources can be collected into the recycling bucket first instead of being executed.

[0173] Step A2: When the drawing of the current frame is detected to be finished, the resource release command collected in the recycling bucket can be executed at the specified time.

[0174] The specified time can be, for example, the end time of drawing the current frame, or a certain time after the end of drawing the current frame; this application does not limit this. For example, at a certain time after multiple sub-task groups to be rendered have been completed, the resource release command collected in the recycling bucket can be executed.

[0175] Optionally, if a recycling bucket corresponds to a thread, then that thread executes the resource release instructions collected in the recycling bucket.

[0176] Alternatively, when a recycle bin corresponds to multiple threads, the target thread among the multiple threads executes the resource release instructions collected in that recycle bin. For example, Figure 8b The recycling bin 1 shown corresponds to thread 1 and thread 2. Therefore, thread 1 of thread 1 and thread 2 can execute the resource release command collected in recycling bin 1. In this case, thread 1 is the target thread among thread 1 and thread 2. Figure 8b Thread 1 and Thread 2 in the code can also be understood as a thread group.

[0177] In some other possible embodiments, based on Figures 6a to 7b The parallel rendering scenario shown illustrates how different threads can copy virtual resources using tokens, enabling mapping of virtual resources to the same physical resources and thus achieving physical resource sharing. Within the operating system, virtual resources are allocated to programs for easier memory management. For example, when a program first accesses a virtual address corresponding to a virtual resource allocated to it, the operating system establishes a mapping between that virtual address and the physical address corresponding to the physical resource. This prevents the program from directly accessing the physical resource, thereby protecting the physical resources of the electronic device.

[0178] For example, Figure 9aThis is a schematic diagram illustrating a resource-sharing scenario applicable to a rendering method provided in this application embodiment. Taking the sharing of resources corresponding to context 1 by context 2 as an example, when physical resource 1 is needed during the processing of context 2, it is retrieved in the cache memory (cache) 2 corresponding to context 2. Optionally, if no virtual resource that can be mapped to physical resource 1 is found, the cache corresponding to other contexts can be queried.

[0179] For example, when context 2 retrieves a virtual resource 1 that can be mapped to physical resource 1 from cache 1 corresponding to context 1, it can copy virtual resource 1 using token 1 to generate virtual resource 2 and store it in cache 2. Token 1 is used to enable access to physical resource 1. It can be understood that since virtual resource 2 is obtained by copying virtual resource 1, virtual resource 2 can also be mapped to physical resource 1. For example, virtual resource 2 and virtual resource 1 share the same virtual address.

[0180] It is understandable that when no virtual resource that can be mapped to physical resource 1 is detected from the cache corresponding to other contexts, a virtual resource that can be mapped to physical resource 1 can be requested for context 2.

[0181] in addition, Figure 9b This is a schematic diagram illustrating another scenario of resource sharing applicable to a rendering method provided in an embodiment of this application. Based on Figure 9a For example, when rendering context 1 is complete, virtual resource 1 stored in cache 1 corresponding to context 1 can be released to the recycle bin. When rendering context 2 is complete, virtual resource 2 stored in cache 2 corresponding to context 2 can be released to the recycle bin. Finally, when the recycle bin is detected to contain both virtual resource 1 and virtual resource 2, the physical resource 1 mapped to virtual resource 1 and virtual resource 2 can be released.

[0182] In this way, during parallel rendering, when processing multiple contexts, if one context has already requested a physical resource, and another context also requests the same physical resource, the virtual resource mapped to that physical resource can be copied using tokens. This reduces the problem of duplicate resource uploads caused by multiple contexts requesting the same physical resource, thus reducing the load on electronic devices and improving their performance. Furthermore, by detecting the reclamation of virtual resources, the accuracy of physical resource management can be ensured, allowing physical resources to be released only after multiple contexts have finished using them.

[0183] Based on the above Figures 4 to 9bThe method described in this application, by employing a parallel architecture, can shorten the time consumed per frame and improve the performance and efficiency of graphics rendering. Thus, the method provided in this application can meet the needs of more graphics rendering scenarios with high screen frame rate requirements.

[0184] Based on the same inventive concept Figure 10 This is a flowchart illustrating a rendering method provided in an embodiment of this application. The method can be applied to electronic devices, and the process may include the following steps:

[0185] Step 1001: Divide the task to be rendered into M subtasks to be rendered, where M is a positive integer greater than or equal to 2.

[0186] For example, the task to be rendered can be understood as the current frame, and the task to be rendered can correspond to a node tree, such as... Figure 4 As shown, the M subtasks to be rendered can each correspond to a node subtree, such as... Figure 5 The node subtrees 1, 2, and 3 are shown.

[0187] Step 1002: Using L threads, generate M sets of drawing commands corresponding to M subtasks to be rendered, where L is less than or equal to M and L is a positive integer greater than or equal to 2; wherein the M sets of drawing commands are stored in M ​​command buffers respectively.

[0188] Step 1003: From the M command buffers, submit the M sets of drawing commands stored therein to the GPU in batches. The GPU will then use these M sets of drawing commands to perform rendering.

[0189] The specific implementation of steps 1001 to 1003 can be referred to the description in the foregoing embodiments, and will not be repeated here.

[0190] Figure 11 This is a flowchart illustrating a rendering method provided in an embodiment of this application. The method can be applied to electronic devices, and the process can be implemented through a graphics rendering service included in the electronic device, and may include the following steps:

[0191] Step 1101: The graphics rendering service divides the tasks to be rendered into a first sub-task group, a second sub-task group, and a third sub-task group. Each sub-task group may include one or more sub-tasks to be rendered, for example... Figure 8a As shown, context M-1 and context M can belong to the same subtask group to be rendered.

[0192] Step 1102: The first thread of the graphics rendering service generates the first set of drawing commands corresponding to the first sub-task group to be rendered.

[0193] Step 1103: The second thread of the graphics rendering service generates the second set of drawing commands corresponding to the second sub-task group to be rendered.

[0194] Step 1104: The graphics rendering service calls the third thread to generate the third set of drawing commands corresponding to the third sub-task group to be rendered.

[0195] Step 1105: The graphics rendering service submits the first set of drawing commands, the second set of drawing commands, and the third set of drawing commands to the GPU. The GPU is used to perform rendering based on the first set of drawing commands, the second set of drawing commands, and the third set of drawing commands.

[0196] The specific implementation of steps 1101 to 1105 can be referred to the description in the foregoing embodiments, and will not be repeated here.

[0197] Based on the above embodiments, this application also provides an electronic device, which includes multiple functional modules. These multiple functional modules interact to implement the functions performed by the electronic device in the methods described in the embodiments of this application. These multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and can be arbitrarily combined or divided based on specific implementations. For example, [the following is an example of implementation details]. Figure 10 Steps 1001 to 1003 are performed by the electronic device in the illustrated embodiment. For example, steps 1001 to 1003 are performed... Figure 11 Steps 1101 to 1105 are performed by the electronic device in the illustrated embodiment.

[0198] Based on the above embodiments, this application also provides an electronic device, which includes at least one processor and at least one memory, wherein the at least one memory stores computer program instructions. When the electronic device is running, the at least one processor executes the functions performed by the electronic device in the methods described in the embodiments of this application. For example, when executing... Figure 10 Steps 1001 to 1003 are performed by the electronic device in the illustrated embodiment. For example, steps 1001 to 1003 are performed... Figure 11 Steps 1101 to 1105 are performed by the electronic device in the illustrated embodiment.

[0199] Based on the above embodiments, this application also provides a computer program product, which includes a computer program (also referred to as code or instructions) that, when run, causes the methods described in the embodiments of this application to be executed.

[0200] Based on the above embodiments, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program (also referred to as code or instructions), which, when executed by a computer, causes the computer to perform the methods described in the embodiments of this application.

[0201] Based on the above embodiments, this application also provides a chip. The chip is used to read a computer program stored in a memory and implement the methods described in the embodiments of this application.

[0202] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the methods described in the embodiments of this application. In one possible embodiment, the chip system further includes a memory for storing programs and data necessary for the computer device. The chip system may be composed of chips or may include chips and other discrete devices. Those skilled in the art will understand that the embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0203] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0204] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0205] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0206] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A rendering method applied to electronic devices, characterized in that, include: The task to be rendered is divided into M subtasks to be rendered, where M is a positive integer greater than or equal to 2; Using L threads, M sets of drawing commands are generated, each corresponding to one of the M subtasks to be rendered. L is less than or equal to M, and L is a positive integer greater than or equal to 2. The M sets of drawing commands are stored in M ​​command buffers. The M sets of drawing commands stored in the M command buffers are submitted in batches to the graphics processing unit (GPU), which then performs rendering based on the M sets of drawing commands.

2. The method according to claim 1, characterized in that, The method further includes: Allocate N recycling buckets to the L threads, where N is less than or equal to L; Each recycling bucket is used to cache resource release instructions from at least one thread.

3. The method according to claim 2, characterized in that, The method further includes: In response to the first set of drawing commands of the first subtask to be rendered, the first thread releases the resource release command of the first resource to the first recycling bucket; Wherein, the first thread is any one of the L threads, the first resource is the resource used by the first thread, and the first recycling bucket is the recycling bucket allocated to the first thread.

4. The method according to claim 2 or 3, characterized in that, The method further includes: The resource release instructions included in the second recycling bucket are executed through the second thread; The second thread is the target thread among at least one thread corresponding to the second recycling bucket.

5. The method according to any one of claims 2 to 4, characterized in that, The method further includes: When it is detected that the GPU has completed rendering the M sub-tasks to be rendered, the resource release instructions included in the N recycling buckets are executed at a specified time; wherein, the specified time is any time after the rendering of the M sub-tasks to be rendered is completed.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: For the third thread, execute: In response to a request instruction for a first virtual resource used by a fourth thread, a second virtual resource is generated using a first token; Wherein, the third thread is any one of the L threads, and the fourth thread is another thread among the L threads that is different from the third thread; the second virtual resource is the same physical resource mapped to the first virtual resource, and the first token is used to manage the virtual resource mapped to the physical resource.

7. The method according to any one of claims 1 to 6, characterized in that, The process of dividing the task to be rendered into M sub-tasks to be rendered includes: The node tree corresponding to the frame to be drawn is divided into M node subtrees; wherein, the rendering task is obtained based on the node tree corresponding to the frame to be drawn, and the M rendering sub-tasks are obtained based on the M node subtrees.

8. The method according to claim 7, characterized in that, The number of nodes in the first node subtree is the same as or the number of nodes in the second node subtree is less than a preset difference.

9. A rendering method applied to electronic devices, characterized in that, include: The graphics rendering service divides the tasks to be rendered into a first sub-task group, a second sub-task group, and a third sub-task group. The first thread of the graphics rendering service generates a first set of drawing commands corresponding to the first sub-task group to be rendered. The second thread of the graphics rendering service generates a second set of drawing commands corresponding to the second sub-task group to be rendered; The third thread of the graphics rendering service generates a third set of drawing commands corresponding to the third sub-task group to be rendered. The graphics rendering service submits the first set of drawing commands, the second set of drawing commands, and the third set of drawing commands to the graphics processor (GPU), which performs rendering based on the first set of drawing commands, the second set of drawing commands, and the third set of drawing commands.

10. The method according to claim 9, characterized in that, The first group of subtasks to be rendered includes at least two subtasks to be rendered.

11. The method according to claim 9 or 10, characterized in that, The method further includes: Allocate a first recycling bin to the first thread and the second thread, and allocate a second recycling bin to the third thread; or, Allocate a first recycling bin to the first thread, the second thread, and the third thread; The first or second recycling bucket is used to cache resource release instructions for the corresponding thread.

12. The method according to claim 9 or 10, characterized in that, The method further includes: Allocate a first recycling bucket to the first thread and the second thread; In response to the first set of drawing commands, the resource release instructions of the first thread for the first resource are collected through the first recycling bucket, where the first resource is the resource used by the first thread; In response to the second set of drawing commands, the resource release instructions of the second thread for the second resource are collected through the first recycling bucket, whereby the second resource is the resource used by the second thread.

13. The method according to claim 12, characterized in that, The method further includes: The resource release instructions for the first resource and the second resource included in the first recycling bin are executed by the first thread. The first thread is the main thread of the graphics rendering service.

14. The method according to any one of claims 9 to 13, characterized in that, The method further includes: In response to a request instruction from a second thread for a first virtual resource used by a first thread, a second virtual resource is generated using a first token; The second virtual resource is the same physical resource mapped to the first virtual resource, and the first token is used to manage the virtual resource mapped to the physical resource.

15. The method according to any one of claims 9 to 14, characterized in that, The graphics rendering service divides the tasks to be rendered into a first sub-task group, a second sub-task group, and a third sub-task group, including: Divide the node tree corresponding to the frame to be drawn into the first node subtree, the second node subtree, and the third node subtree; The task to be rendered is obtained based on the node tree corresponding to the frame to be drawn, the first sub-task group to be rendered is obtained based on the first node subtree, the second sub-task group to be rendered is obtained based on the second node subtree, and the second sub-task group to be rendered is obtained based on the second node subtree.

16. The method according to claim 15, characterized in that, Any two subtrees in the first, second, and third node subtrees contain the same number of nodes or have a difference of less than a preset value.

17. An electronic device, characterized in that, The method includes at least one processor coupled to at least one memory, the at least one processor being configured to read and execute a computer program stored in the at least one memory to perform the method as claimed in any one of claims 1 to 8, or to perform the method as claimed in any one of claims 9 to 16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 8, or cause the computer to perform the method as described in any one of claims 9 to 16.

19. A computer program product comprising instructions, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 8, or causes the computer to perform the method as described in any one of claims 9 to 16.

Images