Rendering method and electronic equipment

Abstract

The invention provides a rendering method and electronic equipment. The to-be-rendered content between the application process and the rendering process can be dynamically balanced, so that the rendering efficiency is improved, and the equipment performance is improved. The method comprises the steps that a first application process generates a first rendering node tree; wherein the first rendering node tree is used for drawing a first application interface of the first application process, and the first rendering node tree comprises a rendering node corresponding to first content in the first application interface and a rendering node corresponding to second content in the first application interface. And the first application process generates a first texture corresponding to the first content according to the first rendering node tree. And the rendering process draws the second content according to the first rendering node tree generated by the first application process. And the rendering process generates a first application interface of the first application process according to the second content and the first texture.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a rendering method and an electronic device. Background Technology

[0002] With the development of terminal technology, the content displayed on the interfaces of electronic devices is becoming increasingly rich. For example, multiple application windows may be displayed in the same frame, each with its own content. Alternatively, a single window may be displayed, but it may contain various styles of content. However, as the content displayed on the interface becomes richer, the pressure on interface rendering also increases.

[0003] Currently, rendering methods include separate rendering and unified rendering. In separate rendering, each application renders its own window content and sends the resulting layers to a surface compositor. The surface compositor then composites these multiple layers to generate the final display interface. Because each application executes its own rendering process, implementing interactive animations between applications is challenging. In unified rendering, each electronic device sends drawing instructions for its window content to a rendering service. The rendering service, based on the received drawing instructions from multiple applications, generates a unified rendering node tree and executes the rendering process to create the final display interface. While unified rendering reduces the difficulty of implementing interactive animations between applications through a globally unified rendering node tree, the rendering load is concentrated in the rendering service, resulting in lower rendering efficiency. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a rendering method and an electronic device. The technical solution provided by this application can dynamically balance the content to be rendered between the application process and the rendering process, thereby improving rendering efficiency and enhancing device performance.

[0005] To achieve the above-mentioned technical objectives, this application provides the following technical solution:

[0006] Firstly, a rendering method is provided, applied to an electronic device. The method includes: a first application process generating a first rendering node tree; wherein the first rendering node tree is used to draw a first application interface of the first application process, and the first rendering node tree includes rendering nodes corresponding to first content in the first application interface, and rendering nodes corresponding to second content in the first application interface. The first application process generates a first texture corresponding to the first content based on the first rendering node tree. The rendering process then draws based on the first texture and the first rendering node tree generated by the first application process to generate the first application interface of the first application process.

[0007] For example, the first application process draws a portion of the rendering nodes to generate the first texture. During the rendering process based on the first rendering node tree, rendering nodes that involve the first texture can directly obtain the first texture; only rendering nodes that do not involve the first texture need to be drawn, such as when the rendering process draws the second content. In this way, the first application process shares some of the rendering load with the rendering process.

[0008] In this way, the first content and the second content in the first application interface are drawn by the first application process and the rendering process respectively, which reduces the problem of concentrated rendering load in a single frame and reduces the processing time of a single frame.

[0009] According to the first aspect, the first content includes static content in the first application interface, and the second content includes dynamic content in the first application interface; or, the first content includes a portion of static content in the first application interface, and the second content includes dynamic content in the first application interface and another portion of static content.

[0010] In this way, static and dynamic content are drawn separately by the application process and the rendering process, reducing the load on the rendering process. Furthermore, during the rendering process, the process avoids waiting for static content to be drawn before drawing dynamic content, thus improving rendering efficiency.

[0011] Furthermore, the rendering process is responsible for drawing dynamic content, reducing the difficulty of creating interactive animations between application processes.

[0012] According to the first aspect, or any implementation of the first aspect above, the first user interface of the electronic device includes a first application interface. It should be understood that the rendering process can synchronously generate the first application interface during the generation of the first user interface. The method further includes: displaying the first user interface. The first application process can generate a second rendering node tree; wherein, the second rendering node tree is used to draw the second application interface of the first application process, and the second rendering node tree includes rendering nodes corresponding to the first content in the second application interface, and rendering nodes corresponding to the third content in the second application interface. The rendering process draws according to the first texture and the second rendering node tree generated by the first application process to generate the second application interface of the first application process. The second user interface is then displayed; wherein, the second user interface includes the second application interface.

[0013] In this way, during the multi-frame interface switching display process, the rendering process can directly reuse the cached textures, thereby reducing the rendering pressure on the rendering process and improving rendering efficiency.

[0014] According to the first aspect, or any implementation of the first aspect above, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process; before the rendering process draws according to the first texture and the first rendering node tree generated by the first application process to generate the first application interface of the first application process, the method further includes: the second application process generating a third rendering node tree; wherein, the third rendering node tree is used to draw the third application interface of the second application process, and the third rendering node tree includes rendering nodes corresponding to the fourth content in the third application interface, and rendering nodes corresponding to the fifth content in the third application interface. The second application process generates a second texture corresponding to the fourth content according to the third rendering node tree. The rendering process draws according to the first texture and the first rendering node tree generated by the first application process to generate the first application interface of the first application process, including: the rendering process generating a target rendering node tree according to the first rendering node tree and the third rendering node tree. The rendering process draws according to the first texture and / or the second texture, and the target rendering node tree to generate the first user interface.

[0015] In this way, the rendering process can combine the rendering node tree of the interfaces of multiple application processes to be displayed, and complete the drawing of the user interface, thereby balancing the rendering pressure between multiple application processes and the rendering process. It should be understood that the rendering process can simultaneously generate the first application interface and the third application interface while generating the first user interface.

[0016] Furthermore, multiple application processes draw some content in parallel, while the rendering process draws another part of the content in a unified manner, reducing the difficulty of implementing cross-application animation effects and achieving refined cascading visual effects.

[0017] According to the first aspect, or any implementation of the first aspect above, the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree, including: when the fourth content is completely covered by the first application interface, removing the drawing operation of the rendering node corresponding to the fourth content in the target rendering node tree; and / or, when the fifth content is completely covered by the first application interface, removing the drawing operation of the rendering node corresponding to the fifth content in the target rendering node tree.

[0018] In this way, drawing is reduced without affecting the display, thereby improving drawing efficiency.

[0019] According to the first aspect, or any implementation of the first aspect above, the first texture includes node textures and / or subtree-level textures, where a node texture corresponds to a rendering node and a subtree-level texture corresponds to a rendering subtree.

[0020] In this way, electronic devices can request a texture from the node subtrees, which can effectively reduce the overhead of texture request and management.

[0021] According to the first aspect, or any implementation of the first aspect above, the parent node of the rendering subtree is a rendering node with animation / visual effects attributes, and the number of rendering nodes in the rendering subtree is greater than or equal to a number threshold. And / or, the root node of the rendering subtree includes a first identifier, which indicates that the rendering subtree is used to generate textures. And / or, the layer containing the rendering subtree includes a second identifier, which indicates that the rendering nodes in the layer are used to generate textures.

[0022] In this way, the application process can effectively reduce the texture allocation overhead by dividing the optimal subtree and requesting a texture corresponding to the rendering subtree, compared to requesting a texture for every node in the rendering subtree.

[0023] According to the first aspect, or any implementation of the first aspect above, after the first application process generates the first texture corresponding to the first content based on the first rendering node tree, the method further includes: the first application process writing the first texture into the first memory space. The rendering process reads the first texture from the first memory space.

[0024] In this way, the application process draws part of the content and generates corresponding textures for caching. Subsequent rendering processes can directly reuse the cached textures during the drawing process, thereby reducing the rendering pressure on the rendering process and improving rendering efficiency.

[0025] According to the first aspect, or any implementation of the first aspect above, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into the first memory space, including: the first application process requests a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0026] In this way, by combining multiple textures to generate a large texture, the number of texture requests and writes is reduced to one, effectively improving the efficiency of texture request and management.

[0027] According to the first aspect, or any implementation of the first aspect above, the first node texture carries the first offset data of the first node texture in the first texture cache, and the second node texture carries the second offset data of the second node texture in the first texture cache.

[0028] In this way, after the rendering process reads the texture from the first memory space, it can determine that the texture includes multiple individual textures, and can determine the cache location of each of the included textures through the offset, thus successfully obtaining the required texture.

[0029] According to the first aspect, or any implementation of the first aspect above, before the first application process generates the first rendering node tree, the method further includes: the rendering process determining to issue a rendering task to the first application process based on one or more of the following: the rendering process's resource utilization rate, the rendering process's historical load, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the load of the central processing unit (CPU), the load of the graphics processing unit (GPU), the hardware performance of the electronic device, and the battery level of the electronic device; the rendering task instructs the first application process to draw the first content.

[0030] For example, based on the high resource utilization of the rendering process, the rendering process determines to allocate the rendering tasks corresponding to a portion of the static content in the first application interface to the first application process. In this way, the rendering process only needs to be responsible for drawing the dynamic content and another portion of the static content in the first application interface, thereby reducing the rendering load of the rendering process.

[0031] For example, if the rendering process has a high resource utilization rate and a high GPU load, it determines to allocate the rendering tasks corresponding to the static content in the first application interface to the first application process. In this way, the rendering process only needs to be responsible for drawing the dynamic content in the first application interface, and does not need to draw the static content in the first application interface, thereby effectively reducing the rendering load of the rendering process.

[0032] In this way, the rendering process can flexibly allocate rendering tasks according to the actual display situation, thereby improving rendering efficiency and balancing the rendering pressure on the application process and the rendering process.

[0033] According to the first aspect, or any of the above implementations of the first aspect, the first content or the second content is one or more of the windows, controls, and properties in the first application interface.

[0034] For example, the content to be drawn can be divided into windows, controls, and properties within controls according to different granularities. For instance, the division between static and dynamic content can be based on the granularity of windows, controls, or properties within controls.

[0035] In this way, electronic devices can flexibly allocate rendering tasks according to different granularities based on the actual display of the content on the interface to be displayed, thereby achieving fine-grained management of rendering tasks and effectively improving rendering efficiency.

[0036] According to the first aspect, or any implementation of the first aspect above, after the first application process generates the first rendering node tree, the method further includes: the first application process writing the first rendering node tree into a second memory space. The rendering process reads the first rendering node tree from the second memory space.

[0037] In this way, the rendering process can obtain the first rendering node tree generated by the first application process, thereby realizing the drawing of the second content.

[0038] Secondly, this application provides a rendering method, which includes: a first application process drawing a first part of a rendering task to generate a first texture; the first texture includes static content in a first application interface; wherein the first texture or at least a portion of the static attributes corresponding to the first texture are cached. The rendering process divides the first rendering node tree into a first part of the rendering task and a second part of the rendering task based on a first rendering node tree of the first application interface of the first application process to be displayed; the second part of the rendering task corresponds to dynamic content in the first application interface. The rendering process draws based on the cached first texture or the at least a portion of the static attributes, and the second part of the rendering task to generate the first application interface.

[0039] In this way, the static and dynamic content in the first application interface to be displayed are rendered separately, reducing the problem of concentrated rendering load within a single frame and reducing the processing time per frame.

[0040] According to the second aspect, the method further includes: the rendering process draws based on the second rendering node tree of the second application interface of the first application process to be displayed, the first texture or the at least part of the static attributes, and the third part of the rendering tasks to generate the second application interface; wherein the second rendering node tree includes the first part of the rendering tasks and the third part of the rendering tasks.

[0041] According to the second aspect, or any implementation of the second aspect above, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process. Before the rendering process draws based on the cached first texture or at least a portion of static attributes and the second part of the rendering task to generate the first application interface, the method further includes: the rendering process assigning a fourth part of the rendering task to the second application process for drawing based on the third rendering node tree of the third application interface of the second application process to be displayed; the third rendering node tree includes the fourth part of the rendering task and the fifth part of the rendering task. The second application process draws the fourth part of the rendering task to generate a third texture; the third texture includes static content in the third application interface; wherein at least a portion of the static attributes in the third texture are cached. The rendering process draws based on the cached first texture or at least a portion of static attributes and the second part of the rendering task to generate the first application interface, including: the rendering process generating a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the third texture and the target rendering node tree to generate the first user interface.

[0042] According to the second aspect, or any implementation of the second aspect above, the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree, including: when the content corresponding to the fourth part of the rendering task is completely covered by the first application interface, removing the drawing operation of the fourth part of the rendering task from the target rendering node tree. And / or, when the content corresponding to the fifth part of the rendering task is completely covered by the first application interface, removing the drawing operation of the fifth part of the rendering task from the target rendering node tree.

[0043] According to the second aspect, or any implementation of the second aspect above, the first texture includes node textures and / or subtree-level textures, where a node texture corresponds to a rendering node and a subtree-level texture corresponds to a rendering subtree.

[0044] According to the second aspect, or any implementation of the second aspect above, the parent node of the rendered subtree is a rendered node with animation / visual effects attributes, and the number of rendered nodes in the rendered subtree is greater than or equal to a number threshold. And / or, the root node of the rendered subtree includes a first identifier, which indicates that the rendered subtree is used to generate textures. And / or, the layer containing the rendered subtree includes a second identifier, which indicates that the rendered nodes in the layer are used to generate textures.

[0045] According to the second aspect, or any implementation of the second aspect above, after the first application process draws the first part of the rendering task and generates the first texture, the method further includes: the first application process writing the first texture or the at least part of the static attributes into a first memory space. The rendering process reads the first texture or the at least part of the static attributes from the first memory space.

[0046] According to the second aspect, or any implementation of the second aspect above, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into the first memory space, including: the first application process requests a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0047] According to the second aspect, or any implementation of the second aspect above, the first node texture carries the first offset data of the first node texture in the first texture cache, and the second node texture carries the second offset data of the second node texture in the first texture cache.

[0048] According to the second aspect, or any implementation of the second aspect above, before the first application process draws the first part of the rendering task and generates the first texture, the method further includes: the rendering process allocating the first part of the rendering task to the first application process for drawing based on one or more of the following: the resource occupancy rate of the rendering process, the historical load of the rendering process, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the load of the central processing unit (CPU), the load of the graphics processing unit (GPU), the hardware performance of the electronic device, the battery power of the electronic device, and the first rendering node tree.

[0049] According to the second aspect, or any implementation of the second aspect above, the first part of the rendering task or the second part of the rendering task corresponds to one or more of the windows, controls, and properties in the first application interface.

[0050] Furthermore, the technical effects of the second rendering method can be referenced from the technical effects of the first rendering method, and will not be elaborated here.

[0051] Thirdly, this application provides a rendering method, which includes: a first application process drawing a first part of a rendering task to generate a first texture; the first texture includes first static content in a first application interface; wherein the first texture or at least a portion of the static attributes corresponding to the first texture are cached. The rendering process divides the first rendering node tree into a first part of the rendering task and a second part of the rendering task based on the first rendering node tree of the first application interface of the first application process to be displayed; the second part of the rendering task corresponds to second static content and dynamic content in the first application interface. The rendering process generates the first application interface based on the cached first texture or the at least a portion of the static attributes, and the second part of the rendering task.

[0052] In this way, the rendering process allocates a portion of the static content in the first application interface to be displayed to the first application process for drawing, and allocates another portion of the static and dynamic content to the rendering process for drawing, thereby reducing the problem of concentrated rendering load within a single frame and reducing the processing time per frame.

[0053] According to the third aspect, or any implementation of the third aspect above, the method further includes: the rendering process draws according to the second rendering node tree of the second application interface of the first application process to be displayed, the first texture or the at least part of the static attributes, and the third part of the rendering tasks to generate the second application interface; wherein, the second rendering node tree includes the first part of the rendering tasks and the third part of the rendering tasks.

[0054] According to the third aspect, or any implementation of the third aspect above, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process. Before the rendering process draws based on the cached first texture or at least a portion of static attributes and the second part of the rendering task to generate the first application interface, the method further includes: the rendering process assigning a fourth part of the rendering task to the second application process for drawing based on the third rendering node tree of the third application interface of the second application process to be displayed; the third rendering node tree includes the fourth part of the rendering task and the fifth part of the rendering task. The second application process draws the fourth part of the rendering task to generate a third texture; the third texture includes the third static content in the third application interface; wherein at least a portion of the static attributes in the third texture are cached. The rendering process draws based on the cached first texture or at least a portion of static attributes and the second part of the rendering task to generate the first application interface, including: the rendering process generating a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the third texture and the target rendering node tree to generate the first user interface.

[0055] According to the third aspect, or any implementation of the third aspect above, the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree, including: when the content corresponding to the fourth part of the rendering task is completely covered by the first application interface, removing the drawing operations of the fourth part of the rendering task from the target rendering node tree. And / or, when the content corresponding to the fifth part of the rendering task is completely covered by the first application interface, removing the drawing operations of the fifth part of the rendering task from the target rendering node tree.

[0056] According to the third aspect, or any implementation of the third aspect above, the first texture includes node textures and / or subtree-level textures, where a node texture corresponds to a rendering node and a subtree-level texture corresponds to a rendering subtree.

[0057] According to the third aspect, or any implementation of the third aspect above, the parent node of the rendered subtree is a rendered node with animation / visual effects attributes, and the number of rendered nodes in the rendered subtree is greater than or equal to a number threshold. And / or, the root node of the rendered subtree includes a first identifier, which indicates that the rendered subtree is used to generate textures. And / or, the layer containing the rendered subtree includes a second identifier, which indicates that the rendered nodes in the layer are used to generate textures.

[0058] According to the third aspect, or any implementation of the third aspect above, after the first application process draws the first part of the rendering task and generates the first texture, the method further includes: the first application process writing the first texture or the at least part of the static attributes into the first memory space. The rendering process reads the first texture or the at least part of the static attributes from the first memory space.

[0059] According to the third aspect, or any implementation of the third aspect above, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into the first memory space, including: the first application process requests a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0060] According to the third aspect, or any implementation of the third aspect above, the first node texture carries the first offset data of the first node texture in the first texture cache, and the second node texture carries the second offset data of the second node texture in the first texture cache.

[0061] According to the third aspect, or any implementation of the third aspect above, before the first application process draws the first part of the rendering task and generates the first texture, the method further includes: the rendering process allocating the first part of the rendering task to the first application process for drawing based on one or more of the following: the resource utilization rate of the rendering process, the historical load of the rendering process, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the load of the central processing unit (CPU), the load of the graphics processing unit (GPU), the hardware performance of the electronic device, the battery power of the electronic device, and the first rendering node tree.

[0062] According to the third aspect, or any of the above implementations of the third aspect, the first part of the rendering task or the second part of the rendering task corresponds to one or more of the windows, controls, and properties in the first application interface.

[0063] Furthermore, the technical effects of the third rendering method can be referenced from the technical effects of the first rendering method, and will not be elaborated here.

[0064] Fourthly, an electronic device is provided. The electronic device includes: a processor and a display screen; a memory coupled to the processor; the memory stores computer program code, including computer instructions; when the processor reads the computer instructions from the memory, the electronic device executes: a first application process generates a first rendering node tree; wherein the first rendering node tree is used to draw a first application interface of the first application process, and the first rendering node tree includes rendering nodes corresponding to first content in the first application interface, and rendering nodes corresponding to second content in the first application interface. The first application process generates a first texture corresponding to the first content based on the first rendering node tree. The rendering process draws based on the first texture and the first rendering node tree generated by the first application process, generating the first application interface of the first application process.

[0065] According to the fourth aspect, the first content includes static content in the first application interface, and the second content includes dynamic content in the first application interface; or, the first content includes a portion of static content in the first application interface, and the second content includes dynamic content in the first application interface and another portion of static content.

[0066] According to the fourth aspect, or any implementation of the fourth aspect above, the first user interface of the electronic device includes a first application interface. When the processor reads computer instructions from memory, it also causes the electronic device to execute: displaying the first user interface; the first application process generates a second rendering node tree; wherein the second rendering node tree is used to draw the second application interface of the first application process, and the second rendering node tree includes rendering nodes corresponding to the first content in the second application interface, and rendering nodes corresponding to the third content in the second application interface. The rendering process draws according to the first texture and the second rendering node tree generated by the first application process to generate the second application interface of the first application process. Displaying the second user interface; wherein the second user interface includes the second application interface.

[0067] According to the fourth aspect, or any implementation of the fourth aspect above, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process; when the processor reads computer instructions from memory, it also causes the electronic device to execute: the second application process generates a third rendering node tree; wherein, the third rendering node tree is used to draw the third application interface of the second application process, and the third rendering node tree includes rendering nodes corresponding to the fourth content in the third application interface, and rendering nodes corresponding to the fifth content in the third application interface. The second application process generates a second texture corresponding to the fourth content based on the third rendering node tree. The rendering process draws based on the first texture and the first rendering node tree generated by the first application process to generate the first application interface of the first application process, including: the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the second texture, and the target rendering node tree to generate the first user interface.

[0068] According to the fourth aspect, or any implementation of the fourth aspect above, the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree, including: when the fourth content is completely covered by the first application interface, removing the drawing operation of the rendering node corresponding to the fourth content in the target rendering node tree; and / or, when the fifth content is completely covered by the first application interface, removing the drawing operation of the rendering node corresponding to the fifth content in the target rendering node tree.

[0069] According to the fourth aspect, or any implementation of the fourth aspect above, the first texture includes node textures and / or subtree-level textures, where a node texture corresponds to a rendering node and a subtree-level texture corresponds to a rendering subtree.

[0070] According to the fourth aspect, or any implementation of the fourth aspect above, the parent node of the rendering subtree is a rendering node with animation / visual effects attributes, and the number of rendering nodes in the rendering subtree is greater than or equal to a number threshold. And / or, the root node of the rendering subtree includes a first identifier, which indicates that the rendering subtree is used to generate textures. And / or, the layer containing the rendering subtree includes a second identifier, which indicates that the rendering nodes in the layer are used to generate textures.

[0071] According to the fourth aspect, or any implementation of the fourth aspect above, when the processor reads computer instructions from memory, it also causes the electronic device to execute: the first application process writes the first texture to the first memory space; the rendering process reads the first texture from the first memory space.

[0072] According to the fourth aspect, or any implementation of the fourth aspect above, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into the first memory space, including: the first application process requests a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0073] According to the fourth aspect, or any of the above implementations of the fourth aspect, the first node texture carries the first offset data of the first node texture in the first texture cache, and the second node texture carries the second offset data of the second node texture in the first texture cache.

[0074] According to the fourth aspect, or any implementation of the fourth aspect above, when the processor reads computer instructions from memory, it also causes the electronic device to execute: the rendering process determines to issue a rendering task to the first application process based on one or more of the following: the resource utilization rate of the rendering process, the historical load of the rendering process, the number of foreground application processes, the rendering load of the first application process, the task load of the first application process, the load of the central processing unit (CPU), the load of the graphics processing unit (GPU), the hardware performance of the electronic device, and the battery level of the electronic device. The rendering task instructs the first application process to draw the first content.

[0075] According to the fourth aspect, or any of the above implementations of the fourth aspect, the first content or the second content is one or more of the windows, controls, and properties in the first application interface.

[0076] According to the fourth aspect, or any implementation of the fourth aspect above, when the processor reads computer instructions from memory, it also causes the electronic device to execute: the first application process writes the first rendering node tree into the second memory space. The rendering process reads the first rendering node tree from the second memory space.

[0077] Fifthly, an electronic device is provided. The electronic device includes: a memory and a processor, the memory being coupled to the processor. The memory stores computer program code, which includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes: a first application process draws a first portion of a rendering task to generate a first texture; the first texture includes static content in a first application interface; wherein the first texture or at least a portion of its corresponding static attributes is cached. The rendering process divides the first rendering node tree into a first portion of the rendering task and a second portion of the rendering task based on the first rendering node tree of the first application interface to be displayed; the second portion of the rendering task corresponds to dynamic content in the first application interface. The rendering process draws based on the cached first texture or the at least a portion of the static attributes, and the second portion of the rendering task, to generate the first application.

[0078] According to the fifth aspect, when the processor reads computer instructions from the memory, it also causes the electronic device to execute: the rendering process draws according to the second rendering node tree of the second application interface of the first application process to be displayed, the first texture or the at least part of the static attributes, and the third part of the rendering tasks, to generate the second application interface; wherein the second rendering node tree includes the first part of the rendering tasks and the third part of the rendering tasks.

[0079] According to the fifth aspect, or any implementation of the fifth aspect above, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process; when the processor reads computer instructions from memory, it also causes the electronic device to execute: the rendering process allocates a fourth part of the rendering task to the second application process for drawing based on the third rendering node tree of the third application interface of the second application process to be displayed; the third rendering node tree includes the fourth part of the rendering task and the fifth part of the rendering task. The second application process draws the fourth part of the rendering task to generate a third texture; the third texture includes static content in the third application interface; wherein at least a portion of the static attributes in the third texture are cached. The rendering process draws based on the cached first texture or at least a portion of the static attributes, and the second part of the rendering task to generate the first application interface, including: the rendering process generating a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the third texture, and the target rendering node tree to generate the first user interface.

[0080] According to the fifth aspect, or any implementation of the fifth aspect above, the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree, including: when the content corresponding to the fourth part of the rendering task is completely covered by the first application interface, removing the drawing operations of the fourth part of the rendering task from the target rendering node tree. And / or, when the content corresponding to the fifth part of the rendering task is completely covered by the first application interface, removing the drawing operations of the fifth part of the rendering task from the target rendering node tree.

[0081] According to the fifth aspect, or any implementation of the fifth aspect above, the first texture includes node textures and / or subtree-level textures, where a node texture corresponds to a rendering node and a subtree-level texture corresponds to a rendering subtree.

[0082] According to the fifth aspect, or any implementation of the fifth aspect above, the parent node of the rendered subtree is a rendered node with animation / visual effects attributes, and the number of rendered nodes in the rendered subtree is greater than or equal to a number threshold. And / or, the root node of the rendered subtree includes a first identifier, which indicates that the rendered subtree is used to generate textures. And / or, the layer containing the rendered subtree includes a second identifier, which indicates that the rendered nodes in the layer are used to generate textures.

[0083] According to the fifth aspect, or any implementation thereof, when the processor reads computer instructions from memory, it further causes the electronic device to execute: a first application process writes a first texture or at least a portion of the static attributes into a first memory space; and a rendering process reads the first texture or at least a portion of the static attributes from the first memory space.

[0084] According to the fifth aspect, or any implementation of the fifth aspect above, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into the first memory space, including: the first application process requests a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0085] According to the fifth aspect, or any implementation of the fifth aspect above, the first node texture carries the first offset data of the first node texture in the first texture cache, and the second node texture carries the second offset data of the second node texture in the first texture cache.

[0086] According to the fifth aspect, or any implementation of the fifth aspect above, when the processor reads computer instructions from memory, it also causes the electronic device to execute: the rendering process allocates a first portion of the rendering task to the first application process for drawing based on one or more of the following: the rendering process's resource utilization rate, the rendering process's historical load, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the load of the central processing unit (CPU), the load of the graphics processing unit (GPU), the hardware performance of the electronic device, the battery level of the electronic device, and the first rendering node tree.

[0087] According to the fifth aspect, or any implementation of the fifth aspect above, the first part of the rendering task or the second part of the rendering task corresponds to one or more of the windows, controls, and properties in the first application interface.

[0088] A sixth aspect provides an electronic device. The electronic device includes: a memory and a processor, the memory being coupled to the processor. The memory stores computer program code, which includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes: a first application process draws a first part of a rendering task to generate a first texture; the first texture includes first static content in a first application interface; wherein the first texture or at least a portion of the static attributes corresponding to the first texture is cached. A rendering process divides the first rendering node tree into a first part of a rendering task and a second part of a rendering task based on the first rendering node tree of the first application interface of the first application process to be displayed; the second part of the rendering task corresponds to second static content and dynamic content in the first application interface. The rendering process generates the first application interface based on the cached first texture or the at least a portion of the static attributes, and the second part of the rendering task. According to the sixth aspect, or any implementation of the sixth aspect above, when the processor reads the computer instructions from the memory, the electronic device further executes: the rendering process draws based on the second rendering node tree of the second application interface of the first application process to be displayed, the first texture or the at least a portion of the static attributes, and a third part of the rendering task to generate the second application interface; wherein the second rendering node tree includes the first part of the rendering task and the third part of the rendering task.

[0089] According to the sixth aspect, or any implementation of the sixth aspect above, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process; when the processor reads computer instructions from memory, it also causes the electronic device to execute: the rendering process assigns a fourth rendering task to the second application process for drawing based on the third rendering node tree of the third application interface of the second application process to be displayed; the third rendering node tree includes the fourth rendering task and the fifth rendering task. The second application process draws the fourth rendering task to generate a third texture; the third texture includes the third static content in the third application interface; wherein at least a portion of the static attributes in the third texture are cached. The rendering process draws based on the cached first texture or at least a portion of the static attributes, and the second rendering task to generate the first application interface, including: the rendering process generating a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the third texture, and the target rendering node tree to generate the first user interface.

[0090] According to the sixth aspect, or any implementation of the sixth aspect above, the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree, including: when the content corresponding to the fourth part of the rendering task is completely covered by the first application interface, removing the drawing operations of the fourth part of the rendering task from the target rendering node tree. And / or, when the content corresponding to the fifth part of the rendering task is completely covered by the first application interface, removing the drawing operations of the fifth part of the rendering task from the target rendering node tree.

[0091] According to the sixth aspect, or any implementation of the sixth aspect above, the first texture includes node textures and / or subtree-level textures, where a node texture corresponds to a rendering node and a subtree-level texture corresponds to a rendering subtree.

[0092] According to the sixth aspect, or any implementation of the sixth aspect above, the parent node of the rendered subtree is a rendered node with animation / visual effects attributes, and the number of rendered nodes in the rendered subtree is greater than or equal to a number threshold. And / or, the root node of the rendered subtree includes a first identifier, which indicates that the rendered subtree is used to generate textures. And / or, the layer containing the rendered subtree includes a second identifier, which indicates that the rendered nodes in the layer are used to generate textures.

[0093] According to the sixth aspect, or any implementation thereof, when the processor reads computer instructions from memory, it further causes the electronic device to execute: a first application process writes a first texture or at least a portion of the static attributes into a first memory space; and a rendering process reads the first texture or at least a portion of the static attributes from the first memory space.

[0094] According to the sixth aspect, or any implementation of the sixth aspect above, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into the first memory space, including: the first application process requests a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0095] According to the sixth aspect, or any implementation of the sixth aspect above, the first node texture carries the first offset data of the first node texture in the first texture cache, and the second node texture carries the second offset data of the second node texture in the first texture cache.

[0096] According to the sixth aspect, or any implementation of the sixth aspect above, when the processor reads computer instructions from memory, it also causes the electronic device to execute: the rendering process allocates a first portion of the rendering task to the first application process for drawing based on one or more of the following: the resource utilization of the rendering process, the historical load of the rendering process, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the load of the central processing unit (CPU), the load of the graphics processing unit (GPU), the hardware performance of the electronic device, the battery power of the electronic device, and the first rendering node tree.

[0097] According to the sixth aspect, or any implementation of the sixth aspect above, the first part of the rendering task or the second part of the rendering task corresponds to one or more of the windows, controls, and properties in the first application interface.

[0098] A seventh aspect provides an electronic device that has the function of implementing the rendering method as described in the first aspect and any of its possible implementations; or, the electronic device has the function of implementing the rendering method as described in the second aspect and any of its possible implementations; or, the electronic device has the function of implementing the rendering method as described in the third aspect and any of its possible implementations. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described function.

[0099] Eighthly, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as instructions or code) that, when executed by an electronic device, causes the electronic device to perform the method of the first aspect or any embodiment of the first aspect; or causes the electronic device to perform the method of the second aspect or any embodiment of the second aspect; or causes the electronic device to perform the method of the third aspect or any embodiment of the third aspect.

[0100] Ninth aspect, a computer program product is provided that, when the computer program product is run on an electronic device, causes the electronic device to perform the method of the first aspect or any one of the embodiments of the first aspect; or causes the electronic device to perform the method of the second aspect or any one of the embodiments of the second aspect; or causes the electronic device to perform the method of the third aspect or any one of the embodiments of the third aspect.

[0101] In a tenth aspect, a circuit system is provided, the circuit system including a processing circuit configured to perform the method of the first aspect or any embodiment of the first aspect; or, the processing circuit is configured to perform the method of the second aspect or any embodiment of the second aspect; or, the processing circuit is configured to perform the method of the third aspect or any embodiment of the third aspect.

[0102] Eleventhly, a chip system is provided, including at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, wherein when the at least one processor executes the instructions, the at least one processor executes the method of the first aspect or any one of the embodiments of the first aspect; or, the at least one processor executes the method of the second aspect or any one of the embodiments of the second aspect; or, the at least one processor executes the method of the third aspect or any one of the embodiments of the third aspect.

[0103] The technical effects of the aforementioned aspects can be referenced from each other, and will not be elaborated further here. Attached Figure Description

[0104] Figure 1 Interface illustration provided for embodiments of this application Figure 1 ;

[0105] Figure 2 This is a schematic diagram of a separate rendering scene provided in an embodiment of this application;

[0106] Figure 3 A schematic diagram of a unified rendering scene provided in the embodiments of this application;

[0107] Figure 4 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0108] Figure 5 A schematic diagram of the software structure of the electronic device provided in the embodiments of this application;

[0109] Figure 6 Interface illustration provided for embodiments of this application Figure 2 ;

[0110] Figure 7 Interface illustration provided for embodiments of this application Figure 3 ;

[0111] Figure 8 A schematic diagram illustrating the process of generating a bitmap using an application provided in an embodiment of this application;

[0112] Figure 9 Schematic diagram of the rendering node tree provided in the embodiments of this application Figure 1 ;

[0113] Figure 10 A schematic diagram of a scene for generating a display interface through separate rendering, as provided in an embodiment of this application;

[0114] Figure 11 A schematic diagram of a scene for generating a display interface using unified rendering, as provided in the embodiments of this application;

[0115] Figure 12 Schematic diagram of the rendering node tree provided in the embodiments of this application Figure 2 ;

[0116] Figure 13 Interface illustration provided for embodiments of this application Figure 4 ;

[0117] Figure 14 A schematic diagram of a balanced rendering scene provided in this application embodiment. Figure 1 ;

[0118] Figure 15 A schematic diagram of a balanced rendering scene provided in this application embodiment. Figure 2 ;

[0119] Figure 16 This is a schematic diagram illustrating resource usage scenarios provided in an embodiment of this application.

[0120] Figure 17 A schematic diagram of a balanced rendering scene provided in this application embodiment. Figure 3 ;

[0121] Figure 18 This is a schematic diagram of the scene to be displayed provided in the embodiments of this application;

[0122] Figure 19 Interface illustration provided for embodiments of this application Figure 5 ;

[0123] Figure 20 Interface illustration provided for embodiments of this application Figure 6 ;

[0124] Figure 21 Interface illustration provided for embodiments of this application Figure 7 ;

[0125] Figure 22 This is a schematic diagram of a single-window display scenario provided in an embodiment of this application;

[0126] Figure 23 This is a schematic diagram of a multi-window display scenario provided in an embodiment of this application;

[0127] Figure 24 This application provides a schematic diagram of a scene for drawing the controls provided in its embodiments.

[0128] Figure 25Schematic diagram of the rendering node tree provided in the embodiments of this application Figure 3 ;

[0129] Figure 26 This is a schematic diagram of the optimal subtree generation scenario provided in an embodiment of this application;

[0130] Figure 27 This is a schematic diagram of a texture application scenario provided in an embodiment of this application;

[0131] Figure 28 Flowchart of the rendering method provided in the embodiments of this application Figure 1 ;

[0132] Figure 29 Flowchart of the rendering method provided in the embodiments of this application Figure 2 ;

[0133] Figure 30 Flowchart of the rendering method provided in the embodiments of this application Figure 3 ;

[0134] Figure 31 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0135] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two).

[0136] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0137] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0138] The term "user interface" (UI) used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0139] As the medium for interaction and information exchange between applications and users, the user interface (UI) requires the electronic device to generate the UI for the foreground application each time a vertical synchronization signal arrives. The frequency of the vertical synchronization signal is related to the refresh rate of the electronic device's screen; for example, the frequency of the vertical synchronization signal may be the same as the screen's refresh rate.

[0140] In other words, before each time an electronic device refreshes the content displayed on its screen, it is necessary to generate the interface for the foreground application so that the newly generated interface of the application can be shown to the user when the screen refreshes.

[0141] The interface displayed on an electronic device can include the interface of one or more applications. That is, the electronic device needs to generate an interface for one or more applications and combine the interfaces to obtain the combined interface displayed on the screen.

[0142] In some embodiments, the content displayed on an electronic device interface may include static content and / or dynamic content. Static content refers to content that does not automatically change over time in consecutive interface frames. Dynamic content refers to content that automatically changes over time in consecutive interface frames. For example, static content includes text, icons, status bars, etc., which may be displayed content that remains unchanged in subsequent consecutive frames after being drawn in a certain frame. Another example is a scaled-down window, such as a full-screen window shrinking to a floating window where the displayed content does not change; this window content is static content. Yet another example is GIFs or images with looping animations displayed on the interface, which are dynamic content.

[0143] For example, such as Figure 1 As shown in (a), the electronic device displays a music application. This interface includes an interface for application 1, such as the album art of the music application; and an interface for application 2, such as the status bar of the operating system. The electronic device detects a user's swipe gesture and determines that the user has instructed a change in the displayed content. For example, as shown... Figure 1 As shown in (b), the user instructs the user to switch to the lyrics display of a music application. It can be seen that during the interface switching process, the operating system's status bar (or part of the text and icons within the status bar) remains static and does not change before or after the switch. Therefore, the electronic device can be determined as follows: Figure 1 The static content in the interface to be displayed shown in (b) is the operating system's status bar. For example, as shown in (b)... Figure 1 In the album cover screen shown in (a), the image displayed is an image with an added looping animation effect. During the display of the multiple frames corresponding to the album cover screen, the status bar remains unchanged. Therefore, during the display of the album cover screen, the electronic device can determine that the static content is the operating system's status bar, and the dynamic content is the image with the added looping animation effect.

[0144] In some embodiments, electronic devices render both static and dynamic content during the rendering of the interface to be displayed, which undoubtedly increases the rendering load.

[0145] In some examples, the rendering methods include separate rendering and unified rendering.

[0146] For example, such as Figure 2 As shown, in the separate rendering process, the electronic device needs the application to render its own bitmap to create the application's interface, and then pass the bitmap to the surface compositor (SurfaceFlinger in this example). For instance, after the application determines the corresponding content to be displayed (including static and dynamic content), it can trigger the drawing process and send the generated rendering node tree to the graphics processing unit (GPU) for rendering to generate a bitmap. The application, acting as the producer, draws and generates the bitmap, storing it in a buffer queue (BufferQueue) provided by SurfaceFlinger; SurfaceFlinger, acting as the consumer, continuously retrieves the bitmap generated by the application from the BufferQueue. The bitmap resides on the surface generated by the application, and this surface is then filled into the BufferQueue.

[0147] After SurfaceFlinger obtains the bitmap of the visible application, it works with the Hardware Composer (HWC) to determine how the bitmap is composited as a layer. SurfaceFlinger can determine the visible application through the Window Manager Service (WMS).

[0148] After SurfaceFlinger obtains the off-screen rendering logic that applies to the application's window from the window management service, such as rounded corners, rotation, color changes, scaling, etc., it copies the application's bitmap to the off-screen buffer for rendering and obtains the bitmap used for layer composition through off-screen rendering.

[0149] After the application's bitmap is composited, SurfaceFlinger / HWC fills the composited bitmap (which can also be called a layer on SurfaceFlinger) into the frame buffer and passes it to the display subsystem (DSS). The DSS can then display the composited bitmap on the screen. This frame buffer can be an on-screen buffer.

[0150] The detailed drawing, rendering, and layer compositing processes are described in the relevant content below and will not be repeated here.

[0151] For example, such as Figure 3 As shown, in the unified rendering process, the electronic device only needs the application to send the drawing instructions for the corresponding static and dynamic content in the interface to be displayed to the rendering service to generate the application's interface. Then, the rendering service will draw the static and dynamic content according to the drawing instructions, generate a unified rendering node tree, and send the generated rendering node tree to the GPU for rendering to generate the interface to be displayed.

[0152] It can be seen that in separate rendering, the independent rendering process of each application makes it difficult to achieve interactive animations between applications. While unified rendering reduces the difficulty of achieving interactive animations between applications through a globally unified rendering node tree, the rendering load is concentrated in the rendering service, resulting in lower rendering efficiency. For example, in unified rendering, if there is a large amount of vector content such as text, and vector content is difficult to render, the sequential rendering process of rendering these text vector contents first, followed by other content, leads to low rendering efficiency and may result in dropped frames or stuttering. Furthermore, the repeated drawing of the same static content across multiple consecutive frames also increases unnecessary rendering power consumption.

[0153] Therefore, this application provides a rendering method that can dynamically balance the content to be rendered between the application and the rendering service, thereby improving rendering efficiency and enhancing device performance.

[0154] In some embodiments, the rendering method provided in this application can be applied to an electronic device 100. In some examples, the electronic device 100 includes, but is not limited to, mobile phones, computers, laptops, tablets, smart screens, projectors, in-vehicle screens, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), wearable devices (such as smartwatches, smart glasses, etc.), artificial intelligence (AI) devices, and other terminal devices. The operating system installed on the electronic device 100 includes, but is not limited to, […]. Alternatively, other operating systems may be used. This application does not limit the specific type of electronic device 100 or the operating system installed on it.

[0155] Figure 4 This is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of this application.

[0156] For example, such as Figure 4As shown, the electronic device 100 may include a processor 410, an external memory interface 420, an internal memory 421, a universal serial bus (USB) interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, an antenna 2, a mobile communication module 450, a wireless communication module 460, an audio module 470, a sensor module 480, a button 490, a motor 491, an indicator 492, a camera 493, a display screen 494, and a subscriber identification module (SIM) card interface 495, etc.

[0157] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0158] For example, electronic device 100 is a laptop computer, and electronic device 100 may not include mobile communication module 450 and SIM card interface 495.

[0159] Processor 410 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0160] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0161] The processor 410 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. This memory can store instructions or data that the processor 410 has just used or that are used repeatedly. If the processor 410 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0162] In some embodiments, the processor 410 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0163] The MIPI interface can be used to connect the processor 410 to peripheral devices such as the display screen 494 and the camera 493. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 410 and the camera 493 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 410 and the display screen 494 communicate via the DSI interface to enable the electronic device 100 to display images.

[0164] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0165] Electronic device 100 implements display functions through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 410 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0166] Display screen 494 is used to display images, videos, etc. Display screen 494 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), such as an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 494, where N is a positive integer greater than 1.

[0167] The electronic device provided in this application embodiment can run an operating system (OS). This operating system can be various operating systems used in industry, such as operating systems developed based on OpenHarmony, like HarmonyOS; or other operating systems such as Android™, iOS mobile operating systems; it can also be various open-source operating systems or their derivatives, such as Linux OS, and other embedded operating systems; or it can be a future new operating system, such as an AI operating system based on artificial intelligence. An operating system is a set of interconnected system software programs that manage and control the operation of electronic devices, utilize and run hardware and software resources, and provide public services to organize user interaction. In electronic devices, the operating system connects downwards to the physical devices at the hardware layer and provides a runtime environment for application software upwards.

[0168] An operating system typically includes a kernel layer, a middleware layer, and an application layer. The application layer includes applications, which can include system applications and third-party applications. The middleware layer includes a suite of software providing various services to application developers, or frameworks providing services such as databases, multimedia, and graphics, or capabilities such as distributed scheduling and system scaling. For example, the middleware layer may include a framework layer and / or a system service layer. The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The system service layer includes the system's core capabilities, providing services to applications through the framework layer. The kernel layer is the layer between hardware and software. The kernel layer may include hardware drivers and the operating system kernel. In addition to providing hardware drivers, the kernel layer also supports functions such as memory management and system process management.

[0169] The electronic devices we use in our daily lives come in various types and forms, and are applied in a wide range of scenarios. Therefore, based on the different forms and functions of electronic devices, different application scenarios, and different user needs, the operating systems used in these devices may also differ. The basic functions implemented by the electronic device provided in this application can be implemented using a general-purpose operating system or a dedicated operating system. To more clearly illustrate the implementation of the embodiments of this application under a specific operating system, the architecture of HarmonyOS is shown below. Those skilled in the art can deduce the implementation of the embodiments of this application under other specific operating systems, such as Android™.

[0170] For example, such as Figure 5 As shown, the software architecture of an electronic device can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. Layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem granularity in different device deployment scenarios, and each subsystem can also be tailored, added, or combined at the functional granularity.

[0171] The kernel layer includes the kernel abstract layer (KAL), kernel subsystems, and driver subsystems.

[0172] The kernel abstraction layer provides basic kernel capabilities to the upper layers by shielding the differences between multiple kernels, including but not limited to process / thread management, memory management, file system, network management, and peripheral device management.

[0173] Kernel Subsystem: Supports the selection of a suitable OS kernel for different resource-constrained devices, including but not limited to Linux kernel, HarmonyOS kernel, LiteOS (Lite Operating System), etc.

[0174] Driver Subsystem: The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: display drivers, camera drivers, audio drivers, Bluetooth drivers, sensor drivers, etc.

[0175] The system service layer comprises the core capabilities of the system, providing services to applications through the framework layer. This layer includes, but is not limited to, the following subsystems:

[0176] The system's basic capability subsystem set provides fundamental capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. This set may include distributed soft bus, distributed data management, distributed task scheduling, and Ark multi-language runtime; it may also include multi-modal input subsystem, graphics subsystem, security subsystem, and AI business subsystem.

[0177] The graphics subsystem primarily includes rendering services, which implement the rendering process. In some examples, the graphics subsystem also includes modules for UI components, layout, animation, fonts, input events, window management, and rendering / drawing. The graphics service provides graphics rendering and display output functions, and internally, through the rational utilization of system hardware resources, it provides a smooth and efficient display experience for the system.

[0178] Basic software service subsystem set: provides public and general software services; the basic software service subsystem set may include event notification subsystem, telephone service subsystem, multimedia subsystem, etc.

[0179] Enhanced software service subsystem suite: Provides differentiated enhanced software services for different devices; the enhanced software service subsystem suite may include smart screen proprietary business subsystem, wearable proprietary business subsystem, IoT proprietary business subsystem, etc.

[0180] Hardware service subsystem set: Provides hardware services; the hardware service subsystem set may include location service subsystem, user IAM (Identity and Access Management) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service subsystem, etc.

[0181] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices.

[0182] Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices.

[0183] The distributed soft bus provides communication-related capabilities for seamless interconnection between multiple devices, including WLAN service capabilities, Bluetooth service capabilities, soft bus, and StarFlash communication capabilities.

[0184] Ark Multilingual Runtime is a unified compilation runtime platform designed to support the joint compilation and execution of multiple programming languages ​​and multiple chip platforms.

[0185] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes: the ArkUI framework (which provides a complete infrastructure for UI development of system applications, including UI functions such as components, layouts, animations, and interactive events, as well as a real-time interface preview tool), the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of Abilities). Different devices may have different operating systems, and the APIs they support may also differ.

[0186] The HarmonyOS API is a series of open capabilities provided to support HarmonyOS application development. The HarmonyOS API can be set at the framework layer or independently of the framework layer. The HarmonyOS API includes the Audio API (audio service), Push API (push service), and Account API (account service), among others.

[0187] The application layer includes multiple applications, which can include system applications and extended / third-party applications. System applications can include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party applications can include social applications, travel applications, etc.

[0188] The rendering method provided in the embodiments of this application will be described in detail below.

[0189] In some embodiments, as described above, the electronic device requires a rendering process before generating the interface to be displayed. During the generation of the interface, the rendering process can be performed by an application and / or a rendering service. In some examples, the electronic device can allocate rendering tasks to the application and rendering service based on factors such as the historical load of the rendering service and / or the application, the resource utilization of the rendering service and / or the application, and the rendering task size of the content to be displayed in the interface.

[0190] For example, an electronic device instructs an application to draw and render all the static content of the interface to be displayed, while a rendering service instructs a rendering service to draw and render all the dynamic content of the interface to be displayed.

[0191] For example, an electronic device instructs an application to draw and render a portion of the static content of the interface to be displayed, while a rendering service instructs a rendering service to draw and render another portion of the static content of the interface to be displayed, as well as all the dynamic content.

[0192] In some examples, during the drawing and rendering process, the application performs the drawing and rendering through the application process, while the rendering service performs the drawing and rendering process through the rendering process.

[0193] Thus, this application provides a load-balanced rendering method. By balancing the load, all drawing and rendering are avoided from being performed by the application or by the rendering service. This reduces the difficulty of creating interactive animations between applications while improving rendering efficiency.

[0194] To facilitate understanding of the process of rendering and drawing the interface to be displayed, the following text will first take the separate rendering process as an example to introduce the overall process of generating the interface to be displayed. It should be understood that in the unified rendering process or the balanced load rendering process, the execution entities in the various processes of generating the interface to be displayed may be different, but the implementation principles are similar.

[0195] First, the following examples illustrate (1) the process of generating bitmaps by the application, (2) the process of compositing bitmaps by SurfaceFlinger / HWC, and (3) off-screen rendering.

[0196] (1) The process of the application generating a bitmap

[0197] Figure 6 , Figure 7 This is a schematic diagram of the interface provided in an embodiment of this application.

[0198] For example, such as Figure 6 As shown, the screen of the electronic device displays the interface of application 1 and the interface of application 2. The interface of application 1 can be referred to as the status bar; application 1 can be the operating system, and application 2 can be a music application.

[0199] Electronic devices displaying such Figure 6 Before the interface shown, each application's bitmap needs to be generated independently. The status bar interface is maintained by the operating system; therefore, the application that generates the status bar bitmap can be considered the operating system itself. For example, the operating system generates the bitmap... Figure 1 Afterwards, the position Figure 1Passed to SurfaceFlinger; Music app generates bits Figure 2 Afterwards, the position Figure 2 Passed to SurfaceFlinger. Among them, bits... Figure 1 The image information of the status bar is displayed on the top; Figure 2 It contains image information for music applications.

[0200] Among them, SurfaceFilter / HWC receives bits Figure 1 and position Figure 2 Afterwards, the position Figure 1 and position Figure 2 As a layer, it is used for layer compositing. The content of layer compositing can be found in the relevant description of the bitmap compositing process in SurfaceFilter / HWC below (2), which will not be repeated here.

[0201] For example, such as Figure 7 As shown, the screen of the electronic device displays the interfaces of application 1, application 2, and application 3. The interface of application 1 can be referred to as the status bar. Application 1 can be the operating system, application 2 can be a music application, and application 3 can be a chat application.

[0202] Electronic devices displaying such Figure 7 Before the interface shown, each application's bitmap also needs to be generated independently. For example, the operating system generates the bitmap... Figure 3 Afterwards, the position Figure 3 Passed to SurfaceFlinger; Music app generates bits Figure 4 Afterwards, the position Figure 4 Passed to SurfaceFlinger; chat application generates bits Figure 5 Afterwards, the position Figure 5 Passed to SurfaceFlinger. Among them, bits... Figure 3 The image information of the status bar is displayed on the top; Figure 4 It contains image information for the music application; bits Figure 5 It contains image information from a chat application.

[0203] Among them, SurfaceFilter / HWC receives bits Figure 3 ,Bit Figure 4 and position Figure 5 Afterwards, the position Figure 3 ,Bit Figure 4 and position Figure 5As a layer, it is used for layer compositing. The content of layer compositing can be found in the relevant description of the bitmap compositing process in SurfaceFilter / HWC below (2), which will not be repeated here.

[0204] The process by which the application generates a bitmap is as follows: Figure 8 As shown.

[0205] Figure 8 This is a schematic diagram illustrating the process of generating a bitmap using an application provided in an embodiment of this application.

[0206] like Figure 8 As shown, after receiving the vertical synchronization signal (Vsync-APP), the application begins to generate a bitmap. The specific steps include three steps:

[0207] ① The view hierarchy is disabled, and the main thread (UI thread) traverses the application's views, identifies and saves the drawing operations for each view, and records the view and its associated drawing operations (DrawOPs) into the displaylist of the render node in the render node tree. This process can also be called the recording process.

[0208] Among them, the view is the basic element that constitutes the application interface, and one control on the interface can correspond to one or more views.

[0209] The drawing operation, as a data structure, is used to draw graphics, such as lines, widths, rectangles, and text. At the rendering node, drawing operations are translated into API calls to image processing libraries, such as OpenGL or Vulkan interfaces. For example, `DrawLineOp` is a data structure containing drawing data such as line length and width.

[0210] The drawing instruction list can be a buffer that records all drawing operations or their identifiers (such as addresses and sequence numbers) included in a single frame of the application's interface. When the application has multiple windows or displays on different display areas, multiple rendering node trees need to be generated independently, resulting in multiple drawing instruction lists corresponding to different windows and display areas.

[0211] In this embodiment, the display area can be a screen or a virtual display. A virtual screen can be the area used by the electronic device to display content on the screen during screen recording.

[0212] The render node tree, generated by the UI thread, is a data structure used to generate the application's interface. It can include multiple render nodes, each containing render attributes and a list of drawing instructions. The render node tree records all the information needed to generate a single frame of the application's interface.

[0213] ②The UI thread passes / synchronizes the rendering node tree to the rendering thread. The rendering node tree is located in the stack of the application's corresponding process and may not be distributed contiguously in physical address space.

[0214] ③ The rendering thread first acquires a hardware canvas and performs drawing operations in the rendering node tree on that canvas to generate a bitmap. This hardware canvas resides within a surface held by the application, which contains bitmap or other data in other formats used to store image information. This process can also be called the playback process.

[0215] ① can be considered the construction phase, which is mainly responsible for determining the size, position, transparency, and other attributes of each view in the application. For example, the drawLine view in the view can be encapsulated into a DrawLineOp during the construction phase. It contains drawing data such as the length and width of the line, and can also contain the interface call corresponding to the drawLineOP of the underlying graphics processing library, which is used to call the underlying graphics library to generate bitmaps during the rendering phase.

[0216] Similarly, ③ can be considered as the rendering stage, which is mainly responsible for traversing the rendering node tree and performing the drawing operation of each rendering node, thereby generating a bitmap on the hardware canvas. In this process, the rendering thread calls the underlying graphics processing library, such as OpenGL or Vulkan, and then calls the GPU to complete the rendering to generate the bitmap.

[0217] Figure 9 This is a schematic diagram of the rendering node tree provided in an embodiment of this application.

[0218] The application's interface consists of multiple nested views, with parent-child relationships between them. Therefore, the parent-child relationships between rendering nodes in the rendering node tree generated by traversing the views are the same as the parent-child relationships between the views. In other words, the parent-child relationships between views determine the nesting relationships between different rendering nodes, thus enabling the rendering thread to correctly render the application's interface when generating bitmaps based on the rendering node tree.

[0219] A view can correspond to one or more rendering nodes, with the root view (DecorView) corresponding to the root rendering node (RootRenderNode). That is, the nesting relationship between rendering nodes corresponds to the parent-child relationship of views. Rendering nodes also include rendering properties, which are used to determine the position, size, and opacity of the view corresponding to that rendering node on the surface when rendering a bitmap.

[0220] For example, the structure of an application's interface is as follows: the application's PhoneWindow contains a root view, and the root view's subviews are views... Figure 1 Heshi Figure 2 ,See Figure 2 The subview is the view Figure 3 Therefore, the structure of the rendering node tree generated by the application's UI thread is as follows: the root rendering node corresponding to PhoneWindow is the root node of the rendering node tree, the child node of the root rendering node is rendering node 0 corresponding to the root view, and the child nodes of rendering node 0 are the child nodes of the view... Figure 1 The corresponding rendering node 1 and view Figure 2 The corresponding rendering node 2, the child nodes of rendering node 2 are the view Figure 3 The corresponding rendering node is 3.

[0221] The relationship between a view and a rendering node means that the rendering node includes all the drawing operations of the corresponding view.

[0222] After receiving the rendering node tree synchronized with the UI thread, the rendering thread calls the OpenGL interface or Vulkan interface to render a bitmap on the application's own surface, and sends the surface to SurfaceFlinger to wait for compositing and display.

[0223] It is worth noting that the build phase requires CPU computing resources, while the rendering phase requires GPU resources.

[0224] It is worth noting that if hardware acceleration is not enabled, the application completes all operations in the build and rendering stages through the UI thread, and does not need to be encapsulated as a rendering node. After traversing the application's views and performing the view drawing operations, it requests an anonymous shared memory from SurfaceFlinger and directly calls the underlying graphics library to generate a bitmap on that memory.

[0225] It is worth noting that, Figure 9 The diagram illustrates the bitmap generation process of an application when hardware acceleration is enabled. When hardware acceleration is not enabled on the electronic device, the application generates the bitmap through software rendering. This software rendering includes:

[0226] ① Disable the view hierarchy, and the UI thread traverses the application's views, recording the drawing operations for each view.

[0227] ② The UI thread obtains a software canvas for drawing through an interface, such as Surface.lockCanvas(), and draws on this software canvas based on a saved list of drawing operations, generating a bitmap. This software canvas resides on the surface generated by the application.

[0228] The surface held by the application is allocated to it by SurfaceFlinger via binder communication. The number of surfaces held by the application can be the same as the number of its current windows (PhoneWindow).

[0229] After introducing the process of generating bitmaps by the application, the process of compositing bitmaps is described by way of example.

[0230] (2) The process of synthesizing bitmaps using SurfaceFilter / HWC

[0231] SurfaceFlinger is a system service on electronic devices used to assign surfaces to applications and composite bitmaps from one or more surfaces as layers. HWC is a functional module of the Hardware Abstraction Layer (HAL) in electronic devices, responsible for compositing and display. It provides an interface to the upper-layer SurfaceFlinger and calls upon the capabilities of the underlying hardware (such as the display driver, excluding the GPU) to perform layer compositing.

[0232] The application's rendering thread generates bitmaps on the surface allocated to it by SurfaceFlinger. SurfaceFlinger maintains one or more application surfaces using a BufferQueue mechanism, meaning SurfaceFlinger can obtain bitmaps from different applications.

[0233] After SurfaceFlinger obtains bitmaps from one or more applications, it can call the GPU to combine multiple bitmaps into a single bitmap (the combination of bitmaps is called layer compositing). This compositing can also be called Client compositing or GLES compositing.

[0234] After SurfaceFilter obtains the application's bitmap, it can use HWC to call the underlying hardware (excluding the GPU) for compositing. This compositing method is also known as Device Compositing.

[0235] Client compositing requires the GPU and can composite multiple layers, performing pixel-by-pixel processing on the layers, such as linear fading.

[0236] Device compositing can composite a limited number of layers and does not support many pixel-by-pixel processing methods. Device compositing, when compositing multiple layers that do not overlap on the screen, can avoid layer compositing and instead read data from different surfaces at different locations on the display screen for display.

[0237] For example, SurfaceFlinger obtains the bit information of application 1. Figure 1 Application 2's bit Figure 2 After the bitmap N of application N, ..., you can access the bits... Figure 1 and position Figure 2 Perform Client synthesis to generate bits Figure 11 Secondly, SurfaceFlinger will be positioned... Figure 11 The bitmap N is synthesized by calling the underlying hardware through HWC.

[0238] Device synthesis will temporarily save the location. Figure 11 And bitmap N, and when the electronic device screen needs to display it, from the bitmap N. Figure 11 / Bitmap N takes the corresponding pixels and displays them on the screen. For example, in Figure 7 In the interface shown, bit Figure 11 Bitmaps are composited for a chat application and a music application, with bitmap N being a bitmap of the status bar.

[0239] For the underlying hardware corresponding to SurfaceFilter or HWC, each bitmap is equivalent to a layer.

[0240] The layer composition method can be determined by the underlying hardware corresponding to HWC, or by SurfaceFilter.

[0241] For example, after acquiring the bitmap, SurfaceFlinger passes the set of layers to the underlying hardware via HWC. The underlying hardware determines which layers are used for client compositing and which are used for device compositing. The underlying hardware labels the layers in the layer list with compositing methods and returns the compositing methods for different layers to SurfaceFlinger. SurfaceFlinger then composites the layers labeled as GPU compositing and places the composited result into a buffer. SurfaceFlinger then passes this buffer, along with other layers labeled as Overlay compositing, to the underlying hardware via HWC, whereby the underlying hardware completes the layer compositing.

[0242] For example, secondly, SurfaceFlinger directly marks layers that trigger off-screen rendering logic, such as window animations, during the layer compositing process as GPU-composited. This off-screen rendering logic includes elements like rounded corners, scaling, rotation, and color conversion—logic that the underlying hardware of HWC cannot handle.

[0243] Because the surface allocated to an application by SurfaceFlinger contains not only bitmaps generated by the application's rendering thread, but also window control information such as layer Z-order obtained from the window management service, SurfaceFlinger can obtain the layer's window control information from the surface, thereby determining whether the layer needs to be rendered by the GPU. The layer Z-order determines the height order of the layers on the Z-axis, where the Z-axis is perpendicular to the screen and is used to calculate the vertical relationship between different layers.

[0244] For example, after obtaining the application's bitmaps, SurfaceFlinger writes each bitmap as a layer into a layer list. SurfaceFlinger passes the layer list to the underlying hardware via HWC. The underlying hardware determines the composition method for different layers based on its own capabilities and returns the result to SurfaceFlinger via HWC.

[0245] After obtaining the results returned by HWC, SurfaceFlinger can determine the compositing method of each layer in the layer list. For layers marked as GPU compositing, SurfaceFlinger performs layer compositing and passes the composited layer, along with the layers marked as overlay compositing, to the underlying hardware via HWC. The underlying hardware then performs layer compositing based on the HWC's corresponding structure.

[0246] For example, after obtaining multiple surfaces, SurfaceFlinger can determine the bitmaps on each surface as the compositing method for layers. Once SurfaceFlinger obtains multiple layers and determines the compositing method to be GPU, it can perform client compositing. The compositing modes for these layers can include: Mode.CLEAR (displays the top-level layer in Z-order), Mode.SRC_OVER (displays layers sequentially in Z-order), Mode.DST_IN (displays the non-overlapping portions of the top-level layer in Z-order and the layers below it), etc.

[0247] For example, when the layer composition mode is Mode.SRC_OVER, SurfaceFilter will combine the contents of three layers, determine the vertical relationship of the different layers according to the Z order of the layers, and stack them onto a single layer in sequence.

[0248] Layer 1 has a Z-order of 'a', Layer 2 has a Z-order of 'a+1', and Layer 3 has a Z-order of 'a+2'. SurfaceFlinger will composite the contents of the three layers, resulting in Layer 1 and Layer 2's contents being partially or completely obscured, while all of Layer 3's contents will be displayed. "Partially or completely obscured" means that due to the obscuring effect of the higher Z-order layer, the view on the lower Z-order layer may be partially or completely hidden.

[0249] Layers composited using the GPU trigger off-screen rendering, which is then used for layer compositing. The following section introduces the relevant concepts of off-screen rendering.

[0250] (3) Off-screen rendering

[0251] If SurfaceFlinger determines that any number of layers need to be composited using the GPU, off-screen rendering needs to be enabled to complete the layer compositing.

[0252] Off-screen rendering involves allocating an off-screen buffer for SurfaceFilter and then using the GPU to perform image processing within that buffer. This off-screen buffer is a memory location outside the current screen buffer, and multiple layers are composited within it.

[0253] Off-screen rendering can include the following steps:

[0254] ① The bitmap in the GPU-composited layer determined by SurfaceFilter needs to be converted into a texture and uploaded to GPU memory (i.e., off-screen buffer) or mapped to an OpenGL or Vulkan texture via shared memory. Then, OpenGL or Vulkan binds the texture (binding the texture includes binding the context).

[0255] ② Render the texture according to the window animation instructions corresponding to the layer. Merge textures from multiple applications, where layer merging is performed pixel-by-pixel rendering based on the layer's compositing mode.

[0256] ③ SurfaceFlinger obtains the rendered texture from GPU memory or directly from shared memory.

[0257] Based on the above content, combined with (1) the process of generating bitmaps by the application, (2) the process of compositing bitmaps by SurfaceFilter / HWC, and (3) the content in off-screen rendering, the process of compositing bitmaps as layers is fully described below.

[0258] Figure 10 This is a schematic diagram of a scene for generating a display interface through separate rendering, as provided in an embodiment of this application.

[0259] like Figure 10 As shown, the UI threads of different application processes independently generate their own rendering node trees and pass the rendering node trees to their respective rendering threads for rendering to generate bitmaps.

[0260] The rendering thread first renders and generates a bitmap based on the attributes of the rendering nodes in the rendering node tree and the list of drawing instructions, and then composites the bitmap as a layer. When the layer compositing method is GPU compositing, SurfaceFlinger merges multiple layers into one layer through off-screen rendering.

[0261] For example, application process 1 generates rendering node tree 1, and then generates bits based on rendering node tree 1. Figure 1 Application process 2 generates rendering node tree 2, and then generates bitwise operations based on rendering node tree 2. Figure 2 SurfaceFlinger receives bits Figure 1 and position Figure 2 After that, the opposite position Figure 1 and position Figure 2 Perform off-screen rendering to generate bitmaps Figure 5 In the off-screen rendering process, the first step is to... Figure 1 Copy to the off-screen buffer, based on window animation information, and store the bits in the off-screen buffer. Figure 1Transform to bit Figure 3 Similarly, the position Figure 2 Transform to bit Figure 4 Finally, the position Figure 3 and position Figure 4 Based on the layer compositing mode, bits are generated by overlaying. Figure 5 .

[0262] For example, after receiving the application's bitmap, SurfaceFlinger first performs off-screen rendering on the application's bitmap involved in the window animation bitmap to complete the transformation, and then copies the transformed layer to the on-screen buffer for display.

[0263] For example, in such Figure 7 In the scenario shown, the bitmap of application 3 needs to be scaled down. Therefore, the bitmap of application 3 cannot be directly copied to the on-screen buffer for transformation, as this may affect the bitmaps of other applications. Instead, it needs to be copied to the off-screen buffer for transformation separately, and the transformed result is copied back to the on-screen buffer.

[0264] Clearly, the correct interface can only be generated by first modifying the bitmaps generated by each application and then overlaying them according to the layer composition mode.

[0265] From the moment the application's main thread generates the rendering node tree until the SurfaceFlinger layer compositing is completed, the GPU needs to be started at least N+1 times because the process calling the GPU keeps changing (from application 1 to application 2, ..., application N, SurfaceFlinger), where N is the number of applications.

[0266] Understandably, in the aforementioned separate rendering process, because different applications independently build and render their own bitmaps, it is quite difficult to achieve interactive animation effects between applications. Furthermore, bitmaps from different applications may occlude each other, and generating occluded bitmaps increases unnecessary power consumption.

[0267] In some examples, corresponding to Figure 8The diagram illustrates the application's bitmap generation process during separate rendering. In unified rendering, the application's main thread traverses the application's views, identifies and saves the drawing operations for each view, and records the view and its associated drawing operations into the drawing instruction list of the rendering node in the rendering node tree. Additionally, the main thread can determine the rendering attributes of the view. The application can then send this drawing instruction list and rendering attributes to the rendering service. Upon receiving the drawing instruction list and rendering attributes, the rendering service can generate the rendering node tree based on them. In some examples, if the rendering service receives drawing instruction lists and rendering attributes from multiple applications, it can integrate these lists and attributes to generate a unified rendering node tree. For example, rendering nodes corresponding to views that might be occluded can be removed. The rendering service can then execute the rendering process based on this unified rendering node tree to generate the interface to be displayed.

[0268] For example, in the iOS rendering architecture, the iOS rendering architecture provides the front-end framework UI Kit and rendering services. UI Kit updates the front-end UI state, while the rendering services provide the image processing framework Core Image, the graphics engine Core Graphics, the animation library Core Animation, and the GPU rendering and computation interface Metal API / OpenGL ES. Based on this, the unified rendering process includes, for example, the following steps: Step 1: UI processing parses drawing instructions, generates a rendering node tree, and distributes the instruction content to the corresponding nodes; Step 2: Traverses the rendering node tree to update node information and replay instructions; Step 3: Traverses the updated rendering node tree and sends drawing instructions to the processor for rendering; Step 4: Multi-window rendering results are composited and sent to the hardware for display.

[0269] For example, such as Figure 11 As shown, the UI thread of application process 1 generates rendering node tree 1, and the UI thread of application process 2 generates rendering node tree 2. Application processes 1 and 2 can pass their respective rendering node trees to the rendering process. The rendering process can generate a new root node, and then connect rendering node tree 1 and rendering node tree 2 to the root node to obtain the target rendering node tree. Afterwards, the rendering process can render the target bitmap based on the target rendering node tree, and the electronic device can display the target bitmap. For example, in... Figure 6In the scenario shown, during the unified rendering process, application 1 and application 2 no longer need to generate their own bitmaps. Application process 1 can generate rendering node tree 1 corresponding to the status bar, and application process 2 can generate rendering node tree 2 corresponding to the album art interface. Then, the rendering process combines rendering node tree 1 and rendering node tree 2 to obtain the target rendering node tree. Based on the target rendering node tree, the target bitmap is then rendered; this target bitmap represents the interface to be displayed, for example... Figure 6 The interface shown.

[0270] As can be seen, during the unified rendering process, when an electronic device generates a user interface within the unified rendering framework, the device's applications can generate a rendering node tree. The rendering service can obtain the rendering node trees of one or more applications and recombine them to generate a target rendering node tree. The rendering service can then directly render based on the target rendering node tree to obtain a bitmap carrying the interface image information of the one or more applications, eliminating the need for layer compositing operations on the electronic device.

[0271] In the process of merging the rendering node trees of one or more applications into a target rendering node tree, the rendering service can determine the animation effects of each layer and modify the child nodes in the target rendering node tree according to the animation effects of each application. This allows the rendering service to directly generate the target bitmap based on the target rendering node tree without off-screen rendering. Furthermore, applications do not need to create their own rendering threads; instead, the rendering service handles the rendering uniformly, thus improving the speed of interface rendering. It should be understood that when there is only one rendering node tree, that tree is the target rendering node tree; when there are more than one rendering node tree, the electronic device can synthesize these multiple rendering node trees to obtain a single target rendering node tree.

[0272] In some examples, compared to separate rendering, electronic devices in a unified rendering approach merge the rendering node trees of one or more applications into a target rendering node tree, eliminating the need to pre-generate multiple bitmaps as layers for compositing. The rendering service can obtain the Z-order of the layers corresponding to each application's rendering node tree during the bitmap generation process based on the target rendering node tree, and use this Z-order as the Z-order of the rendering node tree. The Z-order can be used to identify the hierarchical relationship between the layers.

[0273] In some examples, electronic devices can avoid overdrawing by removing child nodes corresponding to bitmaps that are not displayed or do not affect display, based on the Z-order of the target rendering node tree. After generating the rendering node tree, the application can send it to a rendering service. The rendering service can traverse the child nodes of each rendering node tree, determine the position of the view corresponding to each child node on the surface, and, combined with the Z-order of the rendering node tree, identify completely obscured views and their child nodes. The rendering service can then eliminate the drawing operations on the child nodes corresponding to the completely obscured views in the rendering node tree.

[0274] Understandably, in the unified rendering process described above, the rendering service builds a globally unified rendering node tree, reducing the difficulty of implementing interactive animations between applications and making it easier to eliminate occlusion between applications. However, the load is concentrated on the rendering service, resulting in lower rendering efficiency and impacting the performance of electronic devices.

[0275] Based on this, embodiments of this application provide a rendering method that improves rendering efficiency and enhances device performance by balancing the rendering load between the application and the rendering service.

[0276] It should be understood that during the rendering process, the application performs rendering through its application process, and the rendering service performs rendering through its rendering process. Therefore, balancing the rendering load between the application and the rendering service means balancing the rendering load between the application process and the rendering process. The steps executed by the application in the following text can also be understood as the steps executed by the application process, and the steps executed by the rendering process can also be understood as the steps executed by the rendering service. This will not be elaborated further below.

[0277] In some examples, electronic devices balance the rendering load between the application and the rendering service by assigning a portion of the rendering tasks of the interface to be displayed to the application and another portion to the rendering service.

[0278] In some examples, during load balancing rendering, the application iterates through the views of the interface to be displayed, determines which content needs to be drawn by the application and which needs to be sent to the rendering service for drawing, thus generating a corresponding list of drawing instructions. Subsequently, the application and the rendering service can allocate rendering tasks based on this list of instructions.

[0279] For example, the electronic device includes a balancing strategy module that can output a rendering task allocation result based on input information. The rendering task allocation result may include, for example, which rendering tasks in the interface to be displayed are assigned to the application.

[0280] In some examples, rendering tasks are assigned by a rendering service, the application corresponding to the rendering task, or other system services. For instance, the rendering service may include a load balancing module. Alternatively, the application may include a load balancing module. Yet another example is that the load balancing module is a separate system service or SO (Service Object Model).

[0281] In some examples, the input information includes one or more of the following: resource utilization of the rendering service, historical load of the rendering service, number of foreground applications, rendering load of the application, task load of the application, CPU load, GPU load, hardware performance of the electronic device, and battery level of the electronic device.

[0282] For example, the balancing strategy module determines that the rendering service has a high resource utilization rate based on the rendering service's resource utilization. Then, the balancing strategy module can distribute rendering tasks to the application, allowing the application to share the rendering load of the rendering service.

[0283] For example, application workload refers to the workload of the application excluding rendering tasks. If the application has a heavy workload, the load balancing module can distribute more rendering tasks to the rendering service.

[0284] For example, the balancing strategy module can allocate more rendering tasks to the application based on the high performance of the electronic device hardware and the high battery level, thereby reducing the load on the rendering service.

[0285] In some examples, the application performs rendering tasks, referring to the separate rendering process described above. After generating a bitmap based on the rendering node tree, the application caches the bitmap so that it can be used in subsequent rendering services. In other examples, the application can convert the bitmap into a texture and cache the texture after binding it to a context.

[0286] In some examples, the rendering service performs rendering tasks, following the unified rendering process described above. After the application generates a list of drawing instructions, it sends this list to the rendering service. The rendering service then executes the drawing and rendering process based on this list. During the drawing and rendering process, the rendering service traverses the rendering node tree, determining which rendering node (or subtree) contains content that has already been drawn. This content can be directly retrieved from the cache, such as a bitmap rendered and cached by the application. The rendering service then determines which node (or subtree) contains content to be drawn and executes the drawing and rendering. Thus, after the entire rendering node tree traversal is complete, the rendering service can generate the interface to be displayed.

[0287] For example, such as Figure 12As shown, in the load-balanced rendering implementation, the rendering service can directly draw ordinary rendering nodes in the rendering node tree, while for rendering nodes involving cached content, it does not need to draw them again; instead, it can directly retrieve that content from the cache. The cached content consists of content already drawn by the application, such as textures. This allows the application and the rendering service to share the rendering load.

[0288] For example, in such Figure 13 In the scene shown in the memo application display, the electronic device displays something like... Figure 13 During the process of the memo application interface shown in (a), a swipe operation by the user on the display interface in the direction indicated by the arrow is detected. In response to this swipe operation, the electronic device determines the interface to be displayed as shown in (a). Figure 13 The interface shown in (b) is as follows. It can be seen that the content indicated by reference numeral 131 is as follows: Figure 13 The content already exists in the Memos application interface shown in (a).

[0289] In such Figure 13 In the current frame interface generation process shown in (a), the memo application has already completed the drawing of this part and cached the corresponding textures. Therefore, when the electronic device generates a frame like this... Figure 13 In the process of displaying the next frame of the interface shown in (b), this part of the content can be directly reused. For example, the rendering service directly retrieves the texture corresponding to the content shown in reference numeral 131 from the cache.

[0290] In some examples, in the current sliding scenario, the content shown by reference numeral 132, although in... Figure 13 The content shown in (a) is not displayed in the current frame interface, but it is content created previously and has been drawn and cached by the memo application. During the process of generating the next frame of the interface to be displayed, the rendering service can also directly obtain the texture corresponding to the content shown in reference numeral 132 from the cache, and generate the interface to be displayed through texture reuse.

[0291] In other examples, if the electronic device has not displayed the content shown by reference numeral 132 for an extended period, it may have already removed this content from its cache. The electronic device may also instruct a memo application or rendering service to draw this content during the rendering task allocation process. For example, if this content is static, the electronic device can draw and cache it through a memo application. Alternatively, if this content is dynamic, the electronic device can draw it through a rendering service.

[0292] In this way, the application draws part of the content and generates corresponding textures for caching. The subsequent rendering service can directly reuse the cached textures during the rendering tree process, thereby reducing the rendering pressure on the rendering service and improving rendering efficiency.

[0293] The above text introduces the basic principles of load balancing rendering. The following text describes the detailed implementation process of load balancing.

[0294] In some embodiments, during the display of an electronic device's interface, the content displayed on the interface to be displayed during multi-frame interface switching includes static content and dynamic content. Static content refers to content that does not automatically change over time in consecutive multi-frame interfaces, such as plain text content and images without effects. Content that has already been rendered in previous frames and can be reused in the interface to be displayed is considered static content. Dynamic content refers to content that automatically changes over time in consecutive multi-frame interfaces, such as GIFs and images with looping animations.

[0295] In some examples, the application can be used to draw static content, while the rendering service can be used to draw dynamic content. This way, during the rendering of the interface to be displayed, the rendering service only needs to draw the dynamic content, rather than drawing the static content first and then the dynamic content, thus improving rendering efficiency.

[0296] In some embodiments, the application traverses the application's views in the interface to be displayed, determines and saves the drawing operations for each view, and records the view and the drawing operations involved in that view into the drawing instruction list of the rendering node in the rendering node tree.

[0297] In some examples, the application marks the views corresponding to static content and / or the views corresponding to dynamic content in the drawing instruction list. This allows the application and rendering service to determine the content to be drawn and complete the drawing of their respective rendering nodes in the rendering node tree based on the drawing instruction list.

[0298] In other examples, during the recording of the drawing instruction list, the application records the content that needs to be drawn by the application in drawing instruction list A, such as recording the view corresponding to static content and the drawing operations involved in that view in the drawing instruction list A of the rendering node in the rendering node tree. The application records the content that needs to be drawn by the rendering service in drawing instruction list B, such as recording the view corresponding to dynamic content and the drawing operations involved in that view in the drawing instruction list B of the rendering node in the rendering node tree. Subsequently, the application draws the rendering nodes involved in static content according to drawing instruction list A, and the rendering service draws the rendering nodes involved in dynamic content according to drawing instruction list B. In drawing instruction list A, the rendering nodes corresponding to dynamic content do not have their corresponding drawing operations recorded, but may include a marker 1, which indicates that drawing is not required. In drawing instruction list B, the rendering nodes corresponding to static content do not have their corresponding drawing operations recorded, but may include a marker 2, which indicates that drawing is not required, and the relevant texture can be directly obtained from the cache; for example, marker 2 includes the storage location of the texture.

[0299] For example, such as Figure 14 As shown, the interface to be displayed on an electronic device includes at least one application, such as application 1, application 2, ..., application N. Each application draws static content through its respective main thread and generates a bitmap based on the rendering node tree by calling the GPU through its corresponding rendering thread. Afterward, the application can process the bitmap into a texture and send it to the rendering service. In some examples, the rendering service can also receive a list of drawing instructions sent by each application. The rendering service can then draw dynamic content according to this list of drawing instructions and generate the corresponding bitmap by calling the GPU. Furthermore, the rendering service reuses the textures corresponding to the static content during the drawing process. Thus, after traversing the entire rendering node tree corresponding to the entire list of drawing instructions, the rendering service can generate the interface to be displayed.

[0300] For example, such as Figure 15As shown, the foreground applications for the interface to be displayed include Application 1 and Application 2. After recording a list of drawing instructions, Application 1 draws and renders the static content in the interface to be displayed according to the list, generating corresponding static layers, such as Static Layer 1 and Static Layer 2. Then, the application can write Static Layer 1 and Static Layer 2 into node memory for caching. Application 1 can also write the context of Static Layer 1 and Static Layer 2 into node memory so that subsequent rendering services can reuse them. For example, this node memory is, for instance, Input / Output Node (ION) memory, which can be used to cache static layers. ION memory uses a texture storage method (DMA buffer) to reduce CPU / GPU data transfer. In some examples, Application 1 writes the list of drawing instructions into shared memory. For example, shared memory uses a texture storage method, which can reuse existing system paths and reduce inter-process data transfer. Similarly, after recording a list of drawing instructions, Application 2 completes the drawing of static content according to the list, generates static layers, and writes the static layers into node memory. Furthermore, the drawing instruction list is written to shared memory. It should be understood that the method described above for writing static content textures to ION memory is merely an exemplary method of texture transfer between the application and the rendering service. Applications can also transfer textures through various other methods such as inter-process communication (IPC), shared memory, direct memory access (DMA) channels, and flash memory. Similarly, the drawing instruction list for dynamic content can also use other transfer methods. For example, the application can send the generated drawing instruction list directly to the rendering service without first writing it to shared memory. Figure 15 Shared memory may not need to be included.

[0301] During this process, the rendering service can read a list of drawing instructions written by one or more applications from shared memory and trigger drawing. For example, as Figure 15 As shown, the rendering service reads the drawing instruction lists of application 1 and application 2 from shared memory. Then, based on the drawing instruction lists of application 1 and application 2, the rendering service generates a unified drawing instruction list. During the generation of this unified drawing instruction list, the rendering service can remove occluded content. For example, as... Figure 7In the scenario shown, the interface of application 3 partially overlaps with the interface of application 1. The rendering service can then culle the rendering tree points of the occluded portion, avoiding the drawing of these nodes. This does not affect the interface display and reduces the rendering load. In some examples, the rendering service draws the rendering node tree according to a unified list of drawing instructions. During the drawing process, it generates drawing instructions based on the content to be drawn and calls the GPU to complete the rendering, generating layers. Furthermore, during the drawing process, the rendering service directly reads relevant layers (such as textures) from ION memory based on the content to be drawn by the application. After traversing the rendering node tree corresponding to the unified list of drawing instructions, the rendering service can output the interface to be displayed. During the interface display process, continuous animations may be displayed across multiple frames. Therefore, cached static layers can be reused during the drawing of multiple frames, effectively improving rendering efficiency.

[0302] In some examples, generating a uniform list of drawing instructions can also be described as generating a uniform render node tree. This uniform render node tree differs from the render node tree generated in a uniform rendering approach. The uniform render node tree generated using a balanced rendering approach includes render nodes to be drawn, as well as render nodes involving cached content.

[0303] In some examples, during the generation of a unified list of drawing instructions, the rendering service determines that the covered rendering nodes are those involving cached content. It can choose to retrieve this content from the cache or directly discard it. In some examples, textures generated by the application may be partially covered during the display of the interface in subsequent frames. For example, in a multi-window display, if a window is partially covered, the rendering service cannot directly discard the rendering node corresponding to that window's texture. The rendering service discards the rendering node corresponding to the completely covered content. In some examples, if the window (or other content) is partially covered for a longer period, the electronic device can redraw the texture through the application, generating a new texture for the new window. This new texture only includes the uncovered content other than the partially covered content. The application can then replace the old texture corresponding to that window in the cache with this new texture. This way, when the rendering service retrieves the new texture from the cache, it doesn't need to discard it again, reducing data transfer and cache usage.

[0304] In this way, static and dynamic content in the interface to be displayed are rendered separately, reducing the problem of concentrated rendering load within a single frame and lowering the processing time per frame. Furthermore, static content is drawn in parallel across multiple applications, while dynamic content is rendered uniformly, reducing the difficulty of implementing cross-application animations and enabling refined cascading visual effects.

[0305] The above section describes the load balancing process by directly separating static and dynamic content. In other examples, electronic devices can also more flexibly allocate rendering tasks based on the load of applications and / or rendering services.

[0306] In some embodiments, such as Figure 16 As shown, during the rendering process, each application calls GPU resources through its own application process to achieve rendering, while the rendering service calls GPU resources through its rendering process to achieve rendering. Therefore, to avoid applications and the rendering service competing for GPU resources and to prevent excessive concentration of rendering load in either application or rendering service, during load balancing rendering, the rendering service predicts the subsequent load based on the input scene using a prediction algorithm, determining whether a particular application should use load balancing rendering in the next frame, thus achieving load balancing throughout the entire rendering process.

[0307] In some examples, the rendering service can instruct the application on its rendering tasks. For instance, if it's determined that balanced rendering is required, the rendering service might instruct the application to render all static content, some static content, all static content, and some dynamic content for the next frame. The static content could include, for example, text, scalable vector graphics (SVG), subtrees formed by multiple rendering nodes, custom Canvas, or subtrees. Conversely, if it's determined that balanced rendering is not used, the rendering service might instruct the application to render all content for the next frame, or simply output a list of drawing instructions without rendering.

[0308] In some examples, the rendering service determines whether to use a load-balanced rendering method and how to allocate rendering tasks based on various input scenarios such as resource utilization, historical load, number of foreground applications, application task load, CPU load, GPU load, hardware performance of electronic devices, and battery level of electronic devices.

[0309] For example, if the rendering service has a historically high load, some of the workload can be distributed to applications to reduce the load on the rendering service. Or, if there are many foreground applications, the rendering service can handle more rendering tasks to reduce the applications' consumption of GPU resources.

[0310] In some examples, the prediction algorithms include various time series prediction algorithms such as moving average models and exponential smoothing models.

[0311] For example, such as Figure 17As shown, the foreground applications for the interface to be displayed include Application 1, Application 2, and Application 3. The rendering service predicts, based on the input scenario, that Application 1 and Application 2 will need to use balanced rendering in the next frame, while Application 3 will not. For example, the rendering service may instruct Application 1 and Application 2 to render static content, while Application 3 does not need to perform any rendering tasks.

[0312] like Figure 17 As shown, after generating the drawing instruction list, Application 1 completes the drawing of static content according to the drawing instruction list, and calls the GPU to render the drawn rendering node tree, generating static layer 1 and static layer 2. Afterwards, Application 1 can write static layer 1 and static layer 2 into the node memory. Application 1 can also write the drawing instruction list corresponding to dynamic content into shared memory.

[0313] After generating a drawing instruction list, Application 2 draws the static content according to the list and calls the GPU to render the drawn rendering node tree, generating a static layer. Application 2 can then write the static layer to the node memory. Application 2 can also write the drawing instruction list corresponding to dynamic content to shared memory.

[0314] After application 3 generates the drawing instruction list, it can directly write the drawing instruction list into shared memory without performing drawing and rendering. Then, the rendering service reads the drawing instruction list from shared memory, integrates it, and traverses the rendering node tree starting from the root node. It draws and renders the nodes that need to be drawn and rendered, and for nodes that have already been drawn and rendered, it directly reads the corresponding layers from the node memory to generate the interface to be displayed.

[0315] In this way, during the interface rendering process, electronic devices can flexibly allocate rendering tasks according to the load, thereby improving rendering efficiency and balancing the rendering pressure on applications and rendering services.

[0316] Based on the above, taking the example of a front-end application including application 1 and application 2, where the application performs rendering tasks for static content and the rendering service performs rendering tasks for dynamic content, the complete process of generating the interface to be displayed will be introduced.

[0317] Figure 18 This is a schematic diagram of the scene to be displayed provided in the embodiments of this application.

[0318] like Figure 18As shown, the main thread of the rendering process allocates rendering tasks based on the status of each application process and the rendering process itself. For example, the rendering process instructs application process 1 and application process 2 to perform a balanced rendering process and issues rendering tasks to application process 1 and application process 2 respectively. After receiving the rendering tasks, different applications can generate a rendering node tree through the main thread (such as the UI thread), and mark the rendering nodes that need to be drawn by themselves and / or the rendering nodes that need to be drawn by the rendering process in the rendering node tree according to the rendering tasks. Then, the main thread can pass the rendering node tree to the rendering thread of the application process and the rendering thread of the rendering process to trigger the execution of the drawing process. For example, application process 1 generates rendering node tree 1 through the main thread and sends rendering node tree 1 to the rendering thread of application process 1 and the rendering thread of the rendering process. After receiving rendering node tree 1, the rendering thread of application process 1 can complete the drawing of some rendering nodes according to the markings in it and generate bits. Figure 1 Afterwards, application process 1 can use this bit... Figure 1 Send to the rendering process. It should be understood that application process 1 can use this bit... Figure 1 After being processed into texture 1, it is cached. The rendering process can then retrieve texture 1 from the cache, thereby completing the bitwise operations. Figure 1 Similarly, application process 2 generates rendering node tree 2 through the main thread and sends it to the rendering thread of application process 2 and the rendering thread of the rendering process. After receiving rendering node tree 2, the rendering thread of application process 2 can complete the drawing of some rendering nodes according to the tags in it and generate bits. Figure 2 .

[0319] During this process, after the rendering process receives rendering node tree 1 sent by application process 1 and rendering node tree 2 sent by application process 2, it can call the rendering thread to execute the drawing and rendering process. The rendering thread can generate a new root node, and then connect rendering node tree 1 and rendering node tree 2 to the root node to obtain the target rendering node tree. Afterwards, the rendering thread can render the target bitmap based on the target rendering node tree. During the rendering process, if the rendering node does not involve cached content, the rendering thread can render that node. If the rendering node involves cached content, it can retrieve the required bitmap from the cache. Figure 1 or position Figure 2 Then, the electronic device can display the target bitmap.

[0320] In the process of merging the rendering node trees of one or more applications into a target rendering node tree, the rendering process can determine the animation effects of each layer and change the child nodes in the target rendering node tree according to the animation effects of each application. This allows the rendering process to directly generate the target bitmap based on the target rendering node tree without performing off-screen rendering.

[0321] In some examples, during balanced rendering, the electronic device merges the rendering node trees of one or more applications into a target rendering node tree. The rendering process can obtain the Z-order of the layers corresponding to the rendering node trees of each application during the rendering process based on the target rendering node tree, and use the Z-order as the Z-order of the rendering node tree. The Z-order can be used to identify the hierarchical relationship of each layer.

[0322] In some examples, electronic devices can eliminate drawing operations for child nodes of bitmaps that are not displayed or do not affect display based on the Z-order of the target rendering node tree, thereby avoiding overdraw. Specifically, the rendering service can traverse the child nodes of each rendering node tree, determine the position of the view corresponding to each child node on the surface, and, combined with the Z-order of the rendering node tree, identify completely obscured views and their child nodes. The rendering service can then eliminate drawing operations for child nodes corresponding to completely obscured views in the rendering node tree. For example, if the rendering node corresponding to dynamic content in the target rendering node tree is completely obscured, the rendering process can eliminate the drawing operations for that rendering node. Eliminating the drawing operations for a rendering node can be understood as skipping that node during the drawing process and continuing to traverse subsequent child nodes, instead of performing any drawing operations. This reduces unnecessary drawing operations.

[0323] In this way, electronic devices can share the rendering task through application processes and rendering processes, effectively improving rendering efficiency.

[0324] The following section provides an example of the application scenarios for balanced rendering.

[0325] In some embodiments, the electronic device determines, based on user interaction, that a new window needs to be created in the next frame, and this new window includes dynamic content. The electronic device then determines the rendering task for the interface to be displayed based on the current rendering service and / or application load.

[0326] For example, such as Figure 19 As shown in (a), during the display of a chat application, the electronic device detects the user's operation on control 191 and determines that the user has instructed the display to be as shown in (a). Figure 19The menu bar window 192 is shown in (b). Based on the current input scenario, the electronic device determines and instructs the chat application and rendering service to perform a load balancing process. Specifically, the electronic device uses the rendering service to draw and render the dynamic content corresponding to the menu bar window 192, and reuses static layers that the chat application has already drawn and cached in the previous frame. This reduces redundant drawing by the rendering service and improves the efficiency of generating the interface to be displayed. If the menu bar window 192 also includes static content to be drawn, the electronic device can also instruct the chat application to draw this static content and send the generated texture to the rendering service. Thus, the rendering service, combining this texture with the dynamic content drawn by the rendering service, can generate a complete bitmap corresponding to the menu bar window 192.

[0327] In some embodiments, an electronic device launches a new application window based on user input. According to a preset configuration, the electronic device needs to display a launch animation during the launch process. However, during the display of launch animations across multiple consecutive frames, the cascading refresh of overlapping layers may lead to redundant rendering within frames. To address this, a balanced rendering approach is used, distributing rendering tasks between the application and the rendering service, thereby enabling the reuse of some layers.

[0328] For example, such as Figure 20 As shown in (a), during the process of displaying the desktop, the electronic device detects the user's operation on the Smart Life application icon 201 and determines to launch the Smart Life application. Figure 20 As shown in (a)-(c), during the startup process of the Smart Life application, an animation of the Smart Life application icon 201 enlarging is displayed. During this multi-frame animation, the desktop content may not change, while the dynamically changing content includes the gradually enlarging Smart Life application icon. Therefore, the electronic device can perform the rendering tasks involved in the icon's transformation via a rendering service, reusing the desktop static texture cached in ION memory during this process. If, in consecutive frames, the enlargement of the Smart Life application icon only involves a size change without any change in displayed content, the electronic device can also reuse the texture corresponding to the Smart Life application icon drawn by the desktop application, changing the cached texture through off-screen rendering, thereby improving rendering efficiency.

[0329] In this way, by balancing the rendering load, the problem of increased redraw area caused by the cascading of dirty areas in unified rendering is avoided, effectively improving rendering efficiency.

[0330] In some embodiments, rendering vector content such as text and SVG is quite challenging. If the rendering tasks for vector content are all allocated to the rendering service, the rendering service typically uses a serial rendering method, prioritizing the rendering of this vector content before rendering other dynamic content. However, due to the long rendering time, this cannot match the interface refresh rate, potentially leading to frame dropping anomalies.

[0331] For example, such as Figure 21 As shown, during the display of the Notes application on the electronic device, the text content entered by the user is displayed based on the user's actions. In this process, the next frame includes the text content already displayed in the previous frame. Therefore, the electronic device can draw and render the text content through the Notes application, generate a texture corresponding to the text content, and cache this texture. In this way, the rendering service can directly retrieve the texture corresponding to the text content from the cache when rendering the next frame, without having to draw it again.

[0332] In this way, by sharing the rendering tasks of the rendering service with the application, the rendering pressure on the rendering service is effectively reduced, rendering efficiency is improved, and frame dropping anomalies are avoided.

[0333] In some embodiments, the rendering method provided in this application can also be applied to electronic devices such as PCs that can display multiple windows. These electronic devices typically have a large number of application nodes and windows, resulting in significant rendering pressure. By balancing the load, rendering efficiency can be effectively improved.

[0334] For example, such as Figure 22 In the single-window display scenario shown, the PC displays a browser application window to be displayed. This window contains rich content and involves a large number of nodes in the rendering node tree. If the rendering task is performed solely by the rendering service or solely by the browser application, the rendering load will be concentrated, affecting rendering efficiency. Therefore, by using the rendering method provided in this application embodiment, the electronic device can collaboratively perform the rendering task through the browser application and the rendering service, thereby distributing the rendering load and improving rendering efficiency.

[0335] For example, such as Figure 23In the multi-window display scenario shown, the PC displays an interface with multiple windows, among which the focus window is the browser application window. With a large number of windows, relying solely on a rendering service or the browser application to perform rendering tasks would concentrate the rendering load and affect rendering efficiency. Therefore, using the rendering method provided in this application embodiment, the electronic device can collaboratively perform rendering tasks through the applications and rendering services corresponding to each window, thereby distributing the rendering load and improving rendering efficiency. For example, the rendering service only needs to draw the content corresponding to the focus window, while reusing the textures of other windows that do not refresh the display, thus effectively improving rendering efficiency.

[0336] The rendering scenarios to which the rendering method provided in the embodiments of this application can be applied have been described above. The following describes the various ways that can be implemented in the rendering method implementation process.

[0337] In some embodiments, the content to be drawn can be divided into windows, controls, and properties within controls, etc., according to different granularities. As mentioned above, the division between static and dynamic content can be based on the granularity of windows, controls, or properties within controls.

[0338] For example, such as Figure 23 In the multi-window scenario shown, the electronic device launches a browser application based on user input. The interface to be displayed includes the browser application window, such as the focus window. All or part of the content in the browser application window can be dynamic content. For example, if the focus window displays a GIF image, that GIF image is dynamic content. Or, if the focus window displays a control with animation effects such as gradients, the dynamic content in the interface to be displayed includes a property of that control.

[0339] For example, each UI component's corresponding drawing control may include at least one corresponding rendering node. In some examples, where the drawing control corresponds to multiple rendering nodes, each of the multiple rendering nodes may correspond to different properties. Static properties include, for example, backgrounds related only to the control's content, such as color and images; content, such as text, images, and drawing commands; foregrounds, such as foreground color; and overlays, i.e., the topmost content. Dynamic properties include, for example, geometry related to animation and visual effects, such as size, position, and affine transformations; and appearance properties, such as transparency, shadow, and edge width.

[0340] In some examples, non-cacheable properties include blur / filter effects, which are properties that depend on the content being drawn beforehand, such as blurring or taking the background color. In these cases, these properties need to be classified as dynamic properties.

[0341] For example, such as Figure 24 As shown, in the rendering subtree corresponding to a certain return control, the drawing order of this subtree is geometry, background, content, child nodes, foreground, and top-level content. This return control can implement a gradient color animation effect, which depends on the change of the foreground content as shown in Figure 241. Therefore, the static attributes of this control include geometry, background, content, and child nodes, while the dynamic attributes include foreground and top-level content. The electronic device can allocate the content corresponding to the static attributes to the application for drawing, and the application can request a texture cache for this content to store the generated attribute textures. The electronic device can allocate the content corresponding to the dynamic attributes to the rendering service for drawing. Subsequently, the rendering service can reuse the attribute textures stored by the application during the drawing process.

[0342] In this way, electronic devices can flexibly allocate rendering tasks according to different granularities based on the actual display of the content on the interface to be displayed, thereby achieving fine-grained management of rendering tasks and effectively improving rendering efficiency.

[0343] In some embodiments, the rendering node tree corresponding to the interface to be displayed can contain rendering nodes that involve cached content, such as single nodes, multiple unrelated nodes, or node subtrees. A node subtree includes multiple related nodes. Related nodes indicate that changes to the parent node will affect the child nodes. For example, a node subtree might correspond to a complete control or a portion of a control's properties.

[0344] In some examples, a single node can request a cache independently. A subtree of nodes formed by multiple related nodes can request a single cache.

[0345] For example, such as Figure 25 The rendering node tree shown illustrates how an electronic device assigns rendering tasks to applications for certain nodes and caches the corresponding textures. Nodes A and B are single nodes; an application can request a cache block corresponding to node A and a cache block corresponding to node B. For example, as shown... Figure 25 As shown, nodes C and D form a group of nodes, such as nodes C and D having a parent-child relationship, but nodes C and D are not directly related. Therefore, the application can request a cache block corresponding to node C and a cache block corresponding to node D. As another example, ... Figure 25 As shown, nodes E, F, and G are multiple related nodes that can form a subtree. Therefore, the application can request a cache to store all the content corresponding to this subtree.

[0346] For example, such as Figure 13In the illustrated memo application scenario, each memo entry corresponds to a rendering node. For example, memo entry 1 corresponds to rendering node 1, memo entry 2 corresponds to rendering node 2, ..., memo entry 8 corresponds to rendering node 8. In some examples, the memo application may allocate a cache for each memo entry. In other examples, the memo application may allocate a cache for memo entries 1 through 8.

[0347] The requested cache can be used to cache the drawn textures of the corresponding node, such as textures corresponding to a set or all attributes. The requested cache can also be described as requesting a texture, requesting a texture cache, etc.

[0348] In this way, electronic devices can allocate rendering tasks according to the actual rendering situation. Furthermore, electronic devices can request a texture for a node subtree, which can effectively reduce texture allocation and management overhead.

[0349] In some embodiments, as described above, the rendering node tree includes node subtrees. Generating node subtree-level textures can effectively reduce the overhead of texture allocation, release, and uploading. Then, the application can traverse each rendering node in the rendering node tree to obtain the optimal subtree included within it.

[0350] For example, in such Figure 13 In the sliding scene of the memo application shown, the rendering node tree corresponding to this interface includes a large number of rendering nodes corresponding to static content. If each rendering node generates a texture separately, a large amount of ION memory allocation and release overhead will be generated during the texture generation and caching process, which may also cause ION memory fragmentation.

[0351] In some examples, the application traverses the rendering node tree starting from the root node to determine the first rendering node with animation / visual effects attributes. Then, the application determines the number of child nodes for that rendering node. If the number of child nodes exceeds a preset threshold, a parent node with that rendering node as the optimal subtree can be generated. This optimal subtree can significantly reduce texture allocation overhead. For example, if a rendering node has animation / visual effects attributes, the rendering service needs to draw that node after each frame refresh. If the child nodes of this rendering node are not pre-drawn, the rendering service still needs to draw the child nodes in node order each time the rendering node is drawn, resulting in a large number of unnecessary draws and increasing the rendering load. However, by generating the optimal subtree, the application process can directly draw the optimal subtree and cache the corresponding texture. Thus, after the rendering process draws the rendering node with animation / visual effects attributes, it can directly reuse the cached texture for its child nodes, thereby reducing drawing. Furthermore, by caching the optimal subtree onto a single texture, the rendering service only needs to read the optimal subtree texture once when requesting it, reducing memory release overhead.

[0352] In some examples, the application determines the subtree root node based on identifiers on the rendered nodes. These identifiers may include, for example, labels, attributes, or control names. Alternatively, the application may determine the subtree root node based on the subtree's drawing layer or content. For instance, the application might identify a static layer based on its attributes or labels, and then determine the root node of that static layer as the subtree root node. Or, the application might determine the subtree root node based on the subtree's state across multiple frame views. For example, if the content of a certain subtree does not update across multiple consecutive frame views, the application can determine that this subtree is a node subtree, and its root node is the subtree root node of the optimal subtree to be determined.

[0353] For example, such as Figure 26 As shown, after obtaining the rendering node tree, the application traverses the tree starting from the root node, determining that nodes 3 and 5 are the root nodes of their respective subtrees. Therefore, the application can identify the optimal subtree by including the root node of that subtree and the multiple nodes associated with it. For example, as... Figure 26 As shown, the optimal subtree includes subtree 1 with node 3 as the root node and subtree 2 with node 5 as the root node.

[0354] In this way, by dividing the optimal subtree, the application can request a texture corresponding to the node subtree, which can effectively reduce the texture allocation overhead compared to requesting a texture for every node in the node subtree.

[0355] In some embodiments, as described above, each rendering node needs to request its own corresponding texture during the drawing process. Therefore, if there are many rendering nodes, the problem of excessive texture requests still exists. Based on this, in some embodiments, for subtrees that cannot be divided into optimal subtrees, the application can also request a single texture, thereby effectively reducing the number of texture requests through subtree-level texture requests. In other embodiments, for any multiple rendering nodes in the rendering node tree, the number of texture requests can be reduced by requesting a single texture. For example, for multiple rendering nodes belonging to different rendering subtrees, the application can also request a single texture cache to cache the multiple textures generated after these multiple rendering nodes are drawn, thus eliminating the need to request multiple texture caches.

[0356] In some examples, after generating multiple textures, the application (or application process) can use heuristic algorithms to arrange these textures appropriately in a cache, thus achieving the storage of multiple textures with a single texture cache request. A heuristic algorithm is an algorithm that finds solutions to problems through rules of thumb, intuition, or approximate reasoning. Heuristic algorithms are often used to solve complex or difficult optimization problems, especially those where an exact algorithm cannot find the optimal solution in a reasonable time.

[0357] For example, such as Figure 27 As shown, the subtree includes a root node, subtree node 1, subtree node 2, subtree node 3, and subtree node 4. During the rendering of this subtree, the application generates four textures: texture 1, texture 2, texture 3, and texture 4. If these four textures are cached separately, the application needs to request texture cache from ION memory four times and write the textures to ION memory four times after rendering. Therefore, the application can use a heuristic algorithm to arrange these four textures appropriately, thereby achieving the goal of requesting a single texture cache.

[0358] In some examples, the application obtains the offset of each texture in the texture cache and records that offset into the corresponding texture. Alternatively, the application can use one of the textures as a position reference, obtain the offsets of other textures relative to that texture, and record those offsets into the texture cache.

[0359] Subsequently, after the rendering service reads this texture from ION memory, it can determine that this texture includes multiple individual textures, and can determine the cache location of each of these textures using offsets, thus successfully obtaining the required texture. In some examples, the texture used as the location reference can be the largest texture, the smallest texture, the outermost texture, etc., in the combined texture.

[0360] In this way, by combining multiple textures corresponding to a subtree to generate an Atlas texture, the number of texture requests and writes for that subtree is reduced to one, effectively improving the efficiency of texture request and management.

[0361] Figure 28 This is a flowchart illustrating a rendering method provided in an embodiment of this application. It should be noted that this method does not rely on... Figure 28 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. This method can be applied to electronic devices and includes the following steps:

[0362] S2801, The first application process generates a first rendering node tree; wherein, the first rendering node tree is used to draw the first application interface of the first application process, and the first rendering node tree includes the rendering node corresponding to the first content in the first application interface, and the rendering node corresponding to the second content in the first application interface.

[0363] In some embodiments, when an electronic device is about to display a first user interface, the first application process corresponding to the first application interface in the first user interface can generate a first rendering node tree for the first application interface. The first application interface can record the views in the first application interface and the drawing operations involved in those views into the drawing instruction list of the rendering nodes in the first rendering node tree. Subsequently, the first application process and the rendering process can draw the first application interface according to the drawing instruction list of the rendering nodes in the first rendering node tree.

[0364] In some embodiments, the first application interface may include static content and dynamic content. Static content refers to content that does not automatically change over time in consecutive interface frames, such as plain text or images without effects. Static content includes content that has already been rendered in previous interface frames and can be reused in the interface to be displayed. Dynamic content refers to content that automatically changes over time in consecutive interface frames, such as GIFs or images with looping animations.

[0365] S2802. The first application process generates the first texture corresponding to the first content based on the first rendering node tree.

[0366] In some embodiments, the rendering process can allocate rendering tasks and determine whether the application process needs to participate in executing the rendering tasks. If the rendering process determines that the application process needs to participate in the rendering tasks, it can also allocate rendering tasks to the application process, for example, indicating which rendering nodes the application process needs to draw.

[0367] In some examples, the rendering process determines to issue rendering tasks to the first application process based on one or more of the following: the rendering process's resource utilization, the rendering process's historical load, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the CPU load, the GPU load, the hardware performance of the electronic device, and the battery level of the electronic device. The rendering tasks instruct the first application process to draw the first content.

[0368] In some examples, the first content includes static content in the first application interface, and the second content includes dynamic content in the first application interface. Alternatively, the first content includes a portion of static content in the first application interface, and the second content includes dynamic content in the first application interface as well as another portion of static content.

[0369] For example, based on the high resource utilization of the rendering process, the rendering process determines to allocate the rendering tasks corresponding to a portion of the static content in the first application interface to the first application process. In this way, the rendering process only needs to be responsible for drawing the dynamic content and another portion of the static content in the first application interface, thereby reducing the rendering load of the rendering process.

[0370] For example, if the rendering process has a high resource utilization rate and a high GPU load, it determines to allocate the rendering tasks corresponding to the static content in the first application interface to the first application process. In this way, the rendering process only needs to be responsible for drawing the dynamic content in the first application interface, and does not need to draw the static content in the first application interface, thereby effectively reducing the rendering load of the rendering process.

[0371] In this way, the rendering process can flexibly allocate rendering tasks according to the actual display situation, thereby improving rendering efficiency and balancing the rendering pressure on the application process and the rendering process.

[0372] In some embodiments, the first content or the second content is one or more of the windows, controls, and properties in the first application interface.

[0373] For example, the content to be drawn can be divided into windows, controls, and properties within controls according to different granularities. For instance, the division between static and dynamic content can be based on the granularity of windows, controls, or properties within controls.

[0374] For example, the first content or the second content is, for example, Figure 23 This refers to a window to be drawn in a multi-window scene. For example, the first or second content might be... Figure 22 The control to be drawn in the window of the single-window scene shown. For example, the first content or the second content is... Figure 24The static or dynamic property of the control shown.

[0375] In this way, electronic devices can flexibly allocate rendering tasks according to different granularities based on the actual display of the content on the interface to be displayed, thereby achieving fine-grained management of rendering tasks and effectively improving rendering efficiency.

[0376] In some embodiments, the first application process writes the first texture to the first memory space. Then, the rendering process can read the first texture from the first memory space during the drawing process.

[0377] For example, such as Figure 15 As shown, after recording a list of drawing instructions, application 1 (such as the first application process) draws and renders the static content in the interface to be displayed according to the list of drawing instructions, generating corresponding static layers, such as static layer 1 and static layer 2. Afterwards, the application can write static layer 1 and static layer 2 into node memory for caching. Application 1 can also write the context of static layer 1 and static layer 2 into node memory (such as the first memory space) so that subsequent rendering processes can reuse static layer 1 and static layer 2. For example, this node memory may be ION memory.

[0378] In this way, the application process draws part of the content and generates corresponding textures for caching. Subsequent rendering processes can directly reuse the cached textures during the drawing process, thereby reducing the rendering pressure on the rendering process and improving rendering efficiency.

[0379] In some embodiments, the first texture includes a node texture and / or a subtree-level texture, wherein the node texture corresponds to a rendering node and the subtree-level texture corresponds to a rendering subtree.

[0380] For example, such as Figure 25 The rendering node tree shown illustrates how the application process executes rendering tasks for some nodes and caches the corresponding textures. Nodes A and B are single nodes; the application process can request a cache block corresponding to node A and a cache block corresponding to node B. For example, as shown... Figure 25 As shown, nodes C and D form a group of nodes, such as nodes C and D having a parent-child relationship, but nodes C and D are not directly related. Therefore, the application process can request a cache block corresponding to node C and a cache block corresponding to node D. As another example, ... Figure 25 As shown, nodes E, F, and G are multiple related nodes that can form a subtree. Therefore, the application process can request a cache to store all the content corresponding to this subtree.

[0381] In this way, electronic devices can request a texture from the node subtrees, which can effectively reduce the overhead of texture request and management.

[0382] In some embodiments, the parent node of the rendering subtree is a rendering node with animation / visual effects attributes, and the number of rendering nodes in the rendering subtree is greater than or equal to a number threshold. And / or, the root node of the rendering subtree includes a first identifier indicating that the rendering subtree is used to generate textures. And / or, the layer containing the rendering subtree includes a second identifier indicating that rendering nodes in the layer are used to generate textures.

[0383] In some examples, the first identifier may include a label, attribute, control name, etc. The second identifier may indicate, for example, that the layer is a static layer used to draw static content. Alternatively, the second identifier may indicate, for example, that the content corresponding to the rendered subtree is static content.

[0384] For example, such as Figure 26 As shown, after the application process obtains the rendering node tree, it traverses the tree starting from the root node and determines that nodes 3 and 5 are the root nodes of the subtree. Therefore, the application process can determine the optimal subtree based on the root node of that subtree and the multiple nodes associated with it. For example, as... Figure 26 As shown, the optimal subtree includes subtree 1 with node 3 as the root node and subtree 2 with node 5 as the root node.

[0385] In this way, the application process can effectively reduce the texture allocation overhead by dividing the optimal subtree and requesting a texture corresponding to the rendering subtree, compared to requesting a texture for every node in the rendering subtree.

[0386] In some embodiments, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into the first memory space by: requesting a first texture cache from the first memory space based on the first node texture and the second node texture. The rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes the result into the first texture cache.

[0387] In some examples, after generating multiple textures, the first application process can use heuristic algorithms to arrange these textures appropriately in the cache, thus achieving storage of multiple textures with a single texture cache request. In some examples, these multiple textures may correspond to different rendering nodes within the same rendering subtree; or the rendering nodes corresponding to these multiple textures may be distributed across different rendering subtrees; or some of the rendering nodes corresponding to these multiple textures may be distributed across different rendering subtrees, while others may be distributed across the same rendering subtree. In other words, the first application process does not need to consider the origin of the multiple textures when merging them into a single large texture.

[0388] For example, such as Figure 27 As shown, the subtree includes a root node, subtree node 1, subtree node 2, subtree node 3, and subtree node 4. During the rendering of this subtree, the application process generates four textures: texture 1, texture 2, texture 3, and texture 4. If these four textures are cached separately, the application process needs to request texture caches from the first memory space four times and write the textures to the first memory space four times after rendering. Therefore, the application process can use a heuristic algorithm to arrange these four textures appropriately, thereby achieving the goal of requesting a single texture cache.

[0389] In this way, by combining multiple textures to generate a large texture, the number of texture requests and writes is reduced to one, effectively improving the efficiency of texture request and management.

[0390] In some embodiments, the first node texture carries first offset data of the first node texture in the first texture cache, and the second node texture carries second offset data of the second node texture in the first texture cache.

[0391] For example, the application process obtains the offset of each texture in a large, stitched texture within the texture cache and records the offset in the corresponding texture. Alternatively, the application process can use one of the textures as a positional reference, obtain the offsets of other textures relative to that texture, and record the offsets in the texture cache.

[0392] In this way, after the rendering process reads the texture from the first memory space, it can determine that the texture includes multiple individual textures, and can determine the cache location of each of the included textures through the offset, thus successfully obtaining the required texture.

[0393] S2803. The rendering process draws based on the first texture and the first rendering node tree generated by the first application process, and generates the first application interface of the first application process.

[0394] In some embodiments, after allocating rendering tasks, the rendering process can draw the content to be rendered, such as the second content, according to the first rendering node tree. Furthermore, during the traversal of the first rendering node tree, the rendering process can determine which rendering nodes' corresponding content has been cached, thereby retrieving the texture from the first memory space. Thus, after completing the traversal of the first rendering node tree, the rendering process can generate the first application interface to be displayed.

[0395] In this way, the first content and the second content in the first application interface are drawn by the first application process and the rendering process respectively, which reduces the problem of concentrated rendering load in a single frame and reduces the processing time of a single frame.

[0396] Furthermore, by rendering static and dynamic content separately in the interface to be displayed, the problem of concentrated rendering load within a single frame is reduced, thus lowering the processing time per frame. Additionally, static content is rendered in parallel across multiple applications. Dynamic content is rendered uniformly, reducing the difficulty of implementing cross-application animations and enabling refined cascading visual effects.

[0397] In some embodiments, after the first application process generates the first rendering node tree, it writes the first rendering node tree into a second memory space. The rendering process then reads the first rendering node tree from the second memory space.

[0398] For example, such as Figure 15 As shown, application 1 (such as the first application process) writes the list of drawing instructions corresponding to the first rendering tree node into shared memory (such as the second memory space). This list of drawing instructions includes drawing instructions for dynamic content (such as the second content). In some examples, the application process can also send the first rendering node tree to the rendering process via various transfer methods such as IPC, DMA channels, and flash memory.

[0399] In this way, the rendering process can obtain the first rendering node tree generated by the first application process, thereby realizing the drawing of the second content.

[0400] In some embodiments, the rendering process can cache the generated textures after drawing the second content. This allows the rendering process to reuse these textures during subsequent drawing processes. For example, if the rendering process draws dynamic content frame by frame, the rendering load is high. Therefore, the rendering process can choose to draw dynamic content at intervals between frames. In the frames between drawing operations, the rendering process can reuse the textures of the dynamic content cached in previous frames, thereby reducing the rendering load.

[0401] In some embodiments, the first user interface of the electronic device includes a first application interface, and the electronic device displays the first user interface. A first application process generates a second rendering node tree. The second rendering node tree is used to draw the second application interface of the first application process, and includes rendering nodes corresponding to first content and third content in the second application interface. The rendering process draws based on the first texture and the second rendering node tree generated by the first application process to generate the second application interface of the first application process. The electronic device displays the second user interface; wherein the second user interface includes the second application interface.

[0402] For example, during the display of a first user interface, an electronic device may refresh the interface of a first application process based on user actions, or automatically refresh the interface of the first application process. In response to the interface refresh, the electronic device draws a new interface to be displayed, such as a second user interface. In some examples, the rendering process can allocate rendering tasks for the second application interface based on the current display scene. For example, if the second application interface still includes the first content, then it is not necessary to redraw the first content during the generation of the second application interface. The rendering process only needs to draw the third content and reuse the first texture corresponding to the first content that has been drawn in previous frames to output the second application interface. Here, the third content is, for example, dynamic content in the second application interface. Or the third content is a combination of static and dynamic content in the second application interface.

[0403] In this way, during the multi-frame interface switching display process, the rendering process can directly reuse the cached textures, thereby reducing the rendering pressure on the rendering service and improving rendering efficiency.

[0404] In some embodiments, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process. Before the rendering process draws based on a first texture and a first rendering node tree generated by the first application process to generate the first application interface of the first application process, the second application process generates a third rendering node tree; wherein, the third rendering node tree is used to draw the third application interface of the second application process, and the third rendering node tree includes rendering nodes corresponding to a fourth content in the third application interface and rendering nodes corresponding to a fifth content in the third application interface. The second application process generates a second texture corresponding to the fourth content based on the third rendering node tree. The rendering process draws based on the first texture and the first rendering node tree generated by the first application process to generate the first application interface of the first application process, including: the rendering process generating a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the second texture, and the target rendering node tree to generate the first user interface.

[0405] In some examples, the rendering process generates a target rendering node tree based on the first and third rendering node trees, including: removing the drawing operations of the rendering nodes corresponding to the fourth content in the target rendering node tree when the fourth content is completely covered by the first application interface; and / or removing the drawing operations of the rendering nodes corresponding to the fifth content in the target rendering node tree when the fifth content is completely covered by the first application interface.

[0406] For example, such as Figure 23 In the multi-window scenario shown, the first user interface includes a first application interface of a first application process and a third application interface of a second application process. The first application interface is, for example, the focused window, and the third application interface is, for example, one of the covered windows. Then, as... Figure 14 As shown, the rendering process can allocate rendering tasks to multiple application processes during the task allocation process. After obtaining the rendering node trees sent by multiple application processes, the rendering process can generate a new target rendering node tree based on these trees, and then perform drawing based on this target rendering node tree. During the drawing process, for rendering nodes involving cached content, the rendering process can retrieve textures previously drawn and cached by a certain application process from the first shared space. For rendering nodes not involving cached content, the rendering process can perform drawing. Afterwards, the rendering process can output the first user interface.

[0407] For example, such as Figure 18 As shown, the main thread of the rendering process allocates rendering tasks based on the status of each application process and the rendering process itself. For example, the rendering process instructs application process 1 (e.g., the first application process) and application process 2 (e.g., the second application process) to perform a balanced rendering process, and distributes rendering tasks to application process 1 and application process 2 respectively. After receiving the rendering tasks, different applications can generate a rendering node tree through the main thread (e.g., the UI thread), and mark the rendering nodes that need to be drawn by themselves and / or the rendering nodes that need to be drawn by the rendering process in the rendering node tree according to the rendering tasks. Then, the main thread can pass the rendering node tree to the rendering thread of the application process and the rendering thread of the rendering process to trigger the execution of the drawing process. For example, application process 1 generates rendering node tree 1 (e.g., the first rendering node tree) through the main thread, and sends rendering node tree 1 to the rendering thread of application process 1 and the rendering thread of the rendering process. After receiving rendering node tree 1, the rendering thread of application process 1 can complete the drawing of some rendering nodes according to the markings in it, and generate bitmaps. Figure 1 Afterwards, application process 1 can use this bit... Figure 1 Send to the rendering process. It should be understood that application process 1 can use this bit... Figure 1After being processed into Texture 1 (such as the first texture), it is cached. The rendering process can retrieve this Texture 1 from the cache, thereby completing the bitwise operations. Figure 1 Similarly, application process 2 generates rendering node tree 2 (such as the third rendering node tree) through the main thread and sends this rendering node tree 2 to the rendering thread of application process 2 and the rendering thread of the rendering process. After receiving the rendering node tree 2, the rendering thread of application process 2 can complete the drawing of some rendering nodes according to the tags in it and generate bits. Figure 2 , such as the second texture.

[0408] During this process, after the rendering process receives rendering node tree 1 sent by application process 1 and rendering node tree 2 sent by application process 2, it can call the rendering thread to execute the drawing and rendering process. The rendering thread can generate a new root node, and then connect rendering node tree 1 and rendering node tree 2 to the root node to obtain the target rendering node tree. Afterwards, the rendering thread can perform drawing based on the target rendering node tree. During the rendering process, if the rendering node does not involve cached content, the rendering thread can render that node. If the rendering node involves cached content, it can retrieve the required bits from the cache. Figure 1 or position Figure 2 Examples include the first texture and the second texture. After the rendering process is complete, the electronic device can display the first user interface.

[0409] In this way, the rendering process can combine the rendering node tree of the interfaces of multiple application processes to be displayed, complete the drawing of the user interface, and achieve a balance of rendering pressure between multiple application processes and the rendering process.

[0410] Furthermore, multiple application processes draw some content in parallel, while the rendering process draws another part of the content in a unified manner, reducing the difficulty of implementing cross-application animation effects and achieving refined cascading visual effects.

[0411] For example, during the generation of the target render tree, if the rendering process determines that the content corresponding to certain render nodes is completely overwritten, the rendering process can remove these render nodes from the target render node tree without drawing or retrieving cached textures. It should be understood that removing a render node from the tree does not necessarily mean drawing the corresponding render node.

[0412] In this way, drawing is reduced without affecting the display, thereby improving drawing efficiency.

[0413] Figure 29 This is a flowchart illustrating another rendering method provided in an embodiment of this application. It should be noted that this method does not rely on... Figure 29 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0414] S2901, The first application process draws the first part of the rendering task and generates the first texture; the first texture includes static content in the first application interface; wherein, the first texture or at least a part of the static attributes corresponding to the first texture are cached.

[0415] The first part of the rendering task or the second part of the rendering task corresponds to one or more of the windows, controls, and properties in the first application interface.

[0416] In this way, electronic devices can flexibly allocate rendering tasks according to different granularities based on the actual display of the content on the interface to be displayed, thereby achieving fine-grained management of rendering tasks and effectively improving rendering efficiency.

[0417] In some embodiments, before the first application process draws the first part of the rendering task and generates the first texture, the rendering process assigns the first part of the rendering task to the first application process for drawing based on one or more of the following: the rendering process's resource utilization, the rendering process's historical load, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the CPU load, the GPU load, the hardware performance of the electronic device, the battery level of the electronic device, and the first rendering node tree.

[0418] In this way, the rendering process can flexibly allocate rendering tasks according to the actual display situation, thereby improving rendering efficiency and balancing the rendering pressure on the application process and the rendering process.

[0419] S2902, The rendering process divides the first rendering node tree into a first part of the rendering task and a second part of the rendering task according to the first rendering node tree of the first application interface of the first application process to be displayed; the second part of the rendering task corresponds to the dynamic content in the first application interface.

[0420] S2903. The rendering process draws based on the cached first texture or at least some static attributes, and the second part of the rendering task, to generate the first application interface.

[0421] For example, such as Figure 15 As shown, in the rendering node tree of the first application interface of the application 1 to be displayed (such as the first application process), the first part of the rendering tasks is for drawing static content, and the second part of the rendering tasks is for drawing dynamic content. Therefore, the rendering process allocates rendering tasks according to the current display scene, assigning the first part of the rendering tasks to the first application process. The first application process can execute the drawing of the first rendering tasks, generating the first texture, such as the first texture including static layer 1 and static layer 2. The rendering process can execute the drawing of dynamic content and retrieve the first texture from the node memory, thereby generating the first application interface.

[0422] In this way, the static and dynamic content in the first application interface to be displayed are rendered separately, reducing the problem of concentrated rendering load within a single frame and reducing the processing time per frame.

[0423] Furthermore, after allocating rendering tasks, the rendering process performs the drawing. During the drawing process, the rendering nodes traverse the first rendering node tree, identifying rendering nodes that involve cached content, which can directly access the cached content, such as the first texture; and identifying rendering nodes that do not involve cached content, which can be drawn, such as drawing the second content. In this way, the first application process shares a portion of the rendering load with the rendering process.

[0424] In some embodiments, the first texture includes a node texture and / or a subtree-level texture, wherein the node texture corresponds to a rendering node and the subtree-level texture corresponds to a rendering subtree.

[0425] In this way, electronic devices can request a texture from the node subtrees, which can effectively reduce the overhead of texture request and management.

[0426] In some embodiments, the parent node of the rendering subtree is a rendering node with animation / visual effects attributes, and the number of rendering nodes in the rendering subtree is greater than or equal to a number threshold. And / or, the root node of the rendering subtree includes a first identifier indicating that the rendering subtree is used to generate textures. And / or, the layer containing the rendering subtree includes a second identifier indicating that rendering nodes in the layer are used to generate textures.

[0427] In this way, the application process can effectively reduce the texture allocation overhead by dividing the optimal subtree and requesting a texture corresponding to the rendering subtree, compared to requesting a texture for every node in the rendering subtree.

[0428] In some embodiments, after the first application process draws a first portion of the rendering task and generates a first texture, the first application process writes the first texture or at least some of its static attributes into a first memory space. The rendering process then reads the first texture or at least some of its static attributes from the first memory space.

[0429] In some examples, static content also includes static attributes. During texture generation, the first application process can cache some of the textures corresponding to these static attributes, thus enabling more granular management.

[0430] In this way, the application process draws part of the content and generates corresponding textures for caching. Subsequent rendering processes can directly reuse the cached textures during the drawing process, thereby reducing the rendering pressure on the rendering process and improving rendering efficiency.

[0431] In some embodiments, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into a first memory space by: the first application process requesting a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0432] In some embodiments, the first node texture carries first offset data of the first node texture in the first texture cache, and the second node texture carries second offset data of the second node texture in the first texture cache.

[0433] In this way, by combining multiple textures to generate a large texture, the number of texture requests and writes is reduced to one, effectively improving the efficiency of texture request and management.

[0434] In some embodiments, the rendering process can draw dynamic content directly without generating textures, and during the drawing process, cached static content textures can be used to output the interface to be displayed.

[0435] In this way, static content can be drawn in parallel across multiple applications, while dynamic content can be rendered uniformly, reducing the difficulty of implementing cross-application animation effects and achieving cascading, refined visual effects.

[0436] In some embodiments, the rendering process draws based on the second rendering node tree of the second application interface of the first application process to be displayed, the first texture or at least some static attributes, and the third part of the rendering tasks to generate the second application interface; wherein, the second rendering node tree includes the first part of the rendering tasks and the third part of the rendering tasks.

[0437] In this way, during the multi-frame interface switching display process, the rendering process can directly reuse the cached textures, thereby reducing the rendering pressure on the rendering service and improving rendering efficiency.

[0438] In some embodiments, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process. Before the rendering process draws based on a cached first texture or at least a portion of static attributes and a second portion of rendering tasks to generate the first application interface, the rendering process assigns a fourth portion of rendering tasks to the second application process for drawing based on the third rendering node tree of the third application interface of the second application process to be displayed. The third rendering node tree includes a fourth portion of rendering tasks and a fifth portion of rendering tasks. The second application process draws the fourth portion of rendering tasks to generate a third texture. The third texture includes static content in the third application interface. At least a portion of the static attributes in the third texture are cached. The rendering process draws based on the cached first texture or at least a portion of static attributes and the second portion of rendering tasks to generate the first application interface, including: the rendering process generating a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the third texture and the target rendering node tree to generate the first user interface.

[0439] In some examples, the rendering process generates a target rendering node tree based on the first and third rendering node trees, including: removing the drawing operations of the fourth rendering task from the target rendering node tree when the content corresponding to the fourth rendering task is completely covered by the first application interface; and / or removing the drawing operations of the fifth rendering task from the target rendering node tree when the content corresponding to the fifth rendering task is completely covered by the first application interface.

[0440] In this way, the rendering process can combine the rendering node tree of the interfaces of multiple application processes to be displayed, complete the drawing of the user interface, and achieve a balance of rendering pressure between multiple application processes and the rendering process.

[0441] Furthermore, multiple application processes draw some content in parallel, while the rendering process draws another part of the content in a unified manner, reducing the difficulty of implementing cross-application animation effects and achieving refined cascading visual effects.

[0442] Optionally, Figure 29 For the specific implementation details of each step in the process shown, please refer to the above. Figure 28 The relevant details in the process shown will not be repeated here.

[0443] Figure 30 This is a flowchart illustrating another rendering method provided in an embodiment of this application. It should be noted that this method does not rely on... Figure 30 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0444] S3001, The first application process draws the first part of the rendering task and generates the first texture; the first texture includes the first static content in the first application interface; wherein, the first texture or at least a part of the static attributes corresponding to the first texture are cached.

[0445] The first part of the rendering task or the second part of the rendering task corresponds to one or more of the windows, controls, and properties in the first application interface.

[0446] In this way, electronic devices can flexibly allocate rendering tasks according to different granularities based on the actual display of the content on the interface to be displayed, thereby achieving fine-grained management of rendering tasks and effectively improving rendering efficiency.

[0447] In some embodiments, before the first application process draws the first part of the rendering task and generates the first texture, the rendering process assigns the first part of the rendering task to the first application process for drawing based on one or more of the following: the rendering process's resource utilization, the rendering process's historical load, the number of foreground applications, the rendering load of the first application process, the task load of the first application process, the load of the central processing unit (CPU), the load of the graphics processing unit (GPU), the hardware performance of the electronic device, the battery level of the electronic device, and the first rendering node tree.

[0448] In this way, the rendering process can flexibly allocate rendering tasks according to the actual display situation, thereby improving rendering efficiency and balancing the rendering pressure on the application process and the rendering process.

[0449] S3002, the rendering process divides the first rendering node tree into a first part of the rendering task and a second part of the rendering task according to the first rendering node tree of the first application interface of the first application process to be displayed; the second part of the rendering task corresponds to the second static content and dynamic content in the first application interface.

[0450] S3003: The rendering process generates the first application interface based on the cached first texture or at least some of the static attributes, and the second part of the rendering task.

[0451] For example, in the first rendering node tree of the first application interface of the first application process to be displayed, the first part of the rendering task is to draw a first static content, and the second part of the rendering task is to draw a second static content and dynamic content. Then, the rendering process allocates rendering tasks according to the current display scene, assigning the first part of the rendering task to the first application process. The first application process can execute the drawing of the first rendering task to generate a first texture. The rendering process can execute the drawing of the second static content and dynamic content, and obtain the first texture from the node memory, thereby generating the first application interface. For example, as... Figure 17As shown, the rendering process can also draw static content of application 3.

[0452] In this way, the rendering process allocates a portion of the static content in the first application interface to be displayed to the first application process for drawing, and allocates another portion of the static and dynamic content to the rendering process for drawing, thereby reducing the problem of concentrated rendering load within a single frame and reducing the processing time per frame.

[0453] In some embodiments, the first texture includes a node texture and / or a subtree-level texture, wherein the node texture corresponds to a rendering node and the subtree-level texture corresponds to a rendering subtree.

[0454] In this way, electronic devices can request a texture from the node subtrees, which can effectively reduce the overhead of texture request and management.

[0455] In some embodiments, the parent node of the rendering subtree is a rendering node with animation / visual effects attributes, and the number of rendering nodes in the rendering subtree is greater than or equal to a number threshold. And / or, the root node of the rendering subtree includes a first identifier indicating that the rendering subtree is used to generate textures. And / or, the layer containing the rendering subtree includes a second identifier indicating that rendering nodes in the layer are used to generate textures.

[0456] In this way, the application process can effectively reduce the texture allocation overhead by dividing the optimal subtree and requesting a texture corresponding to the rendering subtree, compared to requesting a texture for every node in the rendering subtree.

[0457] In some embodiments, after the first application process draws a first portion of the rendering task and generates a first texture, the first application process writes the first texture or at least some of its static attributes into a first memory space. The rendering process then reads the first texture or at least some of its static attributes from the first memory space.

[0458] In this way, the application process draws part of the content and generates corresponding textures for caching. Subsequent rendering processes can directly reuse the cached textures during the drawing process, thereby reducing the rendering pressure on the rendering process and improving rendering efficiency.

[0459] In some embodiments, the first texture includes a first node texture and a second node texture. The first application process writes the first texture into a first memory space by: the first application process requesting a first texture cache from the first memory space based on the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering subtree or different rendering subtrees. The first application process concatenates the first node texture and the second node texture and writes them into the first texture cache.

[0460] In this way, by combining multiple textures to generate a large texture, the number of texture requests and writes is reduced to one, effectively improving the efficiency of texture request and management.

[0461] In some embodiments, the first node texture carries first offset data of the first node texture in the first texture cache, and the second node texture carries second offset data of the second node texture in the first texture cache.

[0462] In this way, after the rendering process reads the texture from the first memory space, it can determine that the texture includes multiple individual textures, and can determine the cache location of each of the included textures through the offset, thus successfully obtaining the required texture.

[0463] In some embodiments, the rendering process draws based on the second rendering node tree of the second application interface of the first application process to be displayed, the first texture or at least some static attributes, and the third part of the rendering tasks to generate the second application interface; wherein, the second rendering node tree includes the first part of the rendering tasks and the third part of the rendering tasks.

[0464] In this way, during the multi-frame interface switching display process, the rendering process can directly reuse the cached textures, thereby reducing the rendering pressure on the rendering service and improving rendering efficiency.

[0465] In some embodiments, the first user interface of the electronic device includes a first application interface of a first application process and a third application interface of a second application process. Before the rendering process draws based on a cached first texture or at least a portion of static attributes and a second portion of rendering tasks to generate the first application interface, the rendering process assigns a fourth portion of rendering tasks to the second application process for drawing based on the third rendering node tree of the third application interface of the second application process to be displayed. The third rendering node tree includes the fourth portion of rendering tasks and a fifth portion of rendering tasks. The second application process draws the fourth portion of rendering tasks to generate a third texture. The fifth texture includes the third static content in the third application interface. At least a portion of the static attributes in the third texture are cached. The rendering process draws based on the cached first texture or at least a portion of static attributes and the second portion of rendering tasks to generate the first application interface, including: the rendering process generating a target rendering node tree based on the first rendering node tree and the third rendering node tree. The rendering process draws based on the first texture and / or the third texture and the target rendering node tree to generate the first user interface.

[0466] In some embodiments, the rendering process generates a target rendering node tree based on the first rendering node tree and the third rendering node tree, including: removing the drawing operations of the fourth rendering task from the target rendering node tree when the content corresponding to the fourth rendering task is completely covered by the first application interface; and / or removing the drawing operations of the fifth rendering task from the target rendering node tree when the content corresponding to the fifth rendering task is completely covered by the first application interface.

[0467] In this way, the rendering process can combine the rendering node tree of the interfaces of multiple application processes to be displayed, complete the drawing of the user interface, and achieve a balance of rendering pressure between multiple application processes and the rendering process.

[0468] Furthermore, multiple application processes draw some content in parallel, while the rendering process draws another part of the content in a unified manner, reducing the difficulty of implementing cross-application animation effects and achieving refined cascading visual effects.

[0469] Optionally, Figure 30 For the specific implementation details of each step in the process shown, please refer to the above. Figure 28 The relevant details in the process shown will not be repeated here.

[0470] In some solutions, multiple embodiments of this application can be combined, and the combined solution can be implemented. Optionally, some operations in the processes of each method embodiment may be combined, and / or the order of some operations may be changed. Furthermore, the execution order between the steps of each process is merely exemplary and does not constitute a limitation on the execution order between steps; other execution orders are also possible. It is not intended to indicate that the execution order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. In addition, it should be noted that the process details involved in one embodiment of this document are similarly applicable to other embodiments, or different embodiments may be combined.

[0471] Furthermore, some steps in the method embodiments can be equivalently replaced with other possible steps. Alternatively, some steps in the method embodiments may be optional and can be deleted in certain use cases. Or, other possible steps may be added to the method embodiments.

[0472] Furthermore, the various method embodiments can be implemented individually or in combination.

[0473] The above combination Figures 6-30 The rendering method provided in the embodiments of this application is described in detail below. Figure 31 This application provides a detailed description of the electronic device provided in its embodiments.

[0474] In one possible design, Figure 31 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 31 As shown, the electronic device 3100 may include a processing unit 3101 and a transceiver unit 3102. The electronic device 3100 can be used to implement the functions of the electronic device involved in the above method embodiments.

[0475] Optionally, the processing unit 3101 is used to support the electronic device 3100 in performing operations. Figure 28 S2801-S2803; and / or, for supporting electronic device 3100 to perform Figure 29 S2901-S2903; and / or, for supporting electronic device 3100 to perform Figure 30 S3001-S3003 in the series.

[0476] Optionally, the transceiver unit 3102 is used to support the electronic device 3100 in performing the electronic device's transceiver functions.

[0477] The transceiver unit may include a receiving unit and a transmitting unit, and may be implemented by a transceiver or transceiver-related circuit components, and may be a transceiver or transceiver module. The operation and / or function of each unit in the electronic device 3100 are respectively for implementing the corresponding process of the rendering method described in the above method embodiments. All relevant content of each step involved in the above method embodiments can be referred to the functional description of the corresponding functional unit, and for the sake of brevity, it will not be repeated here.

[0478] Optionally, Figure 31 The illustrated electronic device 3100 may also include a storage unit ( Figure 31 (not shown in the image), this storage unit stores a program or instruction. When the processing unit 3101 and the transceiver unit 3102 execute the program or instruction, it causes... Figure 31 The electronic device 3100 shown can execute the rendering method described in the above method embodiments.

[0479] Figure 31 The technical effects of the electronic device 3100 shown can be referred to the technical effects of the rendering method described in the above method embodiments, and will not be repeated here.

[0480] In addition to being in the form of electronic device 3100, the technical solution provided in this application can also be a functional unit or chip in an electronic device, or a device used in conjunction with an electronic device.

[0481] This application also provides a chip system, including: a processor coupled to a memory, the memory being used to store programs or instructions, wherein when the program or instructions are executed by the processor, the chip system implements the methods in any of the above method embodiments.

[0482] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0483] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0484] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0485] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0486] This application also provides a computer-readable storage medium storing a computer program. When the computer program is run on a computer, it causes the computer to perform the aforementioned steps to implement the rendering method in the above embodiments.

[0487] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the rendering method described in the above embodiments.

[0488] In addition, this application also provides an apparatus. Specifically, the apparatus may be a component or module, and may include one or more processors and a memory connected together. The memory is used to store a computer program. When the computer program is executed by one or more processors, the apparatus performs the rendering methods described in the above-described method embodiments.

[0489] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0490] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an application-specific integrated circuit (ASIC).

[0491] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the division of the above functional modules is only used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0492] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of modules or units may be electrical, mechanical or other forms.

[0493] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0494] Computer-readable storage media include, but are not limited to, any of the following: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media capable of storing program code.

[0495] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A rendering method, characterized by, The method is applied to an electronic device, and the method comprises: A first application process generates a first rendering node tree; wherein the first rendering node tree is used to draw a first application interface of the first application process, and the first rendering node tree comprises a rendering node corresponding to first content in the first application interface and a rendering node corresponding to second content in the first application interface; The first application process generates, according to the first rendering node tree, a first texture corresponding to the first content; A rendering process draws, according to the first texture and the first rendering node tree generated by the first application process, to generate the first application interface of the first application process.

2. The method of claim 1, wherein, The first content comprises static content in the first application interface, and the second content comprises dynamic content in the first application interface; or the first content comprises part of the static content in the first application interface, and the second content comprises dynamic content and another part of the static content in the first application interface.

3. The method according to claim 1 or 2, characterized in that, The first user interface of the electronic device comprises the first application interface, and the method further comprises: Displaying the first user interface; The first application process generates a second rendering node tree; wherein the second rendering node tree is used to draw a second application interface of the first application process, and the second rendering node tree comprises a rendering node corresponding to first content in the second application interface and a rendering node corresponding to third content in the second application interface; The rendering process draws, according to the first texture and the second rendering node tree generated by the first application process, to generate the second application interface of the first application process; Displaying a second user interface; wherein the second user interface comprises the second application interface.

4. The method according to any one of claims 1 to 3, characterized in that, The first user interface of the electronic device comprises the first application interface of the first application process and a third application interface of a second application process; before the rendering process draws, according to the first texture and the first rendering node tree generated by the first application process, to generate the first application interface of the first application process, the method further comprises: The second application process generates a third rendering node tree; wherein the third rendering node tree is used to draw the third application interface of the second application process, and the third rendering node tree comprises a rendering node corresponding to fourth content in the third application interface and a rendering node corresponding to fifth content in the third application interface; The second application process generates, according to the third rendering node tree, a second texture corresponding to the fourth content; The rendering process draws, according to the first texture and the first rendering node tree generated by the first application process, to generate the first application interface of the first application process, comprising: The rendering process generates a target rendering node tree according to the first rendering node tree and the third rendering node tree; The rendering process draws, according to the first texture and / or the second texture and the target rendering node tree, to generate the first user interface.

5. The method of claim 3, wherein, The rendering process generates a target rendering node tree according to the first rendering node tree and the third rendering node tree, including: In the case that the fourth content is completely covered by the first application interface, the drawing operation of the rendering node corresponding to the fourth content in the target rendering node tree is eliminated; And / or, In the case that the fifth content is completely covered by the first application interface, the drawing operation of the rendering node corresponding to the fifth content in the target rendering node tree is eliminated.

6. The method according to any one of claims 1 to 5, characterized in that, The first texture includes a node texture and / or a sub-tree level texture, the node texture corresponds to a rendering node, and the sub-tree level texture corresponds to a rendering sub-tree.

7. The method of claim 6, wherein, The parent node of the rendering sub-tree is a rendering node with animation / effect attribute, and the number of rendering nodes of the rendering sub-tree is greater than or equal to a number threshold; And / or, the sub-tree root node of the rendering sub-tree includes a first identifier, and the first identifier indicates that the rendering sub-tree is used to generate a texture; And / or, the layer where the rendering sub-tree is located includes a second identifier, and the second identifier indicates that the rendering node in the layer is used to generate a texture.

8. The method according to any one of claims 1 to 7, characterized in that, After the first application process generates the first texture corresponding to the first content according to the first rendering node tree, the method further includes: The first application process writes the first texture into a first memory space; The rendering process reads the first texture from the first memory space.

9. The method of claim 8, wherein, The first texture includes a first node texture and a second node texture, and the first application process writes the first texture into a first memory space, including: The first application process applies a first texture cache to the first memory space according to the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering sub-tree or different rendering sub-trees; The first application process writes the first node texture and the second node texture into the first texture cache after splicing.

10. The method of claim 9, wherein, The first node texture carries first offset data of the first node texture in the first texture cache, and the second node texture carries second offset data of the second node texture in the first texture cache.

11. The method according to any one of claims 1-10, characterized in that, Before the first application process generates the first rendering node tree, the method further includes: The rendering process determines to issue a rendering task to the first application process according to one or more of resource occupancy of the rendering process, historical load of the rendering process, number of foreground applications, rendering load of the first application process, task load of the first application process, load of central processing unit (CPU), load of graphics processing unit (GPU), hardware performance of the electronic device, and power of the electronic device, wherein the rendering task instructs the first application process to draw the first content.

12. The method according to any one of claims 1-11, characterized in that, The first content or the second content is one or more of a window, a control, and an attribute in the first application interface.

13. The method of any of claims 1-12, wherein, After the first application process generates the first rendering node tree, the method further includes: The first application process writes the first rendering node tree into a second memory space; The rendering process reads the first rendering node tree from the second memory space.

14. A rendering method, characterized by, The method comprises: The first application process draws the first partial rendering task to generate a first texture; the first texture comprises static content in the first application interface; wherein the first texture or at least a part of static attributes corresponding to the first texture is cached; The rendering process divides the first rendering node tree of the first application interface of the first application process to be displayed into the first partial rendering task and a second partial rendering task according to the first rendering node tree; the second partial rendering task corresponds to dynamic content in the first application interface; The rendering process draws according to the cached first texture or at least a part of the static attributes and the second partial rendering task to generate the first application interface.

15. The method of claim 14, wherein, The method further comprises: The rendering process draws according to the first texture or at least a part of the static attributes, and a third partial rendering task to generate the second application interface according to the second rendering node tree of the second application interface of the first application process to be displayed; wherein the second rendering node tree comprises the first partial rendering task and the third partial rendering task.

16. The method according to claim 14 or 15, characterized in that The first texture comprises a node texture and / or a sub-tree level texture; the node texture corresponds to a rendering node; and the sub-tree level texture corresponds to a rendering sub-tree.

17. The method of claim 16, wherein, The parent node of the rendering sub-tree is a rendering node with animation / effect attributes, and the number of rendering nodes of the rendering sub-tree is greater than or equal to a number threshold; And / or, the sub-tree root node of the rendering sub-tree comprises a first identifier, the first identifier indicating that the rendering sub-tree is used to generate a texture; And / or, the layer in which the rendering sub-tree is located comprises a second identifier, the second identifier indicating that the rendering node in the layer is used to generate a texture.

18. The method according to any of claims 14-17, characterized by, After the first application process draws the first partial rendering task to generate a first texture, the method further comprises: The first application process writes the first texture or at least a part of the static attributes into a first memory space; The rendering process reads the first texture or at least a part of the static attributes from the first memory space.

19. The method of claim 18, wherein, The first texture comprises a first node texture and a second node texture, and the first application process writes the first texture into a first memory space, comprising: The first application process applies a first texture cache to the first memory space according to the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering sub-tree or different rendering sub-trees; The first application process writes the first node texture and the second node texture into the first texture cache after splicing.

20. The method of claim 19, wherein, The first node texture carries first offset data of the first node texture in the first texture cache, and the second node texture carries second offset data of the second node texture in the first texture cache.

21. The method of any of claims 14-20, wherein, Before the first application process draws the first partial rendering task to generate a first texture, the method further comprises: The rendering process assigns the first part of the rendering task to the first application process for rendering according to one or more of resource occupancy of the rendering process, historical load of the rendering process, number of foreground applications, rendering load of the first application process, task load of the first application process, load of a central processing unit (CPU), load of a graphics processing unit (GPU), hardware performance of the electronic device, power of the electronic device, and the first rendering node tree.

22. The method of any of claims 14-21, wherein, The first part of the rendering task or the second part of the rendering task corresponds to one or more of a window, a control, and an attribute in the first application interface.

23. A rendering method, characterized by, The method comprises: The first application process renders the first part of the rendering task to generate a first texture; the first texture comprises first static content in the first application interface; wherein the first texture or at least part of the static attributes corresponding to the first texture is cached; The rendering process divides the first rendering node tree into the first part of the rendering task and a second part of the rendering task according to the first rendering node tree of the first application interface of the first application process to be displayed; the second part of the rendering task corresponds to second static content and dynamic content in the first application interface; The rendering process generates the first application interface according to the cached first texture or at least part of the static attributes and the second part of the rendering task.

24. The method of claim 23, wherein, The method further comprises: The rendering process generates the second application interface according to a second rendering node tree of a second application interface of the first application process to be displayed, the first texture or at least part of the static attributes, and a third part of the rendering task; wherein the second rendering node tree comprises the first part of the rendering task and the third part of the rendering task.

25. The method of claim 23 or 24, wherein, The first texture comprises a node texture and / or a sub-tree level texture; the node texture corresponds to a rendering node; and the sub-tree level texture corresponds to a rendering sub-tree.

26. The method of claim 25, wherein, The parent node of the rendering sub-tree is a rendering node with animation / effect attributes, and the number of rendering nodes of the rendering sub-tree is greater than or equal to a number threshold; And / or, the sub-tree root node of the rendering sub-tree comprises a first identifier, the first identifier indicating that the rendering sub-tree is used to generate a texture; And / or, the layer in which the rendering sub-tree is located comprises a second identifier, the second identifier indicating that the rendering node in the layer is used to generate a texture.

27. The method of any of claims 23-26, wherein, After the first application process renders the first part of the rendering task to generate a first texture, the method further comprises: The first application process writes the first texture or at least part of the static attributes into a first memory space; The rendering process reads the first texture or at least part of the static attributes from the first memory space.

28. The method of claim 27, wherein, The first texture comprises a first node texture and a second node texture, and the first application process writes the first texture into a first memory space, comprising: The first application process applies for a first texture cache in the first memory space according to the first node texture and the second node texture; wherein the rendering node corresponding to the first node texture and the rendering node corresponding to the second node texture belong to the same rendering sub-tree or different rendering sub-trees; The first application process writes the first node texture and the second node texture into the first texture cache after splicing.

29. The method of claim 28, wherein, The first node texture carries first offset data of the first node texture in the first texture cache, and the second node texture carries second offset data of the second node texture in the first texture cache.

30. The method of any of claims 23-29, wherein, Before the first application process draws a first part of rendering task to generate a first texture, the method further comprises: The rendering process allocates the first part of rendering task to the first application process for drawing according to one or more of resource occupancy of the rendering process, historical load of the rendering process, number of foreground application programs, rendering load of the first application process, task load of the first application process, load of central processing unit (CPU), load of graphic processing unit (GPU), hardware performance of the electronic device, power of the electronic device, and the first rendering node tree.

31. The method of any of claims 23-30, wherein, The first part of rendering task or the second part of rendering task corresponds to one or more of windows, controls, and attributes in the first application interface.

32. An electronic device, comprising: Comprise: A processor and a memory, the memory is coupled with the processor, the memory is used to store computer program code, the computer program code comprises computer instructions, when the processor reads the computer instructions from the memory, makes the electronic device execute the method as claimed in any one of claims 1-13, or makes the electronic device execute the method as claimed in any one of claims 14-22, or makes the electronic device execute the method as claimed in any one of claims 23-31.

33. A computer-readable storage medium, comprising: The computer readable storage medium comprises a computer program, when the computer program runs on an electronic device, makes the electronic device execute the method as claimed in any one of claims 1-13, or makes the electronic device execute the method as claimed in any one of claims 14-22, or makes the electronic device execute the method as claimed in any one of claims 23-31.

34. A computer program product, characterised in that, When the computer program product runs on a computer, makes the computer execute the method as claimed in any one of claims 1-13, or makes the computer execute the method as claimed in any one of claims 14-22, or makes the computer execute the method as claimed in any one of claims 23-31.

Images