A rendering method and apparatus

Abstract

Embodiments of the present application provide a rendering method and device, and relate to the field of terminals, which can reduce the memory occupation and performance load of an electronic device. The method comprises: querying a first pointer from a first data structure according to the identifier of a to-be-updated rendering node; and / or querying a second pointer from the first data structure according to the identifier of the parent node of the to-be-updated rendering node; wherein the first data structure is used to record the identifier and the pointer of each rendering node of a first rendering tree, and the first rendering tree is a rendering tree corresponding to a first display interface; updating at least one of the first information of the to-be-updated rendering node and the second information of the parent node of the to-be-updated rendering node in the first rendering tree according to the first pointer and / or the second pointer, to obtain a second rendering tree; and rendering elements on a second display interface according to the second rendering tree, wherein the second display interface is a next frame display interface of the first display interface.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to a rendering method and apparatus. Background Technology

[0002] Currently, the process of displaying a user interface on an electronic device can include: the central processing unit (CPU) in the electronic device draws the content to be displayed on the interface, and then the graphics processing unit (GPU) renders the drawing results to obtain a rendered result; subsequently, the rendering module of the electronic device composites the rendered results and displays them on the screen as the user interface. However, as the display effects of electronic devices become more complex, the amount of content that needs to be rendered when displaying a user interface is increasing. This increase in rendered content leads to excessive memory usage and performance load on the electronic device. Summary of the Invention

[0003] This application provides a rendering method and apparatus that can reduce the memory footprint and performance load of electronic devices.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] In a first aspect, a rendering method is provided, applied to an electronic device. The method includes: acquiring information about a rendering node to be updated, the information including an identifier of the rendering node to be updated and / or an identifier of its parent node; if the information includes the identifier of the rendering node to be updated, querying a first pointer corresponding to the rendering node to be updated from a first data structure based on the identifier of the rendering node to be updated; and if the information includes the identifier of the parent node of the rendering node to be updated, querying a second pointer corresponding to the parent node of the rendering node to be updated from the first data structure based on the identifier of the parent node of the rendering node to be updated. The first data structure is used to record the identifier and pointer of each rendering node in the first rendering tree, which is the rendering tree corresponding to the first display interface. At least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree is updated to obtain the second rendering tree. The first pointer points to a first memory location, which stores the first information of the rendering node to be updated in the first rendering tree. The second pointer points to a second memory location, which stores the second information of the parent node of the rendering node to be updated. Elements on the second display interface are rendered according to the second rendering tree. The second display interface is the next frame of the first display interface.

[0006] The rendering method provided in this application allows an electronic device to perform differential rendering updates based on incremental information (information on rendering nodes whose data has changed (i.e., the rendering nodes to be updated)). This means updating rendering nodes whose data has changed while leaving unchanged ones untouched. This eliminates the need to update the entire rendering tree corresponding to the display interface and to read data from each rendering node in the tree sequentially, thus reducing the memory footprint and performance load of the electronic device. Furthermore, in this application embodiment, the electronic device can quickly locate the pointer to the rendering node to be updated and / or the pointer to its parent node from the first data structure based on the identifier of the rendering node to be updated. Then, based on the corresponding pointer, it can quickly locate the memory location corresponding to the rendering node to be updated or its parent node, and subsequently update the information (first information or second information) in the corresponding memory location. Therefore, compared to traversing the rendering tree sequentially from the root node, quickly locating the memory location of the rendering node to be updated and / or its parent node based on pointers, and then updating the information in the corresponding memory location, effectively saves power consumption.

[0007] In one possible implementation, the first information includes a first rendering instruction for the rendering node to be updated. If the rendering node to be updated is a rendering node to be modified, the information also includes a second rendering instruction corresponding to the rendering node to be updated. Updating at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree includes: deleting the first rendering instruction from the first memory location and writing the second rendering instruction into the first memory location. That is, if the rendering node to be updated is a rendering node to be modified, the first information (first rendering instruction) of the rendering node to be updated in the first rendering tree can be deleted to clear the first memory location and release memory. This eliminates the need to update the entire rendering tree corresponding to the display interface and to read data from each rendering node in the rendering tree one by one, thus reducing the memory usage and performance load of electronic devices.

[0008] In one possible implementation, when the rendering node to be updated is also a rendering node to be deleted, updating at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree includes: deleting the first information in the first memory location and deleting the first pointer of the rendering node to be updated from the second information, where the second information includes the child node pointer information of the parent node of the rendering node to be updated. That is, when the rendering node to be updated is also a rendering node to be deleted, not only can the first information in the first memory location be deleted to clear the first memory location and release memory, but the electronic device can also delete the first pointer of the rendering node to be updated from the child node pointer information of the parent node of the rendering node to be updated. This allows the rendering node to be deleted to be removed from the first rendering tree, preventing the electronic device from mistakenly accessing the cleared first memory location during subsequent rendering, ensuring the correctness of the subsequent rendering order and content. In another possible implementation, the information of the rendering node to be updated also includes the identifiers of the child nodes of the rendering node to be updated. The method further includes: querying the third pointer corresponding to the child node of the rendering node to be updated from the first data structure based on the identifiers of the child nodes of the rendering node to be updated; and deleting the third information in the third memory location pointed to by the third pointer in the first rendering tree. Understandably, when the rendering node to be updated is the same as the rendering node to be deleted, its child nodes also need to be deleted. Therefore, the third pointer corresponding to the child node to be updated can be queried from the first data structure based on the identifier of the child node to be updated. The third information (the rendering data of the child node of the rendering node to be deleted) in the third memory location pointed to by the third pointer in the first rendering tree is deleted to reduce memory usage.

[0009] In one possible implementation, the method further includes deleting the identifier and pointer of the rendering node to be updated from the first data structure. That is, the electronic device can update the first data structure. The updated first data structure can store the identifier and pointer corresponding to each rendering node in the second rendering tree, facilitating the next query of the pointer to the rendering node to be updated (relative to the rendering node to be updated in the second rendering tree) based on the updated first data structure.

[0010] In one possible implementation, when the rendering node to be updated is a newly added rendering node, before updating at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree, the method further includes: determining a fourth pointer corresponding to the rendering node to be updated, the fourth pointer being used to indicate the memory location storing the information of the rendering node to be updated, the information of the rendering node to be updated also including the second rendering instruction corresponding to the rendering node to be updated; updating at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree includes: adding the fourth pointer corresponding to the rendering node to be updated to the second information in the second memory location, the second information including the child node pointer information of the parent node of the rendering node to be updated. That is, when the rendering node to be updated is a newly added rendering node, the electronic device can determine the fourth pointer corresponding to the rendering node to be updated and add the pointer (fourth pointer) of the rendering node to be updated to the child node pointer information of the parent node of the rendering node to be updated. In this way, the newly added rendering node can be mounted to the first rendering tree, ensuring that the electronic device can access the memory location corresponding to the newly added rendering node (the memory location pointed to by the fourth pointer) during subsequent rendering, which can ensure the correctness of the subsequent rendering order and rendering content.

[0011] In one possible implementation, the method further includes adding an identifier and a fourth pointer to the first data structure for the rendering node to be updated. That is, the electronic device can update the first data structure. The updated first data structure can store the identifier and pointer corresponding to each rendering node in the second rendering tree, facilitating the next query of the pointer to the rendering node to be updated (relative to the rendering node to be updated in the second rendering tree) based on the updated first data structure.

[0012] In one possible implementation, the first data structure includes a MAP container. A Map container is an associative container that allows for the rapid creation of key-value (key-value) indexes, enabling quick retrieval of the corresponding value based on the key. In this embodiment, the key in the Map container can correspond to the identifier (unqueid) of a rendering node, and the value can correspond to a pointer to that rendering node.

[0013] In one possible implementation, the electronic device includes a first application process and a rendering service. Obtaining information about the rendering node to be updated includes: the rendering service receiving a first data packet from the first application process. The first data packet includes at least one of a first type identifier, a second type identifier, or a third type identifier. The first type identifier indicates information about the rendering node to be modified, the second type identifier indicates information about the rendering node to be deleted, and the third type identifier indicates information about the rendering node to be added. The rendering method provided in this application embodiment can synchronize incremental information (information about rendering nodes whose data has changed (i.e., the rendering node to be updated)) across processes (across the application process and the rendering service process). The application process (e.g., the Launcher process) can synchronize the information about the rendering node to be updated in the first data packet to the rendering service without synchronizing all the data of the application process (information about all rendering nodes), thus saving memory usage of the data packet (the first data packet). The rendering service can determine the type of the rendering node to be updated (the rendering node to be modified, the rendering node to be deleted, or the rendering node to be added) based on the type identifier (first type identifier, second type identifier, or third type identifier) ​​carried in the first data packet. Therefore, based on the type of the rendering node to be updated, the lifecycle of the rendering node to be updated can be accurately and reasonably managed.

[0014] In one possible implementation, if the information of the rendering node to be updated includes the identifier of the rendering node to be updated, a first pointer corresponding to the rendering node to be updated is retrieved from a first data structure based on the identifier of the rendering node to be updated; if the information of the rendering node to be updated includes the identifier of the parent node of the rendering node to be updated, a second pointer corresponding to the parent node of the rendering node to be updated is retrieved from the first data structure based on the identifier of the parent node of the rendering node to be updated, including: if the rendering node to be updated is a rendering node to be modified, the first data packet includes the identifier of the rendering node to be updated; the rendering service retrieves the first pointer corresponding to the rendering node to be updated from the first data structure based on the identifier of the rendering node to be updated; ... modified is a rendering node to be modified, the first data packet includes the identifier of the rendering node to be updated; the rendering service retrieves the first pointer corresponding to the rendering node to be updated from the first data structure based on the identifier of the rendering node to be updated; the rendering service retrieves the first pointer corresponding to the rendering node to be updated from the first data structure based on the identifier of the rendering node to be updated; the rendering service retrieves the first pointer corresponding to the rendering node to be updated from the first data structure based on the identifier of the rendering node to be updated; the rendering service retrieves the first When the rendering node is a rendering node to be deleted, the first data packet includes the identifier of the rendering node to be updated and the identifier of the parent node of the rendering node to be updated; the rendering service queries the first pointer corresponding to the rendering node to be updated from the first data structure according to the identifier of the rendering node to be updated; and queries the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure according to the identifier of the parent node of the rendering node to be updated; when the rendering node to be updated is a rendering node to be added, the first data packet includes the identifier of the rendering node to be updated and the identifier of the parent node of the rendering node to be updated; the rendering service queries the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure according to the identifier of the parent node of the rendering node to be updated.

[0015] In one possible implementation, updating at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree includes: when the rendering node to be updated is a rendering node to be modified, the first information includes a first rendering instruction for the rendering node to be updated, and the information of the rendering node to be updated also includes a second rendering instruction corresponding to the rendering node to be updated; the rendering service deletes the first rendering instruction from a first memory location and writes the second rendering instruction into the first memory location; when the rendering node to be updated is a rendering node to be deleted, the rendering service deletes the first rendering instruction from the first memory location and deletes the first pointer of the rendering node to be updated from the child node pointer information of the parent node of the rendering node to be updated; when the rendering node to be updated is a rendering node to be added, the rendering service determines the memory location and a fourth pointer corresponding to the rendering node to be updated, the fourth pointer being used to indicate the memory location storing the information of the rendering node to be updated; the rendering service adds the fourth pointer to the child node pointer information of the parent node of the rendering node to be updated. In this way, the rendering service can accurately and reasonably manage the lifecycle of the rendering node to be updated based on its type. When the type of the rendering node to be updated is a rendering node to be modified, only the rendering instructions of that rendering node in the first rendering tree can be updated (i.e., the rendering node can be updated), resulting in a new rendering tree (the second rendering tree). When the type of the rendering node to be updated is a rendering node to be deleted, the first rendering instruction of that rendering node can be deleted, and the rendering node can be deleted from the first rendering tree (i.e., the rendering node can be destroyed), resulting in a new rendering tree (the second rendering tree). When the type of the rendering node to be updated is a rendering node to be added, the rendering node can be added to the first rendering tree (i.e., the rendering node can be created and mounted), resulting in a new rendering tree (the second rendering tree). Rendering the display interface (the second display interface) based on the new rendering tree (the second rendering tree) ensures the correct rendering order and content.

[0016] Secondly, this application provides a chip system including one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via lines. The aforementioned chip system can be applied to electronic devices including communication modules and memory. The interface circuits are used to receive signals from the memory of the electronic device and send the received signals to the processor, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device can perform the methods described in the first aspect and any of its possible design embodiments.

[0017] Thirdly, this application provides a computer-readable storage medium including computer instructions. When the computer instructions are executed on an electronic device (such as a mobile phone), they cause the electronic device to perform the methods described in the first aspect and any of its possible design embodiments.

[0018] Fourthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method described in the first aspect and any possible design thereof.

[0019] Fifthly, embodiments of this application provide a rendering apparatus, including a processor and a memory coupled together. The memory stores program instructions, which, when executed by the processor, cause the apparatus to implement the method described in the first aspect and any possible design of the above. The apparatus may be an electronic device or a server device; or it may be a component of an electronic device or a server device, such as a chip.

[0020] In a sixth aspect, embodiments of this application provide a rendering apparatus, which can be divided into different logical units or modules according to function, each unit or module performing different functions, so that the apparatus performs the method described in the first aspect and any possible design method thereunder.

[0021] It is understood that the beneficial effects achieved by the chip system described in the second aspect, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the apparatus described in the fifth and sixth aspects can be referred to the beneficial effects in the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0022] Figure 1 A schematic diagram of a separate rendering architecture provided in an embodiment of this application;

[0023] Figure 2 A schematic diagram of a unified rendering architecture provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0025] Figure 4 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0026] Figure 5 A schematic diagram illustrating a rendering method provided in an embodiment of this application;

[0027] Figure 6AA schematic diagram of a display interface provided in an embodiment of this application;

[0028] Figure 6B A schematic diagram of a rendering tree provided for an embodiment of this application;

[0029] Figure 7A A schematic diagram of a display interface and rendering tree provided for an embodiment of this application;

[0030] Figure 7B This is a schematic diagram of a data structure corresponding to a rendering tree provided in an embodiment of this application;

[0031] Figure 8A This is a schematic diagram illustrating data updating based on the rendering node to be modified, provided as an embodiment of this application.

[0032] Figure 8B A schematic diagram illustrating yet another display interface provided in an embodiment of this application;

[0033] Figure 9A This is a schematic diagram illustrating data updating based on a rendering node to be deleted, provided as an embodiment of this application.

[0034] Figure 9B A schematic diagram illustrating yet another display interface provided in an embodiment of this application;

[0035] Figure 10A This is a schematic diagram illustrating data updating based on a new rendering node, as provided in an embodiment of this application.

[0036] Figure 10B A schematic diagram illustrating yet another display interface provided in an embodiment of this application;

[0037] Figure 11 A flowchart illustrating a rendering method provided in an embodiment of this application;

[0038] Figure 12 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0039] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant concepts or technologies is given first:

[0040] like Figure 1As shown, Android's traditional rendering framework is a separate rendering approach, where rendering instructions for each application process are isolated from each other, employing an n-rendering, n-layer composition method. For example, a first application process can draw the interface content (e.g., using the `ViewRootImpl.doTraversal` method), and convert the drawing instructions into rendering instructions through Hardware UI (HWUI), then submit the rendering instructions to the graphics processing unit (GPU). The GPU can then render the blank layer according to the rendering instructions to obtain the rendered interface content (e.g., layer 1). The GPU rendering process can include a geometry stage (processing geometric data of the graphics, including vertex positions, primitive assembly, clipping, and screen mapping), a rasterization stage (converting geometry into pixel information), and a fragment shader (calculating the final color of each pixel (fragment), thus obtaining the rendered interface content (layer 1). Similarly, other application processes (e.g., a second application process) can draw and render to obtain layer 2. The GPU can then send the rendering results (including layers 1 and 2) corresponding to each application to the rendering service on the display system side. Rendering services can include an image compositing system (SurfaceFlinger), an output layer, and a rendering engine. The rendering engine can include a graphics library (e.g., SkiaRenderEngine), which can be used for graphics drawing, shading (createRuntimeEffectShader), etc. Rendering services can use HWC or the GPU to composite the application's graphics data into a final screen image, which is then displayed as the user interface on the electronic device's screen (display).

[0041] HWC compositing is a hardware-based image compositing technique that leverages hardware-level image compositing capabilities to improve performance and efficiency. In HWC compositing, the graphics compositing task is performed by a hardware module, eliminating the need for software-level compositing operations via the CPU or GPU. GPU compositing, on the other hand, is a software-based image compositing technique that uses the GPU to perform graphics compositing operations. In GPU compositing, the application's graphics operations are sent from the CPU to the GPU, which then performs the compositing operations at the software level before finally transmitting the final image data to the display. The advantages of GPU compositing are its high flexibility; being software-based, it can support more complex graphics compositing operations such as filters, transformations, and rounded corners. The disadvantages of GPU compositing are its high power consumption and lower performance compared to HWC compositing. HWC compositing can be the default compositing method, while GPU compositing is an optional one.

[0042] With the development of rendering technology, industry vendors have proposed a unified rendering framework. In this unified rendering architecture, rendering commands from multiple application processes can be transmitted across processes and merged into a single process for unified rendering; that is, a one-time rendering and one-time layer compositing approach is adopted. For example, such as... Figure 2 As shown, the first application process can draw the interface content and convert the drawing instructions into rendering instructions through HWUI, then send the rendering instructions to the rendering service on the display system side. Similarly, other application processes can send rendering instructions to the rendering service. The rendering service's rendering node layer can arrange the rendering instructions of each application process in order, and then send the arranged rendering instructions to the GPU. The GPU can perform unified rendering based on the arranged rendering instructions, and then use HWC to composite them into the final screen image (multiple application processes can correspond to the same layer, which includes the application elements of multiple application processes), which is then displayed as the user interface on the screen (display) of the electronic device.

[0043] Compared to a separate rendering architecture, a unified rendering architecture has the following advantages: 1. It can reduce the number of GPU runs, eliminate the isolation between rendering instructions in different processes, and support cross-window animations. 2. It reduces the number of image buffers requested by multiple (e.g., n) application processes, saving direct memory access (DMA) memory. 3. It can reduce the load on application processes.

[0044] In a unified rendering architecture, the RenderService (RS) performs rendering based on the rendering tree corresponding to the display interface. The rendering tree stores the information needed to render the application elements of the display interface in a tree structure. A rendering tree can include one or more render nodes (RNs). Different render nodes can store abstract data structures of different application elements; common types of render nodes include window nodes and control nodes.

[0045] In related technologies, when electronic devices update the display interface, the rendering service needs to receive the full data of the window corresponding to each application process from each application process, traverse the entire rendering tree corresponding to the display interface, and update the data in each rendering node in the rendering tree one by one based on the full data of each application process, resulting in a huge waste of power consumption.

[0046] This application provides a rendering method that can synchronize incremental information (information of rendering nodes whose data has changed) across processes (across application processes and rendering service processes), and perform differential rendering updates based on the incremental information (i.e., update rendering nodes whose data has changed, and do not process rendering nodes whose data has not changed). It does not require updating the entire rendering tree corresponding to the display interface, nor does it require reading data from each rendering node in the rendering tree one by one, which can reduce the memory usage and performance load of electronic devices.

[0047] The rendering method provided in this application can be applied to electronic devices. These electronic devices may include, for example, mobile phones, tablets, desktop computers, handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, etc. This application does not impose any special limitations on the specific form of the electronic device.

[0048] Figure 3 This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application. Figure 3 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0049] The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0050] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0051] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0052] Electronic device 100 can perform shooting functions through an ISP, camera 193, video codec, GPU, display screen 194, and application processor. The ISP processes data fed back from the camera 193. The camera 193 captures still images or video. The digital signal processor processes digital signals, including digital image signals and other digital signals. The video codec compresses or decompresses digital video. Electronic device 100 can support one or more video codecs. Thus, electronic device 100 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0053] Camera 193 may include 1 to N cameras. Each camera includes a photosensitive element (CCD / CMOS), which can sense light, collect photons and convert them into electrical charges.

[0054] The methods described in the following embodiments can all be implemented in the electronic device 100 having the above-described hardware structure.

[0055] The software system of the aforementioned electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of the electronic device 100.

[0056] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the HAL layer, and the kernel layer. It should be noted that this application uses the Android system as an example; however, the solution can also be implemented in other operating systems (such as HarmonyOS, iOS, etc.) as long as the functions implemented by each module are similar to those in the embodiments of this application.

[0057] The application layer can include a series of application packages.

[0058] like Figure 4 As shown, the application package may include desktop applications (Launcher), system user interface applications (system UI), camera, call, SMS, settings, gallery, calendar, maps, navigation, WLAN, Bluetooth, music, video and other applications.

[0059] Desktop applications can handle gesture responses and gesture-based UI loading.

[0060] The system UI application's functions can include status bar information display (such as battery, Wi-Fi signal, 3G / 4G icons), notification panel management (such as system messages, third-party application messages), and recent task bar display panel management (such as long-pressing the recent task shortcut to display recently used applications).

[0061] Optionally, the application package may also include smart capability applications, which can be called by system applications or third-party applications, providing a quick entry point for system applications or third-party applications to launch applications (e.g., floating capsules).

[0062] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example, it may include an activity manager service (AMS), a window manager, a content provider, a view system, a resource manager, a notification manager, etc., but this application embodiment does not impose any limitations on these.

[0063] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0064] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0065] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0066] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0067] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0068] System libraries can include multiple functional modules. For example: graphics library (libEGL), image compositing system (SurfaceFlinger), surface manager, media libraries, 3D graphics processing library, 2D graphics engine, etc.

[0069] The graphics library, also known as a drawing library, defines cross-programming language, cross-platform application programming interfaces (APIs). It contains numerous functions for processing graphics. Taking the Open Graphics Library (OpenGL) as an example, the OpenGL API includes interfaces for drawing 2D or 3D images (including drawing functions such as `glDrawElements()`) and interfaces for displaying the drawn images on the screen (including rendering functions such as `eglSwapBuffers()`). These will not be listed exhaustively in this embodiment. OpenGL functions can be called via instructions. For example, drawing instructions can call drawing functions to draw 2D or 3D images. These drawing instructions are commands written by developers based on functions within the graphics library during game application development, used to call the corresponding graphics library interface.

[0070] Image compositing systems can send images from a buffer queue to an HWC or GPU for compositing based on a periodic signal (e.g., a Vsync signal) used for display.

[0071] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0072] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0073] OpenGL ES is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0074] SGL is a 2D graphics engine.

[0075] The Android runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core libraries consist of two parts: one part contains the functionalities that Java needs to call, and the other part contains the core Android libraries. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer into binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0076] The HAL layer is a wrapper around Linux kernel drivers, providing interfaces to the upper layers and shielding them from the implementation details of the lower-level hardware.

[0077] The HAL layer can include Wi-Fi HAL, audio HAL, camera HAL, etc.

[0078] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, and sensor drivers.

[0079] The hardware layer may include, but is not limited to, GPUs and HWCs. HWCs can be standalone devices or integrated into a system-on-a-chip (SoC).

[0080] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. In the description of this application, unless otherwise stated, "at least one" refers to one or more, and "more than one" refers to two or more. Furthermore, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0081] For ease of understanding, the rendering method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0082] like Figure 5 As shown, this application embodiment provides a rendering method, taking a desktop application (Launcher) process as the first application process as an example, including:

[0083] 501. The Launcher process has identified the rendering nodes to be updated.

[0084] The Launcher process can traverse the rendering tree corresponding to the Launcher window (also known as the Launcher interface or desktop) and identify the rendering nodes to be updated. The rendering tree corresponding to the Launcher window can be maintained by the Launcher process.

[0085] For example, such as Figure 6AAs shown, the Launcher window may include one or more application icons (e.g., clock, calendar, gallery, notes, video, health icons, etc.), folders, a time control, Wi-Fi signal, battery level, and other elements. For ease of description, this embodiment selects a portion of the Launcher window's display area (e.g., display area 601) for illustration. Display area 601 may include a video icon 602, a health icon 603, and a folder 604. Folder 604 may include a weather icon 6041 and a browser icon 6042. Folder 604 may also include more content, which is not limited in this application.

[0086] like Figure 6B The image shows the rendering tree corresponding to a portion of the Launcher window's display area (e.g., display area 601). This rendering tree can be a part of the complete rendering tree corresponding to the Launcher window. The rendering tree can include RN (rendering node) 1, RN2, ..., RN x, etc. Here, RN1 is an abstraction of display area 601. RN2 is the RN corresponding to video icon 602, RN3 is the RN corresponding to fitness and health icon 603, and RN4 is the RN corresponding to folder 604. RN4 can include multiple child nodes, such as RN5, RN6, ..., RN x, etc. Here, RN5 is the RN corresponding to weather icon 6041 in folder 604, and RN6 is the RN corresponding to browser icon 6042 in folder 604. RN x is the RN corresponding to other elements (e.g., XX icons) in folder 604.

[0087] In this embodiment, the rendering node to be updated may include at least one of the following: a rendering node to be modified (rendering node to be changed), a rendering node to be deleted, or a rendering node to be added. Taking the rendering node to be updated as the rendering node corresponding to the Launcher window as an example, the rendering node to be modified may be the RN corresponding to a changed display element on the Launcher window. The rendering node to be deleted is the RN corresponding to a deleted display element on the Launcher window. The RN to be added is the RN corresponding to a newly added display element on the Launcher window.

[0088] In some embodiments, the rendering node to be modified may include a first rendering node, which may be, for example, the React Native (RN) corresponding to the first desktop icon. The first desktop icon changes on specific holidays (e.g., New Year's Day, Mid-Autumn Festival, etc.). For example, the first desktop icon may have its background, color, animation effects, etc., changed based on the original icon. Another example is that the first desktop icon may display a badge (which can be a number badge or a dot badge) when a video application receives a new message. The first desktop icon may include, for example, a fitness icon and a video icon.

[0089] In some embodiments, the render node to be deleted may include a second render node. The second render node may, for example, be the render node corresponding to a second desktop icon. For instance, in response to a user deleting a second desktop icon (e.g., a calendar icon), the Launcher application can delete the render node corresponding to the calendar icon from the render tree corresponding to the Launcher window.

[0090] In some embodiments, the new rendering node to be added may include a third rendering node. The third rendering node may, for example, be the React Native (RN) corresponding to a third desktop icon. The third desktop icon may, for example, be the icon corresponding to a newly downloaded application (e.g., a music application). For example, in response to a user downloading and installing a music application, the Launcher application may add the RN corresponding to the music application's icon to the rendering tree corresponding to the Launcher window.

[0091] 502. The Launcher process sends the first data packet to the rendering service. The first data packet may include information about the rendering node to be updated.

[0092] In this embodiment, the Launcher process can send information about the rendering nodes to be updated in the rendering tree corresponding to the Launcher window to the rendering service. It does not need to send the full information of the entire rendering tree corresponding to the Launcher window (information of all rendering nodes). This simplifies the transmission of full information to on-demand transmission (only sending the information of the rendering nodes to be updated), which can greatly reduce the transmission loss of duplicate data.

[0093] The information of the rendering node to be updated may include the identifier of the rendering node to be updated and / or the identifier of the parent node of the rendering node to be updated.

[0094] It should be noted that, when the rendering node to be updated is a rendering node to be modified, the first data packet may include the identifier of the rendering node to be updated. When the rendering node to be updated is a rendering node to be deleted, the first data packet includes the identifier of the rendering node to be updated and the identifier of its parent node. When the rendering node to be updated is a rendering node to be added, the first data packet may include the identifier of the rendering node to be updated and the identifier of its parent node.

[0095] If the rendering node to be updated is a rendering node to be modified or a rendering node to be added, the first data packet may also include the latest rendering instruction (second rendering instruction) corresponding to the rendering node to be updated.

[0096] If the rendering node to be updated is the rendering node to be modified, and the parent node of the rendering node to be modified has not changed, then it is not necessary to carry the identifier of the parent node of the rendering node to be modified in the first data packet. If the child nodes of the rendering node to be modified have not changed, then it is also not necessary to carry the identifier of the child nodes of the rendering node to be modified in the first data packet.

[0097] When the rendering node to be updated is also the rendering node to be deleted, it is necessary not only to delete the rendering data (i.e., the first information, such as the first rendering instruction) of the rendering node to be deleted, but also to modify the child node pointer information of the parent node of the rendering node to be deleted. This involves removing the pointer information of the rendering node to be deleted from the child node pointer information of the parent node (i.e., removing the rendering node to be deleted from its parent node) to avoid errors in subsequent rendering. Therefore, the identifier of the parent node of the rendering node to be deleted needs to be carried in the first data packet. This allows for the subsequent lookup of the pointer (second pointer) corresponding to the parent node of the rendering node to be deleted based on the identifier, and the determination of the memory location (second memory location) corresponding to the child node pointer information of the parent node of the rendering node to be deleted based on this pointer, so that the pointer information of the rendering node to be deleted can be deleted from that memory location.

[0098] In some cases, when a rendering node to be deleted has child nodes, the identifiers of the child nodes can be carried in the first data packet. This allows subsequent queries to find the pointers (third pointers) corresponding to the child nodes of the rendering node to be updated, based on the identifiers of the child nodes. This enables the deletion of the third information (the rendering data of the child nodes of the rendering node to be deleted) in the third memory location pointed to by the third pointer in the first rendering tree, thereby reducing memory usage.

[0099] When the node to be updated is a newly added node, its parent node needs to be located first. The newly added node is then attached to its parent node (i.e., the pointer information of the child node of the newly added node is added to the child node pointer information of the parent node) to ensure proper rendering later (i.e., the rendering instructions for the newly added node can be found based on the pointer information of the parent node's child nodes, avoiding errors caused by missed execution of these instructions). Therefore, the identifier of the parent node of the newly added node needs to be included in the first data packet. This allows for the subsequent lookup of the pointer (second pointer) corresponding to the parent node of the newly added node, and the determination of the memory location (second memory location) corresponding to the pointer information of the parent node's child nodes, based on this pointer. The pointer information of the newly added node is then added to this memory location.

[0100] 503. The rendering service updates the first rendering tree based on the first data packet to obtain the second rendering tree.

[0101] The first rendering tree can be the rendering tree corresponding to the first display interface. The rendering nodes in the first rendering tree can correspond one-to-one with the elements on the first display interface. For example, the elements on the first display interface can be windows, controls, or elements with finer granularity than controls. The rendering tree corresponding to the first display interface can be maintained by a rendering service.

[0102] For example, such as Figure 7A Image (a) shows an example of a first display interface. The first display interface may include a Launcher window and a wallpaper. Figure 7A As shown in (b) above, this is an example of a Launcher window for the first display interface. Figure 7A As shown in (c) in the image, this is an example of a wallpaper for the first display screen. Figure 7A As shown in (d), this is an example of a rendering tree (first rendering tree) corresponding to the first display interface. The first rendering tree may include a root RN, RN11, RN12, RN1, RN2, ..., RN x, etc. The root RN is an abstraction of the first display interface, and it may include child nodes RN11 and RN12. RN11 may be an abstraction of the Launcher window, and RN12 may be an abstraction of the wallpaper. RN11 may include child nodes RN1, RN2, and more RNs. The details of RN1, RN2, ..., RN x can be found in the relevant description in step 501.

[0103] like Figure 7B As shown, is Figure 7A The rendering tree shown in (d) corresponds to a data structure view. Figure 7B One of the RN Drawables corresponds to, for example Figure 7AIn the (d) section, there is an RN. For example, the root RN corresponds to RN Drawable0, RN11 corresponds to RNDrawable11, RN12 corresponds to RN Drawable12, and so on. Here, RN Drawable is an abstract drawable class. Different RN Drawables can provide different rendering instructions (DrawOps), and different elements can be rendered based on these instructions. For example, the DrawOp corresponding to RN Drawable2 can be used to render a video icon. The DrawOp corresponding to RN Drawable3 can be used to render a fitness icon. The first three DrawOps corresponding to RN Drawable4 can be used to render folder 604 (for example, these three DrawOps can be used to render the background, rounded corners, and blur of folder 604, respectively). The rendering instruction corresponding to RN Drawable5 can be used to render a weather icon. The rendering instruction corresponding to RN Drawable6 can be used to render a browser icon. It can be understood that the rendering service can traverse the data structure corresponding to the rendering tree sequentially from left to right and from top to bottom, executing the corresponding rendering instructions to complete the rendering of elements on the display interface. For example, the rendering service traverses sequentially from left to right and from top to bottom. Figure 7B The data structure shown executes the corresponding rendering instructions, thereby enabling the processing of data such as... Figure 7A The rendering of all elements on the first display interface shown in (a) (including elements corresponding to the wallpaper and Launcher window, such as video icon 602, sports and health icon 603, etc.)).

[0104] In this embodiment of the application, each RN Drawable can correspond to a member variable uniqueid, and each uniqueid can be used to uniquely identify an RN. That is, an RN on the rendering tree can be uniquely determined based on the uniqueid.

[0105] In this embodiment, the rendering service may store a first data structure, which records the identifier and pointer of each rendering node in the rendering tree (e.g., the rendering tree corresponding to the first display interface). The pointer corresponding to each rendering node points to the memory location corresponding to that rendering node. The memory location corresponding to the rendering node stores the rendering data corresponding to that rendering node. The rendering data corresponding to the rendering node may include the rendering instructions corresponding to that rendering node, and pointer information of the parent node and / or child nodes of that rendering node. The parent node pointer information of the rendering node may indicate the memory location corresponding to the parent node of that rendering node. The child node pointer information of the rendering node may indicate the memory location corresponding to the child node of that rendering node.

[0106] For example, the first data structure may include containers, arrays, stacks, queues, linked lists, trees, graphs, heaps, hash tables, etc., and this application does not make any specific limitations.

[0107] In one possible implementation, the first data structure can be a container (e.g., a Map container). A Map container is an associative container that allows for the rapid creation of key-value (key-value) indexes, enabling quick lookup of the corresponding value based on the key. In this embodiment, the key in the Map container can correspond to the identifier (unqueid) of a rendering node, and the value can correspond to a pointer (RN pointer) to the rendering node. Thus, when the rendering service parses the identifier of the rendering node to be updated, it can quickly find the pointer to that node from the Map container based on the identifier, and then quickly locate the memory location corresponding to the rendering data of that node, thereby updating the rendering data of that node. Compared to traversing the rendering tree sequentially from the root node, quickly locating the memory location corresponding to the rendering data of the node to be updated based on the pointer, and then updating the rendering data of that node, effectively saves power consumption.

[0108] In some embodiments, the first data packet may include at least one of a first type identifier, a second type identifier, or a third type identifier; wherein the first type identifier is used to indicate information of a rendering node to be modified, the second type identifier is used to indicate information of a rendering node to be deleted, and the third type identifier is used to indicate information of a rendering node to be added. After parsing the first type identifier, the rendering service determines that the data following the first type identifier is information of a rendering node to be modified; after parsing the second type identifier, the rendering service determines that the data following the first type identifier is information of a rendering node to be deleted; after parsing the first type identifier, the rendering service determines that the data following the first type identifier is information of a rendering node to be added.

[0109] In some embodiments, when the rendering node to be updated is the rendering node to be modified, the first data packet may include the identifier of the rendering node to be updated and the second rendering instruction corresponding to the rendering node to be updated. If the parent node of the rendering node to be modified has not changed, it is not necessary to carry the identifier of the parent node of the rendering node to be modified in the first data packet. The rendering service can query the first pointer corresponding to the rendering node to be updated from the first data structure according to the identifier of the rendering node to be updated; delete the first rendering instruction in the first memory location pointed to by the first pointer, and write the second rendering instruction into the first memory location. This realizes the update of the first rendering tree and obtains the second rendering tree.

[0110] For example, such as Figure 8AThe diagram illustrates a method for updating data based on a rendering node to be updated (or modified). First, the Launcher process identifies the rendering node to be updated (or modified). For example, the rendering node to be modified may include RN2 (the RN corresponding to the video icon) and RN3 (the RN corresponding to the fitness icon). Then, the Launcher process packages the information of the rendering node to be modified into a first data packet and sends the first data packet to the rendering service. The information of the rendering node to be modified may include the node's uniqueid and the latest rendering instruction (i.e., the second rendering instruction). The latest rendering instruction can replace the historical rendering instruction (the first rendering instruction). The historical rendering instruction (the first rendering instruction) is the rendering instruction corresponding to the previous frame's display interface (e.g., the first display interface). For example, the Launcher process can package the uniqueid (e.g., 2) of RN2 and the latest rendering instruction (e.g., DrawOP 2), and the uniqueid (e.g., 3) of RN3 and the latest rendering instruction (e.g., DrawOP 3) into the first data packet and send the first data packet to the rendering service. After receiving the first data packet, the rendering service can parse it to obtain the identifier of the rendering node to be modified and the latest rendering instructions. The rendering service can then update the first rendering tree corresponding to the first display interface based on the first data packet, thus obtaining the second rendering tree. Specifically, the rendering service can use the identifier (uniqueid) of the rendering node to be modified in the first data packet to query the pointer (first pointer) corresponding to that uniqueid in the MAP container, thereby quickly finding the memory location (first memory location) of the rendering node to be modified in the first rendering tree. For example, the uniqueid of RN2 (e.g., 2) can correspond to pointer 2, and the memory location of RN2 in the first rendering tree can be found based on pointer 2. The uniqueid of RN3 (e.g., 3) can correspond to pointer 3, and the memory location of RN3 in the first rendering tree can be found based on pointer 3. Then, based on the latest rendering instructions (second rendering instructions) of the rendering node to be modified, the historical rendering instructions (first rendering instructions) in that memory location are updated. That is, the first rendering instructions in the corresponding memory location can be deleted, and the second rendering instructions can be written. For example, the memory location pointed to by pointer 2 can be cleared, and then DrawOP 2 can be written to the memory location pointed to by pointer 2. Furthermore, the memory location pointed to by pointer 3 can be cleared, and then DrawOP 3 can be written to the memory location pointed to by pointer 3. This updates the first rendering tree corresponding to the first display interface, resulting in the updated rendering tree (the second rendering tree). The second rendering tree is the rendering tree corresponding to the second display interface. The second display interface is the next frame of the first display interface (that is, the first display interface is the previous frame of the second display interface).Subsequently, the rendering service can render a second display interface based on the second rendering tree.

[0111] For example, such as Figure 8B As shown in (a), the first display interface can be interface 801. The video application icon and the fitness and health application icon in display interface 801 have no badges. For example... Figure 8B As shown in (b), the second display interface can be interface 802. Compared to display interface 801, the video application icon and the fitness and health application icon in display interface 802 have been updated. The upper left corner of the video application icon in display interface 802 has a badge (indicating that the video application has received a new message), and the upper left corner of the fitness and health application icon also has a badge (indicating that the fitness and health application has received a new message).

[0112] In some embodiments, when the rendering node to be updated is a rendering node to be deleted, the first data packet may include an identifier of the rendering node to be updated and an identifier of its parent node. The rendering service can query the first pointer corresponding to the rendering node to be updated from the first data structure based on the identifier of the rendering node to be updated; and it can query the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure based on the identifier of its parent node. The rendering service can delete the first rendering instruction in the first memory location pointed to by the first pointer, and update the second information (the child node pointer information of the parent node of the rendering node to be updated) in the second memory location pointed to by the second pointer, that is, delete the first pointer of the rendering node to be updated from the child node pointer information of the parent node of the rendering node to be updated. In this way, the rendering node to be deleted can be removed from the first rendering tree, thereby updating the first rendering tree and obtaining the second rendering tree.

[0113] For example, such as Figure 9AThe diagram illustrates a data update process based on a rendering node to be updated (or deleted). The Launcher process can identify the rendering node to be updated (or deleted). For example, the rendering node to be deleted might include RN5 (the RN corresponding to the weather icon). The Launcher process then packages the information of the rendering node to be deleted into a first data packet and sends this packet to the rendering service. The information of the rendering node to be deleted might include its identifier and the identifier of its parent node. For example, the Launcher process might package the uniqueid of RN5 (e.g., 5) and the uniqueid of RN5's parent node (RN4, the RN corresponding to the folder) (e.g., 4) into the first data packet and send it to the rendering service. Upon receiving the first data packet, the rendering service can parse it to obtain the identifier of the rendering node to be deleted and the identifier of its parent node. The rendering service can then update the first rendering tree corresponding to the first display interface based on the first data packet, thus obtaining the second rendering tree. Specifically, the rendering service can query the pointer (first pointer) corresponding to the uniqueid of the rendering node to be deleted in the MAP container based on the uniqueid in the first data packet, thereby quickly finding the memory location (first memory location) of the rendering node to be deleted in the first rendering tree. For example, the uniqueid of RN5 (e.g., 5) can correspond to pointer 5, and the memory location of RN5 in the first rendering tree can be found based on pointer 5. Then, the memory location pointed to by pointer 5 can be cleared to release memory and avoid unnecessary memory occupation. Furthermore, the rendering service can query the pointer (second pointer) corresponding to the parent node of the rendering node to be deleted in the MAP container based on the identifier of the parent node in the first data packet, and quickly find the memory location (second memory location) of the parent node of the rendering node to be deleted in the first rendering tree based on the pointer corresponding to the parent node. For example, the uniqueid of RN4 (e.g., 4) can correspond to pointer 4. The rendering service can find the memory location of RN4 in the first rendering tree based on pointer 4. The memory location of RN4 in the first rendering tree can store the pointer information of RN4's child nodes. The child node pointer information of RN4 can include pointers to nodes such as RN5 and RN6. The rendering service can delete the pointer to the rendering node to be updated (e.g., RN5) from the child node pointer information of RN4. In this way, RN5 can be removed from the first rendering tree, resulting in the second rendering tree. Compared to the first rendering tree, the second rendering tree has removed the rendering node RN5. This achieves the update of the first rendering tree corresponding to the first display interface, resulting in the updated rendering tree (the second rendering tree). Subsequently, the rendering service can render the second display interface based on the second rendering tree.

[0114] For example, such as Figure 9B As shown in (a), the first display interface can be interface 901, and the folder 604 in display interface 901 includes a weather icon 6041. Figure 9B As shown in (b), the second display interface can be interface 902. Compared to display interface 901, the weather icon in display interface 902 is removed (does not exist).

[0115] In some cases, when a rendering node to be deleted has child nodes, the identifiers of the child nodes can be carried in the first data packet. This allows subsequent queries to find the pointers (third pointers) corresponding to the child nodes of the rendering node to be updated, based on the identifiers of the child nodes. This enables the deletion of the third information (the rendering data of the child nodes of the rendering node to be deleted) in the third memory location pointed to by the third pointer in the first rendering tree, thereby reducing memory usage.

[0116] Optionally, the rendering service can also update the MAP container. For example, it can remove the identifier and pointer of the rendering node to be updated (or deleted) from the MAP container. If the rendering node to be deleted has child nodes, it can also remove the identifiers and pointers of the child nodes of the rendering node to be updated from the MAP container. This saves storage space in the MAP container. The updated MAP container stores the identifier and pointer corresponding to each rendering node in the second rendering tree, which facilitates the next query for the pointer of the rendering node to be updated (relative to the rendering node to be updated in the second rendering tree) based on the MAP container.

[0117] In some embodiments, when the rendering node to be updated is a newly added rendering node, the first data packet may include the identifier of the rendering node to be updated and the identifier of its parent node. After receiving the first data packet, the rendering service can determine the memory location and pointer (fourth pointer) corresponding to the newly added rendering node. The fourth pointer indicates the memory location storing the information of the newly added rendering node. The rendering service can request a block of memory to store the information of the newly added rendering node, or it can determine the memory location of the information based on shared memory technology (the first application process and the rendering service can access the same block of memory (which stores the information of the newly added rendering node)). Furthermore, the rendering service can query the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure based on the identifier of the parent node. The rendering service can update the second information (the child node pointer information of the parent node of the rendering node to be updated) in the second memory location pointed to by the second pointer, that is, add the pointer (fourth pointer) of the rendering node to be updated to the child node pointer information of the parent node of the rendering node to be updated. This updates the first rendering tree and obtains the second rendering tree.

[0118] For example, such as Figure 10A The diagram illustrates a data update process based on a rendering node to be updated (a new rendering node to be added). The Launcher process can identify the rendering node to be updated (e.g., a new rendering node to be added). For example, the new rendering node to be added may include RN10 (e.g., the RN corresponding to the music icon). The Launcher process can then package the information of the new rendering node to be added into a first data packet and send the first data packet to the rendering service. The information of the new rendering node to be added may include the identifier (uniqueid) of the new rendering node and the identifier of its parent node. For example, the Launcher process can package the uniqueid of RN10 (e.g., 10) and the uniqueid of RN10's parent node (RN1) (e.g., 1) into the first data packet and send the first data packet to the rendering service. Upon receiving the first data packet, the rendering service can parse it to obtain the identifier of the new rendering node and the identifier of its parent node. The rendering service can then update the first rendering tree corresponding to the first display interface based on the first data packet to obtain a second rendering tree. Specifically, the rendering service can query the pointer (second pointer) corresponding to the parent node's identifier (uniqueid) in the MAP container based on the first data packet. Based on this pointer, it can quickly locate the memory location (second memory location) of the parent node in the first rendering tree. For example, the uniqueid (e.g., 1) of RN1 can correspond to pointer 1. The rendering service can then locate the memory location of RN1 in the first rendering tree based on pointer 1. The memory location of RN1 in the first rendering tree can store pointer information for RN1's child nodes. This pointer information can include pointers to nodes such as RN2, RN3, and RN4. The rendering service can add a pointer to the child node pointer information of RN4 (e.g., RN10) to be updated. This allows RN10 to be mounted onto RN1 in the first rendering tree, resulting in the second rendering tree. Compared to the first rendering tree, the second rendering tree has the added rendering node RN10. This achieves the update of the first rendering tree corresponding to the first display interface, resulting in the updated rendering tree (second rendering tree). Subsequently, the rendering service can render the second display interface based on the updated rendering tree (second rendering tree), which can ensure that the rendering order and content of the elements in the second display interface are correct.

[0119] For example, such as Figure 10B As shown in (a) above, the first display interface can be interface 1001, which does not include a music icon. Figure 10BAs shown in (b), the second display interface can be interface 1002. Compared to display interface 1001, display interface 1002 adds a music icon 1003.

[0120] Optionally, the rendering service can also update the MAP container, for example, by adding the identifier and pointer (fourth pointer) of the rendering node to be updated (the new rendering node) to the MAP container. In this way, the updated MAP container stores the identifier and pointer corresponding to each rendering node in the second rendering tree, which facilitates the next query based on the MAP container for the pointer of the rendering node to be updated (relative to the rendering node to be updated in the second rendering tree).

[0121] The above embodiments illustrate the process using the rendering node to be updated as a rendering node to be modified, a rendering node to be deleted, or a rendering node to be added. In some other embodiments, the rendering node to be updated may include at least two of the following: a rendering node to be modified, a rendering node to be deleted, and a rendering node to be added. In this case, the first data packet may carry information about at least two types of rendering nodes at once. The first data packet may also carry type identifiers corresponding to the at least two types of rendering nodes. For example, the first data packet may include at least two of a first type identifier, a second type identifier, or a third type identifier. The first type identifier indicates information about the rendering node to be modified, the second type identifier indicates information about the rendering node to be deleted, and the third type identifier indicates information about the rendering node to be added. After parsing the first type identifier, the rendering service determines that the data following the first type identifier is information about the rendering node to be modified; after parsing the second type identifier, the rendering service determines that the data following the first type identifier is information about the rendering node to be deleted; after parsing the first type identifier, the rendering service determines that the data following the first type identifier is information about the rendering node to be added. The processing method of the rendering service for different types of rendering nodes (rendering nodes to be updated) is the same as in the above embodiments and will not be repeated here.

[0122] 504. The rendering service renders the elements on the second display interface according to the second rendering tree. The second display interface is the next frame of the first display interface.

[0123] In this embodiment, the electronic device can render elements on the second display interface based on the updated rendering tree (second rendering tree), thereby presenting the second display interface to the user. For example, the rendering service can sequentially submit the rendering instructions corresponding to each rendering node in the second rendering tree to the GPU, which then executes the rendering instructions sequentially to render the elements on the second display interface, and displays the rendered second display interface on the screen.

[0124] The rendering method provided in this application embodiment can synchronize incremental information (information of rendering nodes whose data has changed (i.e., rendering nodes to be updated)) across processes (across application processes and rendering service processes). The application process (e.g., the Launcher process) can synchronize the information of the rendering nodes to be updated in the first data packet to the rendering service, without needing to synchronize the entire application process's data (information of all rendering nodes), thus saving memory usage of the data packet (the first data packet). The rendering service can perform differential rendering updates based on the incremental information (i.e., updating rendering nodes whose data has changed, and not processing rendering nodes whose data has not changed), without needing to update the entire rendering tree corresponding to the display interface, and without needing to read data from each rendering node in the rendering tree one by one, thus reducing the memory usage and performance load of electronic devices.

[0125] Furthermore, the rendering service can accurately and reasonably manage the lifecycle of a rendering node to be updated based on its type. When the type of the rendering node to be updated is a node to be deleted, it can be deleted from the first rendering tree (i.e., the rendering node can be destroyed), resulting in a new rendering tree (the second rendering tree). When the type of the rendering node to be updated is a rendering node to be added, it can be added to the first rendering tree (i.e., the rendering node can be created and mounted), resulting in a new rendering tree (the second rendering tree). When the type of the rendering node to be updated is a rendering node to be modified, only the rendering instructions of that rendering node in the first rendering tree can be updated (i.e., the rendering node can be updated), resulting in a new rendering tree (the second rendering tree). Rendering the display interface (the second display interface) based on the new rendering tree (the second rendering tree) ensures the correct rendering order and content.

[0126] like Figure 11 As shown in the figure, this application provides a rendering method applied to an electronic device, the method comprising:

[0127] 1101. Obtain the information of the rendering node to be updated. The information of the rendering node to be updated includes the identifier of the rendering node to be updated and / or the identifier of the parent node of the rendering node to be updated.

[0128] In some embodiments, the electronic device includes a first application process and a rendering service. The first application process can send a first data packet to the rendering service. Correspondingly, the rendering service can receive the first data packet from the first application process. The first data packet includes at least one of a first type identifier, a second type identifier, or a third type identifier; wherein the first type identifier is used to indicate information about a rendering node to be modified, the second type identifier is used to indicate information about a rendering node to be deleted, and the third type identifier is used to indicate information about a rendering node to be added.

[0129] The first application process can be, for example, the Launcher process. The process of the Launcher process determining the rendering node to be updated and sending the first data packet to the rendering service can be referred to the relevant descriptions in steps 501-502.

[0130] 1102. Query the first pointer corresponding to the rendering node to be updated from the first data structure according to the identifier of the rendering node to be updated; and / or, query the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure according to the identifier of the parent node of the rendering node to be updated.

[0131] If the information of the rendering node to be updated includes the identifier of the rendering node to be updated, the first pointer corresponding to the rendering node to be updated is retrieved from the first data structure according to the identifier of the rendering node to be updated; if the information of the rendering node to be updated includes the identifier of the parent node of the rendering node to be updated, the second pointer corresponding to the parent node of the rendering node to be updated is retrieved from the first data structure according to the identifier of the parent node of the rendering node to be updated.

[0132] The first data structure records the identifier and pointer of each rendering node in the first rendering tree, which is the rendering tree corresponding to the first display interface. For a detailed explanation, please refer to step 503.

[0133] 1103. Update at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree to obtain the second rendering tree.

[0134] Here, the first pointer points to a first memory location, which stores the first information of the rendering node to be updated in the first rendering tree. The second pointer points to a second memory location, which stores the second information of the parent node of the rendering node to be updated. For a detailed explanation, please refer to step 503.

[0135] 1104. Render the elements on the second display interface according to the second rendering tree. The second display interface is the next frame of the first display interface.

[0136] For related instructions, please refer to step 504.

[0137] The rendering method provided in this application embodiment can synchronize incremental information (information of rendering nodes whose data has changed (i.e., rendering nodes to be updated)) across processes (across application processes and rendering service processes). The application process (e.g., the Launcher process) can synchronize the information of the rendering nodes to be updated in the first data packet to the rendering service, without needing to synchronize the entire application process's data (information of all rendering nodes), thus saving memory usage of the data packet (the first data packet). The rendering service can perform differential rendering updates based on the incremental information (i.e., updating rendering nodes whose data has changed, and not processing rendering nodes whose data has not changed), without needing to update the entire rendering tree corresponding to the display interface, and without needing to read data from each rendering node in the rendering tree one by one, thus reducing the memory usage and performance load of electronic devices.

[0138] Some embodiments of this application provide an electronic device that may include a touchscreen, a memory, and one or more processors. The touchscreen, memory, and processors are coupled. The memory stores computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the electronic device in the above method embodiments. The structure of the electronic device can be referred to... Figure 3 The structure of the electronic device 100 shown.

[0139] This application also provides a chip system (e.g., a system-on-a-chip (SoC)). Figure 12 As shown, the chip system includes at least one processor 1201 and at least one interface circuit 1202. The processor 1201 and the interface circuit 1202 are interconnected via lines. For example, the interface circuit 1202 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 1202 can be used to send signals to other devices (e.g., the processor 1201 or the touchscreen of an electronic device). Exemplarily, the interface circuit 1202 can read instructions stored in the memory and send those instructions to the processor 1201. When the instructions are executed by the processor 1201, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0140] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform various functions or steps performed by the electronic device (e.g., a mobile phone) in the above method embodiments.

[0141] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform various functions or steps performed by the electronic device (e.g., a mobile phone) in the above method embodiments.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0146] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0147] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A rendering method, characterized in that, Applied to electronic devices, the method includes: Obtain information about the rendering node to be updated, including the identifier of the rendering node to be updated and / or the identifier of the parent node of the rendering node to be updated; If the information of the rendering node to be updated includes the identifier of the rendering node to be updated, the first pointer corresponding to the rendering node to be updated is queried from the first data structure according to the identifier of the rendering node to be updated; if the information of the rendering node to be updated includes the identifier of the parent node of the rendering node to be updated, the second pointer corresponding to the parent node of the rendering node to be updated is queried from the first data structure according to the identifier of the parent node of the rendering node to be updated; wherein, the first data structure is used to record the identifier and pointer of each rendering node in the first rendering tree, and the first rendering tree is the rendering tree corresponding to the first display interface; Update at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree to obtain a second rendering tree; wherein, the first pointer points to a first memory location, the first memory location stores the first information of the rendering node to be updated in the first rendering tree, the second pointer points to a second memory location, the second memory location stores the second information of the parent node of the rendering node to be updated; The elements on the second display interface are rendered according to the second rendering tree, and the second display interface is the next frame display interface after the first display interface.

2. The method according to claim 1, characterized in that, The first information includes the first rendering instruction of the rendering node to be updated. If the rendering node to be updated is a rendering node to be modified, the information of the rendering node to be updated also includes the second rendering instruction corresponding to the rendering node to be updated. The step of updating at least one of the first information of the rendering node to be updated in the first rendering tree and the second information of the parent node of the rendering node to be updated includes: Delete the first rendering instruction from the first memory location and write the second rendering instruction into the first memory location.

3. The method according to claim 1 or 2, characterized in that, When the rendering node to be updated is a rendering node to be deleted, at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in updating the first rendering tree includes: Delete the first information from the first memory location, and delete the first pointer of the rendering node to be updated from the second information, wherein the second information includes the child node pointer information of the parent node of the rendering node to be updated.

4. The method according to claim 3, characterized in that, The information of the rendering node to be updated also includes the identifiers of the child nodes of the rendering node to be updated, and the method further includes: Based on the identifier of the child node of the rendering node to be updated, query the third pointer corresponding to the child node of the rendering node to be updated from the first data structure; Delete the third information in the third memory location pointed to by the third pointer in the first rendering tree.

5. The method according to claim 3 or 4, characterized in that, The method further includes: Remove the identifier and pointer of the node to be updated from the first data structure.

6. The method according to any one of claims 1-5, characterized in that, When the rendering node to be updated is a newly added rendering node, before updating at least one of the first information of the rendering node to be updated and the second information of the parent node of the rendering node to be updated in the first rendering tree, the method further includes: Determine the fourth pointer corresponding to the rendering node to be updated. The fourth pointer is used to indicate the memory location where the information of the rendering node to be updated is stored. The information of the rendering node to be updated also includes the second rendering instruction corresponding to the rendering node to be updated. At least one of the first information of the rendering node to be updated in the first rendering tree and the second information of the parent node of the rendering node to be updated includes: Add the fourth pointer corresponding to the rendering node to be updated to the second information in the second memory location. The second information includes the child node pointer information of the parent node of the rendering node to be updated.

7. The method according to claim 6, characterized in that, The method further includes: Add the identifier of the node to be updated and the fourth pointer to the first data structure.

8. The method according to any one of claims 1-7, characterized in that, The first data structure includes a MAP container.

9. The method according to any one of claims 1-8, characterized in that, The electronic device includes a first application process and a rendering service, and obtaining the information of the rendering node to be updated includes: The rendering service receives a first data packet from the first application process, the first data packet including at least one of a first type identifier, a second type identifier, or a third type identifier; The first type identifier is used to indicate the information of the rendering node to be modified, the second type identifier is used to indicate the information of the rendering node to be deleted, and the third type identifier is used to indicate the information of the rendering node to be added.

10. The method according to claim 9, characterized in that, When the information of the rendering node to be updated includes the identifier of the rendering node to be updated, the first pointer corresponding to the rendering node to be updated is queried from the first data structure according to the identifier of the rendering node to be updated. When the information of the rendering node to be updated includes the identifier of the parent node of the rendering node to be updated, querying the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure based on the identifier of the parent node of the rendering node to be updated includes: When the rendering node to be updated is a rendering node to be modified, the first data packet includes the identifier of the rendering node to be updated; the rendering service queries the first pointer corresponding to the rendering node to be updated from the first data structure according to the identifier of the rendering node to be updated; When the rendering node to be updated is a rendering node to be deleted, the first data packet includes the identifier of the rendering node to be updated and the identifier of the parent node of the rendering node to be updated; the rendering service queries the first pointer corresponding to the rendering node to be updated from the first data structure according to the identifier of the rendering node to be updated; and queries the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure according to the identifier of the parent node of the rendering node to be updated. When the rendering node to be updated is a new rendering node to be added, the first data packet includes the identifier of the rendering node to be updated and the identifier of the parent node of the rendering node to be updated; the rendering service queries the second pointer corresponding to the parent node of the rendering node to be updated from the first data structure according to the identifier of the parent node of the rendering node to be updated.

11. The method according to claim 10, characterized in that, The step of updating at least one of the first information of the rendering node to be updated in the first rendering tree and the second information of the parent node of the rendering node to be updated includes: When the rendering node to be updated is a rendering node to be modified, the first information includes the first rendering instruction of the rendering node to be updated, and the information of the rendering node to be updated also includes the second rendering instruction corresponding to the rendering node to be updated. The rendering service deletes the first rendering instruction from the first memory location and writes the second rendering instruction into the first memory location. If the rendering node to be updated is a rendering node to be deleted, the rendering service deletes the first rendering instruction in the first memory location, and deletes the first pointer of the rendering node to be updated from the child node pointer information of the parent node of the rendering node to be updated. When the rendering node to be updated is a new rendering node to be added, the rendering service determines the memory location and fourth pointer corresponding to the rendering node to be updated. The fourth pointer is used to indicate the memory location where the information of the rendering node to be updated is stored. The rendering service adds the fourth pointer to the child node pointer information of the parent node of the rendering node to be updated.

12. An electronic device, characterized in that, The electronic device includes a display screen, a processor, and a memory; the memory stores program instructions; the processor is configured to execute the program instructions stored in the memory, causing the electronic device to perform the method as described in any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes program instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-11.

Images