Image rendering method and electronic device
By pre-drawing static elements to generate pre-drawn images and updating drawing instructions, the problem of lag in electronic device interface display is solved, improving rendering speed and user experience.
Patent Information
- Application Number
- PCT/CN2025/135920
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2025-11-19
- Publication Date
- 2026-11-12
AI Technical Summary
Electronic devices are prone to lag when displaying interfaces, which affects the user experience, especially when processing complex graphics scenes where GPU load can become a bottleneck.
By pre-drawing static elements to generate pre-drawn images and updating their drawing instructions to second drawing instructions, the pre-drawn images can be drawn directly, reducing the amount of computation per frame and improving rendering speed.
It improves the rendering speed of interface images, reduces screen lag on electronic devices, and enhances system responsiveness and user experience.
Smart Images

Figure CN2025135920_12112026_PF_FP_ABST
Abstract
Description
Image rendering methods and electronic devices
[0001] This application claims priority to Chinese patent application filed on May 9, 2025, with application number 202510597680.8 and entitled "Image Rendering Method and Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of image rendering technology, and in particular to an image rendering method and electronic device. Background Technology
[0003] With technological advancements, electronic devices are offering increasingly diverse functions. However, as the amount of data processed and the computational complexity increase, electronic devices are prone to lag and stuttering during interface display, negatively impacting the user experience. Summary of the Invention
[0004] This application provides an image rendering method and an electronic device that can improve the rendering speed of interface images and alleviate the lag problem when the electronic device displays the interface.
[0005] In a first aspect, embodiments of this application provide an image rendering method applied to an electronic device. The method includes: obtaining at least one static element in a first interface image as a first element; obtaining a first drawing instruction for the first element; drawing a pre-drawn image corresponding to the first element according to the first drawing instruction; generating a second drawing instruction for the first element based on the pre-drawn image corresponding to the first element; updating the first drawing instruction of the first element in the first drawing data of the first interface image to the second drawing instruction of the first element to obtain updated first drawing data; and drawing the first interface image according to the updated first drawing data. In this method, the first element in the first interface image is first pre-drawn to obtain a pre-drawn image. Then, the first drawing instruction of the first element in the first drawing data of the first interface image is updated to the second drawing instruction of the first element. Therefore, when drawing the first interface image, it is not necessary to draw the first element according to the first drawing instruction; the pre-drawn image can be drawn directly. Consequently, when the interface image includes the first element in subsequent interfaces, the pre-drawn image of the first element can also be drawn directly, thereby improving the rendering speed of the interface image and alleviating the lag problem when the electronic device displays the interface.
[0006] In one possible implementation, the method further includes: when the second interface image includes a first element, updating the first drawing instruction of the first element in the second drawing data of the second interface image to the second drawing instruction of the first element to obtain updated second drawing data; and drawing the second interface image based on the updated second drawing data. In this method, when the second interface image includes a first element, updating the first drawing instruction of the first element in the second drawing data of the second interface image to the second drawing instruction of the first element allows for direct drawing of the pre-drawn image of the first element when drawing the second interface image, thereby improving the rendering speed of the second interface image and alleviating the lag problem when electronic devices display their interfaces.
[0007] In one possible implementation, before drawing the second interface image based on the updated second drawing data, the method further includes: obtaining a second element in the second interface image, wherein the second element is a static element in the second interface image and does not have a corresponding pre-drawn image; obtaining a first drawing instruction for the second element; drawing the pre-drawn image corresponding to the second element according to the first drawing instruction; generating a second drawing instruction for the second element based on the pre-drawn image corresponding to the second element; and updating the first drawing instruction for the second element in the second drawing data of the second interface image to the second drawing instruction for the second element. In this method, second elements in the second interface image that do not have pre-drawn images can be pre-drawn to obtain pre-drawn images. Then, the first drawing instruction for the second element in the second drawing data of the second interface image is updated to the second drawing instruction for the second element. Therefore, when drawing the second interface image, it is not necessary to draw the second element according to the first drawing instruction; the pre-drawn image can be drawn directly. Consequently, when the interface image includes the second element in subsequent iterations, the pre-drawn image of the second element can also be drawn directly, thereby improving the rendering speed of the interface image and alleviating the lag problem when electronic devices display their interfaces.
[0008] In one possible implementation, generating a second drawing instruction for the first element based on a pre-drawn image corresponding to the first element includes: allocating memory space for the first element in the target memory, storing the pre-drawn image corresponding to the first element in the memory space, and determining the second drawing instruction for the first element based on the pre-drawn image corresponding to the first element and the memory address of the memory space. In this method, storing the pre-drawn image corresponding to the first element in memory thereby enables the generation of the second drawing instruction for the first element based on the pre-drawn image.
[0009] In one possible implementation, the target memory is a DMA cache.
[0010] In one possible implementation, the method further includes: obtaining a first set in a first interface image, the first set including multiple third elements, each of which is a static element in the first interface image, and the display positions of the third elements are relatively fixed; obtaining drawing instructions for the first set, the drawing instructions for the first set including drawing instructions for each third element; drawing a pre-drawn image corresponding to the first set according to the drawing instructions for the first set; generating a second drawing instruction for the first set based on the pre-drawn image corresponding to the first set; and updating the first drawing data of the first interface image according to the second drawing instruction for the first set. In this method, the first set can also be pre-drawn to obtain a pre-drawn image, so that when drawing the first interface image, it is not necessary to draw the first set according to the first drawing instructions; the pre-drawn image of the first set can be drawn directly, thereby improving the rendering speed of the first interface image and alleviating the lag problem when electronic devices display interfaces.
[0011] In one possible implementation, the method further includes: when the second interface image includes the first set, updating the first drawing instruction of the first set in the second drawing data of the second interface image to the second drawing instruction of the first set. In this method, when drawing subsequent second interface images including the first set, it is not necessary to repeatedly draw the first set according to the first drawing instruction; instead, the pre-drawn image of the first set can be drawn directly. This improves the rendering speed of the second interface image and thus alleviates the lag problem when electronic devices display their interfaces.
[0012] In one possible implementation, the first interface image is the application interface image of a first application. The method further includes: upon detecting that the first application has switched to background operation or stopped running, clearing the pre-drawn image corresponding to the first application in the target memory and releasing the memory space occupied by the pre-drawn image corresponding to the first application; and / or, upon detecting that the overall utilization rate of the target memory reaches a preset first threshold, clearing at least a portion of the pre-drawn images corresponding to the target application in the target memory, and / or, stopping the identification of static elements in the application interface image of the target application, wherein the target application is the application involved in the screen interface image where the first interface image is located, and the target application includes the first application; and / or, upon detecting that the utilization rate of the first application in the target memory reaches a preset second threshold, clearing at least a portion of the pre-drawn images corresponding to the first application in the target memory, and / or, stopping the identification of static elements in the application interface image of the first application. This method provides a memory control strategy for the target memory, improving the processing speed and efficiency of the target memory.
[0013] In one possible implementation, obtaining at least one static element in a first interface image as a first element includes: obtaining at least one element of a preset type from first drawing data of the first interface image as a first element; and / or obtaining at least one static element in the first interface image as a first element based on a drawing interface called by a first application to which the first interface image belongs. This achieves the acquisition of the first element in the first interface image.
[0014] In a second aspect, embodiments of this application provide an electronic device, including: a processor and a memory; wherein one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the processor, cause the electronic device to perform the method of any one of the first aspects.
[0015] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method of any one of the first aspects.
[0016] Fourthly, embodiments of this application provide a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method of any one of the first aspects. Attached Figure Description
[0017] Figure 1 is a schematic diagram of an electronic device provided in an embodiment of this application;
[0018] Figure 2 is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application;
[0019] Figure 3 is a schematic diagram of the first type of image rendering method provided in the embodiment of this application;
[0020] Figure 4 is a schematic diagram of a second process of the image rendering method provided in an embodiment of this application;
[0021] Figures 5A to 5C are schematic diagrams of the interfaces provided in the embodiments of this application;
[0022] Figure 5D is a schematic diagram of the third process of the image rendering method provided in the embodiment of this application;
[0023] Figure 5E is a schematic diagram of the fourth process of the image rendering method provided in the embodiments of this application;
[0024] Figure 5F is a schematic diagram of the fifth process of the image rendering method provided in the embodiments of this application;
[0025] Figure 5G is a schematic diagram of a pre-drawn image provided in an embodiment of this application;
[0026] Figure 6 is a schematic diagram of an electronic device provided in an embodiment of this application;
[0027] Figure 7A is a schematic diagram of the sixth process of the image rendering method provided in the embodiments of this application;
[0028] Figure 7B is a schematic flowchart of a specific method shown in Figure 7A provided in an embodiment of this application;
[0029] Figure 7C is a schematic diagram of another specific process of the method shown in Figure 7A provided in an embodiment of this application;
[0030] Figure 8 is a schematic diagram of the seventh process of the image rendering method provided in the embodiments of this application;
[0031] Figure 9 is a schematic diagram of the eighth process of the image rendering method provided in the embodiments of this application;
[0032] Figure 10 is a schematic diagram of the ninth type of image rendering method provided in the embodiments of this application;
[0033] Figure 11 is a schematic diagram of the structure of the drawing tree provided in an embodiment of this application;
[0034] Figure 12 is a schematic diagram of the tenth process of the image rendering method provided in the embodiment of this application;
[0035] Figure 13 is a schematic diagram of the eleventh process of the image rendering method provided in the embodiments of this application;
[0036] Figure 14 is a schematic diagram of the twelfth process of the image rendering method provided in the embodiment of this application;
[0037] Figure 15 is a schematic diagram of the thirteenth process of the image rendering method provided in the embodiment of this application;
[0038] Figure 16 is a schematic diagram of the fourteenth process of the image rendering method provided in the embodiment of this application;
[0039] Figure 17 is a schematic diagram of the fifteenth type of image rendering method provided in the embodiments of this application;
[0040] Figure 18 is a schematic diagram of the sixteenth process of the image rendering method provided in the embodiments of this application. Detailed Implementation
[0041] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.
[0042] Currently, with the continuous development of computer technology, especially the increasing demands in fields such as graphics processing and artificial intelligence, the computing power of the central processing unit (CPU) and graphics processing unit (GPU) has gradually become a significant bottleneck affecting the system performance of electronic devices. Particularly in applications such as graphics rendering, video processing, game development, virtual reality (VR), and augmented reality (AR), the increasing volume of processed data and computational complexity leads to a significant increase in application interface lag, directly impacting the smoothness of system operation and user experience.
[0043] Currently, although the computing power of CPUs and GPUs continues to improve, the rate of improvement is gradually slowing down due to physical limitations such as hardware architecture, energy efficiency, and heat dissipation. Graphics processing tasks typically require a large amount of parallel computing. While GPUs have certain advantages in this area, their computing resources are still limited, and for some complex graphics scenes, GPU load can easily become a bottleneck, leading to latency and stuttering during rendering. More commonly, in some mid-to-low-end products, CPU and GPU computing power become the core issue for user experience.
[0044] To address this issue, graphics load optimization has become a critical challenge. Optimization strategies include efficient algorithm design, rational scheduling of hardware resources, load distribution, and data compression techniques. These techniques can not only effectively reduce the demand for computing resources but also improve the parallelism and resource utilization in graphics processing, thereby alleviating or eliminating stuttering and improving system responsiveness and user experience.
[0045] Against this background, embodiments of this application provide an image rendering method that can improve image rendering speed and alleviate the problem of interface lag in electronic devices.
[0046] The image rendering method provided in this application can be applied to electronic devices, such as mobile phones, tablet computers (PADs), personal computers (PCs), smart wearable devices such as smartwatches, smart glasses, smart screens, and in-vehicle devices.
[0047] Figure 1 shows a schematic diagram of the structure of electronic device 100. As shown in Figure 1, electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0048] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0049] Processor 110 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.
[0050] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.
[0051] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0052] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0053] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.
[0054] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.
[0055] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.
[0056] The electronic device provided in this application embodiment can run an operating system (OS). This operating system can be various operating systems used in industry, such as an operating system based on OpenHarmony, like HarmonyOS; or other operating systems, such as... A mobile operating system can also refer to various open-source operating systems or their derivatives, such as Linux OS and other embedded operating systems; it can also be a future type of operating system, such as an AI operating system based on artificial intelligence. An operating system is a set of interconnected system software programs that manage and control the operation of electronic devices, utilize and run hardware and software resources, and provide public services to organize user interaction. In electronic devices, the operating system connects downwards to the physical devices at the hardware layer and upwards to provide a runtime environment for application software.
[0057] An operating system typically includes a kernel layer, a middleware layer, and an application layer. The application layer includes applications, which can include system applications and third-party applications. The middleware layer includes a suite of software providing various services to application developers, or frameworks providing services such as databases, multimedia, and graphics, or capabilities such as distributed scheduling and system scaling. For example, the middleware layer may include a framework layer and / or a system service layer. The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The system service layer includes the system's core capabilities, providing services to applications through the framework layer. The kernel layer is the layer between hardware and software. The kernel layer may include hardware drivers and the operating system kernel. In addition to providing hardware drivers, the kernel layer also supports functions such as memory management and system process management.
[0058] The electronic devices we use in our daily lives come in various types and forms, and are applied in a wide range of scenarios. Therefore, based on the different forms and functions of electronic devices, different application scenarios, and different user needs, the operating systems used in these devices may also differ. The basic functions implemented by the electronic device provided in this application can be implemented using a general-purpose operating system or a dedicated operating system. To more clearly illustrate the implementation of the embodiments of this application under a specific operating system, Figure 2 shows the architecture of HarmonyOS. Those skilled in the art can deduce the implementation of the embodiments of this application under other specific operating systems, such as Android™.
[0059] The software architecture of electronic devices can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. Layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem level depending on the deployment scenario of different device forms. Each subsystem can also be tailored, added, or combined at the functional level.
[0060] The kernel layer, also known as the kernel abstraction layer (KAL), provides basic kernel capabilities to the upper layers by shielding the differences between multiple kernels. These capabilities include, but are not limited to, process / thread management, memory management, file system, network management, and peripheral device management.
[0061] Kernel Subsystem: Supports the selection of a suitable OS kernel for different resource-constrained devices, including but not limited to Linux kernel, HarmonyOS kernel, LiteOS (Lite Operating System), etc.
[0062] Driver Subsystem: The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: display drivers, camera drivers, audio drivers, Bluetooth drivers, sensor drivers, etc.
[0063] The system service layer comprises the core capabilities of the system, providing services to applications through the framework layer. This layer includes, but is not limited to, the following subsystems:
[0064] The system's basic capability subsystem set provides fundamental capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. This set may include distributed soft bus, distributed data management, distributed task scheduling, and Ark multi-language runtime; it may also include multi-modal input subsystem, graphics subsystem, security subsystem, and AI subsystem.
[0065] Basic software service subsystem set: provides public and general software services; the basic software service subsystem set may include event notification subsystem, telephone service subsystem, multimedia subsystem, etc.
[0066] Enhanced software service subsystem suite: Provides differentiated enhanced software services for different devices; the enhanced software service subsystem suite may include smart screen proprietary business subsystem, wearable proprietary business subsystem, IoT proprietary business subsystem, etc.
[0067] Hardware service subsystem set: Provides hardware services; the hardware service subsystem set may include location service subsystem, user IAM (Identity and Access Management) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service subsystem, etc.
[0068] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices.
[0069] Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices.
[0070] The distributed soft bus provides communication-related capabilities for seamless interconnection between multiple devices, including: WLAN service capabilities, Bluetooth service capabilities, soft bus, inter-process communication RPC (Remote Procedure Call), and StarFlash communication capabilities.
[0071] Ark Multilingual Runtime is a unified compilation runtime platform designed to support the joint compilation and execution of multiple programming languages and multiple chip platforms.
[0072] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes: the ArkUI framework (which provides a complete infrastructure for UI development of system applications, including UI functions such as components, layouts, animations, and interactive events, as well as a real-time interface preview tool), the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of Abilities). Different devices may have different operating systems, and the APIs they support may also differ.
[0073] The HarmonyOS API is a series of open capabilities provided to support HarmonyOS application development. The HarmonyOS API can be set at the framework layer or independently of the framework layer. The HarmonyOS API includes the Audio API (audio service), Push API (push service), and Account API (account service), among others.
[0074] The application layer can include applications. Applications can include system applications and extended / third-party applications. System applications can include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party applications can include social applications, travel applications, etc.
[0075] The image rendering method of this application embodiment will be described in detail in conjunction with the structure of the above-described electronic device in the following embodiments.
[0076] In the image rendering method of this application embodiment, the processed image may be, for example, an interface image displayed by an electronic device to a user.
[0077] A user interface (UI) is the graphical interface through which users interact with the operating system or applications of an electronic device (such as a mobile phone). An UI can include visual elements such as icons, buttons, menus, and animations. In related technologies, these visual elements may also be referred to as elements, controls, or UI elements. When displaying an UI, an electronic device needs to continuously draw, render, and display UI images according to a certain frame rate. In other words, the UI displayed by an electronic device is achieved by displaying UI images at a certain frame rate.
[0078] In this embodiment of the application, the complete image displayed on the screen of the electronic device is referred to as the screen interface, and the image displayed by the application to the user is referred to as the application interface.
[0079] Visual elements on a screen interface may be provided by one or more applications. In other words, a screen interface can be a single application interface or a combination of multiple application interfaces. In this embodiment, the application providing visual elements for the screen interface is referred to as the application involved in the screen interface image. In other words, the application involved in the screen interface is the application to which each of the several application interfaces constituting the screen interface belongs. Example 1: The screen interface includes: a status bar provided by a status bar application, which displays information such as battery level, WLAN, data traffic, and carrier information through text or icons; and an application interface provided by a browser application. In this case, the applications involved in the screen interface include: the status bar application and the browser application. Example 2: If the application interface provided by the browser application is displayed in full screen, then the screen interface only includes the application interface of the browser application. In this case, the applications involved in the screen interface include: the browser application.
[0080] In some embodiments, to display the screen interface, the applications involved in the screen interface need to draw their own application interface images, and then composite the application interface images of each application as layer images to obtain a complete screen interface image, which is then sent to the display screen for display. This process of sending the screen interface image to the display screen can also be called display delivery. In this embodiment, the rendering process of the screen interface image is shown in Figure 3. After the application determines the drawing data for a frame of application interface image, it sends the drawing data to the user interface (UI) framework of the framework layer. The UI framework constructs the drawing node data of the application interface image based on the drawing data and sends the drawing node data to the 2D engine module. The 2D engine module performs graphics drawing and accelerated graphics rendering, converts the drawing node data into a sequence of instructions that the GPU can execute, and sends the instruction sequence of the application interface to the GPU driver. The GPU is then driven to draw the application interface image, obtaining the application interface image, which is then sent to the layer compositing module. The layer compositing module composites the application interface images of each application involved in the screen interface to obtain the screen interface image.
[0081] In other embodiments, to improve the display efficiency of the screen interface, instead of each application drawing its own interface image, the screen interface image drawing data is obtained based on the drawing data of each application's interface image. This drawing data is then used for centralized drawing, resulting in the final screen interface image, which is then displayed. The rendering process for the screen interface image in this embodiment is illustrated in Figure 4. Each application involved in the screen interface can send its application interface image drawing data to the graphics logic processing module. The graphics logic processing module merges this data into drawing node data for the screen interface image and sends it to the 2D engine module. The 2D engine module performs graphics drawing and accelerated graphics rendering, converts the drawing node data into a sequence of instructions executable by the GPU, and then sends this sequence of instructions to the GPU driver. This drives the GPU to draw the screen interface image, thus obtaining the final screen interface image.
[0082] A drawing node represents a visual element in the interface. Drawing node data records the information needed to draw interface images (such as application interface images or screen interface images), such as the hierarchy of drawing nodes, attribute information, and drawing command information. The aforementioned hierarchy and attribute information can also be collectively referred to as layout information.
[0083] In some embodiments, drawing node data can be implemented using a drawing tree. Taking an application interface as an example, the drawing tree includes multiple drawing nodes, each representing a visual element in the application interface. The drawing nodes in the drawing tree have a hierarchical relationship, corresponding to the hierarchical relationship between visual elements in the application interface.
[0084] Each drawing node can include drawing data, which records the node's attribute information and drawing instructions. The attribute information of a drawing node can include, for example, the position, size, angle, and opacity of the corresponding visual element. The drawing instructions can include, for example, the shape, material, and texture of the control.
[0085] The elements on a screen interface can be divided into dynamic elements and static elements. Dynamic elements are those with dynamic effects; in other words, the content displayed by dynamic elements changes over time. Static elements are those without dynamic effects; their content remains unchanged during display.
[0086] For example, the desktop of an electronic device can include dynamic elements and static elements. Dynamic elements can include dynamic icons, animated components, animated backgrounds, etc., while static elements can include static icons, static components, static backgrounds, etc. Taking the desktop interface of the electronic device shown in Figure 5A as an example, the events displayed in the time display control 500 change with time, so it is a dynamic element. The clock application icon 501 can change the position of the hour and minute hands in the icon with time, so it is a dynamic element. The icons of applications such as music applications and video applications do not change with time, so they are static elements.
[0087] For example, Figure 5B shows the settings interface provided by an electronic device. The names of the various settings options, such as text 506, the settings controls, such as control 503, and the icons, such as icon 504, are static elements. The status display controls can include states such as on and off, which can change over time, as shown in control 505, and are dynamic elements.
[0088] For example, Figure 5C shows a news interface in a browser. The news title 507 and the text 508 in the news body do not change over time and are considered static elements. The video 509 in the news body can change its content over time and is considered a dynamic element.
[0089] In some embodiments, the process of generating screen interface images or application interface images, including lighting, textures, rendering effects, etc., is referred to as image rendering. The generation of interface images and the generation of various elements in the interface images during the rendering process can be referred to as drawing. It can be understood that the rendering process includes the drawing process of the interface images and the drawing process of the elements in the interface images. It should be noted that in some related technologies, rendering and drawing can also be used interchangeably, or collectively referred to as drawing and rendering.
[0090] In some embodiments provided in this application, when rendering a frame of interface image (e.g., an application interface frame), for at least one static element in the interface image, a pre-drawn image corresponding to the static element can be drawn according to the first drawing instruction of the static element. Based on the pre-drawn image, a second drawing instruction of the static element is generated. When the frame of interface image and each subsequent frame of interface image including the static element are drawn, the first drawing instruction of the static element in the drawing data of the interface image is replaced with the second drawing instruction based on the pre-drawn image. Therefore, it is not necessary to execute the first drawing instruction of the static element in the drawing of each frame of interface image, but to directly execute the second drawing instruction based on the pre-drawn image, thereby improving the drawing speed of the interface image, thereby improving the display efficiency of the screen interface and reducing the lag of the electronic device in displaying the interface.
[0091] For example, referring to Figure 5D, for a frame of interface image that the target application needs to display, the elements in the interface image can be identified and divided into static elements and dynamic elements. For dynamic elements, no processing is performed. For static elements, the GPU performs a drawing based on the first drawing instruction of the static element to obtain a pre-drawn image of the static element, and then generates a second drawing instruction of the static element. Finally, the drawing data of the interface image is updated according to the second drawing instruction of the static element. The GPU performs drawing processing based on the updated drawing data and sends it to the display.
[0092] Taking the settings interface in Figure 5B as an example, when the user scrolls up and down on the settings interface, the icons and text in each frame of the interface image can be identified as static elements. As the interface images are continuously displayed, a pre-drawn image of each static element is generated and can be reused after generation. Thus, during the repeated frame refresh process of scrolling up and down on the settings interface, when the GPU draws these frame interface images, it only needs to draw the pre-drawn images of these static elements, without having to draw the static elements into images. Since the GPU is more efficient at processing images than elements, it can improve the efficiency of the GPU in drawing each frame of the interface image, thereby improving the display efficiency of the screen interface and reducing the stuttering of electronic devices when displaying the interface.
[0093] This method can improve CPU performance. Specifically, by pre-drawing static elements as images, the static elements become static images, eliminating the need to calculate the layout and style of static elements every frame during rendering, thus reducing the CPU's computational burden per frame. This is particularly effective when the interface needs to be frequently updated or displays large amounts of text, significantly improving rendering performance.
[0094] This method can also improve GPU performance. Specifically, GPUs currently process images more effectively and efficiently than static elements, especially for complex static element layouts. The GPU's rendering capabilities can be directly utilized to accelerate the display of interface images, reducing repetitive GPU processing.
[0095] In other embodiments provided in this application, when rendering a frame of interface image (e.g., a frame of application interface), a pre-drawn image corresponding to the set of static elements in the interface image can be drawn first according to the first drawing instruction of the static elements in the set. A second drawing instruction of the set is generated based on the pre-drawn image. When the frame of interface image and each subsequent frame of interface image including the set are drawn, the second drawing instruction of the set is used to realize the drawing of the set in the interface image. Therefore, it is not necessary to execute the first drawing instruction of the set in the drawing of each frame of interface image, but to directly execute the second drawing instruction based on the pre-drawn image, thereby improving the drawing speed of the interface image, thereby improving the display efficiency of the screen interface and reducing the lag of the electronic device in displaying the interface.
[0096] For example, referring to Figure 5E, for a frame of interface image that the target application needs to display, the elements in the interface image can be identified and divided into sets (including multiple static elements) and other elements. For other elements, no processing is performed. For sets, the GPU performs a drawing based on the first drawing instruction of the set to obtain a pre-drawn image of the set, and then generates a second drawing instruction for the set. Finally, the drawing data of the interface image is updated according to the second drawing instruction of the set. The GPU performs drawing processing based on the updated drawing data and sends it to the display.
[0097] In some embodiments provided in this application, the solutions of the two embodiments described above can be combined. That is, when rendering a frame of interface image (e.g., a frame of application interface), a pre-drawn image is drawn for at least one static element in the interface image according to the static element, and a second drawing instruction for the static element is generated based on the pre-drawn image. For a set of multiple static elements in the interface image, a pre-drawn image is drawn according to the set, and a second drawing instruction for the set is generated based on the pre-drawn image. Thus, when drawing the current frame of interface image and each subsequent frame of interface image, if the frame of interface image includes the above-mentioned static elements, the static elements can be drawn directly based on the second drawing instruction for the static elements. If the frame of interface image includes the above-mentioned set, the set can be drawn directly based on the second drawing instruction for the set. This improves the drawing speed of the interface image, thereby improving the display efficiency of the screen interface and reducing the lag of the electronic device's interface display.
[0098] For example, referring to Figure 5F, for a frame of interface image that the target application needs to display, the elements in the interface image can be identified and divided into sets (including multiple static elements), other static elements, and dynamic elements. Dynamic elements are not processed. For sets, the GPU performs a first drawing based on the set's first drawing instruction to obtain a pre-drawn image of the set, and then generates a second drawing instruction for the set. For the other static elements, the GPU performs a first drawing based on the static element's first drawing instruction to obtain a pre-drawn image of the static element, and then generates a second drawing instruction for the static element. Finally, the drawing data of the interface image is updated according to the second drawing instructions of the set and other static elements. The GPU then performs drawing processing based on the updated drawing data and sends it for display.
[0099] As shown in Figure 5B, text 506, icon 504, and control 503 are all static elements, and their display positions are relatively fixed. Therefore, they can be identified as a set, which includes text 506, icon 504, and control 503. Thus, a pre-drawn image is drawn based on the first drawing instruction of the three elements, as shown in Figure 5G.
[0100] This embodiment enables the GPU to change from drawing images of multiple static elements to drawing a single image, reducing the GPU's overhead of multiple drawing operations and further reducing the GPU's drawing load.
[0101] The following examples provide a more detailed description of the possible implementations of the image rendering method of this application.
[0102] Figure 6 is a schematic diagram of an electronic device to which the image rendering method of this application is applicable. As shown in Figure 6, the electronic device may include: a target application, a static element recognition module, a static element cache processing module, an image instruction generation module, a memory management module, a graphics logic processing module, a drawing execution module, and a display driver.
[0103] The implementation structure of the rendering execution module can differ depending on the rendering strategy. Taking the rendering strategy shown in Figure 3 as an example, the rendering execution module may include a 2D engine module, a GPU driver, and a layer compositing module. Taking the rendering strategy shown in Figure 4 as an example, the rendering execution module may include a 2D engine module and a GPU driver. In Figure 6, taking the centralized rendering strategy shown in Figure 4 as an example, the rendering execution module includes a 2D engine module and a GPU driver.
[0104] Referring to Figures 3 and 4 and their corresponding descriptions, the specific functions implemented by the graphics logic processing module may differ depending on the rendering strategy, which will not be elaborated here.
[0105] The target application can be any application involved in the screen interface. When the electronic device needs to display the application interface image, the drawing data of the application interface image is sent down, including layout information and drawing instructions.
[0106] The static element recognition module is used to identify static elements in the interface image, such as static elements in the application interface image provided by the target application. It can also be used to identify a first set of static elements in the interface image, where the display positions of the static elements in the first set are relatively fixed. In some embodiments, the static element recognition module can also be used to send the first drawing instruction for the static element and / or the first drawing instruction for the first set to the GPU driver. In some embodiments, sending the first drawing instruction for the static element and / or the first drawing instruction for the first set to the GPU driver can be implemented via a graphics logic processing module, a 2D engine module, etc. The aforementioned first drawing instruction is the drawing instruction recorded in the drawing data of the application interface image sent by the target application.
[0107] The static element caching module is used to allocate memory space for static elements to support the storage of pre-drawn images corresponding to the static elements.
[0108] The memory management module is used for memory management. For example, the memory management module can be used to manage memory provided by DDR, which is a type of memory.
[0109] The drawing execution module is used to draw the pre-drawn image corresponding to the static element according to the first drawing instruction of the static element, draw the pre-drawn image corresponding to the first set according to the first drawing instruction of the first set composed of static elements, and draw the interface image.
[0110] The image instruction generation module is used to generate a second drawing instruction for a static element based on a pre-drawn image corresponding to the static element. It can also be used to generate a second drawing instruction for a first set based on a pre-drawn image corresponding to the first set.
[0111] The graphics logic processing module is used to receive the drawing data of the interface image and update the drawing data of the interface image according to the second drawing instruction of the static element and / or the second drawing instruction of the first set.
[0112] The drawing execution module is used to draw the interface image based on the drawing data of the interface image. For specific implementation details, please refer to Figures 3 and 4, or other relevant drawing and rendering technologies. This application embodiment does not impose any limitations. The 2D engine module is mainly used for graphics drawing and accelerated graphics rendering.
[0113] GPU drivers are used to drive the GPU. A GPU is hardware used to process graphics and render images; it is the physical device in computers and other electronic devices responsible for generating and rendering images.
[0114] The display driver is used to drive the monitor and display the screen interface image, enabling the monitor to interact directly with the user.
[0115] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may combine certain modules or split certain modules, etc. For example, as shown in FIG6, the static element identification module and the static element cache processing module may be merged into a single static element processing module.
[0116] In some embodiments, the image rendering method of this application can be executed when a preset trigger condition is detected. Specifically, the trigger condition can be detected by a static element recognition module, and the image rendering method of this application can be triggered upon detection of the trigger condition. It is understood that before the image rendering method of this application is triggered, the application interface images to be displayed by the target application can be drawn using application interface image rendering methods in related technologies.
[0117] In some embodiments, the target application may send a startup notification message to the static element recognition module at startup, thereby enabling the static element recognition module to determine that a triggering condition has been detected.
[0118] In some embodiments, the static element recognition module can monitor the startup of each application, thereby determining that a triggering condition has been detected when the target application is detected to be starting.
[0119] In some embodiments, the electronic device may provide the user with controls for controlling whether the electronic device performs interface display acceleration, including on and off states. When the user sets the control to the on state, the static element recognition module can detect the trigger condition. Correspondingly, when the user sets the control to the off state, the static element recognition module can detect the stop execution trigger condition and stop executing the image rendering method of this application embodiment.
[0120] When the target application sends out the rendering data of the application interface image, it can send out the complete rendering data of each frame of the application interface image, or it can send out the incremental rendering data of each frame of the application interface image. The following uses Figures 7A and 8 to illustrate the possible implementation of the image rendering method of this application embodiment when the target application sends out the complete rendering data of each frame of the application interface image, and uses Figure 9 to illustrate the possible implementation of the image rendering method of this application embodiment when the target application sends out the incremental rendering data of each frame of the application interface image.
[0121] Figure 7A is a flowchart illustrating an image rendering method provided by an embodiment of this application in conjunction with the electronic device shown in Figure 6. As shown in Figure 7A, the method may include:
[0122] Step 701: The target application sends the complete drawing data of the first application interface image to the graphics logic processing module.
[0123] The target application in this step can be any application involved in the screen interface.
[0124] The first application interface image can be any frame of application interface image that the target application needs to display. In some embodiments, the first application interface image can be the first frame of application interface image that the target application will display when the image rendering method of this application starts to execute.
[0125] Step 702: The static element recognition module obtains at least one static element in the first application interface image as the first target element.
[0126] It is understood that the static element recognition module in this step can obtain all or part of the static elements in the first application interface image, and this application embodiment does not impose any limitations.
[0127] In some embodiments, the first application interface image may include multiple types of elements, some of which are static elements and some of which are dynamic elements. The static element recognition module can obtain elements of a preset type in the first application interface image, thereby obtaining some or all of the static elements in the first application interface image as the first target elements. The preset type may be one type or multiple types, and the preset type may include, for example, text, scalable vector graphics (SVG) icons, etc.
[0128] In some embodiments, the static element recognition module can obtain all or part of the static elements in the first application interface image as the first target element.
[0129] In some embodiments, the complete drawing data of the first application interface image may record the drawing data of each element in the first application interface image. The drawing data of each element may include the type of each element. The element type can distinguish whether the element is a static element or a dynamic element. In this step, the static element recognition module can obtain a static element of a preset type based on the complete drawing data of the first application interface image, and use it as the first target element.
[0130] This application does not limit how the static element recognition module obtains the complete drawing data of the first application interface image. In some embodiments, as shown in FIG7B, the static element recognition module can obtain the complete drawing data of the first application interface image from the target application. In other embodiments, the static element recognition module can monitor the data sent by the target application to the graphics logic processing module to obtain the complete drawing data of the first application interface image. In still other embodiments, as shown in FIG7C, the static element recognition module can obtain the complete drawing data of the first application interface image from the graphics logic processing module.
[0131] In other embodiments, the electronic device can provide drawing interfaces for various types of elements for the target application. After determining the UI layout of the first application interface image, the target application can call the corresponding drawing interface for each element to realize the drawing of each element. Correspondingly, the drawing interface can maintain element information corresponding to the first application interface image of the target application, which is used to record which elements of the first application interface image the target application called the drawing interface for. In this step, the static element recognition module can obtain the element information corresponding to the first application interface image of the target application from the drawing interface of the preset type, and summarize it to obtain the element of the preset type as the first target element. The preset type can be one or more types included in static elements. For example, an electronic device can provide text drawing interfaces, SVG icon drawing interfaces, and animated icon drawing interfaces. If the first application interface image includes elements 1-6, where elements 1-3 are text, element 4 is an SVG icon, and elements 5-6 are animated icons, then the target application can call the text drawing interface for elements 1-3, the SVG icon drawing interface for element 4, and the animated icon drawing interface for elements 5-6. Correspondingly, the text drawing interface can record the element information corresponding to the first application interface image of the target application: element 1, element 2, and element 3. The SVG icon drawing interface can record the elements corresponding to the first application interface image of the target application. Information: Element 4, the dynamic icon drawing interface can record: element information corresponding to the first application interface image of the target application: element 5, element 6. In this step, if the static element recognition module wants to recognize two types of static elements in the first application interface image of the target application, namely text and SVG icons, then in this step, the static element recognition module can obtain the element information corresponding to the first application interface image of the target application from the text drawing interface: element 1, element 2, element 3, and obtain the element information corresponding to the first application interface image of the target application from the SVG icon drawing interface: element 5, element 6, thereby summarizing elements 1 to 3 and elements 5 to 6 as the first target element.
[0132] It is understandable that there may be situations where the static element recognition module fails to recognize static elements in the first application interface image. In this case, the process of static element recognition and image pre-drawing for the first application interface image can be terminated, and the first application interface image can be drawn directly based on the first drawing data.
[0133] In some embodiments, the static element identification module may also transmit information such as the number and size of the static elements included in the first target element to the static element cache processing module.
[0134] Step 703: The static element recognition module obtains the first drawing instruction of the first target element and sends the first drawing instruction of the first target element to the static element cache processing module.
[0135] In some embodiments, the first drawing instruction in this step may be a drawing instruction for the first target element recorded in the complete drawing data of the first application interface image. In some embodiments, the static element recognition module may obtain the first drawing instruction for the first target element from the complete drawing data of the first application interface image.
[0136] The method for the static element recognition module to obtain the complete drawing data of the first application interface image can be found in the corresponding description in step 702, and will not be repeated here. It should be noted that if the static element recognition module has already obtained the complete drawing data of the first application interface image in step 702, then it is not necessary to obtain it again in this step, and the first drawing instruction of the first target element can be obtained directly from the complete drawing data of the first application interface image.
[0137] Step 704: The static element cache processing module sends memory request 1 to the memory management module to request memory space for the first target element.
[0138] The memory space requested for the first target element can be used to store the pre-drawn image obtained by the first drawing instruction based on the first target element in subsequent steps.
[0139] Step 705: In response to memory request 1, the memory management module allocates a first memory space for the first target element and sends the first memory address of the first memory space to the static element cache module.
[0140] In some embodiments, the memory space allocated by the memory management module for the first target element can specifically be a direct memory access (DMA) buffer. It can be understood that when the first target element includes multiple static elements, a corresponding first memory space can be allocated for each static element to obtain the first memory address corresponding to each static element. For example, if the first target element includes static element 1 and static element 2, the memory management module can allocate a first memory space for static element 1 to obtain the first memory address corresponding to static element 1, and allocate a first memory space for static element 2 to obtain the first memory address corresponding to static element 2.
[0141] In some embodiments, the hardware memory of the first memory space described above may be provided specifically through double data rate (DDR) memory.
[0142] Step 706: The static element cache processing module sends a drawing request 1 to the drawing execution module.
[0143] Drawing request 1 is used to request the drawing of a first target element. Drawing request 1 may include: a first drawing instruction and a first memory address of the first target element. Optionally, drawing request 1 may also include: an identifier of the first target element. The identifier of the first target element is used to identify the first target element. The identifier of the first target element may be set in the complete drawing data of the first application interface image, and obtained by the static element identification module and transmitted to the static element cache processing module during the aforementioned processing. It is understood that when the first target element includes multiple static elements, drawing request 1 may include: a first drawing instruction and a first memory address of each static element. For example, if the first target element includes: static element 1 and static element 2, the drawing instruction may include: a first drawing instruction and a first memory address of static element 1, and a first drawing instruction and a first memory address of static element 2.
[0144] In some embodiments, the static element cache processing module can send drawing request 1 to the drawing execution module via the graphics logic processing module in this step.
[0145] In some embodiments, the static element cache processing module can send the first memory address of the first memory space allocated for the first target element to the static element identification module, and the static element identification module can send a drawing request 1 to the drawing execution module.
[0146] Step 707: The drawing execution module draws the pre-drawn image corresponding to the first target element based on the first drawing instruction of the first target element, and stores the pre-drawn image corresponding to the first target element in the first memory address corresponding to the first target element.
[0147] For example, if the first target element includes static element 1 and static element 2, a pre-drawn image corresponding to static element 1 can be drawn based on the first drawing instruction of static element 1, and the pre-drawn image corresponding to static element 1 can be stored in the first memory address corresponding to static element 1. Similarly, a pre-drawn image corresponding to static element 2 can be drawn based on the first drawing instruction of static element 2, and the pre-drawn image corresponding to static element 2 can be stored in the first memory address corresponding to static element 2.
[0148] It should be noted that the term "pre-drawn image" is used only to distinguish the image from the application interface image, screen interface image, etc., in the embodiments of this application.
[0149] In some embodiments, the pre-drawn image described above may be a bitmap.
[0150] In this step, the drawing execution module draws the pre-drawn image corresponding to the first target element. This can be achieved by referring to the drawing process of the application interface image, for example, by the method flow shown in Figures 3 and 4.
[0151] Step 708: The drawing execution module sends an instruction generation request 1 to the image instruction generation module.
[0152] Instruction generation request 1 is used to request the generation of a second drawing instruction for the first target element. In some embodiments, instruction generation request 1 may include: a first memory address corresponding to the first target element. Optionally, instruction generation request 1 may also include: an identifier of the first target element.
[0153] Step 709: The image instruction generation module generates a second drawing instruction for the first target element based on the first memory address corresponding to the first target element and the pre-drawn image, and sends the second drawing instruction for the first target element to the graphics logic processing module.
[0154] The second drawing instruction for the first target element can also be called the drawing instruction for the pre-drawn image corresponding to the first target element.
[0155] Step 710: The graphics logic processing module updates the complete drawing data of the first application interface image according to the second drawing instruction of the first target element, and obtains the updated complete drawing data.
[0156] Specifically, in this step, the graphics logic processing module can replace the first drawing instruction of the first target element in the complete drawing data of the first application interface image with the second drawing instruction of the first target element, thereby realizing the update of the complete drawing data.
[0157] Step 711: The drawing execution module draws the first application interface image based on the updated complete drawing data.
[0158] The specific implementation of the drawing execution module drawing the first application interface image based on the updated complete drawing data in this step is not limited in this application embodiment, and can be implemented using methods from related technologies. For example, the first application interface image can be drawn directly based on the updated complete drawing data using the method shown in Figure 3; or, the screen interface image can be drawn centrally based on the drawing data of each application interface image included in the screen interface image using the method shown in Figure 4, where the first application interface image is one of the application interface images included in the screen interface image.
[0159] In this method, the first target element in the first application interface image is pre-drawn to obtain a pre-drawn image. Then, the first drawing instruction of the first target element in the complete drawing data of the first application interface image is updated to the second drawing instruction of the first target element. Thus, when drawing the first application interface image, it is not necessary to draw the first target element according to the first drawing instruction, but to draw the pre-drawn image directly. In addition, when the application interface image includes the first target element in subsequent applications, the pre-drawn image of the first target element can also be drawn directly, thereby improving the rendering speed of the application interface image and alleviating the lag problem when the electronic device displays the interface.
[0160] Figure 8 is a flowchart illustrating an image rendering method provided by an embodiment of this application in conjunction with the electronic device shown in Figure 6. As shown in Figure 8, the method may include:
[0161] Step 801: The target application sends the complete drawing data of the second application interface image to the graphics logic processing module.
[0162] The second application interface image can be any frame of application interface image that the target application needs to display. In some embodiments, the second application interface image can be any frame of application interface image that the target application needs to display after the image rendering method of this application embodiment starts execution. It is understood that before the image rendering method of this application embodiment is executed, each frame of application interface image that the target application needs to display can be drawn using the application interface image drawing and rendering method in the related art.
[0163] Step 802: The static element recognition module obtains at least one static element in the second application interface image as the second target element.
[0164] The second target element can be one or more static elements.
[0165] The implementation of this step can refer to step 702, which will not be repeated here. The only difference is to replace the corresponding terms. For example, replace the first application interface image with the second application interface image in this step, and replace the first target element with the second target element in this step.
[0166] Step 803: The static element recognition module determines whether there is a pre-drawn image corresponding to the second target element. If it exists, the branch process for the second target element ends. If it does not exist, the second target element is used as the third target element, and step 804 is executed.
[0167] When the second target element includes multiple static elements, the static element identification module can perform the judgment in this step for each second target element separately.
[0168] In some embodiments, static element information corresponding to the target application may be recorded. This static element information is used to record static elements identified from the application interface images already displayed by the target application (hereinafter referred to as identified static elements), that is, to record static elements with pre-drawn images in the application interface images already displayed by the target application. The static element information corresponding to the target application can be managed by a static element identification module, a static element caching module, or a memory management module; this application embodiment does not impose any limitations. Accordingly, in this step, the static element identification module can determine whether a second target element is recorded in the static element information corresponding to the target application. If so, a pre-drawn image corresponding to the second target element exists; otherwise, a pre-drawn image corresponding to the second target element does not exist.
[0169] Step 804: The static element recognition module obtains the first drawing instruction of the third target element and sends the first drawing instruction of the third target element to the static element cache processing module.
[0170] Step 805: The static element cache processing module sends memory request 2 to the memory management module to request memory space for the third target element.
[0171] The memory space requested for the third target element can be used to store the pre-drawn image obtained by the first drawing instruction based on the third target element in subsequent steps.
[0172] Step 806: In response to memory request 2, the memory management module allocates a second memory space for the third target element and sends the second memory address of the second memory space to the static element cache module.
[0173] Step 807: The static element cache processing module sends drawing request 2 to the drawing execution module.
[0174] Drawing request 2 is used to request the drawing of a third target element. Drawing request 2 may include: a first drawing instruction and a second memory address of the third target element. Optionally, drawing request 2 may also include: an identifier of the third target element. The identifier of the third target element is used to identify the third target element. The identifier of the third target element can be set in the complete drawing data of the second application interface image, and is obtained by the static element recognition module and transmitted to the static element cache processing module during the aforementioned processing. It is understood that when the third target element includes multiple static elements, drawing request 2 may include: a first drawing instruction and a second memory address of each static element. For example, if the third target element includes: static element 3 and static element 4, drawing instruction 2 may include: a first drawing instruction and a second memory address of static element 3, and a first drawing instruction and a second memory address of static element 4.
[0175] In some embodiments, the static element cache processing module can send drawing request 2 to the drawing execution module via the graphics logic processing module in this step.
[0176] Step 808: The drawing execution module draws the pre-drawn image corresponding to the third target element based on the first drawing instruction of the third target element, and stores the pre-drawn image corresponding to the third target element to the second memory address corresponding to the third target element.
[0177] Step 809: The drawing execution module sends instruction generation request 2 to the image instruction generation module.
[0178] Instruction generation request 2 is used to request the generation of a second drawing instruction for a third target element. In some embodiments, instruction generation request 2 may include: a second memory address corresponding to the third target element. Optionally, instruction generation request 2 may also include: an identifier of the third target element.
[0179] Step 810: The image instruction generation module generates a second drawing instruction for the third target element based on the second memory address corresponding to the third target element and the pre-drawn image, and sends the second drawing instruction for the third target element to the graphics logic processing module.
[0180] The second drawing instruction for the third target element can also be called the drawing instruction for the pre-drawn image corresponding to the third target element.
[0181] The implementation of steps 804 to 810 above can refer to steps 703 to 709, which will not be repeated here. The only difference is to perform corresponding replacements of terms, such as replacing the first target element with the third target element, and replacing the first memory address with the second memory address, etc.
[0182] Step 811: The graphics logic processing module updates the complete drawing data of the second application interface image according to the second drawing instruction of the third target element, and updates the complete drawing data of the second application interface image according to the second drawing instruction of the identified static element corresponding to the target application, so as to obtain the updated complete drawing data of the second application interface image.
[0183] Specifically, in this step, the graphics logic processing module can replace the first drawing instruction of the second target element in the complete drawing data of the second application interface image with the second drawing instruction of the second target element. Furthermore, if the complete drawing data of the second application interface image contains the aforementioned identified static element, the first drawing instruction of the identified static element in the complete drawing data of the second application interface image can be replaced with the second drawing instruction of the identified static element, thereby updating the complete drawing data of the second application interface image and obtaining the updated complete drawing data of the second application interface image.
[0184] In some embodiments, the graphics logic processing module may store, for the target application, a second drawing instruction for the identified static element. Taking the second application interface image as the next frame image of the first application interface image in the aforementioned embodiments as an example, the graphics logic processing module may store, for the target application, a second drawing instruction for the first target element.
[0185] It is understandable that the identified static elements recorded in the graphics logic processing module for the target application are consistent with the static elements recorded in the static element information of the target application. Therefore, through the judgment step 803, the second target element can be divided into a third target element and a fourth target element. The third target element includes static elements newly identified from the second application interface image, and the fourth target element includes identified static elements corresponding to the target application. For example, suppose the second target element identified in step 802 includes static elements 1 to 4, and the static information corresponding to the target application includes static element 4. The graphics logic processing module stores the second drawing instruction for static element 4 for the target application. In the judgment step of step 803, the judgment result for static element 4 is yes, and the process ends. The judgment results for static elements 1 to 3 are no, and they are respectively used as the third target elements to continue executing subsequent steps 804 to 810, thereby obtaining the second drawing instruction corresponding to static elements 1 to 3. In this step, the graphics logic processing module replaces the first drawing instruction of static elements 1 to 3 in the complete drawing data of the second application interface image with the second drawing instruction, and according to the second drawing instruction of static element 4 corresponding to the target application, replaces the first drawing instruction of static element 4 in the complete drawing data of the second application interface image with the second drawing instruction, thereby obtaining the updated complete drawing data of the second application interface image.
[0186] Step 812: The drawing execution module draws the second application interface image based on the complete drawing data of the updated second application interface image.
[0187] The implementation of this step can refer to step 811, which will not be repeated here. The only difference is to replace the corresponding terms. For example, replace the first application interface image with the second application interface image in this step, and replace the first target element with the second target element in this step.
[0188] In the method shown in Figure 8, the third target element in the second application interface image is pre-drawn to obtain a pre-drawn image. Then, the first drawing instruction of the third target element in the complete drawing data of the second application interface image is updated to the second drawing instruction of the third target element, and the first drawing instruction of the previously identified fourth target element is updated to the second drawing instruction of the fourth target element. Thus, when drawing the second application interface image, it is not necessary to draw the third and fourth target elements according to the first drawing instruction, but to directly draw the pre-drawn image, thereby improving the rendering speed of the second application interface image and alleviating the lag problem when the electronic device displays the interface.
[0189] In the embodiment shown in Figure 9 below, the possible implementation flow of the image rendering method of this application embodiment is illustrated when the target application instructs the graphics logic processing module on the drawing data of the application interface image of each frame through an incremental indication method.
[0190] Figure 9 is a flowchart illustrating an image rendering method provided by an electronic device shown in Figure 6 according to an embodiment of this application. As shown in Figure 9, in this embodiment, the method may include:
[0191] Step 901: The target application sends the incremental drawing data of the third application interface image to the graphics logic processing module.
[0192] The incremental rendering data of the third application interface image is used to record the rendering data of the changes in the third application interface image relative to the previous frame application interface image.
[0193] The third application interface image can be any frame of the application interface image that needs to be displayed after the target application starts. If the third application interface image is the first frame of the application interface image that needs to be displayed after the target application starts, the incremental rendering data of the third application interface image can be the complete rendering data of the third application interface image.
[0194] The third application interface image is a frame of application interface image that the target application needs to display. In some embodiments, the third application interface image can be any frame of application interface image that the target application will display after the image rendering method of this application embodiment starts execution. It is understood that before the image rendering method of this application embodiment is executed, each frame of application interface image that the target application needs to display can be drawn using the application interface image rendering method in related technologies.
[0195] Step 902: The static element recognition module obtains the fifth target element in the third application interface image.
[0196] The aforementioned fifth target element is at least one of the static elements added to the third application interface image relative to the previous frame application interface image.
[0197] In some embodiments, the incremental drawing data of the third application interface image may record the drawing data of the newly added elements of the third application interface image relative to the previous frame application interface image. The drawing data of the elements may include the type of each element. The element type can distinguish whether the element is a static element or a dynamic element. In this step, the static element recognition module can obtain a static element of a preset type based on the incremental drawing data of the third application interface image, and use it as the fifth target element.
[0198] This application does not limit how the static element recognition module obtains the incremental rendering data of the third application interface image. In some embodiments, the static element recognition module can obtain the incremental rendering data of the third application interface image from the target application. In other embodiments, the static element recognition module can monitor the data sent by the target application to the graphics logic processing module to obtain the incremental rendering data of the third application interface image. In still other embodiments, the static element recognition module can obtain the incremental rendering data of the third application interface image from the graphics logic processing module.
[0199] In other embodiments, the electronic device can provide drawing interfaces for various types of elements for the target application. After determining the UI layout of the first application interface image, the target application can call the corresponding drawing interface for each element to realize the drawing of each element. Correspondingly, the drawing interface can maintain element information corresponding to the first application interface image of the target application, used to record which elements of the first application interface image the target application called the drawing interface for. In this step, the static element recognition module can obtain the element information added to the third application interface image of the target application relative to the previous frame application interface image from the drawing interface of the preset type, and summarize it to obtain the static element of the preset type as the fifth target element. The preset type can be one or more types included in the static element.
[0200] Step 903: The static element recognition module obtains the first drawing instruction of the fifth target element and sends the first drawing instruction of the fifth target element to the static element cache processing module.
[0201] In some embodiments, the first drawing instruction in this step may be the drawing instruction of the fifth target element recorded in the incremental drawing data of the third application interface image. In some embodiments, the static element recognition module may obtain the first drawing instruction of the fifth target element from the incremental drawing data of the third application interface image.
[0202] The method for the static element recognition module to obtain incremental drawing data of the third application interface image can be found in the corresponding description in step 902, and will not be repeated here. It should be noted that if the static element recognition module has already obtained the incremental drawing data of the third application interface image in step 202, it is not necessary to obtain it again in this step. The first drawing instruction of the fifth target element can be obtained directly from the incremental drawing data of the third application interface image.
[0203] Step 904: The static element cache processing module sends memory request 3 to the memory management module to request memory space for the fifth target element.
[0204] The memory space requested for the fifth target element can be used to store the pre-drawn image of the fifth target element obtained by the first drawing instruction based on the fifth target element in subsequent steps.
[0205] Step 905: The memory management module allocates a third memory space for the fifth target element and sends the third memory address of the third memory space to the static element cache module.
[0206] Step 906: The static element cache processing module sends a drawing request 3 to the drawing execution module.
[0207] Drawing request 3 is used to request the drawing of the fifth target element. Drawing request 3 may include: a first drawing instruction and a third memory address of the fifth target element. Optionally, drawing request 3 may also include: an identifier of the fifth target element. The identifier of the fifth target element is used to identify the fifth target element.
[0208] Step 907: The drawing execution module draws the pre-drawn image corresponding to the fifth target element based on the first drawing instruction of the fifth target element, and stores the pre-drawn image corresponding to the fifth target element to the third memory address corresponding to the fifth target element.
[0209] Step 908: The drawing execution module sends an instruction generation request 3 to the image instruction generation module.
[0210] Instruction generation request 3 is used to request the generation of a second drawing instruction for the fifth target element. In some embodiments, instruction generation request 3 may include: a third memory address corresponding to the fifth target element. Optionally, instruction generation request 3 may further include: an identifier of the fifth target element.
[0211] Step 909: The image instruction generation module generates a second drawing instruction for the fifth target element based on the third memory address corresponding to the fifth target element and the pre-drawn image, and sends the second drawing instruction for the fifth target element to the graphics logic processing module.
[0212] The second drawing instruction for the fifth target element can also be called the drawing instruction for the pre-drawn image corresponding to the fifth target element.
[0213] Step 910: The graphics logic processing module updates the incremental drawing data of the third application interface image according to the second drawing instruction of the fifth target element, and obtains the updated incremental drawing data of the third application interface image.
[0214] Specifically, in this step, the graphics logic processing module can replace the first drawing instruction of the fifth target element in the incremental drawing data of the third application interface image with the second drawing instruction of the fifth target element, thereby realizing the update of the incremental drawing data.
[0215] Step 911: The graphics logic processing module determines the complete drawing data of the third application interface image based on the incremental drawing data of the updated third application interface image.
[0216] In some embodiments, the rendering execution module can determine the complete rendering data of the third application interface image based on the complete rendering data of the application interface image of the previous frame and the incremental rendering data of the third application interface image. It can be understood that the determined complete rendering data of the third application interface image includes the second rendering instruction for the newly added fifth target element of the third application interface image. If the third application interface image also includes identified static elements, the rendering instruction for the identified static elements in the complete rendering data of the third application interface image is also the second rendering instruction. Therefore, in step 912, the rendering execution module does not need to repeat the rendering of the fifth target element and the identified static elements, but only needs to render the corresponding pre-rendered image, thereby saving the rendering execution module the rendering time of the third application interface image and improving the rendering speed.
[0217] Step 912: The graphics logic processing module sends the complete drawing data of the third application interface image to the drawing execution module, and the drawing execution module draws the third application interface image based on the complete drawing data of the third application interface image.
[0218] The specific implementation of the drawing execution module drawing the third application interface image based on the complete drawing data in this step is not limited in this application embodiment, and can be implemented using methods from related technologies. For example, the third application interface image can be drawn directly based on the complete drawing data using the method shown in Figure 3; or, the screen interface image can be drawn centrally based on the drawing data of each application interface image included in the screen interface image using the method shown in Figure 4, where the third application interface image is one of the application interface images included in the screen interface image.
[0219] In the method shown in Figure 9, the fifth target element in the third application interface image is pre-drawn to obtain a pre-drawn image. Then, the first drawing instruction of the fifth target element in the incremental drawing data of the third application interface image is updated to the second drawing instruction of the fifth target element. Based on the updated incremental drawing data of the third application interface image, the complete drawing data of the third application interface image is determined. Thus, in the complete drawing data of the third application interface image, the drawing instruction of the previously identified target element is also the second drawing instruction of that target element. Therefore, when drawing the third application interface image, it is not necessary to draw the fifth target element and the previously identified target element according to the first drawing instruction. The pre-drawn image can be drawn directly, thereby improving the rendering speed of the third application interface image and alleviating the lag problem when electronic devices display the interface.
[0220] In some embodiments, the display positions of some static elements in the application interface image to be displayed by the target application are relatively fixed. Therefore, in another embodiment provided in this application, in addition to pre-drawing the static elements to obtain corresponding pre-drawn images and replacing the drawing instructions in the drawing data of the application interface image accordingly, multiple static elements with relatively fixed display positions can be treated as a set, and the set can be pre-drawn to obtain a pre-drawn image corresponding to the set. In other words, the multiple static elements with relatively fixed display positions are drawn into a pre-drawn image, and then the drawing data of multiple static elements in the drawing data of the application interface image is replaced with the drawing data of this image. The method shown in Figure 10 is illustrated below. Figure 10 still uses the example of the target application indicating the drawing data of the application interface image in an incremental manner.
[0221] As shown in Figure 10, the method may include:
[0222] Step 1001: The target application sends the incremental drawing data of the third application interface image to the graphics logic processing module.
[0223] Step 1002: The static element recognition module obtains the fifth target element in the third application interface image.
[0224] The implementation of steps 1001 to 1002 can be referred to steps 901 to 902, which will not be repeated here.
[0225] Step 1003: The static element recognition module obtains the first target set from the fifth target element.
[0226] The first target set may include multiple fifth target elements, and the display positions of the multiple fifth target elements are relatively fixed.
[0227] The first target set can be one set or multiple sets, and this application embodiment does not impose any restrictions.
[0228] Through this step, the fifth target element can be divided into: the first target set and the fifth target element outside the first target set. To distinguish between the fifth target element in the first target set and the fifth target element outside the first target set, in this embodiment, the fifth target element in the first target set is referred to as the sixth target element, and the fifth target element outside the first target set is referred to as the seventh target element.
[0229] For example, suppose the fifth target element includes elements 1 to 8, the first target set includes set 1 and set 2, set 1 includes elements 1 and 2, and set 2 includes elements 6 to 8. Then, elements 1, 2, and 6 to 8 can be called the sixth target element, and elements 3 to 5 can be called the seventh target element.
[0230] In this context, the multiple sixth target elements included in the first target set can be elements at the same level or elements at different levels. For example, set 1 includes element 1 and element 2. Element 1 and element 2 can be two elements under the same root node, or element 1 can be the root node and element 2 can be a leaf node of element 1, and so on.
[0231] Step 1004: Perform image pre-drawing and second drawing instruction generation process on the seventh target element to obtain the second drawing instruction of the seventh target element, and then execute step 1012.
[0232] This step can be implemented through steps 903 to 909 in the method shown in Figure 9. The main difference is that the corresponding replacement of the nouns is performed. For example, the fifth target element in steps 903 to 909 is replaced with the seventh target element in this step.
[0233] Step 1005: The static element recognition module obtains the first drawing instruction of the first target set and sends the first drawing instruction of the first target set to the static element cache processing module.
[0234] The first drawing instruction for the first target set includes: the first drawing instruction for each sixth target element in the first target set. Continuing the example in step 1004, if set 1 includes element 1 and element 2, then the first drawing instruction for set 1 includes: the first drawing instruction for element 1 and the first drawing instruction for element 2; if set 2 includes elements 6 to 8, then the first drawing instruction for set 2 includes: the first drawing instruction for element 6, the first drawing instruction for element 7, and the first drawing instruction for element 8.
[0235] In some embodiments, the static element recognition module can obtain the first drawing instruction of the first target set from the incremental drawing data of the third application interface image. Specifically, for each first target set, it can obtain the first drawing instruction of each sixth target element in the first target set from the incremental drawing data of the third application interface image.
[0236] The method for the static element recognition module to obtain incremental drawing data of the third application interface image can be found in the corresponding description in step 1002, and will not be repeated here. It should be noted that if the static element recognition module has already obtained the incremental drawing data of the third application interface image in step 1002, then it is not necessary to obtain it again in this step, and the first drawing instruction of the first target set can be obtained directly from the incremental drawing data of the third application interface image.
[0237] Step 1006: The static element cache processing module sends memory request 4 to the memory management module to request memory space for the first target set.
[0238] The memory space requested for the first target set can be used to store the pre-drawn image of the first target set obtained by the first drawing instruction based on the first target set in subsequent steps.
[0239] Step 1007: The memory management module allocates a fourth memory space for the first target set and sends the fourth memory address of the fourth memory space to the static element cache module.
[0240] Step 1008: The static element cache processing module sends a drawing request 4 to the drawing execution module.
[0241] Drawing request 4 is used to request the drawing of the first target set. Drawing request 4 may include: a first drawing instruction for the first target set and a fourth memory address.
[0242] Step 1009: The drawing execution module draws the pre-drawn image corresponding to the first target set based on the first drawing instruction of the first target set, and stores the pre-drawn image corresponding to the first target set in the fourth memory address corresponding to the first target set.
[0243] Step 1010: The drawing execution module sends an instruction generation request 4 to the image instruction generation module.
[0244] Instruction generation request 4 is used to request the generation of a second drawing instruction for the first target set. In some embodiments, instruction generation request 4 may include: an identifier of the first target set and a fourth memory address corresponding to the first target set.
[0245] Step 1011: The image instruction generation module generates a second drawing instruction for the first target set based on the fourth memory address corresponding to the first target set and the pre-drawn image, and sends the second drawing instruction for the first target set to the graphics logic processing module.
[0246] The second drawing instruction for the first target set can also be called the drawing instruction for the pre-drawn image corresponding to the first target set.
[0247] Step 1012: The graphics logic processing module updates the incremental drawing data of the third application interface image according to the second drawing instruction of the first target set, and updates the incremental drawing data of the third application interface image according to the second drawing instruction of the seventh target element, so as to obtain the updated incremental drawing data of the third application interface image.
[0248] In this step, the incremental drawing data of the third application interface image is updated according to the second drawing instruction of the first target set. There may be different implementation methods depending on the first target set.
[0249] For example, if multiple sixth target elements included in the first target set are leaf nodes under the same root node, then an eighth target element can be added under that root node. The layout data of the eighth target element is determined based on the layout data of the multiple sixth target sets included in the first target set, and the drawing instruction of the eighth target element is set as the second drawing instruction of the first target set. The drawing data of the multiple sixth target elements included in the first target set is then deleted. Taking the drawing tree shown in Figure 11(a) as an example, the first target set includes elements 7 and 8, which are leaf nodes under element 6. At this time, as shown in Figure 11(b), a leaf node - element 9 can be added to element 6. The layout data of element 9 is determined based on the layout data of elements 7 and 8, the second drawing instruction of the first target set is used as the drawing instruction of element 9, and elements 7 and 8 are then deleted.
[0250] For example, if the first target set includes multiple sixth target elements, and there exists a root node and at least one leaf node under that root node, and leaf nodes under the same root node, then the first drawing instruction for the sixth target element that serves as the root node can be replaced with the second drawing instruction for the first target set, and the leaf nodes corresponding to the other sixth target elements included in the first target set can be deleted. Taking the drawing tree shown in Figure 11(a) as an example, the first target set includes elements 6 to 8. In this case, as shown in Figure 11(c), the first drawing instruction for element 6 can be modified to the second drawing instruction, and elements 7 and 8 can be deleted.
[0251] It is understandable that the multiple sixth target elements included in the first target set can have other relationships and implementations in the drawing tree, as long as the incremental drawing data of the third application interface image is updated based on the principle of not changing the third application interface image.
[0252] Step 1013: The graphics logic processing module determines the complete drawing data of the third application interface image based on the incremental drawing data of the updated third application interface image.
[0253] Step 1014: The graphics logic processing module sends the complete drawing data of the third application interface image to the drawing execution module, and the drawing execution module draws the third application interface image based on the complete drawing data of the third application interface image.
[0254] The implementation of steps 1013 to 1014 can be referred to steps 911 to 912, which will not be repeated here.
[0255] In the method shown in Figure 10, in addition to pre-drawing static elements as images, combinations of static elements can also be pre-drawn as images, thereby further improving the rendering speed of the third application interface image and thus better alleviating the lag problem when electronic devices display the interface.
[0256] It should be noted that the method shown in Figure 10 can also be extended to the scenario where the target application distributes complete drawing data of the application interface image. For example, the method shown in Figure 10 can be used to obtain a target set consisting of multiple first target elements from the first target element shown in Figure 7A, or to obtain a target set consisting of multiple second target elements from the second target element shown in Figure 8, and so on. The specific process can be adapted by referring to the method shown in Figure 10, and will not be listed here.
[0257] It should be noted that in other embodiments, the target set can also be obtained only from the application interface image, the target set can be pre-drawn as an image, and the static elements in the application interface image that are not included in the target set can be drawn directly by executing the first drawing instruction during the rendering process of the application interface image according to the method in the related technology. This method can also improve the rendering speed of the interface image to a certain extent, thereby better alleviating the lag problem when the electronic device displays the interface.
[0258] To ensure better operation of the target memory of electronic devices, embodiments of this application provide a control strategy for the memory occupied by pre-drawn images corresponding to static elements.
[0259] Referring to Figure 12, in some embodiments, when the target application moves from the foreground to the background or exits, the pre-drawn images corresponding to the static elements generated during the application interface image rendering process of the target application can be cleared and the corresponding memory can be reclaimed.
[0260] As shown in Figure 12, the method may include:
[0261] Step 1201: The memory management application detects that target application 1 has been moved to the background or exited.
[0262] This application does not limit how the memory management application detects that the target application 1 has been moved to the background or exited.
[0263] Step 1202: The memory management application clears the pre-drawn image corresponding to target application 1 in the target memory and releases the corresponding memory space.
[0264] In some embodiments, a preset mark can be set on the memory space allocated for the pre-drawn image corresponding to the target application 1 to distinguish whether the data stored in the memory space is a static element pre-drawn image of the target application. Then, in this step, the memory space with the preset mark corresponding to the target application can be obtained, the data in the memory space can be deleted, and the memory space can be released.
[0265] By reclamating the corresponding memory space when the target application 1 is running in the background or exiting, memory can be dynamically managed and excess memory can be cleared in a timely manner.
[0266] In some embodiments, a corresponding memory control strategy may be executed when the overall utilization of the target memory exceeds a preset threshold. For example, as shown in Figure 13, the method may include:
[0267] Step 1301: The memory management application detects that the overall utilization of the target memory exceeds a preset first threshold.
[0268] The specific value of the first threshold mentioned above is not limited in this application embodiment.
[0269] Step 1302: The memory management application sends a first instruction to the static element recognition module, instructing the static element recognition module to stop recognizing the static elements of each target application, and / or the memory management application clears the infrequently used pre-drawn images in the pre-drawn images corresponding to each target application and releases the corresponding memory.
[0270] The first instruction mentioned above is used to instruct the static element recognition module to stop recognizing static elements of each target application.
[0271] In this step, "infrequently used pre-drawn images" refers to pre-drawn images with relatively low usage rates. In some embodiments, the memory management application can sort the pre-drawn images corresponding to each target application according to their usage frequency, and determine the first number of pre-drawn images at the bottom of the sort as infrequently used pre-drawn images, or determine pre-drawn images whose usage frequency is below a threshold as infrequently used pre-drawn images, and so on.
[0272] In some embodiments, when the memory usage of the target application exceeds a preset threshold, a corresponding memory control strategy can be executed. For example, as shown in Figure 14, the method may include:
[0273] Step 1401: The memory management application detects that the memory usage of the pre-drawn image corresponding to the target application 2 exceeds the preset second threshold.
[0274] The specific value of the second threshold mentioned above is not limited in the embodiments of this application.
[0275] Step 1402: The memory management application sends a second instruction to the static element recognition module, instructing the static element recognition module to stop recognizing static elements of target application 1, and / or, the memory management application clears infrequently used pre-drawn images in the pre-drawn images corresponding to target application 2 and releases the corresponding memory.
[0276] The second instruction mentioned above is used to instruct the static element recognition module to stop recognizing static elements of the target application 2.
[0277] In this step, "infrequently used pre-drawn images" refers to pre-drawn images with relatively low usage rates. In some embodiments, the memory management application can sort the pre-drawn images corresponding to the target application 2 according to their usage frequency, and determine the second-to-last number of pre-drawn images in the sorted list as infrequently used pre-drawn images, or determine pre-drawn images whose usage frequency is below a threshold as infrequently used pre-drawn images, and so on.
[0278] The above methods enable fine-grained management of target memory to better utilize memory resources. On the one hand, by combining with user scenarios, excess memory can be reclaimed in a timely manner. On the other hand, by detecting memory status, the unlimited use of memory can be controlled, ensuring a dynamic balance between performance and memory.
[0279] Figure 15 is a flowchart illustrating an image rendering method provided in an embodiment of this application. As shown in Figure 15, the method may include:
[0280] Step 1501: Obtain at least one static element in the first interface image as the first element.
[0281] The first interface image may be, for example, the first application interface image, the second application interface image, or the third application interface image in the aforementioned embodiments.
[0282] The aforementioned first element may be, for example, the first target element, the third target element, the fifth target element, etc. in the aforementioned embodiments.
[0283] Step 1502: Obtain the first drawing instruction of the first element, and draw the pre-drawn image corresponding to the first element according to the first drawing instruction of the first element.
[0284] Step 1503: Generate a second drawing instruction for the first element based on the pre-drawn image corresponding to the first element.
[0285] Step 1504: Update the first drawing instruction of the first element in the first drawing data of the first interface image to the second drawing instruction of the first element to obtain the updated first drawing data.
[0286] The first drawing data mentioned above may be, for example, the complete drawing data or the incremental drawing data in the aforementioned embodiments.
[0287] Step 1505: Draw the first interface image based on the updated first drawing data.
[0288] The steps in this embodiment can be referred to the corresponding descriptions in the foregoing embodiments, and will not be repeated here.
[0289] In some embodiments, as shown in FIG16, the method may further include:
[0290] Step 1506: When the second interface image includes the first element, update the first drawing instruction of the first element in the second drawing data of the second interface image to the second drawing instruction of the first element to obtain the updated second drawing data.
[0291] The second interface image mentioned above can be, for example, the second application interface image in the foregoing embodiments. The second drawing data mentioned above can be, for example, the complete drawing data of the second interface image.
[0292] Step 1507: Draw the second interface image based on the updated second drawing data.
[0293] In some embodiments, as shown in FIG17, the steps prior to step 1507 may further include:
[0294] Step 1701: Obtain the second element in the second interface image. The second element is a static element in the second interface image and does not have a corresponding pre-drawn image.
[0295] The second element mentioned above may be, for example, the third target element in the aforementioned embodiments.
[0296] Step 1702: Obtain the first drawing instruction of the second element, and draw the pre-drawn image corresponding to the second element according to the first drawing instruction of the second element.
[0297] Step 1703: Generate a second drawing instruction for the second element based on the pre-drawn image corresponding to the second element.
[0298] Step 1704: Update the first drawing instruction of the second element in the second drawing data of the second interface image to the second drawing instruction of the second element.
[0299] In some embodiments, the method shown in FIG15 may further include the following after step 1505:
[0300] The third drawing data of the second interface image is determined. When the second interface image includes the first element, the third drawing data includes the second drawing instruction of the first element.
[0301] The second interface image is drawn based on the third drawing data.
[0302] In some embodiments, the third drawing data may be complete drawing data. The above-mentioned determination of the third drawing data of the second interface image may include: determining the third drawing data of the second interface image based on the incremental drawing data of the second interface image and the complete drawing data of the interface image of the previous frame of the second interface image.
[0303] In some embodiments, the above-described second drawing instruction for generating the first element based on the pre-drawn image corresponding to the first element may include:
[0304] Allocate memory space for the first element in the target memory, and store the pre-drawn image corresponding to the first element into the memory space;
[0305] The second drawing instruction for the first element is determined based on the pre-drawn image corresponding to the first element and the memory address of the memory space.
[0306] The aforementioned memory space may be, for example, the first memory space, the third memory space, etc. in the aforementioned embodiments.
[0307] In some embodiments, the target memory is a DMA cache.
[0308] In some embodiments, as shown in FIG18, the method may further include:
[0309] Step 1801: Obtain the first set in the first interface image. The first set includes multiple third elements. Each third element is a static element in the first interface image, and the display positions of the third elements are relatively fixed.
[0310] The first set mentioned above may be, for example, the first target set in the foregoing embodiments, and the third element mentioned above may be, for example, the sixth target element in the foregoing embodiments.
[0311] Step 1802: Obtain the drawing instructions for the first set, which includes the drawing instructions for each third element;
[0312] Step 1803: Draw the pre-drawn image corresponding to the first set according to the drawing instructions of the first set;
[0313] Step 1804: Generate a second drawing instruction for the first set based on the pre-drawn images corresponding to the first set;
[0314] Step 1805: Update the first drawing data of the first interface image according to the second drawing instructions of the first set.
[0315] In some embodiments, the method may further include:
[0316] When the second interface image includes the first set, the drawing data of the first set in the second drawing data of the second interface image is updated to the drawing data of the second image.
[0317] In some embodiments, the first interface image is an application interface image of the first application; the method may further include:
[0318] Upon detecting that the first application has moved to the background or stopped running, clear the pre-drawn image corresponding to the first application from the target memory and release the memory space occupied by the pre-drawn image corresponding to the first application; and / or,
[0319] If the overall utilization rate of the target memory is detected to have reached a preset first threshold, at least a portion of the pre-drawn images corresponding to the target application in the target memory are cleared, and / or, the identification of static elements in the application interface image of the target application is stopped. The target application is the application involved in the screen interface image where the first interface image is located, and the target application includes the first application; and / or,
[0320] If the memory usage of the first application in the target memory reaches a preset second threshold, at least a portion of the pre-drawn images corresponding to the first application in the target memory are cleared, and / or, the identification of static elements in the application interface image of the first application is stopped.
[0321] In some embodiments, obtaining at least one static element in the first interface image, as the first element, may include:
[0322] Obtain at least one element of a preset type from the first drawing data of the first interface image, as the first element; and / or,
[0323] At least one static element in the first interface image is obtained from the drawing interface called by the first application to which the first interface image belongs, and is used as the first element.
[0324] The image rendering method of this application can improve CPU performance. Specifically, after pre-drawing static elements as images, the static elements become static images. During rendering, it is no longer necessary to calculate the layout and style of static elements every frame, thereby reducing the computational burden on the CPU per frame. This can significantly improve rendering performance, especially when the interface needs to be frequently updated or display a large amount of text.
[0325] The image rendering method in this application embodiment can also improve GPU performance. Specifically, currently, GPUs process images more effectively and efficiently than static elements, especially for complex static element layouts, where the GPU's rendering capabilities can be directly utilized to accelerate the display of interface images and reduce repetitive GPU processing.
[0326] This application also provides an electronic device for implementing the method provided in this application.
[0327] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the method provided in this application.
[0328] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method provided in this application.
[0329] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.
[0330] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0331] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0332] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0333] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.
Claims
1. An image rendering method, characterized by, The method is applied to an electronic device, and comprises the following steps: obtaining at least one static element in a first interface image as a first element; obtaining a first drawing instruction of the first element, and drawing a pre-drawing image corresponding to the first element according to the first drawing instruction of the first element; generating a second drawing instruction of the first element based on the pre-drawing image corresponding to the first element; updating the first drawing instruction of the first element in first drawing data of the first interface image to the second drawing instruction of the first element to obtain updated first drawing data; drawing the first interface image according to the updated first drawing data.
2. The method of claim 1, wherein, Further comprising: when the first element is included in a second interface image, updating the first drawing instruction of the first element in second drawing data of the second interface image to the second drawing instruction of the first element to obtain updated second drawing data; drawing the second interface image according to the updated second drawing data.
3. The method of claim 2, wherein, Before the step of drawing the second interface image according to the updated second drawing data, the method further comprises the following steps: obtaining a second element in the second interface image, the second element being a static element in the second interface image, and the second element not having a corresponding pre-drawing image; obtaining a first drawing instruction of the second element, and drawing a pre-drawing image corresponding to the second element according to the first drawing instruction of the second element; generating a second drawing instruction of the second element based on the pre-drawing image corresponding to the second element; updating the first drawing instruction of the second element in second drawing data of the second interface image to the second drawing instruction of the second element.
4. The method according to any one of claims 1 to 3, characterized in that, The step of generating the second drawing instruction of the first element based on the pre-drawing image corresponding to the first element comprises the following steps: applying for a memory space for the first element in a target memory, and storing the pre-drawing image corresponding to the first element into the memory space; determining the second drawing instruction of the first element based on the pre-drawing image corresponding to the first element and a memory address of the memory space.
5. The method of claim 4, wherein, The target memory is a DMA cache.
6. The method according to any one of claims 1 to 5, characterized in that, Further comprising: obtaining a first set in the first interface image, the first set comprising a plurality of third elements, each of the third elements being a static element in the first interface image, and the display positions of the third elements being relatively fixed; obtaining a drawing instruction of the first set, the drawing instruction of the first set comprising a drawing instruction of each of the third elements; drawing a pre-drawing image corresponding to the first set according to the drawing instruction of the first set; generating a second drawing instruction of the first set based on the pre-drawing image corresponding to the first set; updating first drawing data of the first interface image according to the second drawing instruction of the first set.
7. The method of claim 6, wherein, Further comprising: when the first set is included in a second interface image, updating the first drawing instruction of the first set in second drawing data of the second interface image to the second drawing instruction of the first set.
8. The method according to any one of claims 4 to 7, characterized in that, The first interface image is an application interface image of a first application; and the method further comprises the following steps: when it is detected that the first application is running in the background or is stopped, clearing the pre-rendered images corresponding to the first application in the target memory, and releasing the memory space occupied by the pre-rendered images corresponding to the first application; and / or, when it is detected that the overall usage of the target memory reaches a preset first threshold, clearing at least part of the pre-rendered images corresponding to the target application in the target memory, and / or stopping identifying the static elements in the application interface image of the target application, the target application being an application involved in the screen interface image where the first interface image is located, and the target application including the first application; and / or, when it is detected that the usage of the target memory by the first application reaches a preset second threshold, clearing at least part of the pre-rendered images corresponding to the first application in the target memory, and / or stopping identifying the static elements in the application interface image of the first application.
9. The method of claim 2, wherein, The obtaining of the at least one static element in the first interface image as the first element includes: obtaining at least one element of a preset type from the first rendering data of the first interface image as the first element; and / or, obtaining at least one static element in the first interface image based on a rendering interface called by a first application to which the first interface image belongs as the first element.
10. An electronic device, comprising: comprise: a processor, a memory; one or more computer programs are stored in the memory, and the one or more computer programs include instructions which, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 9.
11. A computer readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and when it runs on the computer, it causes the computer to perform the method of any one of claims 1 to 9.
12. A computer program product, characterised in that, The computer program product comprises a computer program, and when it runs on the computer, it causes the computer to perform the method of any one of claims 1 to 9.
