Data display method

Abstract

This application relates to the field of three-dimensional data processing, and more particularly to a data display method applied to a terminal device including a first drawing unit and a second drawing unit. The method includes: receiving a target invalid display area determined by a graphical user interface library; determining a drawing task for updating the target invalid display area based on the target invalid display area; determining the expected energy consumption of the first drawing unit and the second drawing unit in performing the drawing task, and selecting the drawing unit with the lower expected energy consumption as the target drawing unit; controlling the target drawing unit to perform drawing on the target invalid display area, and refreshing the drawing result to the area corresponding to the target invalid display area. Thus, through dynamic scheduling of the drawing units, combined with a burst drawing mechanism driven by the invalid display area, rendering smoothness is improved and overall power consumption is reduced while ensuring interface responsiveness.

Description

Technical Field

[0001] This application relates to the field of three-dimensional data processing, and in particular to data display methods. Background Technology

[0002] With the increasing demands for human-computer interaction in embedded terminals and mobile devices, terminals need to implement graphical user interface (GUI) displays under conditions of limited battery power, computing power, and storage. Common interface rendering methods include CPU software rendering and GPU hardware-accelerated rendering. The former is simple to implement and highly adaptable, typically involving a GUI library generating invalid display areas (dirty areas) that need updating when the interface display state changes, and then redrawing and refreshing those areas. However, when the content being rendered is complex or updates are frequent, the CPU load is high, easily causing interface stuttering and increased power consumption. The latter can utilize the parallel capabilities of the GPU to improve rendering efficiency, but in embedded terminals, GPU wake-up, driver calls, and data transmission introduce additional energy consumption and overhead. Adopting a fixed strategy for long-term GPU use or large-area refreshes is also not conducive to reducing power consumption and improving battery life.

[0003] Many existing CPU / GPU hybrid rendering solutions rely on fixed thresholds or coarse-grained switching, which usually fail to take into account the differences in task such as the size and spatial distribution of the invalid display area and the type of rendering, and to comprehensively consider the energy efficiency and wake-up cost of different rendering units, thus making it difficult to achieve a better balance between refresh smoothness and energy consumption. Summary of the Invention

[0004] The purpose of this application is to provide a data display method that effectively improves rendering smoothness and reduces overall power consumption while ensuring interface responsiveness.

[0005] The objective of this application is achieved through the following technical solution: In a first aspect, embodiments of this application provide a data display method applied to a terminal device including a first drawing unit and a second drawing unit. The method includes: receiving a target invalid display area determined by a graphical user interface library; determining a drawing task for updating the target invalid display area based on the target invalid display area; determining the expected energy consumption of the first drawing unit and the second drawing unit in performing the drawing task, and selecting the drawing unit with the lower expected energy consumption as the target drawing unit; controlling the target drawing unit to perform drawing on the target invalid display area, and refreshing the drawing result to the area corresponding to the target invalid display area.

[0006] In some embodiments, the invalid display area is determined by the graphical user interface library based on changes in the interface display state; receiving the target invalid display area generated by the graphical user interface library includes: receiving at least one invalid display area determined by the graphical user interface library within a refresh cycle; if an invalid display area is received, using the invalid display area as the target invalid display area; or, if multiple invalid display areas are received, merging the multiple invalid display areas and using the merged area as the target invalid display area; wherein merging the multiple invalid display areas includes: merging intersecting invalid display areas into a bounding rectangle, and / or merging invalid display areas with a spacing less than a preset threshold into a bounding rectangle.

[0007] In some embodiments, determining the drawing task for updating the target invalid display area based on the target invalid display area includes: extracting the drawing task associated with the target invalid display area from the drawing task queue generated by the graphical user interface library; and using the drawing task associated with the target invalid display area as the drawing task for updating the target invalid display area.

[0008] In some embodiments, the task characteristics of the drawing task include at least one of the following: the number of pixels covered by the target invalid display area, the spatial distribution characteristics of the target invalid display area, and the drawing operation type of the drawing task; the spatial distribution characteristics are used to characterize the degree of dispersion of the target invalid display area within the display area, and the degree of dispersion is determined based on the number of the target invalid display areas and / or the geometric position of the target invalid display areas; the drawing operation type includes at least one of the following: fill drawing, text drawing, image drawing, and transparent blending drawing.

[0009] In some embodiments, determining the first estimated energy consumption of the first drawing unit and the second estimated energy consumption of the second drawing unit includes: obtaining the number of pixels covered by the target invalid display area corresponding to the drawing task; and determining the first estimated energy consumption of the first drawing unit and the second estimated energy consumption of the second drawing unit based on the number of pixels and the unit pixel energy consumption parameters corresponding to the first drawing unit and the second drawing unit, respectively.

[0010] In some embodiments, determining the first estimated energy consumption and the second estimated energy consumption further includes: obtaining wake-up energy consumption parameters corresponding to the first drawing unit and the second drawing unit respectively; for each drawing unit, adding the unit pixel energy consumption obtained based on the number of pixels and the corresponding unit pixel energy consumption parameter to the wake-up energy consumption obtained based on the corresponding wake-up energy consumption parameter to determine the estimated energy consumption of the corresponding drawing unit.

[0011] In some embodiments, determining the first estimated energy consumption and the second estimated energy consumption respectively includes: determining a correction coefficient based on the spatial distribution characteristics and / or the drawing operation type; and correcting the first estimated energy consumption and the second estimated energy consumption according to the correction coefficient.

[0012] In some embodiments, refreshing the drawing results to the region corresponding to the target invalid display region includes: dividing the target invalid display region into multiple sub-regions when the display buffer capacity used for drawing does not meet the buffer capacity required by the target invalid display region; controlling the target drawing unit to draw each sub-region sequentially; and refreshing the drawing results corresponding to each sub-region to the corresponding sub-region in the target invalid display region.

[0013] In some embodiments, the method further includes: during the process of the target drawing unit performing drawing and / or refreshing, controlling the drawing units in the first drawing unit and the second drawing unit that are not determined to be the target drawing unit to remain in an unwake-up state or a low-power state.

[0014] Secondly, embodiments of this application provide a terminal device, including: a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the method as described in any one of the first aspects.

[0015] Thirdly, embodiments of this application provide a computer storage medium for storing a program, wherein the program, when running, controls the device where the computer storage medium is located to execute the method described in any one of the first aspects.

[0016] Fourthly, embodiments of this application provide a computer program product for implementing the method as described in any one of the first aspects.

[0017] This application provides a data display method. This application addresses the problems of pure CPU software rendering becoming a bottleneck, having high resource consumption and high power consumption, and affecting touch and IPC response in complex interfaces / 1080P, as well as the high power consumption of pure GPU full-screen high frame rate refresh. After detecting the updated area generated by the change in interface state, the method analyzes the number of pixels covered by the updated area, its spatial distribution, and the drawing type (fill / text / image / transparent blending, etc.). Combining the energy efficiency characteristics of each drawing unit, the method dynamically selects the CPU or GPU with lower expected power consumption for execution and adopts burst drawing driven by dirty regions. After the drawing is completed, the unselected units are kept idle / low power, thereby balancing frame rate, response, and power consumption. Attached Figure Description

[0018] This application will be further described below with reference to the accompanying drawings and specific embodiments.

[0019] Figure 1 This is a flowchart illustrating a data display method provided in an embodiment of this application.

[0020] Figure 2 This is a flowchart illustrating another data display method provided in the embodiments of this application.

[0021] Figure 3 This is a structural block diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0024] In related implementations, terminal devices can perform interface rendering based on graphical user interface libraries (GUIs). These GUIs typically favor software rendering using the Central Processing Unit (CPU) and support local redrawing based on invalid display areas (dirty areas). Alternatively, terminal devices can also call the Graphics Processing Unit (GPU) through a graphics interface to perform hardware-accelerated rendering. However, the single rendering path has its limitations. When the interface is complex, has many dynamic effects, or has a high resolution, CPU software rendering can easily generate high pixel-level computation and memory access overhead, leading to increased rendering latency, higher processor usage, lower frame rate, and potentially affecting the terminal's response to external events such as touch and inter-process communication. While using the GPU for full-screen high refresh rate rendering can improve rendering efficiency, the GPU's prolonged operation can also cause high power consumption.

[0025] To address the aforementioned issues, this application embodiment performs feature analysis on the drawing tasks corresponding to interface updates and dynamically selects a more suitable drawing unit to complete the drawing and refreshing process, based on the energy efficiency characteristics of different drawing units, thereby reducing energy consumption and improving display smoothness. Specifically, when the terminal device's interface display state changes, the graphical user interface library determines a target invalid display area and triggers a drawing schedule based on the target invalid display area. A drawing task for updating that area is determined based on the target invalid display area, and features are extracted from the drawing task. These features include at least: the number of pixels covered by the target invalid display area, the spatial distribution characteristics of the target invalid display area, and the type of drawing operation (e.g., fill drawing, text drawing, image drawing, transparent blending drawing, etc.).

[0026] Based on this, the expected energy consumption of the first and second drawing units in performing drawing tasks is evaluated separately. The drawing unit with lower expected energy consumption is selected as the target drawing unit, and the target drawing unit is controlled to perform drawing on the target invalid display area. The drawing result is then refreshed to the area corresponding to the target invalid display area. A burst drawing mechanism triggered by invalid display areas is adopted, that is, drawing and refreshing are only started when a target invalid display area exists, and the unselected drawing units are kept in an idle or low-power state after drawing is completed, thereby reducing the processor's continuous running time and avoiding multiple drawing units from operating in a high-power state simultaneously.

[0027] Through the above technical solution, this application can adaptively schedule between CPU software rendering and GPU hardware accelerated rendering based on the characteristics of different interface update tasks: for rendering tasks suitable for GPU acceleration, it can reduce the bottleneck caused by the CPU performing pixel-level calculations for a long time, alleviate frame rate drops, and improve the response to external events; for scenarios that are not suitable for continuous GPU full-screen refresh, it can avoid the increased power consumption caused by the GPU working for a long time. Therefore, this application improves the smoothness and responsiveness of interface display while helping to reduce the overall power consumption of terminal devices.

[0028] The data display method mentioned in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0029] See Figure 1 , Figure 1 This is a flowchart illustrating a data display method provided in an embodiment of this application.

[0030] This application provides a data display method applied to a terminal device including a first drawing unit and a second drawing unit, the method including steps S101-S104: S101: Receive the target invalid display area determined by the graphical user interface library.

[0031] S102: Based on the target invalid display area, determine a drawing task for updating the target invalid display area.

[0032] S103: Determine the expected energy consumption of the first drawing unit and the second drawing unit in performing the drawing task, and select the drawing unit with the smaller expected energy consumption as the target drawing unit.

[0033] S104: Control the target drawing unit to draw the target invalid display area, and refresh the drawing result to the area corresponding to the target invalid display area.

[0034] Terminal devices can be understood as embedded mobile terminals (such as handheld laser scanners) with display and interaction capabilities. They typically integrate processors, memory, displays / display terminal devices, and run graphical user interface libraries to achieve interface display and interaction.

[0035] It is understood that the first rendering unit can be a first type of processing unit in the terminal device capable of performing interface rendering tasks. In some embodiments, the first rendering unit can be a CPU, for example, the 2D software renderer of the graphical user interface library calculates and generates rendering results pixel by pixel on the CPU side and writes them to the frame buffer. The second rendering unit can be a second type of processing unit in the terminal device capable of performing interface rendering tasks. In some embodiments, the second rendering unit can be a GPU, for example, the GPU is called through graphics APIs such as OpenGL / Vulkan to complete graphics processing and rendering (including geometric drawing, texture mapping and special effects rendering, etc.).

[0036] Graphical user interface (GUI) libraries can provide basic capabilities such as interface component management, interface state maintenance, drawing, and refreshing. In some embodiments, the GUI library can be a lightweight GUI library such as LVGL (Light and Versatile Graphics Library), which supports CPU rendering and local area redrawing.

[0037] The target invalid display area can be a region determined by the graphical user interface library (GUI) when it detects a UI update request, which is considered to be the UI update area / dirty region. In some specific implementations, the target invalid display area can be represented by the coordinate range of one or more rectangles (or other polygons) to indicate the scope of subsequent drawing and refreshing, and can serve as the basis for triggering a drawing schedule. By triggering drawing schedules based on the target invalid display area and adopting a bursty drawing method driven by the dirty region, meaningless continuous drawing and refreshing can be reduced, decreasing the processor's continuous runtime, thereby reducing overall power consumption and ensuring UI responsiveness.

[0038] In some embodiments, the process by which the terminal device obtains information about the target invalid display area from the graphical user interface library can be implemented through function calls, callback interfaces, message queues, or shared memory. For example, after the graphical user interface library generates an updated area, parameters such as the area coordinates / size are passed to the thread or task where the scheduling logic resides.

[0039] A drawing task can be a set of drawing operations that need to be performed to update an invalid display area of ​​a target.

[0040] In some embodiments, the determination of drawing tasks is related to the target invalid display area. That is, the terminal device can determine which drawing operations need to be performed and to what extent based on information such as the geometric range of the target invalid display area, the interface elements involved, or the drawing order. For example, drawing operations that affect the target invalid display area can be extracted from the drawing instructions / drawing task queue generated by the graphical user interface library to avoid repeatedly drawing irrelevant areas.

[0041] The estimated energy consumption can characterize the amount of energy that a certain rendering unit may consume when performing this rendering task. It can be estimated by the terminal device based on the characteristics of the rendering task and the energy efficiency characteristics of the rendering unit.

[0042] In some embodiments, heterogeneous selection with energy consumption as the optimization objective involves the terminal device evaluating the estimated energy consumption of the first and second rendering units performing the same rendering task, and determining the unit with the lower estimated energy consumption as the execution subject for this rendering, thereby achieving dynamic selection between CPU software rendering and GPU hardware rendering. Thus, by performing feature analysis on the rendering task and combining the energy efficiency characteristics of the heterogeneous rendering units, the estimated energy consumption of the first and second rendering units performing the rendering task is evaluated separately, and the unit with the lower estimated energy consumption is selected to execute the rendering task. This achieves a more suitable execution unit selection among different tasks (e.g., different update regions and different rendering operations), balancing rendering efficiency and energy consumption performance.

[0043] In some embodiments, if the target rendering unit is a CPU, the CPU-side renderer can calculate the pixels in the invalid display area of ​​the target point by point / line by line and write them to the frame buffer; if the target rendering unit is a GPU, rendering commands, texture / vertex data, etc. can be submitted through the graphics API, and the GPU can complete rasterization and pixel processing and output the rendering results.

[0044] Refreshing refers to the process of updating the drawn image data to the final display path, so that the position on the display screen corresponding to the target invalid display area presents the updated content. In some embodiments, refreshing may include copying / compositing the drawing results from the drawing buffer to the frame buffer, triggering a partial update of the display terminal device, or sending the target area pixel data to the display panel through a display interface (such as SPI / DSI / parallel port, etc.), thereby realizing updating only the target area instead of a full-screen refresh.

[0045] It is understood that, in this embodiment, the graphical user interface library in the terminal device maintains the display state of interface elements and determines the invalid display area corresponding to the interface update when the interface display state changes. The terminal device can receive the target invalid display area determined by the graphical user interface library and trigger a drawing schedule based on the target invalid display area. Subsequently, the terminal device determines the drawing task for updating the area based on the target invalid display area and performs feature analysis on the drawing task. On this basis, the terminal device combines the energy efficiency characteristics of the first drawing unit and the second drawing unit, evaluates the expected energy consumption of each unit in performing the drawing task, selects the drawing unit with lower expected energy consumption as the target drawing unit to perform drawing, and refreshes the drawing result to the area corresponding to the target invalid display area. After the drawing is completed, the terminal device controls the unselected drawing units to remain in an idle or low-power state and adopts a burst drawing method triggered by dirty areas to reduce the continuous running time of the processor.

[0046] It is understood that the embodiments of this application address the problems of prolonged processing time, high CPU utilization, decreased frame rate and high power consumption, slow response to external events, and high power consumption of pure GPU full-screen high frame rate refresh under complex interface / high resolution rendering in existing LVGL pure CPU software rendering. The embodiments adopt dynamic scheduling based on rendering task characteristics and the energy efficiency characteristics of heterogeneous rendering units, and combine a burst rendering mechanism driven by dirty regions, thereby improving rendering smoothness and reducing overall power consumption while ensuring interface response performance.

[0047] In some embodiments, the invalid display area may be determined by the graphical user interface library based on changes in the interface display state. Receiving the target invalid display area generated by the graphical user interface library may include: receiving at least one invalid display area determined by the graphical user interface library within a refresh cycle; and, if an invalid display area is received, using that invalid display area as the target invalid display area.

[0048] In some embodiments, receiving the target invalid display area generated by the graphical user interface library may further include: receiving at least one invalid display area determined by the graphical user interface library during a refresh cycle; and, if multiple invalid display areas are received, merging the multiple invalid display areas and using the merged area as the target invalid display area. The merging of the multiple invalid display areas includes: merging intersecting invalid display areas into a bounding rectangle, and / or merging invalid display areas with a spacing less than a preset threshold into a bounding rectangle.

[0049] Changes in the display state of an interface can be understood as changes in the displayed content or properties of interface elements / controls. Examples include changes in text content, icon switching, progress bar values, animation frame advancement, control position / size changes, and transparency changes. In some implementations, graphical user interface libraries can generate an interface update area (dirty area) upon detecting such state changes and trigger drawing scheduling accordingly.

[0050] Invalid display areas are used to represent areas where displayed content needs to be updated; they can also be called interface update areas or dirty areas. Essentially, they describe the range within the display area where old content is outdated and needs to be redrawn and refreshed. This can be represented using rectangular coordinates (…). x , y , w , h It can be represented by a set of multiple rectangles, or a set of regions approximated by several rectangles, and serves as the scope of subsequent drawing and refresh processing.

[0051] It can be understood that the target invalid display area is the final area used to drive drawing / refreshing in a current refresh cycle, obtained by the terminal device after sorting through one or more invalid display areas received within a refresh cycle. It can be equivalent to a single invalid display area, or it can be an area merged from multiple invalid display areas. It is used to reduce repeated drawing and repeated refresh triggering within the same refresh cycle, making it easier to uniformly determine drawing tasks and select drawing units in the future.

[0052] The refresh cycle can be understood as a time window during which the terminal device updates the display, such as a preset time range between two display refresh processes. Within this time window, the graphical user interface library may continuously generate one or more invalid display areas. The terminal device can collect these invalid display areas within the refresh cycle and form a target invalid display area within or at the cycle boundary so that drawing and refreshing can be completed in a single refresh process.

[0053] When only one update area is generated within the refresh cycle, the terminal device can directly reuse that area as the target range for this drawing and refresh without additional merging calculations, thereby reducing unnecessary area processing overhead and shortening the scheduling link.

[0054] When multiple update regions appear within a refresh cycle, the terminal device can first organize these regions into one or more merged regions, and then perform drawing and refresh accordingly. This reduces the frequent scheduling and multiple transmission overhead caused by triggering drawing / refresh for multiple small regions separately, and also facilitates the subsequent unified determination of drawing tasks and assessment of energy consumption based on the target invalid display region.

[0055] In some embodiments, the merged region is the result area output by the merging algorithm, which can be a single rectangle, a set of multiple rectangles, or the smallest enclosing region used to cover the original multiple invalid display areas. In some specific implementations, the merged region is represented in the form of an outer rectangle, so as to uniformly represent the update range with regular rectangular boundaries, which facilitates subsequent drawing and refreshing (especially the implementation of the display interface updating by rectangular window).

[0056] The circumscribed rectangle can be a rectangular region used to enclose one or more invalid display areas. It can be understood as the smallest (or approximately smallest) axis-aligned rectangle capable of covering the merged region. For example, the circumscribed rectangle for the union of two rectangular regions can be determined by the smallest axis alignment of the two rectangles. x / y With the largest x / y Confirm. A new rectangular window will be generated for subsequent drawing and refreshing.

[0057] Intersecting invalid display areas indicate that two invalid display areas overlap geometrically (i.e., they share a common coverage area). When invalid display areas intersect, refreshing them separately often results in duplicate coverage and redrawing. Therefore, merging intersecting areas into a bounding rectangle allows multiple intersecting areas to be covered with a single target area processing, thereby reducing redundant processing.

[0058] In some embodiments, the spacing can be defined as the minimum horizontal and / or vertical distance between two rectangular boundaries, or as the minimum distance between the nearest points of two regions, used to determine whether regions that are not intersecting but are sufficiently close are worth merging.

[0059] When the spacing between multiple invalid display areas is less than a preset threshold, it indicates that these areas are very close together. Refreshing them separately may result in too many refreshes or frequent switching of the transmission window. Therefore, they can be merged into an outer rectangle to reduce refresh overhead. In some embodiments, the preset threshold can be a fixed pixel threshold (e.g., a number of pixels, a number of rows / columns), or it can be adaptively set according to screen resolution, refresh bandwidth, or system load.

[0060] To address this, by aggregating and uniformly processing invalid display areas within the refresh cycle, directly reusing an invalid display area when only one is received, and merging multiple invalid display areas into a bounding rectangle based on intersection and / or distance thresholds, the overhead of repeated scheduling and multiple refreshes caused by multiple scattered invalid display areas within the same refresh cycle can be reduced. This reduces the trigger frequency of drawing and refreshing, alleviates the load on system bus / storage access and display updates, and provides a more stable input range for subsequent drawing task analysis and drawing unit selection based on the target invalid display area, further improving the overall efficiency and energy consumption of interface updates.

[0061] It is understood that, in this embodiment, the graphical user interface library in the terminal device maintains the interface display state and determines one or more invalid display areas (interface update areas / dirty areas) when the interface display state changes. The terminal device can receive at least one invalid display area within a refresh cycle and generate a target invalid display area based on the invalid display areas received within that refresh cycle. If only one invalid display area is received, it is directly determined as the target invalid display area. If multiple invalid display areas are received, a merging process is performed to obtain the target invalid display area. This merging process may include merging intersecting invalid display areas into a bounding rectangle and / or merging invalid display areas with a spacing less than a preset threshold into a bounding rectangle. Subsequently, the target invalid display area serves as the scope and triggering basis for subsequent drawing scheduling and refresh updates, realizing a drawing process triggered by the update area.

[0062] In summary, this application addresses the problem that changes in the interface display state lead to a large number of scattered invalid display areas, which can easily cause frequent drawing and high refresh overhead. It adopts a region aggregation mechanism that receives and merges invalid display areas within the refresh cycle and uses the merged outer rectangle as the target invalid display area. This reduces the number of scheduling and refreshes and improves the overall display efficiency while ensuring the coverage of the updated area.

[0063] In some embodiments, determining the drawing task for updating the target invalid display area based on the target invalid display area may include: extracting the drawing task associated with the target invalid display area from the drawing task queue generated by the graphical user interface library; and using the drawing task associated with the target invalid display area as the drawing task for updating the target invalid display area.

[0064] A drawing task queue can be a collection of drawing operations organized by a graphical user interface library (GUI) to complete a single interface update. It typically records each drawing operation and its parameters in the form of a queue, list, or command stream. In some embodiments, the drawing task queue can be a list of drawing commands generated by the rendering thread, a list of objects to be drawn obtained by traversing the control tree within the GUI library, or a collection of drawing commands built based on a display list or command buffer.

[0065] In some embodiments, extraction may include traversing the drawing task queue, obtaining the corresponding drawing range / target control / layer information for each drawing task, and filtering based on the association with the target invalid display area. The output of the extraction may be a sub-queue, a task set, or a regenerated sequence of drawing tasks for subsequent execution and energy consumption assessment.

[0066] In some embodiments, the update association between the drawing task and the target invalid display area may include at least any of the following situations: the drawing range of the drawing task overlaps or covers the target invalid display area; the target control / interface element corresponding to the drawing task is located within or crosses the target invalid display area; the layer / texture area corresponding to the drawing task intersects with the target invalid display area; or the drawing task changes the display attributes of pixels within the target invalid display area, such as color, transparency, depth / occlusion relationship.

[0067] In some embodiments, after the terminal device completes the extraction, it determines the set of drawing tasks obtained by filtering as the final set of drawing tasks for updating the target invalid display area, and uses this set as the input for subsequent processes (such as task feature statistics, expected energy consumption assessment, selection of target drawing units, and execution of drawing and refresh), thereby avoiding the inclusion of irrelevant drawing operations in the subsequent scheduling chain and improving the targeting of subsequent heterogeneous drawing unit selection and refresh processing.

[0068] In this way, by extracting drawing tasks associated with the target invalid display area from the drawing task queue generated by the graphical user interface library, the scope of subsequent analysis and execution tasks is narrowed, avoiding the inclusion of drawing operations unrelated to the target invalid display area in energy consumption assessment and drawing scheduling. This reduces the unnecessary computation and processing overhead in the drawing scheduling chain and improves the pertinence and accuracy of drawing task feature analysis (such as pixel count, spatial distribution, drawing operation type, etc.) and expected energy consumption assessment. This, in turn, facilitates a more suitable selection between CPU and GPU, alleviating problems such as frame rate drop and slow response to external events caused by high CPU load in complex interface scenarios.

[0069] It is understood that the embodiments of this application address the problem that there are many types and a large range of drawing operations during the interface update process, and that irrelevant drawing operations are easily introduced, thereby increasing the scheduling and analysis overhead. The embodiments adopt a task extraction method that uses the target invalid display area as an index to extract related drawing tasks from the drawing task queue. This reduces the invalid processing overhead while ensuring the correctness of the update of the target invalid display area, and provides more accurate input for subsequent dynamic scheduling based on the characteristics of drawing tasks and the energy efficiency characteristics of drawing units.

[0070] In some embodiments, the task characteristics of the drawing task may include at least one of the following: the number of pixels covered by the target invalid display area, the spatial distribution characteristics of the target invalid display area, and the drawing operation type of the drawing task.

[0071] The spatial distribution features can be used to characterize the degree of dispersion of the target invalid display area within the display area, and the degree of dispersion is determined based on the number of the target invalid display areas and / or the geometric position of the target invalid display areas.

[0072] The drawing operation type may include at least one of the following: fill drawing, text drawing, image drawing, and transparent blending drawing.

[0073] In some embodiments, task characteristics are used to characterize the nature of the current rendering task, its workload, and whether its execution is more computationally or bandwidth-intensive. These characteristics serve as inputs for subsequent energy consumption assessments and rendering unit selection. Task characteristics can be a single indicator or a combination of multiple indicators, and can be obtained through statistical analysis, identification, or table lookup methods, enabling the scheduling logic to differentiate its processing for different interface update patterns.

[0074] The number of pixels can be used to characterize the pixel scale covered by the target invalid display area, and can usually be calculated from the geometric dimensions of the target invalid display area (e.g., the width × height of a rectangle). When there are multiple invalid display areas (or the target invalid display area consists of multiple sub-regions), the number of pixels can be the sum of the pixel counts of each area, or the number of pixels in the bounding rectangle after merging. The number of pixels can be used to roughly estimate the drawing workload and data transfer volume; for example, the larger the number of pixels, the greater the total overhead of processing each pixel and writing to the frame buffer / display buffer.

[0075] Spatial distribution features are used to describe the spatial morphology of the target invalid display areas within the entire display area, such as whether they are concentrated or dispersed, close together or far apart, to reflect the additional overhead that discrete updates may cause. In some embodiments, spatial distribution features may include (but are not limited to) the number of invalid display areas, the geometric position of the center point / boundary box of each area, the minimum or average spacing between areas, whether the areas are clustered, and the ratio of the area of ​​the bounding rectangle of the target invalid display area to the sum of the areas of all areas, etc., to help determine whether the current update is a multi-block scattered refresh or a few-block concentrated refresh.

[0076] Discreteness is a quantitative expression of spatial distribution characteristics, used to indicate the degree of dispersion of target invalid display areas. In some embodiments, discreteness can be determined based on the number of target invalid display areas (e.g., the more areas, the higher the discreteness), or it can be determined in combination with geometric location (e.g., the larger the average distance between areas or the larger the coverage area, the higher the discreteness), or it can be calculated using more comprehensive indicators (e.g., the variance of the area center point, or the ratio of the area of ​​the circumscribed rectangle to the effective area), thereby providing a comparable quantitative basis for energy consumption assessment or scheduling strategies. By characterizing discreteness with spatial distribution characteristics, the overhead differences of different update forms such as multi-small-block dispersed refresh and single large-block concentrated refresh can be identified, thereby reducing the additional scheduling and data transmission overhead caused by frequent small-area rendering / refreshing; The type of drawing operation characterizes the main types of drawing operations involved in a drawing task, thus reflecting the computational and memory access characteristics of the task. Different drawing operations typically exhibit different acceleration effects and unit energy consumption differences on CPUs and GPUs. Therefore, identifying the type of drawing operation helps to more accurately estimate the expected energy consumption of different drawing units and improves the rationality of selecting target drawing units. To this end, by extracting task characteristics that include at least the number of pixels, spatial distribution features, and drawing operation types, a more comprehensive basis is provided for drawing unit energy consumption assessment and heterogeneous scheduling, thereby avoiding mismatches in drawing unit selection caused by relying on only a single indicator (such as only pixel scale).

[0077] In some embodiments, fill drawing can refer to a drawing operation that fills a specified area with color / pattern, such as background color filling, rectangular / rounded rectangle filling, gradient filling, etc. In some specific implementations, fill drawing typically involves writing to consecutive pixel blocks, exhibiting strong data bandwidth characteristics, making it suitable for improving efficiency through hardware acceleration or other methods.

[0078] Text rendering refers to the drawing operation of rendering characters / strings at a specified location. For example, it involves rasterizing glyphs based on bitmap or vector fonts and blending the glyph pixels into the target area. Text rendering may involve font caching, glyph outline calculation, anti-aliasing, and alpha blending, and its computational and memory access characteristics differ from those of pure fill rendering.

[0079] Image drawing refers to the operation of drawing bitmap / texture content onto a target area, such as icon drawing, image mapping, and drawing scaled / cropped images. Image drawing typically involves texture sampling, pixel copying, scaling interpolation, and may be combined with opacity / blending operations.

[0080] Transparent blending refers to a drawing operation that combines foreground pixels with transparency with background pixels using alpha blending or other blending rules. Examples include semi-transparent control overlays, shadow / gradient effects, and anti-aliased edge blending. This type of operation often introduces an additional read-modify-write process, impacting memory access and computing power. Therefore, identifying it as an independent drawing type helps to make energy consumption assessments more realistic. Thus, by distinguishing between fill, text, image, and transparent blending drawing operation types, we can reflect the differences in computational and memory access characteristics of different drawing operations on the CPU and GPU. This improves the accuracy of predicted energy consumption assessments and enhances the rationality of target drawing unit selection, thereby improving interface refresh smoothness and reducing overall energy consumption while ensuring correct display updates.

[0081] This application embodiment employs a mechanism for extracting task features from a drawing task used to update a target invalid display area. After determining the drawing task, the terminal device acquires and forms a set of task features that characterize the workload and form of the drawing task. On one hand, it counts the number of pixels covered by the target invalid display area to characterize the pixel scale involved in this update. On the other hand, it determines the spatial distribution characteristics of the target invalid display area within the display area to characterize the dispersion of the updated area. This dispersion can be determined based on the number and / or geometric location of the target invalid display areas (e.g., the degree of dispersion, relative spacing, and coverage of multiple areas). Simultaneously, it identifies the drawing operation type of the drawing task and categorizes it into at least one of the following types: fill drawing, text drawing, image drawing, and transparent blending drawing. Subsequently, the terminal device uses these task features as input for subsequent estimated energy consumption assessment and drawing unit selection, enabling drawing scheduling to perform differentiated processing for different task forms.

[0082] This application addresses the problem that interface update tasks vary significantly in pixel size, regional dispersion, and drawing operation type, making it difficult to make reasonable drawing unit selection based on a single indicator, thus causing poor energy consumption or performance. It adopts a method of extracting multi-dimensional task features of drawing tasks and using task features to drive energy consumption assessment and scheduling, so that the terminal device can adaptively select a more suitable drawing unit for different update forms and different drawing operations, taking into account both display smoothness and low power consumption goals.

[0083] In some embodiments, determining the first estimated energy consumption of the first drawing unit and the second estimated energy consumption of the second drawing unit may include: obtaining the number of pixels covered by the target invalid display area corresponding to the drawing task; and determining the first estimated energy consumption of the first drawing unit and the second estimated energy consumption of the second drawing unit based on the number of pixels and the unit pixel energy consumption parameters corresponding to the first drawing unit and the second drawing unit, respectively.

[0084] The first estimated energy consumption characterizes the energy that the first rendering unit may consume when performing the rendering task, and the second estimated energy consumption characterizes the energy that the second rendering unit may consume when performing the rendering task. Both are typically calculated or estimated in the same energy consumption unit (e.g., joules (J), millijoules (mJ), or normalized energy consumption units) for comparison. In some embodiments, the estimated energy consumption can be an absolute value of energy, or a relative value or rating value (e.g., an energy consumption index) used for comparison.

[0085] In some embodiments, a drawing task may include multiple drawing operations, but its overall update range can be characterized by a target invalid display area, which is then used as the basis input for energy consumption estimation.

[0086] The unit pixel energy consumption parameter characterizes the energy consumption characteristics of a rendering unit when processing a unit pixel (or writing / blending / filling a unit pixel), and is a parameter for energy efficiency. In some embodiments, the unit pixel energy consumption parameter can be a pre-calibrated constant (e.g., obtained by offline testing and statistical analysis of energy consumption under different pixel counts and fitting), or it can be a parameter that is dynamically adjusted according to operating frequency, voltage level, temperature, or power consumption mode. The unit pixel energy consumption parameter can be set separately for the CPU rendering unit and the GPU rendering unit to reflect the difference in energy consumption between the two under the same pixel workload.

[0087] For example, when the first rendering unit is a CPU software rendering unit, its unit pixel energy consumption parameter can reflect the energy consumption of CPU pixel-by-pixel calculation and memory access writing; when the second rendering unit is a GPU hardware accelerated rendering unit, its unit pixel energy consumption parameter can reflect the energy consumption characteristics of GPU parallel processing and texture / blending operations, thus providing differentiation for heterogeneous selection. In this way, by obtaining the number of pixels covered by the target invalid display area and combining it with the unit pixel energy consumption parameters of different rendering units to determine the first and second expected energy consumption respectively, the calculation of expected energy consumption can simultaneously reflect the pixel scale difference of the rendering task and the energy efficiency difference between the first and second rendering units. This avoids the distortion of energy consumption assessment caused by coarse-grained selection based solely on a fixed threshold or the number of pixels, and improves the specificity and interpretability of subsequent target rendering unit selection among heterogeneous rendering units. This, in turn, helps to reduce overall energy consumption and improve display smoothness while ensuring the correctness of interface updates.

[0088] In some embodiments, the expected energy consumption can be estimated using a multiplicative relationship (e.g., expected energy consumption = number of pixels × energy consumption per pixel).

[0089] This application embodiment uses the number of pixels covered by the target invalid display area to represent the pixel scale of the current update, and introduces a mechanism that uses unit pixel energy consumption parameters corresponding to the first and second drawing units to represent the energy efficiency characteristics of different drawing units. After determining the drawing task, the terminal device first obtains the number of pixels covered by the target invalid display area corresponding to the drawing task. Subsequently, the terminal device calls (or obtains from a table) the unit pixel energy consumption parameters of the first drawing unit and the second drawing unit, respectively, and calculates the first estimated energy consumption of the first drawing unit and the second estimated energy consumption of the second drawing unit to perform the drawing task based on the number of pixels and the corresponding unit pixel energy consumption parameters. This allows for a unified evaluation of the task scale (number of pixels) and unit energy efficiency (unit pixel energy consumption parameter) under the same energy consumption metric, providing a comparable basis for subsequent selection of the target drawing unit.

[0090] This application addresses the problem in existing heterogeneous rendering selections that lack unified quantification of the differences in rendering task scale and rendering unit energy efficiency, which can easily lead to mismatched rendering unit selection, resulting in high power consumption or poor performance. It adopts a method that measures task scale by the number of pixels and unit energy efficiency by the energy consumption parameter per unit pixel, and calculates the expected energy consumption accordingly. This provides comparable energy consumption indicators for the first and second rendering units and supports the selection of execution units with lower energy consumption, thereby achieving comprehensive optimization of energy consumption and display performance.

[0091] In some embodiments, the estimated energy consumption can be further superimposed with fixed or corrective terms (e.g., considering wake-up energy consumption, bus transmission overhead, transparency blending ratio, etc.). For example, in addition to determining the estimated energy consumption based on the number of pixels and the energy consumption parameter per unit pixel, factors such as drawing operation type, spatial distribution characteristics, and wake-up energy consumption parameters can be introduced as supplementary factors to improve the accuracy and adaptability of energy consumption estimation.

[0092] Specifically, in some embodiments, determining the first estimated energy consumption and the second estimated energy consumption may further include: obtaining wake-up energy consumption parameters corresponding to the first drawing unit and the second drawing unit respectively; for each drawing unit, adding the unit pixel energy consumption obtained based on the number of pixels and the corresponding unit pixel energy consumption parameter to the wake-up energy consumption obtained based on the corresponding wake-up energy consumption parameter to determine the estimated energy consumption of the corresponding drawing unit.

[0093] Wake-up energy consumption parameters characterize the energy consumption of a rendering unit when it switches from a non-working state (e.g., shutdown, sleep, clock-gated, low-power standby, etc.) to a working state capable of performing rendering tasks. It is a parameter reflecting startup / wake-up overhead. In some embodiments, wake-up energy consumption parameters can be obtained through offline calibration (e.g., by statistically analyzing the energy integral of the rendering unit from low power to working state), or different levels of wake-up energy consumption parameters can be dynamically selected based on current operating conditions such as power domain, voltage frequency level, temperature, and power consumption mode.

[0094] For example, when the first rendering unit is a CPU software rendering unit, its wake-up energy consumption may be mainly related to the CPU core waking up from idle state, frequency increase, cache / bus activity, etc.; when the second rendering unit is a GPU hardware rendering unit, its wake-up energy consumption may be related to the GPU power domain power-on, clock start, driver initialization, command queue activation, etc.

[0095] In some embodiments, the terminal device can estimate the energy consumed by the drawing unit in pixel processing and writing to complete the current drawing task based on the number of pixels covered by the target invalid display area and the unit pixel energy consumption parameter of the drawing unit. In some embodiments, the energy consumption can be estimated using a multiplicative relationship (e.g., execution energy consumption = number of pixels × unit pixel energy consumption parameter) to obtain an energy consumption item proportional to the task size.

[0096] The terminal device can determine the wake-up energy required for the current drawing unit to perform the drawing task based on the current power / clock state of the drawing unit and the corresponding wake-up energy consumption parameters. In some embodiments, if the drawing unit is already in a working state, the wake-up energy consumption can be zero or close to zero; if the drawing unit is in a low-power state, the wake-up energy consumption can be a fixed energy consumption corresponding to that state or an energy consumption accumulated in stages according to the wake-up process. By obtaining the wake-up energy consumption parameters corresponding to the first and second drawing units respectively, and adding the wake-up energy consumption to the expected energy consumption calculation, the energy consumption assessment can cover the fixed overhead caused by drawing unit switching and wake-up. This avoids the situation where the switching energy consumption is higher in small-area or short-term drawing tasks due to frequent selection of drawing units to be woken up based solely on the energy consumption per unit pixel. It also improves the stability and rationality of the target drawing unit selection, thereby reducing unnecessary unit wake-ups and frequent switching, reducing the overall system power consumption, and improving the continuity and responsiveness of the interface update process.

[0097] By superimposing the execution energy consumption (the part related to the number of pixels) with the wake-up energy consumption (the fixed / semi-fixed overhead related to state switching), the expected energy consumption of the drawing unit to perform the drawing task can be obtained, reflecting how much energy is needed to do the task and how much energy is needed to start doing the task. This avoids selection distortion caused by only looking at the execution energy consumption and ignoring the wake-up overhead in small task scenarios.

[0098] It is understood that in this embodiment, the estimated energy consumption is an energy consumption estimate that comprehensively considers execution energy consumption and wake-up energy consumption, and is used for subsequent decision-making to select the rendering unit with the lower estimated energy consumption as the target rendering unit. The estimated energy consumption can be an absolute energy value or an energy consumption score / index for comparison. The key is that it can reflect the overall energy consumption difference of different rendering units under the current task and current power consumption state.

[0099] In this embodiment, a mechanism is introduced to further introduce a wake-up energy consumption parameter on top of the expected energy consumption obtained based on the number of pixels and the unit pixel energy consumption parameter. After calculating the unit pixel energy consumption of the first drawing unit and the second drawing unit respectively, the terminal device obtains the wake-up energy consumption parameters corresponding to the first drawing unit and the second drawing unit respectively. Subsequently, for each drawing unit, the wake-up energy consumption when the drawing unit switches from the idle / low power state to execute the drawing task is determined based on the corresponding wake-up energy consumption parameter. The wake-up energy consumption is then added to the unit pixel energy consumption obtained based on the number of pixels and the corresponding unit pixel energy consumption parameter to obtain the expected energy consumption of the drawing unit executing the drawing task. This ensures that the expected energy consumption reflects both the task execution energy consumption and the unit wake-up overhead.

[0100] This application addresses the problem that in heterogeneous rendering unit scheduling, only the energy consumption of task execution is considered while the wake-up / switching overhead of rendering units is ignored, which can easily lead to selection mismatch and additional power consumption in small task scenarios. The embodiment adopts energy consumption modeling that adds wake-up energy consumption on top of the energy consumption per unit pixel, so that the expected energy consumption is closer to the actual energy consumption and supports the selection of rendering units with lower overall energy consumption, further reducing power consumption and improving the display update experience.

[0101] Specifically, it may also include: in some embodiments, determining the first expected energy consumption and the second expected energy consumption respectively may include: determining a correction coefficient based on the spatial distribution characteristics and / or the drawing operation type; and correcting the first expected energy consumption and the second expected energy consumption according to the correction coefficient.

[0102] In some embodiments, the correction factor can be a parameter used to correct the base estimated energy consumption, incorporating spatial distribution and rendering type into the calculation model of estimated energy consumption to make the estimation results closer to the actual energy consumption under different task modes. In some embodiments, the correction factor can be a multiplier (e.g., >1 indicates greater additional overhead, <1 indicates relatively smaller overhead) or a segmentation factor / weighting factor (e.g., different coefficients are selected according to the degree of dispersion interval or according to the rendering type category). The correction factor can be determined through offline calibration, empirical configuration, online learning / adaptive updating, or lookup table methods.

[0103] Terminal devices can first obtain spatial distribution features and drawing operation types from the drawing task, and then map the features to the corresponding correction coefficients; for example, they can map by region quantity / dispersion degree → discrete correction coefficients, or by fill / text / image / transparency blending → type correction coefficients, or combine the two into a comprehensive correction coefficient (e.g., sequential multiplication, weighted fusion, or multidimensional lookup table).

[0104] It is understood that, in this embodiment, the first and second estimated energy consumptions can be basic estimates based on the number of pixels and the energy consumption per pixel (and optionally, wake-up energy consumption), used to characterize the energy consumption of the first and second drawing units performing the same drawing task, respectively. These two basic estimates will be corrected by introducing additional influencing factors through correction coefficients to improve discriminability and accuracy.

[0105] Terminal devices can adjust the basic estimated energy consumption using correction coefficients, ensuring that the estimated energy consumption reflects not only pixel size and unit energy efficiency but also differences in task form and operation type. In some embodiments, the correction may include a scaling correction (e.g., E'=E×k), a stacking correction (e.g., E'=E+ΔE), or a segmented correction (e.g., selecting different model parameters according to type), thereby obtaining a first and second estimated energy consumption that are closer to reality. This provides a more reliable basis for comparison when selecting a rendering unit with lower estimated energy consumption. Thus, by determining the correction coefficients based on spatial distribution characteristics and / or rendering operation type and correcting the first and second estimated energy consumption, it is possible to distinguish energy consumption differences in scenarios with the same number of pixels but different update forms / rendering types. This avoids estimation distortion caused by relying solely on pixel size, thereby improving the accuracy and interpretability of heterogeneous rendering unit energy consumption assessment and enhancing the rationality of selecting rendering units with lower estimated energy consumption. This makes the scheduling strategy more consistent with dynamic selection based on rendering task characteristics and rendering unit energy efficiency characteristics, thereby reducing overall power consumption while ensuring interface responsiveness.

[0106] In this embodiment, based on the first and second estimated energy consumption obtained from the number of pixels and the energy consumption per unit pixel (and optionally, wake-up energy consumption), a correction coefficient determined by spatial distribution characteristics and / or drawing operation type is further introduced. After obtaining the drawing task characteristics (including the spatial distribution characteristics of the interface update area and the drawing operation type), the terminal device can apply the correction coefficient to the first and second estimated energy consumption to correct energy consumption estimation deviations under different drawing task conditions. For example, for drawing tasks where the update area is distributed in multiple blocks, the correction coefficient can reflect the additional overhead caused by multiple local processing operations; for specific drawing operation types such as transparent blending, the correction coefficient can reflect the additional energy consumption impact caused by read-write, mixed computation, etc., thereby making the estimated energy consumption closer to the actual energy consumption performance under different task characteristic conditions.

[0107] This application addresses the problem that interface update tasks vary significantly in terms of regional dispersion and drawing operation type, and that estimation based solely on pixel scale can easily lead to mismatched drawing unit selection. It adopts an energy consumption correction method that uses task characteristics to drive correction coefficients and corrects the expected energy consumption, thereby improving the accuracy of the expected energy consumption assessment and supporting more reasonable dynamic selection of CPU / GPU, while taking into account both display smoothness and low power consumption goals.

[0108] In some embodiments, refreshing the drawing results to the region corresponding to the target invalid display region may include: dividing the target invalid display region into multiple sub-regions when the display buffer capacity used for drawing does not meet the buffer capacity required by the target invalid display region; controlling the target drawing unit to draw each sub-region sequentially; and refreshing the drawing results corresponding to each sub-region to the corresponding sub-region in the target invalid display region.

[0109] In some embodiments, the refresh function applies to the display location corresponding to the target invalid display area; that is, it only updates the image data obtained in this drawing to the area that needs updating, rather than refreshing the entire screen or irrelevant areas. In some embodiments, the refresh may include copying / compositing the drawing result from the drawing buffer to the frame buffer, triggering a partial update of a specified window by the display terminal device, or sending the pixel data of a specified area to the display panel through the display interface, thereby implementing the principle of partial update driven by dirty regions.

[0110] The display buffer can be a storage space used to temporarily store drawing output, and can be understood as a part of the draw buffer / display buffer / frame buffer or a dedicated buffer. In some embodiments, the display buffer can be a drawing buffer in CPU-side RAM, a render target in GPU-side video memory, or a DMA buffer maintained by the display driver. The display buffer can be used to carry the rendering output of the target drawing unit and serve as an intermediate carrier for subsequent refresh to the display panel or frame buffer.

[0111] Display buffer capacity characterizes the size of pixel data that the display buffer can hold at one time, and is typically related to the available memory size and pixel format. For example, the same memory size can hold different numbers of pixels in different pixel formats such as RGB565 and ARGB8888. In some embodiments, display buffer capacity may also be related to implementation details such as double / triple buffering, buffer alignment, and row stride.

[0112] The buffer capacity required for the target invalid display area refers to the capacity needed to hold the drawing results of the target invalid display area in the display buffer all at once. It can be calculated from the number of pixels covered by the target invalid display area and the pixel format (bytes per pixel), and can further consider factors such as line span, alignment padding, and whether additional intermediate buffers (such as temporary buffers for transparent blending) are needed. When the required buffer capacity is greater than the display buffer capacity, it means that the drawing results of the area cannot be fully held in a single draw / refresh.

[0113] In some embodiments, the terminal device can compare the display buffer capacity with the buffer capacity required for the target invalid display area before refreshing, and use this as a criterion for determining whether to perform area division, so that drawing and refreshing can still be completed in one go when resources are sufficient, avoiding the introduction of unnecessary block overhead.

[0114] Dividing the target invalid display area into multiple smaller region units ensures that the drawing results corresponding to each sub-region fall within the display buffer's capacity. In some embodiments, the sub-region can be multiple rectangular blocks, such as by row (strip), by column, or by grid. It can also be divided according to a preset maximum width / height threshold. The division rules can be fixed or adaptively determined based on the current display buffer capacity to determine the size and number of sub-regions.

[0115] The resulting sub-regions can be local areas after the target invalid display area has been segmented, with each sub-region corresponding to a portion of the display position within the target invalid display area. In some embodiments, the sub-regions can be defined by coordinate ranges (starting from...). x , beginning yThe width and height of the sub-region represent the area and can correspond one-to-one with the write window in the display buffer. The introduction of sub-regions allows terminal devices to complete large-area updates with limited buffers by using block drawing and block refresh.

[0116] In some embodiments, the terminal device may drive the target drawing unit to draw one by one in the order of sub-regions (e.g., from top to bottom, from left to right, or in scan line order). In some embodiments, each sub-region may be processed serially, or a pipelined approach may be adopted when synchronization is satisfied (e.g., preparing to draw the next sub-region when one sub-region is refreshed), but overall, it is guaranteed that each sub-region is drawn and can be positioned in the correct display position.

[0117] The drawing output for each sub-region is only refreshed to the display position within the target invalid display area that matches the coordinates of that sub-region, avoiding misalignment or overlap errors between sub-regions. In some embodiments, this refresh can be achieved by setting the display window coordinates / clipping the window, or by writing the corresponding area in the frame buffer at an offset address, thereby ensuring that the final display effect is equivalent to drawing and refreshing the entire target invalid display area at once.

[0118] In some embodiments, the target invalid display area can be divided into several strips according to height, and pixel data for only one strip is generated in the display buffer each time and immediately refreshed to the screen. In other embodiments, the area can be divided into grid blocks, which are rendered and refreshed block by block. In still other embodiments, the size of the sub-region can be adjusted according to the display interface bandwidth or DMA transfer granularity to achieve a trade-off between the number of refreshes and the amount of data transferred per time, thereby adapting to different terminal hardware conditions.

[0119] In this embodiment, when the display buffer capacity used by the terminal device for drawing is insufficient to accommodate the buffer capacity required for the target invalid display area at once, the terminal device divides the target invalid display area into multiple sub-regions according to a preset division rule, so that the pixel data corresponding to each sub-region can fall within the range that the display buffer can bear; then, the terminal device controls the target drawing unit to draw each sub-region in sequence, obtain the drawing result of each sub-region, and after the drawing of each sub-region is completed, refresh the drawing result corresponding to the sub-region to the corresponding sub-region position in the target invalid display area, until the block drawing and block refresh of the target invalid display area are completed.

[0120] Thus, by dividing the target invalid display area into multiple sub-regions when the display buffer capacity is insufficient, large-area updates that would otherwise be impossible to handle all at once can be completed in a block-by-block manner. This reduces the peak buffer capacity required for a single refresh and improves adaptability to terminals with small memory / small video memory. By controlling the target drawing unit to draw each sub-region sequentially and refresh block by block, buffer overflow or refresh failure caused by drawing large areas can be avoided, thus ensuring the integrity and correctness of the target invalid display area update. By directing the sub-region drawing results to the corresponding sub-regions within the target invalid display area, unnecessary full-screen caching and full-screen refresh overhead can be reduced while maintaining the principle of updating only the areas that need updating, thereby helping to reduce storage bandwidth pressure and overall power consumption. To address the problem of a large target invalid display area and limited display buffer capacity, which prevents drawing and refreshing from being completed all at once, a mechanism of dividing the target invalid display area into blocks, drawing blocks, and refreshing blocks is adopted. This enables reliable updates to the target invalid display area under limited buffer conditions, while balancing display update efficiency and resource consumption.

[0121] In some embodiments, the method may further include: during the process of the target drawing unit performing drawing and / or refreshing, controlling the drawing units in the first drawing unit and the second drawing unit that are not determined to be the target drawing unit to remain in an unwake-up state or a low-power state.

[0122] It is understandable that drawing units that are not identified as target drawing units (or non-target drawing units) do not undertake drawing tasks (or participate in refresh-related processing) in this drawing schedule, and therefore can be controlled to be inactive by power management strategies to avoid unnecessary energy consumption.

[0123] The "unwake-up" state describes a rendering unit that is not activated and has not entered a working state where it can perform rendering tasks. In some embodiments, the unwake-up state may correspond to states such as the power domain not being powered on, the clock being off, the driver not being initialized / activated, or the command queue not being started. For example, when the GPU is used as a non-target rendering unit, its power domain can be kept off or the driver not being woken up, thereby avoiding the fixed power consumption overhead caused by GPU startup.

[0124] Low-power states can be used to describe a rendering unit in a state where it can recover quickly but consumes less power. In some embodiments, low-power states may include clock gating, frequency and voltage reduction (low DVFS), mild sleep / idle state, partial shutdown of power islands, etc. For example, when the CPU is used as a non-target rendering unit, its core frequency can be reduced to a low level or it can enter an idle state to keep it responsive to system scheduling while minimizing static and dynamic power consumption. If the non-target rendering unit supports complete non-wake-up (e.g., the power domain can be shut down), it can remain in an unwake-up state. If responsiveness or system constraints require it to recover quickly, it can remain in a low-power state. This provides a trade-off between power consumption reduction and recovery latency, and improves the adaptability of the solution to different terminal devices.

[0125] In some embodiments, the terminal device can manage the power supply, clock, frequency, and operating status of the rendering unit through a power management mechanism. In some embodiments, the control can be implemented by the operating system power management module, driver, PMU (Power Management Unit), or graphics subsystem scheduling logic. Specifically, it can include sending power state switching instructions, setting clock gating, setting frequency voltage levels, disabling / delaying driver wake-up, etc., thereby achieving state constraints on non-target rendering units.

[0126] While the target rendering unit is performing rendering and / or refreshing, constraints are imposed on non-target rendering units to keep them from being woken up or to operate at low power. This avoids unnecessary wake-ups or entry into high-power states by non-target rendering units during the operation of the target rendering unit, which would result in superimposed power consumption. It also avoids resource contention (such as bus / memory bandwidth contention) caused by parallel activation from affecting the rendering efficiency of the target rendering unit.

[0127] In this embodiment, by implementing power consumption state control on non-target drawing units after determining the target drawing unit, the terminal device, while controlling the target drawing unit to perform drawing and / or refreshing of the target invalid display area, simultaneously identifies another drawing unit among the first and second drawing units that has not been determined as the target drawing unit, and applies power consumption management control to the non-target drawing unit to keep it in an unwake-up state or a low-power state. The unwake-up state can correspond to a state where the power domain is not powered on, the clock is off, or the driver is not activated; the low-power state can correspond to states such as clock gating, frequency and voltage reduction, or light sleep. This ensures that the drawing link is primarily handled by the target drawing unit during the same period, and the non-target drawing unit does not participate in the drawing / refreshing process.

[0128] Thus, by controlling non-target drawing units to remain in an unwake-up or low-power state during the drawing and / or refreshing process of the target drawing unit, unnecessary static and wake-up power consumption is avoided when the first and second drawing units are simultaneously in working state, thereby reducing the total power consumption of the heterogeneous system during interface updates. Simultaneously, by reducing unnecessary wake-ups and frequent switching of non-target drawing units, power management jitter can be reduced and the stability of the drawing scheduling strategy can be improved, enabling the energy efficiency advantage of selecting execution units by task to be truly realized. Addressing the issue that non-target drawing units in heterogeneous drawing scenarios may generate additional power consumption due to driver activation, state switching, or default always-on operation, resulting in insignificant overall power consumption reduction, a power convergence mechanism is adopted to maintain non-wake-up / low-power status for non-target drawing units during drawing. This ensures that the target drawing unit completes drawing and refreshing while suppressing unnecessary power consumption, further improving the energy efficiency and battery life of the terminal device.

[0129] See Figure 2 , Figure 2 This is a flowchart illustrating a data display method provided in an embodiment of this application.

[0130] In some specific application scenarios, the main body of this application embodiment can be the aforementioned terminal device, which includes a first drawing unit and a second drawing unit. During the interface state maintenance and update area generation stage, the current interface display state is continuously maintained. When the interface display state changes, an update area that needs to be redrawn / refreshed (i.e., the scope of subsequent drawing) is generated, and this update information is submitted to the subsequent scheduling stage, subsequently entering drawing scheduling. A path selection is made based on the current update requirements and scheduling strategy: either the CPU path or the GPU path is selected. When the CPU path is selected, the process enters the left branch: the CPU drawing path (first drawing unit) undertakes the current drawing task, and the CPU drawing execution completes the specific drawing execution (e.g., generating pixel results in the buffer based on the update area and drawing instructions). When the GPU path is selected, the process enters the right branch: the GPU drawing path (second drawing unit) undertakes the current drawing task, and the GPU drawing execution completes the specific drawing execution (e.g., submitting commands through the graphics API, completing texture / blending / rasterization, etc., to generate the corresponding image results). Regardless of whether CPU or GPU is selected, the two branches will eventually converge the drawing results and output the drawing results to the display chain (such as performing partial / window refresh, updating the frame buffer, or driving the display panel rendering).

[0131] After the display output is completed, the process enters the drawing completion feedback and scheduling update stage: the drawing / refresh completion status is summarized, and scheduling-related feedback information is generated (such as drawing time, energy consumption estimate, load status, whether there is insufficient buffering / block refresh, etc.). This feedback is transmitted back through the feedback scheduling status loop shown by the dotted line on the right, which is used to update the scheduling status and strategy parameters. This allows the terminal device to make a more appropriate CPU / GPU path selection based on historical execution results when the next interface update arrives, forming a closed-loop scheduling mechanism for continuous iterative optimization.

[0132] This application also provides a computer storage medium for storing a program, wherein the program controls the device where the computer storage medium is located to execute any of the methods described above during runtime.

[0133] This application also provides a computer program product for implementing the methods described in any of the above embodiments.

[0134] See Figure 3 , Figure 3 This is a structural block diagram of a terminal device provided in an embodiment of this application.

[0135] This application also provides a terminal device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method provided in the above embodiments.

[0136] The terminal device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through an internal connection path.

[0137] The memory 110 is used to store computer programs, which in some implementations may include code for implementing the methods of the embodiments of this application.

[0138] The processor 120 executes the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, and output operation results and other data. In some implementations, when the solutions of the embodiments of this application are implemented by software or firmware, the computer program used to implement the solutions of the embodiments of this application can be stored in the processor 120 and executed by the processor 120.

[0139] The memory 110 may be volatile or non-volatile, or may include both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes random access memory (RAM), cache memory, and read-only memory (ROM). The memory 110 stores a computer program that can be executed by processor 120, causing processor 120 to implement the steps of any of the methods described above.

[0140] The processor 120 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can be any conventional processor.

[0141] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 120 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor 120. The software modules can reside in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory 110, and the processor 120 reads the information in memory 110 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0142] In some implementations, in addition to the hardware units described above, the terminal device may also include software modules, such as an operating system, a basic input / output system (BIOS), and application software.

[0143] An operating system is used to manage one or more of the hardware and software resources of a terminal device; it is the kernel and foundation of the terminal device. The operating system needs to handle basic tasks such as managing and configuring memory, determining the priority of system resource allocation and demand, controlling input and output devices, operating the network, and managing the file system. To facilitate user operation, most operating systems provide a user interface for interaction with the system.

[0144] The BIOS is used to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display processor temperature and execute temperature protection strategies.

[0145] Application software, also known as an application program, can be understood as software written for a specific user application purpose, and is one of the main categories of computer software. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0146] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0147] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0148] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0149] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "one or more" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "one," "above," and "that" as used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0150] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the embodiments described above can be referred to the corresponding processes and beneficial effects in other embodiments, and will not be repeated here.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0154] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0155] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data display method, applied to a terminal device including a first drawing unit and a second drawing unit, characterized in that, The method includes: Receive the target invalid display area determined by the graphical user interface library; Based on the target invalid display area, a drawing task for updating the target invalid display area is determined; The expected energy consumption of the first drawing unit and the second drawing unit in performing the drawing task is determined respectively, and the drawing unit with the smaller expected energy consumption is selected as the target drawing unit. The target drawing unit is controlled to draw the target invalid display area, and the drawing result is refreshed to the area corresponding to the target invalid display area.

2. The data display method according to claim 1, characterized in that, The invalid display area is determined by the graphical user interface library based on changes in the interface display state; the target invalid display area received by the graphical user interface library includes: Receive at least one invalid display area determined by the graphical user interface library during the refresh cycle; If an invalid display area is received, the invalid display area is taken as the target invalid display area; or, If multiple invalid display areas are received, the multiple invalid display areas are merged, and the merged area is taken as the target invalid display area; The merging of the multiple invalid display areas includes: merging intersecting invalid display areas into a bounding rectangle, and / or merging invalid display areas with a spacing less than a preset threshold into a bounding rectangle.

3. The data display method according to claim 1, characterized in that, The step of determining the drawing task for updating the target invalid display area based on the target invalid display area includes: Extract the drawing tasks associated with the target invalid display area from the drawing task queue generated by the graphical user interface library; The drawing task associated with the target invalid display area is used as a drawing task for updating the target invalid display area.

4. The data display method according to claim 1, characterized in that, The task characteristics of the drawing task include at least one of the following: the number of pixels covered by the target invalid display area, the spatial distribution characteristics of the target invalid display area, and the drawing operation type of the drawing task; The spatial distribution feature is used to characterize the degree of dispersion of the target invalid display area within the display area, and the degree of dispersion is determined based on the number of the target invalid display areas and / or the geometric position of the target invalid display areas; The drawing operation type includes at least one of the following: fill drawing, text drawing, image drawing, and transparent blending drawing.

5. The data display method according to claim 4, characterized in that, The step of determining the first estimated energy consumption of the first rendering unit and the second estimated energy consumption of the second rendering unit includes: Obtain the number of pixels covered by the target invalid display area corresponding to the drawing task; Based on the number of pixels and the unit pixel energy consumption parameters corresponding to the first drawing unit and the second drawing unit respectively, the first expected energy consumption of the first drawing unit and the second expected energy consumption of the second drawing unit are determined.

6. The data display method according to claim 5, characterized in that, The step of determining the first estimated energy consumption and the second estimated energy consumption respectively also includes: Obtain the wake-up energy consumption parameters corresponding to the first drawing unit and the second drawing unit, respectively; For each rendering unit, the unit pixel energy consumption obtained based on the number of pixels and the corresponding unit pixel energy consumption parameter is added to the wake-up energy consumption obtained based on the corresponding wake-up energy consumption parameter to determine the expected energy consumption of the corresponding rendering unit.

7. The data display method according to claim 5 or 6, characterized in that, The steps of determining the first estimated energy consumption and the second estimated energy consumption respectively include: The correction coefficient is determined based on the spatial distribution characteristics and / or the drawing operation type; The first and second projected energy consumption are corrected according to the correction factor.

8. The data display method according to claim 1, characterized in that, The step of refreshing the drawing result to the area corresponding to the target invalid display area includes: If the display buffer capacity used for drawing is insufficient to meet the buffer capacity required by the target invalid display area, the target invalid display area is divided into multiple sub-regions; The target rendering unit is controlled to sequentially render each sub-region; The drawing results corresponding to each sub-region are refreshed to the corresponding sub-region in the target invalid display area.

9. The data display method according to claim 1, characterized in that, The method further includes: During the process of the target drawing unit performing drawing and / or refreshing, the drawing units in the first drawing unit and the second drawing unit that are not identified as the target drawing unit are controlled to remain in an unwake-up state or a low-power state.

10. A terminal device, characterized in that, include: A memory and a processor, wherein the memory stores at least one computer program, the at least one computer program being loaded and executed by the processor to implement the method as described in any one of claims 1-9.

11. A computer storage medium, characterized in that, The computer storage medium is used to store a program, wherein, when the program is running, it controls the device where the computer storage medium is located to execute the method according to any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product is used to implement the method as described in any one of claims 1-8.

Images