Optimizing combiner workloads in steady state in processor device

Abstract

Optimizing combiner workloads in a steady state in a processor device is disclosed herein. In some aspects, a processor device is configured to perform image synthesis by executing a synthesizer pipeline, the synthesizer pipeline comprising a synthesizer, the synthesizer comprising a workload handler; a synthesizer hardware abstraction layer (HAL); a workload regulator communicatively coupled to the synthesizer HAL; and a display driver. The workload regulator detects that the image synthesis has entered a steady state and sends an indication to the workload handler to enter an acceleration mode. Upon receiving the indication, the workload handler places the synthesizer pipeline in an acceleration mode. While in an acceleration mode, the synthesizer bypasses the synthesizer HAL and sends acceleration mode data directly to the display driver.

Description

Priority application

[0001] This application claims priority to U.S. Patent Application Serial No. 18 / 466,498, filed on September 13, 2023, entitled “OPTIMIZING COMPOSITOR WORKLOADIN STEADY STATE IN PROCESSOR DEVICES”, the entire contents of which are incorporated herein by reference. background I. Technical Field The present disclosure relates in general to a compositor pipeline for generating composite images for display in a processor-based device, and more specifically to optimizing the performance of the compositor pipeline when performing image compositing.

[0002] II. Background Technology Processor-based devices (such as those running the Android operating system (OS)) can be configured to execute a compositor pipeline to generate composite images for display on a display device. These composite images can include multiple layers, such as a background layer, a status bar layer, an application layer, etc., which are combined by the compositor pipeline into a single image for display. A typical compositor pipeline includes a compositor (e.g., as a non-limiting example, the SurfaceFlinger system service provided by the Android OS) that processes image data from multiple buffers into image compositor data, thereby aggregating all layer geometry updates, such as position changes, buffer flips (i.e., switching between buffers storing image data), modified "dirty" areas, etc. This image compositor data is then passed to the compositor hardware abstraction layer (HAL), which interfaces between the execution software and the hardware. The compositor HAL is responsible for allocating the hardware resources of the display processing unit (DPU) of the processor-based device to balance power consumption and system performance. Based on the received image synthesizer data, the synthesizer HAL generates a resource configuration that reflects the optimal allocation of hardware resources and forwards the resource configuration to the display driver for programming the DPU.

[0003] In normal operation, image compositing performed by the compositor pipeline may include relatively short periods of layer geometry updates (e.g., applying startup or exit animations, transition animations, foreground or background visibility modifications, etc.), followed by a relatively long period in which the image composition remains in a steady state. As used herein, a "steady state" refers to a state in which the only layer geometry updates that occur are buffer flips or repeated dirty region updates (such as blinking a cursor) and in which resource allocation remains unchanged. However, in many use cases, even when image compositing is in a steady state, the processor load caused by the compositor pipeline can account for a significant portion of power consumption. Therefore, it is desirable to optimize the overall processor power consumption of the compositor pipeline while minimizing the negative impact on performance. Summary of the Invention

[0004] The aspects disclosed in the detailed description include optimizing the synthesizer workload in a stable state in a processor device. Related apparatus, methods, and computer-readable media are also disclosed. In this regard, a processor device executes a synthesizer pipeline including a synthesizer, the synthesizer including a workload handler; a synthesizer hardware abstraction layer (HAL); a workload conditioner communicatively coupled to the synthesizer HAL; and a display driver. The synthesizer pipeline executed by the processor device performs image compositing, during which the workload conditioner detects that the image compositing has entered a stable state. In some aspects, detecting that the image compositing has entered the stable state may include: the workload conditioner detecting that each update to the image compositing includes one of buffer flipping and repeated dirty region updates. In some aspects, the workload conditioner of the synthesizer pipeline may also collect application performance statistics, and the detection of the stable state may be further based on the application performance statistics.

[0005] In response to detecting that the image composition has entered the steady state, the workload conditioner sends an indication to the workload processor to enter accelerated mode. The workload processor then places the compositor pipeline in accelerated mode, during which the compositor bypasses the compositor HAL and sends accelerated mode data (e.g., speculative fences and / or latched buffer handles and fences) directly to the display driver. In some aspects, the compositor pipeline can also maintain the latest resource configuration for the display driver generated by the compositor HAL, instead of using the compositor HAL to generate new resource configurations as in normal operation. By bypassing the compositor HAL in accelerated mode, the power consumption generated by the processor device executing the compositor pipeline during the steady state can be reduced without negatively impacting performance.

[0006] In some respects, the processor device may later detect that the image composition has exited the stable state (e.g., by detecting layer geometry updates to the image composition other than buffer flips and repeated dirty region updates via the workload processor, or by detecting an indication from either the display driver or the processor device via the workload regulator). In response to detecting that the image composition has exited the stable state, the processor device puts the compositor pipeline into normal mode, and the compositor sends image composition data to the compositor HAL in a normal manner.

[0007] In another aspect, a processor device is provided. The processor device is configured to perform image compositing by executing a compositor pipeline including a compositor comprising a workload processor; a compositor HAL; a workload regulator communicatively coupled to the compositor HAL; and a display driver. The processor device is further configured to use the workload regulator to detect that the image compositing has entered a stable state. The processor device is further configured to use the workload regulator to send an indication to the workload processor to enter an accelerated mode in response to detecting that the image compositing has entered the stable state. The processor device is further configured to use the workload processor to place the compositor pipeline into the accelerated mode in response to receiving the indication. The processor device is further configured to use the compositor to directly send accelerated mode data to the display driver when in the accelerated mode.

[0008] In another aspect, a processor device is provided. The processor device includes components for performing image compositing. The processor device also includes components for detecting that the image compositing has entered a stable state. The processor device further includes components for sending an indication to enter an accelerated mode in response to detecting that the image compositing has entered the stable state. The processor device further includes components for placing the compositor pipeline into the accelerated mode. The processor device also includes components for directly sending accelerated mode data to a display driver when in the accelerated mode.

[0009] On the other hand, a method for optimizing compositor workload in a stable state is provided. The method includes: performing image compositing via a compositor pipeline executed by a processor device. The method further includes: detecting that the image compositing has entered a stable state via a workload regulator communicatively coupled to the compositor pipeline's HAL. The method further includes: in response to detecting that the image compositing has entered the stable state, sending an indication to the compositor's workload handler in the compositor pipeline to enter an accelerated mode via the workload regulator. The method further includes: placing the compositor pipeline into the accelerated mode via the workload handler. The method further includes: when in the accelerated mode, sending accelerated mode data directly to the display driver via the compositor.

[0010] In another aspect, a non-transitory computer-readable medium is disclosed. This non-transitory computer-readable medium stores computer-executable instructions that, when executed, cause a processor of a processor-based device to: perform image compositing by executing a compositor pipeline, the compositor pipeline including: a compositor including a workload processor; a compositor HAL; a workload regulator communicatively coupled to the compositor HAL; and a display driver. The computer-executable instructions also cause the processor to: use the workload regulator to detect that the image compositing has entered a stable state. The computer-executable instructions further cause the processor to: in response to detecting that the image compositing has entered the stable state, use the workload regulator to send an indication to the workload processor to enter an accelerated mode. The computer-executable instructions further cause the processor to: in response to receiving the indication, use the workload processor to place the compositor pipeline into the accelerated mode. The computer-executable instructions further cause the processor to: when in the accelerated mode, use the compositor to directly send accelerated mode data to the display driver. Attached Figure Description

[0011] Figure 1 This is a block diagram of an exemplary processor-based system according to some aspects, the exemplary processor-based system including a processor device that executes a synthesizer pipeline and is configured to optimize the synthesizer workload in a steady state; Figures 2A to 2B Examples are provided based on some aspects. Figure 1 A flowchart illustrating exemplary operations performed by the processor device to optimize the synthesizer workload in a steady state; and Figure 3 Yes, it can include Figure 1 An exemplary block diagram of a processor-based device. Detailed Implementation

[0012] Several exemplary aspects of this disclosure will now be described with reference to the accompanying drawings. The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or superior to other aspects. The terms “first,” “second,” etc., are used herein to distinguish similarly named elements and should not be construed as indicating an ordering relationship between such elements unless so explicitly described herein.

[0013] The aspects disclosed in the detailed description include optimizing the synthesizer workload in a stable state in a processor device. Related apparatus, methods, and computer-readable media are also disclosed. In this regard, a processor device executes a synthesizer pipeline including a synthesizer, the synthesizer including a workload handler; a synthesizer hardware abstraction layer (HAL); a workload conditioner communicatively coupled to the synthesizer HAL; and a display driver. The synthesizer pipeline executed by the processor device performs image compositing, during which the workload conditioner detects that the image compositing has entered a stable state. In some aspects, detecting that the image compositing has entered the stable state may include: the workload conditioner detecting that each update to the image compositing includes one of buffer flipping and repeated dirty region updates. In some aspects, the workload conditioner of the synthesizer pipeline may also collect application performance statistics, and the detection of the stable state may be further based on the application performance statistics.

[0014] In response to detecting that the image composition has entered the steady state, the workload conditioner sends an indication to the workload processor to enter accelerated mode. The workload processor then places the compositor pipeline in accelerated mode, during which the compositor bypasses the compositor HAL and sends accelerated mode data (e.g., speculative fences and / or latched buffer handles and fences) directly to the display driver. In some aspects, the compositor pipeline can also maintain the latest resource configuration for the display driver generated by the compositor HAL, instead of using the compositor HAL to generate new resource configurations as in normal operation. By bypassing the compositor HAL in accelerated mode, the power consumption generated by the processor device executing the compositor pipeline during the steady state can be reduced without negatively impacting performance.

[0015] In some respects, the processor device may later detect that the image composition has exited the stable state (e.g., by detecting layer geometry updates to the image composition other than buffer flips and repeated dirty region updates via the workload processor, or by detecting an indication from either the display driver or the processor device via the workload regulator). In response to detecting that the image composition has exited the stable state, the processor device puts the compositor pipeline into normal mode, and the compositor sends image composition data to the compositor HAL in a normal manner.

[0016] In this respect, Figure 1 , Figure 1 This is a block diagram of an exemplary processor-based device 100. The processor-based device 100 includes a processor device 102, which may be an ordered or unordered processor (OoP) and / or may be one of a plurality of processor devices 102 provided by the processor-based device 100. Examples of processor devices 102 may include, but are not limited to, digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (PGAs), or other equivalent integrated or discrete logic circuits.

[0017] like Figure 1 As seen, the processor-based device 100 includes a display processing unit (DPU) 104 for performing graphics operations and generating images for display on a display device 106. As a non-limiting example, the DPU 104 may include dedicated hardware units with fixed functionality and programmable components for rendering images and executing DPU applications. The DPU 104 may also include a DSP, a general-purpose microprocessor, an ASIC, an FPGA, or other equivalent integrated or discrete logic circuitry; for clarity, these devices are... Figure 1 Not shown in the diagram. Note that although processor device 102 and DPU 104 are in... Figure 1 In the examples, they are illustrated as separate units, but in some examples, the processor device 102 and the DPU 104 may be integrated into the same hardware element.

[0018] Figure 1 The processor-based device 100 may encompass any of known digital logic elements, semiconductor circuits, processing cores, and / or memory structures, as well as other elements or combinations thereof. The aspects described herein are not limited to any particular arrangement of elements, and the disclosed techniques can be readily extended to various structures and layouts on semiconductor dies or packages. It will be understood that some aspects of the processor-based device 100 may include, in addition to… Figure 1 Elements other than those exemplified, and / or may include Figure 1The illustrated components may be more or fewer. For example, processor-based device 100 may also include additional processor device 102, processor core, cache, controller, communication bus and / or persistent storage device, which, for clarity, are described in... Figure 1 The middle part is omitted.

[0019] Figure 1 The processor-based device 100 is configured to generate images, such as user interfaces, application interfaces, media streams, etc., for display on the display device 106. In this regard, the processor device 102 of the processor-based device 100 executes a compositor pipeline 108, which includes multiple system services to coordinate the compositing of multiple layers of image data. The compositor pipeline 108 includes a compositor 110, a compositor HAL 112, and a display driver 114. The compositor 110 (as a non-limiting example, it may include, for example, the SurfaceFlinger system service provided by the Android operating system (OS)) is responsible for compositing image data from multiple buffers (not shown) into image compositor data that aggregates all layer geometry updates, such as position changes, buffer flips, dirty regions, etc. The compositor HAL 112 receives the image compositor data from the compositor 110 via inter-process communication (IPC) and determines, based on the image compositor data, the optimal allocation of hardware resources for the DPU 104 to optimize power consumption and system performance. The synthesizer HAL 112 then generates a resource configuration that reflects the optimal allocation of hardware resources and forwards this resource configuration to the display driver 114 (e.g., a direct rendering manager (DRM) driver, as a non-limiting example) for programming the DPU 104.

[0020] As described above, in normal operation, image compositing performed by the compositor pipeline 108 may include relatively short periods of layer geometry updates (such as applying startup or exit animations, transition animations, foreground or background visibility modifications, etc.), followed by a relatively long period in which the image compositing remains in a steady state. As used herein, a "steady state" refers to a state in which the only layer geometry updates that occur are buffer flips or repeated dirty region updates (such as blinking a cursor) and in which resource allocation does not change. However, even when the image compositing is in a steady state, the processor load caused by the compositor pipeline 108 can account for a considerable portion of the power consumed by the processor-based device 100.

[0021] Therefore, in order to optimize the overall processor power consumption of the synthesizer pipeline 108 while minimizing the negative impact on performance, the processor device 102 is configured to optimize the synthesizer workload in a steady state. Figure 1In the example, the compositor pipeline 108 includes a workload regulator 116 communicatively coupled to the compositor HAL 112. The workload regulator 116 receives data from the compositor HAL 112 regarding layer geometry updates and, based on the received data, identifies when image compositing performed by the compositor pipeline 108 has entered a steady state. When the workload regulator 116 detects that image compositing is in a steady state, it transmits an instruction 118 to the workload handler 120 of the compositor 110 to enter an accelerated mode.

[0022] Then, workload processing program 120 puts synthesizer pipeline 108 into accelerated mode. When in accelerated mode, synthesizer 110 bypasses synthesizer HAL 112 and transmits accelerated mode data 122 directly to display driver 114. Accelerated mode data 122 includes data to be used by display driver 114 and / or DPU 104 to perform buffer flipping and / or dirty region updates, and, as a non-limiting example, may include, for example, speculative fences and / or latched buffer handles and fences to be programmed on DPU 104. By bypassing synthesizer HAL 112, the processor load caused by synthesizer pipeline 108 is reduced, thereby reducing the power consumption of processor-based device 100. In addition, in some aspects, synthesizer HAL 112 of synthesizer pipeline 108 maintains the latest resource configuration for display driver 114 (in... Figure 1 The configuration is marked as "latest configuration" 124 (i.e., instead of generating a new resource configuration as in normal operation). Therefore, the latest resource configuration 124 remains valid, and the synthesizer pipeline 108 continues to operate in accelerated mode.

[0023] In some aspects, the compositor pipeline 108 continues to operate in accelerated mode until image compositing has exited a stable state. In some such aspects, the workload handler 120 can detect that image compositing has exited a stable state by detecting layer geometry updates to image compositing other than buffer flips and repeated dirty region updates. In some such aspects, the workload regulator 116 can detect that image compositing has exited a stable state by detecting an instruction 126 from one of the display driver 114 and the processor device 102 (e.g., an instruction for kernel reset as a non-limiting example). In response to detecting that image compositing has exited a stable state, the processor device 102 (i.e., the workload handler 120 or the workload regulator 116) puts the compositor pipeline 108 back to normal mode. When in normal mode, the compositor pipeline 108 operates in a normal manner, wherein the compositor 110 sends image compositing data (in the compositor HAL 112 of the compositor pipeline 108) to the compositor pipeline 108. Figure 1 (128, labeled as "image composite data")

[0024] According to some aspects, the workload regulator 116 of the synthesizer pipeline 108 also collects application performance statistics (in Figure 1 The data is labeled "Application Performance Statistics" 130. As a non-limiting example, Application Performance Statistics 130 may include data such as: application execution time, frequency and duration of accelerated mode and normal mode activity for each application, power consumption estimates for each application during accelerated mode and normal mode, etc. In this respect, the workload conditioner 116 may also base its detection of a steady state on Application Performance Statistics 130 by determining whether entering accelerated mode may provide a power consumption benefit.

[0025] To illustrate, based on some aspects, Figure 1 The processor device 102 performs exemplary operations to optimize the synthesizer workload in a steady state. Figures 2A to 2B A flowchart illustrating exemplary operation 200 is provided. For clarity, in the description... Figures 2A to 2B When quoting Figure 1 The components. It should be understood that, in some respects, exemplary operation 200 may be performed in a different order than that illustrated herein, and / or may be omitted.

[0026] Exemplary operation 200 in Figure 2A The process begins with the synthesizer pipeline executed by the processor device (e.g., by...). Figure 1 The processor device 102 executes a compositor pipeline 108 to perform image compositing (box 202). According to some aspects, the workload regulator of the compositor pipeline 108 (such as...) Figure 1 The workload regulator 116 collects application performance statistics (e.g., Figure 1 Application performance statistics 130 (box 204). The workload regulator 116 then detects that image synthesis has entered a steady state (box 206). In some aspects, the operation of box 206 for detecting that image synthesis has entered a steady state may include the workload regulator 116 detecting one of buffer flipping and repeated dirty region updates for each update of image synthesis (box 208). In some aspects, the operation of box 206 for detecting that image synthesis has entered a steady state may also be based on application performance statistics 130 (box 210).

[0027] In response to detecting that image compositing has entered a stable state, the workload regulator 116 adjusts the workload processing procedure of the compositor in the compositor pipeline 108 (e.g., ...). Figure 1 The workload handler 120 of the synthesizer 110 sends an instruction to enter acceleration mode (such as...). Figure 1The instruction 118 (box 212). Then, the workload handler 120 puts the synthesizer pipeline 108 into accelerated mode (box 214). Exemplary operation in Figure 2B Continue at frame 216.

[0028] Now go to Figure 2B When the synthesizer pipeline 108 is in accelerated mode, multiple operations are performed (box 216). The synthesizer 110 feeds directly to the display driver (e.g., ...). Figure 1 The display driver 114 sends acceleration mode data (such as...) Figure 1 Acceleration mode data 122 (box 218). Depending on some aspects, synthesizer pipeline 108 also maintains the latest resource configuration 124 for display driver 114 (box 220).

[0029] In some aspects, processor device 102 may later detect that image composition has exited a steady state (box 222). Some of these aspects may specify that the operation of box 222 for detecting that image composition has exited a steady state includes workload processor 120 detecting layer geometry updates to the image composition other than buffer flipping and repeated dirty region updates (box 224). According to some of these aspects, the operation of box 222 for detecting that image composition has exited a steady state may include workload regulator 116 detecting indications from one of display driver 114 and processor device 102 (such as...). Figure 1 Instruction 126 (box 226). In response to detecting that image compositing has exited the steady state, processor device 102 puts compositor pipeline 108 into normal mode (box 228). When in normal mode, compositor 110 feeds data to compositor HAL (e.g., ...) in compositor pipeline 108. Figure 1 The synthesizer HAL 112 sends image synthesis data (such as...) Figure 1 Image synthesis data 128 (box 230).

[0030] Based on the information disclosed in this article and referenced Figure 1The processor devices discussed in these aspects can be located in or integrated into any processor-based device. Examples, without limitation, include: set-top boxes, entertainment units, navigation devices, communication devices, fixed location data units, mobile location data units, Global Positioning System (GPS) devices, mobile phones, cellular phones, smartphones, Session Initiation Protocol (SIP) phones, tablets, phablets, servers, computers, portable computers, mobile computing devices, laptops, wearable computing devices (e.g., smartwatches, health or fitness trackers, glasses, etc.), desktop computers, personal digital assistants (PDAs), monitors, computer monitors, televisions, tuners, radios, satellite radios, music players, digital music players, portable music players, digital video players, video players, digital video disc (DVD) players, portable digital video players, automobiles, vehicle components, avionics systems, drones, and multirotor aircraft.

[0031] In this respect, Figure 3 An example of a processor-based device 300 (as opposed to) Figure 1 The example is illustrated and described. In this example, functionally corresponding to Figure 1 The processor-based device 100 and the processor-based device 300 include a central processing unit (CPU) 302, which includes one or more processors 304 coupled to a cache memory 306. The processors 304 are also coupled to a system bus 308 and can be interactively coupled to devices included in the processor-based device 300. As is well known, the processors 304 communicate with these other devices by exchanging address, control, and data information on the system bus 308. For example, the processors 304 may communicate bus transaction requests to a memory controller 310. Although in Figure 3 Not illustrated, but multiple system buses 308 may be provided, each of which constitutes a different architecture.

[0032] Other devices can be connected to system bus 308. For example... Figure 3As illustrated, these devices may include a memory system 312, one or more input devices 314, one or more output devices 316, one or more network interface devices 318, and one or more display controllers 320. Input devices 314 may include any type of input device, including but not limited to input keys, switches, voice processors, etc. Output devices 316 may include any type of output device, including but not limited to audio, video, other visual indicators, etc. Network interface devices 318 may be any device configured to allow bidirectional data exchange with network 322. Network 322 may be any type of network, including but not limited to wired or wireless networks, private or public networks, local area networks (LANs), wireless local area networks (WLANs), wide area networks (WANs), and Bluetooth. ™ Networks and the Internet. Network interface device 318 can be configured to support any type of communication protocol desired. Memory system 312 may include a memory controller 310 coupled to one or more memory arrays 324. Display controller may include, for example... Figure 1 DPU 104.

[0033] The processor 304 can also be configured to access the display controller 320 via the system bus 308 to control the transmission of information to one or more displays 330. The display controller 320 transmits information to be displayed to the displays 330 via one or more video processors 332, which process the information to be displayed into a format suitable for the displays 330. The displays 330 may include any type of display, including but not limited to cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light-emitting diode (LED) display, etc.

[0034] Those skilled in the art will further understand that the various exemplary logic blocks, modules, circuits, and algorithms described in connection with the aspects disclosed herein can be implemented as electronic hardware, stored in memory or another computer-readable medium and executed by a processor or other processing device, or a combination of both. As an example, the master and slave devices described herein can be employed in any circuit, hardware component, integrated circuit (IC), or IC chip. The memory disclosed herein can be of any type and size and can be configured to store any type of information desired. To clearly illustrate this interchangeability, the functionality of the various exemplary components, blocks, modules, circuits, and steps has been generally described above. How such functionality is implemented depends on the specific application, design choices, and / or design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such specific implementation decisions should not be construed as departing from the scope of this disclosure.

[0035] The various exemplary logic blocks, modules, and circuits described in conjunction with the aspects disclosed herein may be implemented or executed using a processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic unit, discrete hardware component, or any combination thereof, designed to perform the functions described herein. The processor may be a microprocessor, but in alternative embodiments, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration).

[0036] The aspects disclosed herein may be embodied in hardware and instructions stored in the hardware, and may reside in, for example, random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CD-ROMs, or any other form of computer-readable medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a remote station. Alternatively, the processor and storage medium may reside as discrete components in a remote station, base station, or server.

[0037] It should also be noted that the operational steps described in any of the exemplary aspects of this document are described for the purpose of providing examples and discussion. The described operations may be performed in many different orders other than the order illustrated. Furthermore, the operations described in a single operational step may actually be performed in multiple different steps. In addition, one or more operational steps discussed in the exemplary aspects may be combined. It should be understood that, as will be apparent to those skilled in the art, many different modifications may be made to the operational steps illustrated in the flowcharts. Those skilled in the art will also understand that any of a variety of different techniques and arts can be used to represent information and signals. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0038] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0039] Specific implementation examples are described in the following numbered clauses: 1. A processor device, the processor device being configured to: Image compositing is performed by executing a synthesizer pipeline, the synthesizer pipeline comprising: A synthesizer, the synthesizer including a workload processing program; Synthesizer Hardware Abstraction Layer (HAL); A workload regulator, communicatively coupled to the synthesizer HAL; and Display driver; The workload regulator is used to detect that the image synthesis has entered a stable state. In response to detecting that the image synthesis has entered the stable state, the workload conditioner is used to send an indication to the workload processor to enter the acceleration mode; In response to receiving the instruction, the workload handler is used to place the synthesizer pipeline into the accelerated mode; and When in the accelerated mode, the synthesizer is used to send accelerated mode data directly to the display driver.

[0040] 2. The processor device according to Clause 1, wherein the processor device is configured to detect that the image synthesis has entered the steady state by performing the following operation: using the workload regulator to detect each update to the image synthesis including one of buffer flipping and repeated dirty region updates.

[0041] 3. The processor device according to any one of Clauses 1 to 2, wherein the processor device is further configured to: maintain the latest resource configuration for the display driver when in the acceleration mode.

[0042] 4. The processor device according to any one of clauses 1 to 3, wherein the processor device is further configured to: The image synthesis has been detected to have exited the stable state; In response to detecting that the image synthesis has exited the stable state, the synthesizer pipeline is placed in normal mode; and When in the normal mode, the synthesizer is used to send image synthesis data to the synthesizer HAL.

[0043] 5. The processor device according to Clause 4, wherein the processor device is configured to detect that the image composition has exited the steady state by using the workload processing program to detect layer geometry updates to the image composition other than buffer flipping and repeated dirty region updates.

[0044] 6. The processor device according to any one of Clauses 4 to 5, wherein the processor device is configured to detect that the image composition has exited the steady state by using the workload regulator to detect an indication from either the display driver or the processor device.

[0045] 7. The processor device according to any one of clauses 1 to 6, wherein: The processor device is further configured to use the workload conditioner to collect application performance statistics; and The processor device is configured to detect that the image synthesis has entered the stable state based on the application performance statistics.

[0046] 8. The processor device according to any one of Clauses 1 to 7, wherein the processor device is integrated into a device selected from the group consisting of: set-top boxes; entertainment units; navigation devices; communication devices; fixed location data units; mobile location data units; global positioning system (GPS) devices; mobile phones; cellular phones; smartphones; session initiation protocol (SIP) phones; tablet computers; tablet phones; servers; computers; portable computers; mobile computing devices; wearable computing devices; desktop computers; personal digital assistants (PDAs); monitors; computer monitors; televisions; tuners; radios; satellite radios; music players; digital music players; portable music players; digital video players; video players; digital video disc (DVD) players; portable digital video players; automobiles; vehicle components; avionics systems; unmanned aerial vehicles; and multi-rotor aircraft.

[0047] 9. A processor device, the processor device comprising: Components used to perform image composition; A component used to detect that the image synthesis has entered a stable state; A component for sending an indication to enter an acceleration mode in response to detecting that the image synthesis has entered the stable state; Components for placing the synthesizer pipeline into the accelerated mode; and A component for sending acceleration mode data directly to the display driver when the acceleration mode is in effect.

[0048] 10. A method for optimizing synthesizer workload in a steady state, the method comprising: Image compositing is performed through a synthesizer pipeline executed by the processor device; The image synthesis has reached a stable state by using the workload regulator of the synthesizer hardware abstraction layer (HAL) in the synthesizer pipeline; In response to detecting that the image synthesis has entered the stable state, the workload regulator sends an instruction to the workload processing program of the synthesizer in the synthesizer pipeline to enter the acceleration mode. The synthesizer pipeline is placed in the accelerated mode via the workload processing procedure; and When in the acceleration mode, acceleration mode data is sent directly to the display driver via the synthesizer.

[0049] 11. The method according to Clause 10, wherein detecting that the image synthesis has entered the steady state comprises: detecting, via the workload regulator, that each update to the image synthesis includes either buffer flipping or repeated dirty region updates.

[0050] 12. The method according to any one of Clauses 10 to 11, the method further comprising: maintaining the latest resource configuration for the display driver when in the acceleration mode.

[0051] 13. The method according to any one of clauses 10 to 12, wherein the method further comprises: The image synthesis has been detected to have exited the stable state; In response to detecting that the image synthesis has exited the stable state, the synthesizer pipeline is placed in normal mode; and When in the normal mode, image synthesis data is sent to the synthesizer HAL via the synthesizer.

[0052] 14. The method according to Clause 13, wherein detecting that the image synthesis has exited the steady state comprises: detecting layer geometry updates to the image synthesis other than buffer flips and repeated dirty region updates via the workload processing procedure.

[0053] 15. The method according to any one of Clauses 13 to 14, wherein detecting that the image synthesis has exited the steady state comprises: detecting an indication from one of the display driver and the processor device via the workload regulator.

[0054] 16. The method according to any one of clauses 10 to 15, the method further comprising collecting application performance statistics via the workload regulator; The detection that the image synthesis has entered the stable state is based on the application performance statistics.

[0055] 17. A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause a processor of a processor-based device to: Image compositing is performed by executing a synthesizer pipeline, the synthesizer pipeline comprising: A synthesizer, the synthesizer including a workload processing program; Synthesizer Hardware Abstraction Layer (HAL); A workload regulator, communicatively coupled to the synthesizer HAL; and Display driver; The workload regulator is used to detect that the image synthesis has entered a stable state. In response to detecting that the image synthesis has entered the stable state, the workload conditioner is used to send an indication to the workload processor to enter the acceleration mode; In response to receiving the instruction, the workload handler is used to place the synthesizer pipeline into the accelerated mode; and When in the accelerated mode, the synthesizer is used to send accelerated mode data directly to the display driver.

[0056] 18. The non-transitory computer-readable medium according to Clause 17, wherein the computer-executable instructions cause the processor to detect that the image composition has entered the steady state by causing the processor to perform the following operation: using the workload regulator to detect that each update to the image composition includes one of buffer flipping and repeated dirty region updates.

[0057] 19. A nontransitory computer-readable medium according to any one of Clauses 17 to 18, wherein the computer-executable instructions further cause the processor to: maintain the latest resource configuration for the display driver when in the accelerated mode.

[0058] 20. A non-transitory computer-readable medium according to any one of clauses 17 to 19, wherein the computer-executable instructions further cause the processor to: The image synthesis has been detected to have exited the stable state; In response to detecting that the image synthesis has exited the stable state, the synthesizer pipeline is placed in normal mode; and When in the normal mode, the synthesizer is used to send image synthesis data to the synthesizer HAL.

[0059] 21. The non-transitory computer-readable medium according to Clause 20, wherein the computer-executable instructions cause the processor to detect that the image composition has exited the steady state by causing the processor to: use the workload processing program to detect layer geometry updates to the image composition other than buffer flipping and repeated dirty region updates.

[0060] 22. A non-transitory computer-readable medium according to any one of Clauses 20 to 21, wherein the computer-executable instructions cause the processor to detect that the image composition has exited the steady state by causing the processor to perform the following operation: using the workload regulator to detect an instruction from one of the display driver and the processor device.

[0061] 23. A non-transitory computer-readable medium according to any one of clauses 17 to 22, wherein: The computer-executable instructions also cause the processor to use the workload regulator to collect application performance statistics; and The computer-executable instructions cause the processor to detect, based on the application performance statistics, that the image synthesis has entered the stable state.

Claims

1. A processor device configured to: perform image composition by executing a compositor pipeline, the compositor pipeline comprising: a compositor comprising a workload handler; a compositor hardware abstraction layer (HAL); a workload regulator communicatively coupled to the compositor HAL; and a display driver; detect, using the workload regulator, that the image composition has entered a steady state; in response to detecting that the image composition has entered the steady state, send, using the workload regulator, an indication to the workload handler to enter an accelerated mode; in response to receiving the indication, put, using the workload handler, the compositor pipeline in the accelerated mode; and while in the accelerated mode, send, using the compositor, accelerated mode data directly to the display driver.

2. The processor device of claim 1, configured to detect that the image composition has entered the steady state by being configured to detect, using the workload regulator, that each update to the image composition comprises one of a buffer flip and a repeated dirty region update.

3. The processor device of claim 1, further configured to, while in the accelerated mode, maintain an up-to-date resource configuration for the display driver.

4. The processor device of claim 1, further configured to: detect that the image composition has exited the steady state; in response to detecting that the image composition has exited the steady state, put the compositor pipeline in a regular mode; and while in the regular mode, send, using the compositor, image composition data to the compositor HAL.

5. The processor device of claim 4, configured to detect that the image composition has exited the steady state by being configured to detect, using the workload handler, a layer geometry update to the image composition other than a buffer flip and a repeated dirty region update.

6. The processor device of claim 4, configured to detect that the image composition has exited the steady state by being configured to detect, using the workload regulator, an indication from one of the display driver and the processor device.

7. The processor device of claim 1, wherein: the processor device is further configured to collect, using the workload regulator, application performance statistics; and the processor device is configured to detect that the image composition has entered the steady state based on the application performance statistics.

8. The processor device of claim 1 integrated into a device selected from the group consisting of: a set top box; an entertainment unit; a navigation device; a communications device; a fixed location data unit; a mobile location data unit; a global positioning system (GPS) device; a mobile phone; a cellular phone; a smartphone; a session initiation protocol (SIP) phone; a tablet; a phablet; a server; a computer; a portable computer; a mobile computing device; a wearable computer; a desktop computer; a personal digital assistant (PDA); a monitor; a computer monitor; a television; a tuner; a radio; a satellite radio; a music player; a digital music player; a portable music player; a digital video player; a video player; a digital video disc (DVD) player; a portable digital video player; an automobile; a transportation component; avionics; a drone; and a multicopter.

9. A processor device comprising: means for performing image composition; means for detecting that the image composition has entered a steady state; means for sending an indication to enter an accelerated mode in response to detecting that the image composition has entered the steady state; means for placing a compositor pipeline in the accelerated mode; and means for sending accelerated mode data directly to a display driver while in the accelerated mode.

10. A method for optimizing compositor workloads in steady state, the method comprising: performing image composition by a compositor pipeline executed by a processor device; detecting, by a work load regulator of a compositor hardware abstraction layer (HAL) of the compositor pipeline, that the image composition has entered a steady state; sending, by the work load regulator, an indication to enter an accelerated mode to a work load handler of a compositor of the compositor pipeline in response to detecting that the image composition has entered the steady state; placing, by the work load handler, the compositor pipeline in the accelerated mode; and sending, by the compositor, accelerated mode data directly to a display driver while in the accelerated mode.

11. The method of claim 10, wherein detecting that the image composition has entered the steady state comprises: detecting, by the work load regulator, each update to the image composition includes one of a buffer flip and a repeated dirty region update.

12. The method of claim 10, further comprising: maintaining, while in the accelerated mode, an up to date resource configuration for the display driver.

13. The method of claim 10, the method further comprising: detecting that the image composition has exited the steady state; placing, in response to detecting that the image composition has exited the steady state, the compositor pipeline in a regular mode; and sending, by the compositor, image composition data to the compositor HAL while in the regular mode.

14. The method of claim 13, wherein detecting that the image composition has exited the steady state comprises: detecting, by the work load handler, layer geometry updates to the image composition other than a buffer flip and a repeated dirty region update.

15. The method of claim 13, wherein detecting that the image composition has exited the steady state comprises: detecting, by the work load regulator, an indication from one of the display driver and the processor device.

16. The method of claim 10, further comprising collecting application performance statistics by the workload regulator; wherein detecting that the image composition has entered the steady state is based on the application performance statistics.

17. A non-transitory computer-readable medium having stored thereon computer- executable instructions that, when executed, cause a processor of a processor-based device to: perform image composition by executing a compositor pipeline, the compositor pipeline comprising: a compositor comprising a workload handler; a compositor hardware abstraction layer (HAL); a workload regulator communicatively coupled to the compositor HAL; and a display driver; detect, using the workload regulator, that the image composition has entered a steady state; in response to detecting that the image composition has entered the steady state, send, using the workload regulator, an indication to enter an accelerated mode to the workload handler; in response to receiving the indication, place, using the workload handler, the compositor pipeline in the accelerated mode; and when in the accelerated mode, send, using the compositor, accelerated mode data directly to the display driver.

18. The non-transitory computer-readable medium of claim 17, wherein the computer- executable instructions cause the processor to detect that the image composition has entered the steady state by causing the processor to detect, using the workload regulator, that each update to the image composition comprises one of a buffer flip and a repeated dirty region update.

19. The non-transitory computer-readable medium of claim 17, wherein the computer- executable instructions further cause the processor to, when in the accelerated mode, maintain an up-to-date resource configuration for the display driver.

20. The non-transitory computer-readable medium of claim 17, wherein the computer- executable instructions further cause the processor to: detect that the image composition has exited the steady state; in response to detecting that the image composition has exited the steady state, place the compositor pipeline in a regular mode; and when in the regular mode, send, using the compositor, image composition data to the compositor HAL.

21. The non-transitory computer-readable medium of claim 20, wherein the computer- executable instructions cause the processor to detect that the image composition has exited the steady state by causing the processor to detect, using the workload handler, a layer geometry update to the image composition other than a buffer flip and a repeated dirty region update.

22. The non-transitory computer-readable medium of claim 20, wherein the computer- executable instructions cause the processor to detect that the image composition has exited the steady state by causing the processor to: use the workload regulator to detect an indication from one of the display driver and the processor device.

23. The non-transitory computer-readable medium of claim 17, wherein: the computer-executable instructions further cause the processor to use the workload regulator to collect application performance statistics; and the computer-executable instructions cause the processor to detect that the image composition has entered the steady state based on the application performance statistics.

Images