Dynamic video / camera power framework

Abstract

This disclosure provides systems, devices, apparatuses, and methods for dynamic video / camera power framing, including computer programs encoded on a storage medium. A processor obtains (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with the video content. The processor maps the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile of the device. The processor outputs an indication of the device's power profile based on the mapping.

Description

Technical Field

[0001] This disclosure relates generally to processing systems, and more specifically to one or more techniques for display processing. Background Technology

[0002] Computing devices typically perform graphics and / or display processing (e.g., utilizing a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices can include, for example, computer workstations, mobile phones (such as smartphones), embedded systems, personal computers, tablet computers, and video game controllers. A GPU is configured to execute a graphics processing pipeline that includes one or more processing stages that operate together to execute graphics processing commands and output frames. A CPU controls the operation of a GPU by issuing one or more graphics processing commands to it. Modern CPUs are typically capable of executing multiple applications concurrently, each of which may require the use of a GPU during execution. A display processor can be configured to convert digital information received from the CPU into analog values ​​and can issue commands to a display panel to display visual content. Devices that provide content for visual presentation on a display can utilize a CPU, GPU, and / or display processor.

[0003] Current technologies for video playback may not be able to handle video scene transitions. Improved technologies are needed for video playback when video scene transitions occur. Summary of the Invention

[0004] The following is a simplified summary of one or more aspects to provide a basic understanding of these aspects. This summary is not a broad overview of all anticipated aspects, nor is it intended to identify key or essential elements of all aspects, nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that follows.

[0005] In one aspect of this disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus includes: a memory; and a processor coupled to the memory, and configured, based on information stored in the memory, to: obtain (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with the video content; map the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile of the device; and output an indication of the power profile of the device based on the mapping.

[0006] To achieve the foregoing and related objectives, one or more aspects include the features fully described below and specifically pointed out in the claims. The following description and drawings set forth some exemplary features of one or more aspects in detail. However, these features indicate only a few of the various ways in which the principles of the various aspects may be employed, and this description is intended to include all such aspects and their equivalents. Attached Figure Description

[0007] Figure 1 This is a block diagram illustrating an example of a system for generating content based on one or more techniques of this disclosure.

[0008] Figure 2 Example graphics processors (e.g., graphics processing units (GPUs)) according to one or more technologies according to this disclosure are illustrated.

[0009] Figure 3 An example display frame including a display processor and a display is illustrated according to one or more technologies of this disclosure.

[0010] Figure 4 This is an illustration illustrating examples of video scene switching according to one or more technologies disclosed herein.

[0011] Figure 5 This is a diagram illustrating an example of a dynamic video power framework according to one or more techniques of this disclosure.

[0012] Figure 6 This is a diagram illustrating an example of a dynamic power profile according to one or more techniques of this disclosure.

[0013] Figure 7 This is a call flow diagram illustrating example communication between a central processing unit (CPU) and a power control component according to one or more technologies of this disclosure.

[0014] Figure 8 This is a flowchart illustrating an example method of processing according to one or more techniques disclosed herein.

[0015] Figure 9 This is a flowchart illustrating an example method of processing according to one or more techniques disclosed herein. Detailed Implementation

[0016] Various aspects of the systems, apparatuses, computer program products, and methods will be described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function presented throughout this disclosure. Rather, these aspects are provided to make this disclosure comprehensive and complete, and to fully convey the scope of this disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art will understand that the scope of this disclosure is intended to cover any aspect of the systems, apparatuses, computer program products, and methods disclosed herein, whether implemented independently of or in combination with other aspects of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Furthermore, the scope of this disclosure is intended to cover such apparatuses or methods implemented using structures, functionalities, or structures and functionalities other than or different from the various aspects of the disclosure set forth herein. Any aspect disclosed herein may be embodied by one or more elements of the claims.

[0017] Although various aspects are described herein, many variations and substitutions of these aspects fall within the scope of this disclosure. While some potential benefits and advantages of the aspects of this disclosure are mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. Rather, the aspects of this disclosure are intended to be broadly applicable to different wireless technologies, system configurations, processing systems, networks, and transmission protocols, some of which are illustrated by way of example in the accompanying drawings and the description below. The detailed description and drawings are merely illustrative and not limiting of this disclosure, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0018] Several aspects are presented with reference to various apparatuses and methods. These apparatuses and methods are described in detail and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0019] For example, an element, any part of an element, or any combination of elements can be implemented as a “processing system” including one or more processors (which may also be referred to as processing units). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, system-on-a-chip (SoCs), baseband processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic units, discrete hardware circuits, and other suitable hardware configured to perform the various functionalities described throughout this disclosure. One or more processors in a processing system can execute software. Whether referred to as software, firmware, middleware, microcode, hardware description languages, or other names, software is broadly understood to mean instructions, instruction sets, code, code segments, program code, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc.

[0020] The term "application" can refer to software. As described herein, one or more technologies can refer to an application (e.g., software) configured to perform one or more functions. In such examples, the application may be stored in memory (e.g., on-chip memory of a processor, system memory, or any other memory). Hardware described herein, such as a processor, may be configured to execute the application. For example, an application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more technologies described herein. As an example, the hardware may access and execute code accessed from memory to perform one or more technologies described herein. In some examples, components are identified in this disclosure. In such examples, a component may be hardware, software, or a combination thereof. Each component may be a separate component or a subcomponent of a single component.

[0021] In one or more examples described herein, the described functionality can be implemented in hardware, software, or any combination thereof. If implemented in software, the functionality can be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media can be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disc storage devices, magnetic disk storage devices, other magnetic storage devices, combinations of computer-readable media of the types described above, or any other medium that can be used to store computer-executable code in the form of instructions or data structures accessible by a computer.

[0022] As used herein, instances of the term "content" may refer to "graphic content," "image," etc., regardless of whether the term is used as an adjective, noun, or other part of speech. In some examples, as used herein, the term "graphic content" may refer to content produced by one or more processes in a graphics processing pipeline. In other examples, as used herein, the term "graphic content" may refer to content produced by a processing unit configured to perform graphics processing. In yet another example, as used herein, the term "graphic content" may refer to content produced by a graphics processing unit.

[0023] Video playback applications present video content to users via a display. Some video playback applications offer features such as the ability to adjust video quality and participate in interactive activities (such as bullet comments). Bullet comments can refer to comments that appear at a specific video timestamp, are displayed directly above the video content, and are presented as a scroll bar (or window). However, these features can pose power consumption challenges for battery-constrained devices (e.g., smartphones, tablets, etc.). Some devices can reduce the power consumption of running video playback applications by optimizing for a fixed set of features (e.g., optimizing for a specific resolution, optimizing for a specific frame rate, and / or optimizing for a specific set of interactive features). However, such optimizations may not handle video scene transitions (e.g., switching from 30 frames per second (FPS) video playback to 60 FPS video playback). If a device loses power optimization due to video scene transitions, its power consumption may be affected, potentially reducing the amount of time the device can play video content. Furthermore, applying uniform parameters to all types of video playback scenarios can lead to performance degradation, especially for high-quality video.

[0024] This document describes various techniques related to dynamic video / camera power framing. In an example, a device (e.g., a CPU) obtains (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with video content. The device (e.g., the CPU) maps the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile of the device. As used herein, a “power profile” can refer to a device setting that (1) is designed to reduce the device’s battery consumption to a specific level at the expense of performance (e.g., reduced frame rate, reduced resolution, other reduced graphics settings, etc.) or (2) is designed to maintain a specific level of performance (e.g., a specific frame rate, a specific resolution, other specific graphics settings, etc.) at the expense of battery consumption (e.g., increased battery consumption). In some respects, a power profile designed to reduce the device’s battery consumption to a specific level at the expense of performance can also be referred to as a “power profile”, and a power profile designed to maintain a specific level of performance at the expense of battery consumption (e.g., (2) above) can be referred to as a “performance profile”. The device outputs an indication of the device’s power profile based on the mapping. By mapping a set of frame rate and resolution characteristics, as well as a set of workload characteristics, to a device's power profile and outputting an instruction on the device's power profile based on this mapping, the device may be able to select a power profile suitable for the current video playback scenario. Therefore, through this technique, the device may be able to conserve battery resources while maintaining a specific performance level.

[0025] In the aspects presented in this article, power profiles can be set for different video scenarios to optimize device power usage. Video decoding, screen refresh rate, and workload size (co-located and top threads) can also be monitored to allocate power decisions. For example, heavy workloads or dropped threads can be used to adjust power settings. Learning algorithms can be used to process expected FPS, target FPS, and clip resolution. Workload learning algorithms can be used to learn the expected workload of the profile and determine thresholds for power decisions. Therefore, an initial profile can be used, but performance can be monitored to increase / decrease power decisions.

[0026] The examples described herein may relate to the use and functionality of a graphics processing unit (GPU). As used herein, a GPU can be any type of graphics processor, and a graphics processor can be any type of processor designed or configured to process graphical content. For example, a graphics processor or GPU can be a dedicated circuit designed to process graphical content. As an additional example, a graphics processor or GPU can be a general-purpose processor configured to process graphical content.

[0027] Figure 1This is a block diagram illustrating an example content generation system 100 configured to implement one or more technologies of this disclosure. The content generation system 100 includes a device 104. Device 104 may include one or more components or circuitry for performing the various functions described herein. In some examples, one or more components of device 104 may be components of a System-on-a-Chip (SOC). Device 104 may include one or more components configured to perform one or more technologies of this disclosure. In the illustrated example, device 104 may include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, device 104 may include multiple components (e.g., a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131). Display 131 may refer to one or more displays 131. For example, display 131 may include a single display or multiple displays, which may include a first display and a second display. The first display may be a left-eye display, and the second display may be a right-eye display. In some examples, the first and second displays may receive different frames for presentation on the first and second displays. In other examples, the first and second displays may receive the same frames used for rendering on both displays. In yet another example, the results of graphics processing may not be displayed on the device; for example, the first and second displays may not receive any frames used for rendering on either display. Instead, the frames or graphics processing results may be transferred to another device. In some respects, this is referred to as split rendering.

[0028] Processing unit 120 may include internal memory 121. Processing unit 120 may be configured to perform graphics processing using graphics processing pipeline 107. Content encoder / decoder 122 may include internal memory 123. In some examples, device 104 may include a processor configured to perform one or more display processing techniques on one or more frames generated by processing unit 120, and then display those frames through one or more displays 131. Although the processor in example content generation system 100 is configured as display processor 127, it should be understood that display processor 127 is one example of a processor and other types of processors, controllers, etc., may be used instead of display processor 127. Display processor 127 may be configured to perform display processing. For example, display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by processing unit 120. One or more displays 131 may be configured to display or otherwise present the frames processed by display processor 127. In some examples, one or more displays 131 may include one or more of the following: liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, projection display device, augmented reality display device, virtual reality display device, head-mounted display, or any other type of display device.

[0029] Memory (such as system memory 124) external to processing unit 120 and content encoder / decoder 122 may be accessible to processing unit 120 and content encoder / decoder 122. For example, processing unit 120 and content encoder / decoder 122 may be configured to read from and / or write to external memory (such as system memory 124). Processing unit 120 may be communicatively coupled to system memory 124 via a bus. In some examples, processing unit 120 and content encoder / decoder 122 may be communicatively coupled to internal memory 121 via the bus or via a different connection.

[0030] Content encoder / decoder 122 can be configured to receive graphic content from any source, such as system memory 124 and / or communication interface 126. System memory 124 can be configured to store received encoded or decoded graphic content. Content encoder / decoder 122 can be configured to receive encoded or decoded graphic content from system memory 124 and / or communication interface 126, for example, in the form of encoded pixel data. Content encoder / decoder 122 can be configured to encode or decode any graphic content.

[0031] Internal memory 121 or system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, internal memory 121 or system memory 124 may include RAM, static random access memory (SRAM), dynamic random access memory (DRAM), erasable programmable ROM (EPROM), EEPROM, flash memory, magnetic data media or optical storage media, or any other type of memory. According to some examples, internal memory 121 or system memory 124 may be a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be construed as meaning that internal memory 121 or system memory 124 is not removable or that its contents are static. For example, system memory 124 may be removed from device 104 and moved to another device. Alternatively, system memory 124 may not be removable from device 104.

[0032] Processing unit 120 may be a CPU, GPU, GPGPU, or any other processing unit configured to perform graphics processing. In some examples, processing unit 120 may be integrated into the motherboard of device 104. In other examples, processing unit 120 may reside on a graphics card mounted in a port on the motherboard of device 104, or may otherwise be incorporated into a peripheral device configured to interoperate with device 104. Processing unit 120 may include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, arithmetic logic units (ALUs), DSPs, discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, processing unit 120 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 121) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors.

[0033] The content encoder / decoder 122 can be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 may be integrated into the motherboard of device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the content encoder / decoder 122 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 123) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors.

[0034] In some aspects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any of the receiving functions described herein with respect to device 104. Additionally, the receiver 128 may be configured to receive information from another device, such as eye or head positioning information, rendering commands, and / or location information. The transmitter 130 may be configured to perform any of the transmitting functions described herein with respect to device 104. For example, the transmitter 130 may be configured to transmit information to another device, which may include a request for content. The receiver 128 and the transmitter 130 may be combined to form a transceiver 132. In such an example, the transceiver 132 may be configured to perform any of the receiving and / or transmitting functions described herein with respect to device 104.

[0035] Refer again Figure 1 In some aspects, processing unit 120 may include profile switcher 198, which is configured to obtain (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with video content; map the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile of the device; and output an indication of the power profile of the device based on the mapping. Although the following description may focus on display processing, the concepts described herein are applicable to other similar processing techniques. Furthermore, although the following description may focus on power optimization for video playback, the concepts described herein are also applicable to power optimization for camera applications (i.e., video streaming applications that use cameras to capture video streamed to another device via a network connection).

[0036] Devices such as device 104 can refer to any device, apparatus, or system configured to perform one or more of the technologies described herein. For example, a device can be a server, base station, user equipment, client device, station, access point, computer (such as a personal computer, desktop computer, laptop computer, tablet computer, computer workstation, or mainframe computer), end product, apparatus, telephone, smartphone, server, video game platform or controller, handheld device (such as a portable video game device or personal digital assistant (PDA)), wearable computing device (such as a smartwatch, augmented reality device, or virtual reality device), non-wearable device, display or display device, television, set-top box, intermediate network device, digital media player, video streaming device, content streaming device, in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more of the technologies described herein. The processes described herein may be described as being performed by a specific component (e.g., GPU), but in other embodiments, other components (e.g., CPU) consistent with the disclosed embodiments may be used to perform them.

[0037] A GPU can process various types of data or data packets within its pipeline. For example, in some aspects, a GPU can process two types of data or data packets, such as context register packets and draw call data. Context register packets can be a set of global state information, such as information about global registers, shaders, or constant data, which can adjust how the graphics context will be processed. For example, a context register packet may include information about the color format. In some aspects of a context register packet, there may be one or more bits indicating which workload belongs to the context register. Additionally, multiple functions or programs can run simultaneously and / or in parallel. For example, a function or program may describe an operation, such as a color mode or color format. Therefore, context registers can define various states of the GPU.

[0038] Context states can be used to determine how individual processing units (e.g., vertex extractors (VFDs), vertex shaders (VSs), shader processors, or geometry processors) operate and / or in which mode they operate. To do this, the GPU uses context registers and programming data. In some aspects, the GPU can generate workloads in the pipeline based on the context register definitions of modes or states, such as vertex or pixel workloads. Certain processing units (e.g., VFDs) can use these states to determine certain functions, such as how to aggregate vertices. Because these modes or states can change, the GPU may need to modify the corresponding context. Additionally, the workload corresponding to a mode or state may follow the changed mode or state.

[0039] Figure 2 Example GPU 200 is illustrated according to one or more technologies according to this disclosure. For example... Figure 2 As shown, GPU 200 includes a command processor (CP) 210, a draw call group 212, a VFD 220, a VS 222, a vertex cache (VPC) 224, a triangle setup engine (TSE) 226, a rasterizer (RAS) 228, a Z-process engine (ZPE) 230, a pixel interpolator (PI) 232, a fragment shader (FS) 234, a rendering backend (RB) 236, an L2 cache (UCHE) 238, and system memory 240. Although Figure 2 The GPU 200 includes processing units 220 to 238, but the GPU 200 may include multiple additional processing units. Additionally, processing units 220 to 238 are merely examples, and the GPU may use any combination or order of processing units in accordance with this disclosure. The GPU 200 also includes a command buffer 250, a context register group 260, and a context state 261.

[0040] like Figure 2 As shown, the GPU can use a CP (e.g., CP 210) or a hardware accelerator to resolve the command buffer into context register groups (e.g., context register group 260) and / or draw call data groups (e.g., draw call group 212). Subsequently, CP 210 can transfer the context register group 260 or the draw call group 212 to a processing unit or block in the GPU via a separate path. Furthermore, the command buffer 250 can alternate between different states of the context registers and draw calls. For example, the command buffer can simultaneously store the following information: the context register of context N, the draw call of context N, the context register of context N+1, and the draw call of context N+1.

[0041] GPUs can render images in a variety of different ways. In some cases, GPUs can render images using direct rendering and / or tiled rendering. In a tiled rendering GPU, an image can be divided or separated into different parts or tiles. After the image is divided, each part or tile can be rendered individually. A tiled rendering GPU can divide a computer graphics image into a grid format, so that each part of the grid (i.e., a tile) is rendered individually. In some aspects of tiled rendering, the image can be divided into different bins or tiles during binning passes. In some aspects, a visibility stream can be constructed during binning passes, where visible primitives or draw calls can be identified. A rendering pass can be performed after a binning pass. In contrast to tiled rendering, direct rendering does not divide a frame into smaller bins or tiles. Instead, in direct rendering, the entire frame is rendered at once (i.e., without binning passes). Additionally, some types of GPUs allow both tiled rendering and direct rendering (e.g., flex rendering).

[0042] In some respects, a GPU can apply the drawing or rendering process to different bins or tiles. For example, a GPU can render a bin and perform all drawing for the primitives or pixels within that bin. During the bin-based rendering process, the rendering target can be located in GPU Internal Memory (GMEM). In some instances, after rendering a bin, the contents of the rendering target can be moved to system memory, and GMEM can be freed to render the next bin. Additionally, a GPU can render another bin and perform drawing for the primitives or pixels within that bin. Thus, in some respects, there may be a small number of bins covering all the drawing on a surface, for example, four bins. Furthermore, a GPU can loop through all the drawing in a bin but perform drawing only for visible drawing calls, i.e., drawing calls that include visible geometry. In some respects, a visibility stream can be generated, for example, in binning passes, to determine the visibility information of each primitive in an image or scene. For example, such a visibility stream can identify whether a primitive is visible. In some respects, this information can be used to remove invisible primitives, such that, for example, invisible primitives are not rendered in a rendering pass. Additionally, at least some primitives that are marked as visible can be rendered in the rendering pass.

[0043] In some aspects of tile rendering, there can be multiple processing stages or passes. For example, rendering can be performed in two passes, such as a binning, visibility, or box visibility pass and a rendering or box rendering pass. During a visibility pass, the GPU can input a rendering workload, record the positions of primitives or triangles, and then determine which primitives or triangles fall into which bins or regions. In some aspects of a visibility pass, the GPU can also identify or mark the visibility of each primitive or triangle in the visibility stream. During a rendering pass, the GPU can input a visibility stream and process one bin or region at a time. In some aspects, the visibility stream can be analyzed to determine which primitives or primitive vertices are visible or invisible. Thus, visible primitives or primitive vertices can be processed. By doing so, the GPU can reduce the unnecessary workload of processing or rendering invisible primitives or triangles.

[0044] In some aspects, certain types of primitive geometry, such as localized geometry, can be processed during visibility passes. Additionally, primitives can be categorized into different bins or regions based on their localization or position. In some instances, categorizing primitives or triangles into different bins can be performed by determining visibility information for those primitives or triangles. For example, the GPU can determine the visibility information for each primitive in each bin or region or write it to, for example, system memory. This visibility information can be used to determine or generate a visibility stream. In a rendering pass, the primitives in each bin can be rendered individually. In these cases, the visibility stream can be retrieved from memory and used to remove primitives that are not visible to that bin.

[0045] Some aspects of a GPU or GPU architecture can provide multiple different options for rendering (e.g., software rendering and hardware rendering). In software rendering, the driver or CPU can process each view... Figure 1 The entire frame geometry is copied each time. Additionally, some different states can change depending on the viewpoint. Therefore, in software rendering, the software can copy the entire workload by changing some states that can be used for rendering for each viewpoint in the image. In some respects, this can lead to increased overhead because the GPU may submit the same workload multiple times for each viewpoint in the image. In hardware rendering, the hardware or GPU may be responsible for copying or processing the geometry for each viewpoint in the image. Therefore, the hardware can manage the copying or processing of primitives or triangles for each viewpoint in the image.

[0046] Figure 3 This is a block diagram 300 illustrating an example display frame including a processing unit 120, system memory 124, display processor 127, and display 131, as identified by incorporable device 104.

[0047] A GPU may be included in a device that provides content for visual presentation on a display. For example, processing unit 120 may include GPU 310 configured to render graphics data for display on a computing device (e.g., device 104), which may be a computer workstation, mobile phone, smartphone or other intelligent device, embedded system, personal computer, tablet computer, video game console, etc. The operation of GPU 310 may be controlled based on one or more graphics processing commands provided by CPU 315. CPU 315 may be configured to execute multiple applications concurrently. In some cases, each of the concurrently executing applications may utilize GPU 310 simultaneously. Processing techniques may be executed by outputting frames over a physical or wireless communication channel via processing unit 120.

[0048] System memory 124, executable by processing unit 120, may include user space 320 and kernel space 325. User space 320 (sometimes referred to as "application space") may include software applications and / or application frameworks. For example, software applications may include operating systems, media applications, graphics applications, workspace applications, etc. Application frameworks may include frameworks used by one or more software applications, such as libraries, services (e.g., display services, input services, etc.), application programming interfaces (APIs), etc. Kernel space 325 may further include display driver 330. Display driver 330 may be configured to control display processor 127. For example, display driver 330 may cause display processor 127 to synthesize frames and send data of the frames to the display.

[0049] Display processor 127 includes display control block 335 and display interface 340. Display processor 127 can be configured to (e.g., based on input received from display driver 330) manipulate the functions of display 131. Display control block 335 can also be configured to output image frames to display 131 via display interface 340. In some examples, display control block 335 may additionally or alternatively perform post-processing of image data provided based on the processing unit 120's execution of system memory 124.

[0050] Display interface 340 can be configured to cause display 131 to display image frames. Display interface 340 can output image data to display 131 according to an interface protocol, such as, for example, MIPI DSI (Mobile Industry Processor Interface, Display Serial Interface) . That is, display 131 can be configured according to the MIPI DSI standard. The MIPI DSI standard supports video mode and command mode. In an example where display 131 operates in video mode, display processor 127 can continuously refresh the graphic content of display 131. For example, the entire graphic content can be refreshed in each refresh cycle (e.g., line by line). In an example where display 131 operates in command mode, display processor 127 can write the graphic content of a frame to buffer 350.

[0051] In some such examples, display processor 127 may not continuously refresh the graphics content of display 131. Instead, display processor 127 may use a vertical sync (Vsync) pulse to coordinate the rendering and consumption of graphics content at buffer 350. For example, when a Vsync pulse is generated, display processor 127 may output new graphics content to buffer 350. Thus, the generation of a Vsync pulse indicates that current graphics content has been rendered at buffer 350.

[0052] A frame is displayed on the monitor 131 based on the display controller 345, the display client 355, and the buffer 350. The display controller 345 can receive image data from the display interface 340 and store the received image data in the buffer 350. In some examples, the display controller 345 can output the image data stored in the buffer 350 to the display client 355. Therefore, the buffer 350 can represent local memory of the monitor 131. In some examples, the display controller 345 can directly output the image data received from the display interface 340 to the display client 355.

[0053] Display client 355 may be associated with a touch panel that senses interaction between the user and display 131. When the user interacts with display 131, one or more sensors in the touch panel may output signals to display controller 345 indicating which of the one or more sensors is active, the duration of the sensor activity, the pressure applied to the one or more sensors, etc. Display controller 345 may use the sensor outputs to determine how the user interacts with display 131. Display 131 may further be associated with / include other devices such as cameras, microphones, and / or speakers that operate in conjunction with display client 355.

[0054] Some processing techniques of device 104 can be performed through three stages (e.g., stage 1: rendering stage; stage 2: compositing stage; and stage 3: display / transfer stage). However, other processing techniques can combine the compositing stage and the display / transfer stage into a single stage, allowing the processing technique to be performed based on a total of two stages (e.g., stage 1: rendering stage; and stage 2: compositing / display / transfer stage). During the rendering stage, GPU 310 can process the content buffer based on the execution of the application that generates content on a pixel-by-pixel basis. During the compositing and display stages, pixel elements can be assembled to form a frame, which is then transferred to the physical display panel / subsystem (e.g., display 131) that displays the frame.

[0055] Instructions executed by the CPU (e.g., software instructions) or by the display processor can cause the CPU or display processor to search for and / or generate compositing strategies for compositing frames based on dynamic priorities and runtime statistics associated with one or more compositing strategy groups. A frame to be displayed by a physical display device (such as a display panel) may include multiple layers. Furthermore, frame compositing may be based on combining multiple layers into a frame (e.g., based on a frame buffer). After combining the multiple layers into a frame, the frame can be provided to the display panel for display on that display panel. The process of combining each of the multiple layers into a frame may be referred to as compositing, frame compositing, compositing process, compositing handling, etc.

[0056] A frame compositing process or strategy can correspond to a technique used to combine different layers from multiple layers into a single frame. Multiple layers can be stored in double data rate (DDR) memory. Each of the multiple layers can further correspond to a separate buffer. A compositor or hardware compositor (HWC) associated with a block or function determines the input to each layer / buffer and performs the frame compositing process to generate an output indicating the composite frame. That is, the input can be layers, and the output can be a frame compositing process used to synthesize the frame to be displayed on a display panel.

[0057] Some aspects of display processing can utilize different types of mask layers, such as shape masks. A mask layer is a layer that can represent a portion of a display or display panel. For example, an area of ​​the mask layer may correspond to an area of ​​the display, but the entire mask layer can depict a portion of the content actually displayed on the display or panel. For example, a mask layer may include a top and bottom portion of the display area, but the middle portion of the mask layer may be empty. In some examples, multiple mask layers may exist to represent different portions of the display area. Furthermore, for certain portions of the display area, the contents of different mask layers may overlap each other. Therefore, a mask layer can represent a portion of the display area that may or may not overlap with other mask layers.

[0058] Figure 4Figure 400 illustrates an example 402 of video scene switching according to one or more technologies of this disclosure. Device 104 (e.g., smartphone, tablet, etc.) can be connected via a video playback application ( Figure 4 (Not depicted in the text) Plays video content 404 on display 131. In this example, video content 404 may be streaming video content. The video playback application may have features that allow the user to set / adjust the video quality (or other characteristics of video content 404). For example, the video playback application may receive input from the user specifying a frame rate and / or resolution, and the video playback application may play video content 404 at that frame rate and / or resolution. As used herein, "frame rate" may refer to the number of frames rendered per second for display. In this example, the frame rate may be 30 FPS, 60 FPS, 90 FPS, or 120 FPS. In this example, the resolution may be 1920×1080 pixels, 2400×1080 pixels, or 2532×1170 pixels.

[0059] In the example, device 104 may experience a switching event 406 that causes device 104 to change the video quality of video content 404. In one example, switching event 406 may be or include manual input (e.g., text input, voice input, etc.) received from a user specifying an adjustment to the quality of the video content (e.g., resolution adjustment, frame rate adjustment). In another example, switching event 406 may be or include video content 404 reaching a specific playback point. In yet another example, switching event 406 may include a change in network conditions (e.g., an increase or decrease in the available bandwidth of device 104). In yet another example, switching event 406 may be a change in the availability of computing resources of device 104.

[0060] When device 104 experiences a switching event 406, device 104 may change the video quality of video content 404. In the example, when switching event 406 occurs, device 104 may be playing video content 404 at a first resolution and a first frame rate. Based on switching event 406, device 104 may switch to playing video content 404 at a second resolution and a second frame rate, wherein the second resolution may be different from the first resolution, and wherein the second frame rate may be different from the first frame rate.

[0061] In one aspect, the switching event 406 could be the point in time when video content 404 reaches the point where bullet comments 408 are to be displayed. Bullet comments can refer to comments that appear at a specific video timestamp of video content 404, displayed directly above the video content, and presented as a scrollbar (or window). Bullet comments can act as timed feedback to the user about what they are watching on the screen at that video timestamp. Bullet comments may increase power consumption on the device.

[0062] In one aspect, device 104 may optimize the playback of video content 404 for a fixed set of characteristics (e.g., fixed resolution, fixed frame rate, etc.). In the example, device 104 may optimize the playback of video content 404 for a fixed 30 FPS video playback. However, in such an aspect, if a switching event 406 occurs and the frame rate of video content 404 changes from 30 FPS to 60 FPS, device 104 may lose power optimization, which could result in increased power consumption or reduced performance. Furthermore, applying a fixed power parameter to all video playback scenarios can lead to performance degradation, especially for high-quality video content. The aspect presented herein relates to a dynamic video / camera power framework that can maintain an optimal power level under given changing device conditions (e.g., given switching event 406). As used herein, "power level" can refer to the amount of battery power consumed by the device.

[0063] In one aspect, this paper describes a dynamic video power technique (i.e., a solution). Dynamic video power technology allows switching power profiles between different video scenarios. Power profiles can be categorized into multiple levels, each representing a balance between power benefits and performance impact. Dynamic video power technology helps ensure broad optimization across video playback scenarios without performance regression.

[0064] In one aspect, this paper describes a power decision framework. This framework can be configured to align with a specific power optimization scenario. In an example, when a video playback session begins, the power decision framework analyzes the session to determine the video decoding rate and / or screen refresh frame rate. The framework can then allocate (i.e., determine) power decisions based on the video decoding rate and / or screen refresh frame rate, enabling dynamic switching of power optimization to match the specific video playback scenario.

[0065] In one aspect, this paper describes a workload monitor. The workload monitor evaluates the application encapsulation behavior of co-located and top-threaded applications. When a task with a computationally heavy workload is detected, power settings can be adjusted (e.g., negative boosts can be applied) to achieve optimal power settings for the current task.

[0066] In one aspect described herein, feedback from the video frame can be used to trigger a frame drop warning. The FPS monitor checks the stability of the video decoding rate and / or screen refresh rate. If the FPS monitor detects instability in the video decoding rate and / or screen refresh rate, it can trigger a new round of power decision selection. When the FPS monitor confirms the stability of the video decoding rate and / or screen refresh rate, the device can attempt to switch to a profile that reduces power consumption. The FPS monitor can then re-examine the stability of the video decoding rate and / or screen refresh rate. The device can perform at least one round of adjustments to select a suitable power profile that balances performance and power consumption.

[0067] Figure 5 Figure 500 illustrates an example of a dynamic video power framework 502 according to one or more techniques of this disclosure. The dynamic video power framework 502 may be implemented by device 104. In the example, the CPU of the device may implement some or all of the dynamic video power framework 502. The dynamic video power framework 502 may implement FPS learning, workload learning, and power level control to adjust to an appropriate power profile.

[0068] At 504, device 104 may begin playing video (i.e., video content) via a video playback application. At 506, device 104 may perform FPS learning based on video playback. In the example, device 104 may perform FPS learning based on a cue FPS 508, a target FPS 510, and / or a clip resolution 512. The cue FPS 508 may be an indication of the frame rate from a video frame (e.g., a video frame supporting 15 FPS, 30 FPS, 60 FPS, and / or 120 FPS). As used herein, "video frame" may refer to software configured to facilitate the display of video content on the device. The target FPS 510 may be an indication of the frame rate from a frame compositor. "Compositor" or "frame compositor" may refer to software and / or hardware that synthesizes multiple layers to generate frames. The target FPS 510 may be based on the refresh rate of display 131. The clip resolution 512 may be the resolution of a video clip generated from video playback.

[0069] At 514, device 104 may perform workload learning based on video playback. For example, device 104 may learn task workload 516 (e.g., task group workload) based on video playback. In the example, task workload 516 may be a workload performed by one or more threads for playing video. As used herein, "thread" may refer to a sequence of programming instructions that can be independently managed by a scheduler (e.g., a minimal sequence of programming instructions). As used herein, "task" may refer to an action performed by a thread. The scheduler may be part of the operating system. The device may evaluate task workload 516 based on a workload threshold 518. As used herein, "workload threshold" may refer to the percentage utilization of one or more threads.

[0070] At 520, device 104 may make power decisions based on FPS learning at 506 and / or workload learning at 514. For example, device 104 may process a combination of cue FPS 508, target FPS 510, and / or clip resolution 512 to map cue FPS 508, target FPS 510, and / or clip resolution 512 to profile 522 (e.g., a power profile). Additionally or alternatively, device 104 may map task workload 516 to profile 522 based on an evaluation of task workload 516 according to workload threshold 518. At 524, device 104 may implement profile 522. For example, device 104 may send instructions to adjust the power consumption of the device based on profile 522. In the example, device 104 may send instructions to power level controller 526 of device 104 based on profile 522, wherein power level controller 526 may adjust the power consumption of device 104 based on the instructions.

[0071] In one aspect, a given power decision can be mapped to an initial profile (e.g., profile 522). After device 104 implements the initial profile, device 104 can monitor the device's frame rate to increase or decrease the power step and update the initial profile. For example, power level controller 526 can receive an indication of video frame drop 528 from the video frame. Based on the indication of video frame drop 528, at 530, power level controller 526 can apply a power step 532 to profile 522. For example, power step 532 can be a positive power step that increases the power consumption of device 104 or a negative power step that decreases the power consumption of device 104. At 534, an FPS monitor can monitor the frame rate of device 104 after power step 532 has been applied to profile 522. If the frame rate is stable, at 524, device 104 can implement profile 522 with power step 532 applied. If the frame rate is unstable, device 104 can repeat the aforementioned steps until a stable frame rate is achieved.

[0072] Figure 6 Figure 600 illustrates an example 602 of a dynamic power profile according to one or more techniques of this disclosure. Device 104 may include a default level profile 604. In the example, the default level profile 604 balances battery consumption and performance. Device 104 may include a first performance level profile 606 and a second performance level profile 608. In the example, when device 104 displays video content, the first performance level profile 606 may be designed to maintain a first performance level (e.g., a first resolution and a first frame rate), and the second performance level profile 608 may be designed to maintain a second performance level (e.g., a second resolution and a second frame rate), wherein the second performance level is greater than the first performance level (e.g., the second resolution is greater than the first resolution, and the second frame rate is greater than the first frame rate). Device 104 may also include a first power level profile 610 and a second power level profile 612. In the example, when device 104 displays video content, the first power level profile 610 may be designed to maintain a first battery consumption of device 104, and the second power level profile 612 may be designed to maintain a second battery consumption of device 104, wherein the second battery consumption is less than the first battery consumption.

[0073] In one aspect presented in this paper, dynamic video power technology (i.e., the solution) provides a set of profiles with power benefits and corresponding performance impacts. In scenarios requiring high video playback performance, the initial optimized profile can prioritize performance over power savings. The initial optimized profile can be determined by power decisions (e.g., the decision at 520). Table 1 below illustrates the power decision mapping.

[0074]

[0075] Figure 7 This is a call flow diagram 700 illustrating example communication between a CPU 702 and a power control component 704 according to one or more technologies of this disclosure. In the example, the CPU 702 and / or the power control component 704 may be included in the device 104. In the example, the power control component 704 may be implemented in hardware and / or software on the device 104.

[0076] At 710, CPU 702 may obtain (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with video content. As used herein, “frame rate and resolution characteristics” may refer to information about the device’s frame rate and screen resolution when the device plays back video content. As used herein, “video content” may refer to a sequence of sequential images displayed on a display. As used herein, “workload characteristics” may refer to the utilization of one or more threads associated with presenting video content on a display. At 712, CPU 702 may map the set of frame rate and resolution characteristics and the set of workload characteristics to the device’s power profile. At 714, CPU 702 may output an instruction to the device’s power profile based on the mapping. For example, at 716, CPU 702 may send an instruction to the power control component 704 regarding the device’s power profile. At 718A, power control component 704 may adjust the device’s power level based on the instruction. In another example, at 718B, CPU 702 can adjust the device's power level based on the device's power profile.

[0077] At 720, CPU 702 may receive a first instruction regarding at least one of a change in the resolution of the video content, a change in the frame rate of the video content, or frame dropping of the video content. As used herein, "change in resolution" may refer to an increase or decrease in the display resolution of the device. As used herein, "change in frame rate" may refer to an increase or decrease in the frame rate of the device. As used herein, "frame dropping" may refer to a frame in a frame sequence that is not displayed due to one or more reasons (e.g., heavy computational workload of the device, network problems, etc.). At 722, CPU 702 may modify the power profile of the device based on the first instruction. At 724, CPU 702 may output a second instruction regarding the modified power profile. For example, at 726, CPU 702 may send the second instruction to power control component 704. At 728, power control component 704 may adjust the power level of the device based on the second instruction.

[0078] In the example, the video content may include streaming video content, and at 706, CPU 702 may receive the streaming video content from the server. As used herein, "streaming video content" may refer to video content displayed by the device via continuous transmission of audio and / or video from the server to the device. At 708, CPU 702 may render the streaming video content on the device's display.

[0079] Figure 8 This is a flowchart 800 illustrating an example method of processing according to one or more techniques of this disclosure. The method can be performed by, for example, combining... Figures 1 to 7The method is executed by various devices used in the process, such as devices for display processing, GPUs, CPUs (e.g., CPU 702), display processors, device 104, wireless communication devices, etc. In the example, the method may be executed by a configuration file switcher 198.

[0080] At 802, the device (e.g., CPU) obtains (1) a set of frame rate and resolution characteristics associated with the video content and (2) a set of workload characteristics associated with the video content. For example, Figure 7 At 710, it is shown that CPU 702 can obtain (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with video content. In the example, the video content may be or include video content 404. In the example, the set of frame rate and resolution characteristics may be or include clip resolution 512, cue FPS 508, and / or target FPS 510. In the example, the set of workload characteristics may be or include task load characteristics 516. In the example, 802 may be performed by profile switcher 198.

[0081] At 804, the device (e.g., the CPU) maps a set of frame rate and resolution characteristics, along with a set of workload characteristics, to the device's power profile. For example, Figure 7 At 712, it is shown that CPU 702 can map a set of frame rate and resolution characteristics, as well as a set of workload characteristics, to a device's power profile. In the example, the device's power profile may be or include a first power level profile 610 or a second power level profile 612. In the example, the device's power profile may be or include a first performance level profile 606 or a second performance level profile 608. In the example, the device's power profile may be profile 522. In the example, the mapping may correspond to... Figure 5 In the example, 804 can be executed by the configuration file switcher 198.

[0082] At 806, the device (e.g., the CPU) outputs an indication of the device's power profile based on the mapping. For example, Figure 7 At 714, it is shown that CPU 702 can output (e.g., to power control component 704) an indication of the device's power profile based on a mapping. In the example, 806 can be performed by profile switcher 198.

[0083] Figure 9 This is a flowchart 900 illustrating an example method of processing according to one or more techniques disclosed herein. The method can be performed by, for example, combining... Figures 1 to 7The method is executed by various devices used in the process (such as devices for display processing, GPUs, CPUs, display processors, device 104, wireless communication devices, etc.). In the example, the method (including the various aspects detailed below) may be executed by a profile switcher 198.

[0084] At 906, the device (e.g., the CPU) obtains (1) a set of frame rate and resolution characteristics associated with the video content and (2) a set of workload characteristics associated with the video content. For example, Figure 7 At 710, it is shown that CPU 702 can obtain (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with video content. In the example, the video content may be or include video content 404. In the example, the set of frame rate and resolution characteristics may be or include clip resolution 512, cue FPS 508, and / or target FPS 510. In the example, the set of workload characteristics may be or include task load characteristics 516. In the example, 906 may be performed by profile switcher 198.

[0085] At 908, the device (e.g., the CPU) maps a set of frame rate and resolution characteristics, along with a set of workload characteristics, to the device's power profile. For example, Figure 7 At 712, it is shown that CPU 702 can map a set of frame rate and resolution characteristics, as well as a set of workload characteristics, to a device's power profile. In the example, the device's power profile may be or include a first power level profile 610 or a second power level profile 612. In the example, the device's power profile may be or include a first performance level profile 606 or a second performance level profile 608. In the example, the device's power profile may be profile 522. In the example, the mapping may correspond to... Figure 5 520 in the example. In the example, 908 can be executed by the configuration file switcher 198.

[0086] At 910, the device (e.g., the CPU) outputs an indication of the device's power profile based on the mapping. For example, Figure 7 At 714, it is shown that the CPU 702 can output (e.g., to the power control component 704) an indication of the device's power profile based on a mapping. In the example, 910 can be performed by the profile switcher 198.

[0087] In one aspect, outputting an indication of the device's power profile may include outputting a first indication for adjusting the device's power level. For example, outputting an indication of the device's power profile at 714 may include outputting a first indication for adjusting the device's power level. In the example, power control component 704 may adjust the device's power level based on the first indication.

[0088] In one aspect, at 912, the device (e.g., CPU) can adjust the device's power level based on the device's power profile. For example, Figure 7 At 718B, it is shown that CPU 702 can adjust the power level of the device based on the device's power profile. In the example, 912 can be performed by profile switcher 198.

[0089] In one aspect, at 914, the device (e.g., the CPU) may receive a first indication of at least one of a change in the resolution of the video content, a change in the frame rate of the video content, or frame dropping of the video content. For example, Figure 7 At 720, it is shown that CPU 702 can receive a first indication of at least one of a change in the resolution of the video content, a change in the frame rate of the video content, or frame dropping of the video content. In the example, the foregoing aspect may correspond to switching event 406. In the example, 914 may be executed by profile switcher 198.

[0090] In one aspect, at 916, the device (e.g., the CPU) can modify the device's power profile based on a first instruction. For example, Figure 7 At 722, it is shown that CPU 702 can modify the power profile of the device based on a first instruction. In the example, the aforementioned aspect may correspond to power step 532. In the example, 916 may be performed by profile switcher 198.

[0091] In one aspect, at 918, the device (e.g., the CPU) can output a second indication of the modified power profile. For example, Figure 7 At 724, it is shown that CPU 702 can output a second instruction for the modified power profile. In the example, 918 can be performed by profile switcher 198.

[0092] In one aspect, obtaining the first indication may include obtaining a third indication of frame dropping from a frame rate monitor, and wherein modifying the power profile may include applying a power step to the power profile based on the first indication. As used herein, a "frame rate monitor" may refer to software and / or hardware configured to measure the frame rate of displayed video content and detect frame dropping. In the example, frame dropping may be video frame dropping 528, and the power step may be a power step 532.

[0093] In one aspect, the set of frame rate and resolution characteristics may include at least one of a first frame rate from a frame compositor associated with the device, a second frame rate from a video frame associated with the device, or the resolution of the video content. For example, the first frame rate may be a cue FPS of 508, the second frame rate may be a target FPS of 510, and the resolution of the video content may be a clip resolution of 512.

[0094] In one aspect, the set of workload characteristics may include indications of tasks performed by threads of the device. For example, indications of tasks performed by threads of the device may include task workload 516.

[0095] In one aspect, a power profile may be associated with a device’s first power consumption and a device’s first graphics performance. For example, a power profile may be a first power level profile 610. The first power level profile 610 may be associated with a device’s first power consumption and a device’s first graphics performance. As used herein, “power consumption” may refer to the amount of power consumed by the device. As used herein, “graphics performance” may refer to the characteristics of the displayed video content (e.g., frame rate, resolution, other graphics characteristics, etc.).

[0096] In one aspect, a power profile may include a set of power profiles, and mapping a set of frame rate and resolution characteristics as well as a set of workload characteristics to a power profile may include mapping a first power profile in the set of power profiles to a second performance profile in the set of power profiles. For example, the first power profile may be a first power level profile 610, and the second power profile may be a first performance level profile 606.

[0097] In one aspect, the set of frame rate and resolution characteristics obtained may include a set of frame rate and resolution characteristics determined based on video content, wherein the set of workload characteristics obtained may include a set of workload characteristics determined based on video content. For example, the set of frame rate and resolution characteristics obtained at 710 may include a set of frame rate and resolution characteristics determined based on video content, and the set of workload characteristics obtained at 710 may include a set of workload characteristics determined based on video content. In the example, the foregoing aspects may correspond to 506 and 514.

[0098] In one aspect, determining the set of frame rate and resolution characteristics may include determining the refresh rate of the device's display and the device's video decoding rate. For example, obtaining the set of frame rate and resolution characteristics may include determining the refresh rate of the device's display and the device's video decoding rate. As used herein, "refresh rate" may refer to the number of times the display refreshes per second. As used herein, "video decoding rate" may refer to the amount of information decoded per second into the portion of video content to be displayed.

[0099] In one aspect, determining the set of workload characteristics may include determining a workload threshold, and wherein mapping the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile may include mapping the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile based on the workload threshold. For example, the workload threshold may be a workload threshold 518. In the example, mapping the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile at 712 may include mapping the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile based on the workload threshold 518.

[0100] In one aspect, the video content may have enabled bullet screen features, and mapping a set of frame rate and resolution characteristics as well as a set of workload characteristics to a power profile may include mapping the set of frame rate and resolution characteristics as well as the set of workload characteristics to a power profile based on the bullet screen features being enabled. For example, the foregoing aspect may correspond to bullet screen 408. In the example, mapping the set of frame rate and resolution characteristics as well as the set of workload characteristics to a power profile at 712 may include mapping the set of frame rate and resolution characteristics as well as the set of workload characteristics to a power profile based on the bullet screen features being enabled.

[0101] In one aspect, the video content may include streaming video content, and at 902, a device (e.g., a CPU) may receive the streaming video content from a server. For example, the video content may be video content 404, and Figure 7 At 706, it is shown that CPU 702 can receive streaming video content from the server. In the example, 902 can be performed by profile switcher 198.

[0102] In one aspect, at position 904, the device can display streaming video content on the device's display. For example, Figure 7 At 708, it is shown that CPU 702 can present streaming video content on the device's display (e.g., display 131). In the example, 904 can be performed by profile switcher 198.

[0103] In one aspect, outputting an indication of the device's power profile may include: sending an indication of the device's power profile; or storing an indication of the device's power profile. For example, outputting an indication of the device's power profile at 716 may include: sending (e.g., to power control component 704) an indication of the device's power profile; or storing an indication of the device's power profile.

[0104] In each configuration, a method or apparatus for display processing is provided. The apparatus may be a GPU, a CPU, or some other processor capable of performing graphics processing. In various aspects, the apparatus may be a processing unit 120 within device 104, or some other hardware within device 104 or another device. The apparatus may be a DPU, a display processor, or some other processor capable of performing display processing. In various aspects, the apparatus may be a display processor 127 within device 104, or some other hardware within device 104 or another device. The apparatus may include components for obtaining (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with video content. The apparatus may also include components for mapping the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile of the device. The apparatus may also include components for outputting an indication of the power profile of the device based on the mapping. The apparatus may also include components for adjusting the power level of the device based on the power profile of the device. The apparatus may also include components for obtaining a first indication of at least one of a resolution change of video content, a frame rate change of video content, or frame dropping of video content. The apparatus may further include components for modifying the device's power profile based on a first instruction. The apparatus may also include components for outputting a second instruction to the modified power profile. The apparatus may further include components for receiving streaming video content from a server. The apparatus may further include components for displaying the streaming video content on the device's display.

[0105] It should be understood that the specific order or hierarchy of boxes / steps in the processes, flowcharts, and / or call flowcharts disclosed herein are merely illustrative of example methods. It should be understood that the specific order or hierarchy of boxes / steps in these processes, flowcharts, and / or call flowcharts may be rearranged based on design preferences. Furthermore, some boxes / steps may be combined or omitted. Other boxes / steps may also be added. The appended method claims provide the elements of various boxes / steps in an exemplary order, but are not intended to limit one to the given specific order or hierarchy.

[0106] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims, wherein, unless specifically stated otherwise, references to elements in the singular are not intended to mean “one and only one,” but rather “one or more.” The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0107] Unless otherwise specified, the term "some" refers to one or more, and the term "or" may be interpreted as "and / or" unless otherwise specified in the context. Combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, which may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, wherein any such combination may contain one or more members of A, B, or C. All structural and functional equivalents of the elements throughout the various aspects described herein that are known to or will later be known to a person skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly recited in the claims. The words “module,” “mechanism,” “element,” “device,” etc., cannot replace the word “component.” Therefore, no claim element will be construed as a functional component unless the element is explicitly recited using the phrase “component for…”. Unless otherwise stated, the phrase “processor” may refer to “any processor in one or more processors” (e.g., one processor in one or more processors, a plurality (more than one) of one or more processors, or all processors in one or more processors), and the phrase “memory” may refer to “any memory in one or more memories” (e.g., one memory in one or more memories, a plurality (more than one) of one or more memories, or all memories in one or more memories).

[0108] In one or more examples, the functionality described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any functionality, processing unit, technique, or other module described herein is implemented in software, then such functionality, processing unit, technique, or other module may be stored on or transmitted on a computer-readable medium as one or more instructions or code.

[0109] Computer-readable media may include computer data storage media and communication media, including any media that facilitates the transfer of computer programs from one place to another. In this way, computer-readable media may generally correspond to: (1) a non-transitory tangible computer-readable storage medium; or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to extract instructions, code, and / or data structures for implementing the techniques described in this disclosure. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, compressed optical disc read-only memory (CD-ROM) or other optical disc storage devices, magnetic disk storage devices, or other magnetic storage devices. As used herein, magnetic disks and optical discs include compressed optical discs (CD), laser optical discs, optical discs, digital versatile optical discs (DVD), floppy disks, and Blu-ray discs, wherein magnetic disks typically magnetically copy data, while optical discs optically copy data using lasers. Combinations of the above should also be included within the scope of computer-readable media. Computer program products may include computer-readable media.

[0110] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. Rather, as described above, various units can be combined in any hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with suitable software and / or firmware. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0111] The following aspects are merely illustrative and may be combined with other aspects or teachings described herein without limitation.

[0112] Aspect 1 is a display processing method, the method comprising: obtaining (1) a set of frame rate and resolution characteristics associated with video content and (2) a set of workload characteristics associated with the video content; mapping the set of frame rate and resolution characteristics and the set of workload characteristics to a power profile of a device; and outputting an indication of the power profile of the device based on the mapping.

[0113] Aspect 2 can be combined with aspect 1, wherein the output of the indication to the power profile of the device includes the output of a first indication for adjusting the power level of the device.

[0114] Aspect 3 may be combined with any of aspects 1 to 2, and further includes: adjusting the power level of the device based on the power profile of the device.

[0115] Aspect 4 may be combined with any of aspects 1 to 3, and further includes: obtaining a first indication of at least one of a resolution change of the video content, a frame rate change of the video content, or frame dropping of the video content; modifying the power profile of the device based on the first indication; and outputting a second indication of the modified power profile.

[0116] Aspect 5 may be combined with aspect 4, wherein obtaining the first indication includes obtaining a third indication of frame dropping from a frame rate monitor, and wherein modifying the power profile includes applying a power ladder to the power profile based on the first indication.

[0117] Aspect 6 may be combined with any of aspects 1 to 5, wherein the set of frame rate and resolution characteristics includes at least one of a first frame rate from a frame synthesizer associated with the device, a second frame rate from a video frame associated with the device, or the resolution of the video content.

[0118] Aspect 7 may be combined with any of aspects 1 to 6, wherein the set of workload characteristics includes an indication of a task performed by a thread of the device.

[0119] Aspect 8 may be combined with any of aspects 1 to 7, wherein the power profile is associated with a first power consumption of the device and a first graphics performance of the device.

[0120] Aspect 9 may be combined with any of aspects 1 to 8, wherein the power profile includes a set of power profiles, and wherein mapping the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile includes mapping the set of frame rate and resolution characteristics and the set of workload characteristics to a first power profile in the set of power profiles or a second power profile in the set of power profiles.

[0121] Aspect 10 may be combined with any of aspects 1 to 9, wherein the set of obtaining the frame rate and resolution characteristics includes a set of determining the frame rate and resolution characteristics based on the video content, and wherein the set of obtaining the workload characteristics includes a set of determining the workload characteristics based on the video content.

[0122] Aspect 11 may be combined with aspect 10, wherein determining the set of frame rate and resolution characteristics includes determining the refresh rate of the device's display and the video decoding rate of the device.

[0123] Aspect 12 may be combined with any of aspects 10 to 11, wherein determining the set of workload characteristics includes determining a workload threshold, and wherein mapping the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile includes mapping the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile based on the workload threshold.

[0124] Aspect 13 may be combined with any of aspects 1 to 12, wherein the video content has enabled bullet screen features, and wherein mapping the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile includes mapping the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile based on the bullet screen features being enabled.

[0125] Aspect 14 may be combined with any of aspects 1 to 13, wherein the video content includes streaming video content, and the method further includes: receiving the streaming video content from a server; and displaying the streaming video content on a display of the device.

[0126] Aspect 15 may be combined with any of aspects 1 to 14, wherein outputting the indication of the power profile of the device includes sending the indication of the power profile of the device; or storing the indication of the power profile of the device.

[0127] Aspect 16 is an apparatus for display processing, the apparatus including a processor coupled to a memory, and configured, based on information stored in the memory, to implement the method according to any one of aspects 1 to 15.

[0128] Aspect 17 may be combined with aspect 16 and includes: the device is a wireless communication device, the wireless communication device including at least one of a transceiver or an antenna coupled to the processor, wherein in order to receive the streaming video content, the processor is configured to receive the streaming video content via at least one of the transceiver or the antenna.

[0129] Aspect 18 is an apparatus for display processing, the apparatus including components for implementing the method according to any one of aspects 1 to 15.

[0130] Aspect 19 is a computer-readable medium (e.g., a non-transitory computer-readable medium) storing computer-executable code that, when executed by a processor, causes the processor to implement the method according to any one of aspects 1 to 15.

[0131] Various aspects have been described herein. These and other aspects are within the scope of the following claims.

Claims

1. An apparatus for display processing, the apparatus comprising: Memory; and A processor, coupled to the memory, and configured based on information stored in the memory, to: Obtain (1) a set of frame rate and resolution characteristics associated with the video content and (2) a set of workload characteristics associated with the video content; Map the set of frame rate and resolution characteristics and the set of workload characteristics to at least one power profile in the device's power profile; as well as The power profile of the device is output based on the mapping.

2. The apparatus of claim 1, wherein, in order to output the indication of the power profile of the device, the processor is configured to output a first indication for adjusting the power level of the device.

3. The apparatus of claim 1, wherein the processor is further configured to: The power level of the device is adjusted based on the power profile of the device.

4. The apparatus of claim 1, wherein the processor is further configured to: Obtain a first indication of at least one of a resolution change of the video content, a frame rate change of the video content, or frame dropping of the video content; Modify the power profile of the device based on the first instruction; as well as Output a second instruction for the modified power profile.

5. The apparatus of claim 4, wherein in order to obtain the first indication, the processor is configured to obtain a third indication of frame dropping from a frame rate monitor, and wherein in order to modify the power profile, the processor is configured to apply a power step to the power profile based on the first indication.

6. The apparatus of claim 1, wherein the set of frame rate and resolution characteristics includes at least one of a first frame rate from a frame synthesizer associated with the device, a second frame rate from a video frame associated with the device, or the resolution of the video content.

7. The apparatus of claim 1, wherein the set of workload characteristics includes indications of tasks performed by threads of the apparatus.

8. The apparatus of claim 1, wherein the power profile is associated with a first power consumption of the device and a first graphics performance of the device.

9. The apparatus of claim 1, wherein the power profile comprises a set of power profiles, and wherein, in order to map the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile, the processor is configured to map the set of frame rate and resolution characteristics and the set of workload characteristics to a first power profile in the set of power profiles or a second power profile in the set of power profiles.

10. The apparatus of claim 1, wherein, in order to obtain the set of frame rate and resolution characteristics, the processor is configured to determine the set of frame rate and resolution characteristics based on the video content, and wherein, in order to obtain the set of workload characteristics, the processor is configured to determine the set of workload characteristics based on the video content.

11. The apparatus of claim 10, wherein, in order to determine the set of frame rate and resolution characteristics, the processor is configured to determine the refresh rate of the display of the device and the video decoding rate of the device.

12. The apparatus of claim 10, wherein in order to determine the set of workload characteristics, the processor is configured to determine a workload threshold, and wherein in order to map the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile, the processor is configured to map the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile based on the workload threshold.

13. The apparatus of claim 1, wherein the video content has enabled bullet screen features, and wherein, in order to map the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile, the processor is configured to map the set of frame rate and resolution characteristics and the set of workload characteristics to the power profile based on the bullet screen features being enabled.

14. The apparatus of claim 1, wherein the video content comprises streaming video content, and the processor is further configured to: Receive the streaming video content from the server; and The streaming video content is displayed on the device's screen.

15. The apparatus of claim 14, wherein the apparatus is a wireless communication device, the wireless communication device including at least one of a transceiver or an antenna coupled to the processor, and wherein, in order to receive the streaming video content, the processor is configured to receive the streaming video content via at least one of the transceiver or the antenna.

16. The apparatus of claim 1, wherein, in order to output the instruction on the power profile of the apparatus, the processor is configured to: Send the instruction for the power profile of the device; or The instruction for the power profile of the device is stored.

17. A method for display processing, the method comprising: Obtain (1) a set of frame rate and resolution characteristics associated with the video content and (2) a set of workload characteristics associated with the video content; Map the set of frame rate and resolution characteristics and the set of workload characteristics to the device's power profile; as well as The power profile of the device is output based on the mapping.

18. The method of claim 17, wherein outputting the indication of the power profile of the device includes outputting a first indication for adjusting the power level of the device.

19. The method of claim 17, further comprising: The power level of the device is adjusted based on the power profile of the device.

20. A computer-readable medium storing computer-executable code, which, when executed by a processor, causes the processor to: Obtain (1) a set of frame rate and resolution characteristics associated with the video content and (2) a set of workload characteristics associated with the video content; Map the set of frame rate and resolution characteristics, as well as the set of workload characteristics, to the device's power profile; and The power profile of the device is output based on the mapping.

Images