Performance power management by proactive frame rate monitoring

Abstract

This disclosure describes systems and techniques for performative power management by proactive frame rate monitoring. In aspects, a frame stream is rendered, the frame stream including a plurality of frames. A frame timeline guardian receives frame timing data for the frame stream prior to the rendering of the frame stream. The frame timeline guardian compares the frame timing information with a threshold value and, based on the comparison, determines to adjust a resource of an electronic device rendering the frame stream. The frame timeline guardian generates an output configured to adjust the resource of the electronic device. The adjustment adjusts a parameter of the rendering such that it is closer to a target parameter value.

Description

PERFORMATIVE POWER MANAGEMENT BY PROACTIVE FRAME RATE MONITORINGBACKGROUND

[0001] Video rendering on electronic devices often suffers from performance issues, such as device jiter, dropped frames, input lag. and inefficient power usage. These undesirable effects are often the result of a video being rendered to the device with a frame rate or frame time that is sub-optimal for the application. For example, when a video is rendered to the device from a video game, a low frames-per-second (FPS) may result in a lag between a user input and a display element, which may lead to a frustrating user experience. In another example, a streaming video may overuse device hardware, such as frames being rendered for too short a time resulting in excess draining of a batery of the device.

[0002] Hardware monitoring is an existing solution to sub-optimal frame rendering, but this is a reactive strategy. For example, if a game is having its video rendered at too slow a rate, a hardware monitor may assess this and increase a processor frequency of the device. However, the game may already have suffered from significant lag, the post-render corrections may not be suitable for the rendering of the frames that are pre-rendered, and the hardware monitoring may cause undesirable overhead, resulting in additional adverse performance characteristics.SUMMARY

[0003] This disclosure describes systems and techniques for performative power management by proactive frame rate monitoring. In aspects, a plurality of frames are rendered. A frame timeline guardian receives frame timing data for the frame stream prior to the rendering of the frame stream. The frame timeline guardian compares the frame timing information with a threshold value and, based on the comparison, determines to adjust a resource of an electronic device rendering the frame stream. The frame timeline guardian generates an output configured to adjust the resource of the electronic device. The adjustment adjusts a parameter of the rendering such that it is closer to a target parameter value.

[0004] In some aspects, a method is described for adjusting display resources, the method including receiving, by one or more processors, frame timing data, the frame timing data based on one or more pre-rendered frames configured to be rendered to a display. The method further includes comparing, by the one or more processors, the frame timing data to a pre-render threshold value. In aspects, the method includes determining, by the one or more processors, to adjust one or more resources of an electronic device, the determination based on the comparison of the frame timing data to the pre-render threshold value. The method further includes generating, by the one or more processors, an adjustment output, where the adjustment output is configured to be outputto the electronic device and includes one or more adjustment instructions configured to adjust one or more resources of the electronic device.

[0005] The method may further, for example, include receiving, by the one or more processors, post-render timing data, the post-render timing data based on one or more postrendered frames rendered to the display. The method may further include comparing, by the one or more processors, the post-render timing data to a post-render threshold value. In some examples, the method further includes determining, by the one or more processors, to adjust the one or more of the resources of the electronic device, the determination based on the comparison of the post-render timing data to the post-render threshold value. The method may further include generating, by the one or more processors, a tuning output, the tuning output configured to be output to the electronic device and including one or more tuning instructions configured to adjust the one or more resources of the electronic device.

[0006] This document also describes computer-readable media having instructions for performing the above-summarized method and other methods set forth herein, as well as systems, means, and program products for performing these methods.

[0007] This summary is provided to introduce simplified concepts for performative power management by proactive frame rate monitoring, which is further described below in the Detailed Description and Drawings. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The details of one or more aspects of performative power management by proactive frame rate monitoring are described in this document with reference to the following drawings. The same numbers are used throughout the drawings to reference like features and components:

[0009] FIG. 1 illustrates an example environment, in which techniques for performative power management by proactive frame rate monitoring may be implemented;

[0010] FIG. 2 illustrates an example device for implementing performative power management by proactive frame rate monitoring;

[0011] FIG. 3 illustrates an example of an algorithm for optimizing frame rendering without the use of performative power management by proactive frame rate monitoring;

[0012] FIG. 4 illustrates an example of an algorithm for optimizing frame rendering using performative power management by proactive frame rate monitoring;

[0013] FIG. 5 illustrates an example buffer queue as a source for pre-rendered frames and frame timing;

[0014] FIG. 6 illustrates an example performance increase from employing performative power management by proactive frame rate monitoring;

[0015] FIG. 7 illustrates another example performance increase from employing performative power management by proactive frame rate monitoring;

[0016] FIG. 8 illustrates an example resource optimization from employing performative power management by proactive frame rate monitoring;

[0017] FIG. 9 illustrates an example method for performative power management by proactive frame rate monitoring;

[0018] FIG. 10 illustrates another example method for performative power management by proactive frame rate monitoring.DETAILED DESCRIPTION

[0019] This disclosure describes techniques and systems for performative power management by proactive frame rate monitoring. Electronic devices may render a frame stream to a display to produce a video. The rate at which the frame stream is rendered, in some examples, may cause undesired effects. For instance, the frame stream rendered at a lower frame rate or frames-per-second (FPS) than a target value may result in lag, jittery' video, or other glitches undesirable to an end user. For example, a video rendered for a video game at 45 FPS may not be as responsive to user inputs as a designer of the video game intends or as the user expects should the video be rendered at 60 FPS.

[0020] In another example, the user accesses a video conference on a device. The video being rendered for the video conference may have a target frame-time of 15 ms, the frame-time being the average amount of time for each frame to be rendered to the display. In some examples, the device may allocate too many resources to rendering the video, such as processing cycles or power. For example, the device may increase a frequency of the processor to render the video and render each frame for an average of 10 milliseconds (ms) each. The manufacturer may have determined that a user, in this application, cannot tell a difference between the frames being rendered for an average time of 10 ms each and the frames being rendered for an average time of 15 ms each. In such an example, the increased frequency of the processor unnecessarily wastes resources as the faster frame time is of no discernable benefit to the user. Further, in some examples the extra processor use may result in ancillary unwanted behavior, such as increased battery usage, laggy secondary applications, or excess heat in a mobile display device.

[0021] One technique that may be used in some examples to optimize rendering of the frame stream is to monitor the hardware while rendering. For instance, a hardware monitor may monitor the displayed FPS of a rendered frame stream, resource usage such as processor dutycycle, or other performance characteristics. One weakness with this reactive sty de of optimization is that it may be resource intensive, inefficient, and / or poorly compensate for unwanted effects of off-target FPS rendering or frame time rendering. This is an example of a reactive approach to FPS and frame time rendering characteristics.

[0022] The systems and techniques discussed herein for performative power management by proactive frame rate monitoring outline a proactive approach to FPS and frame time rendering characteristics. In aspects, a frame timeline guardian is disclosed. The frame timeline guardian collects statistics, information or frame timing data from pre-rendered frames from the frame stream and initiates hardware corrections prior to frame rendering. For example, the frame timeline guardian gathers information from a pre-rendering side of the device, such as the frame timing data. The frame timing data, in some examples, is based on one or more times that one or more frames are entered into a buffer queue. The buffer queue adds and removes the one or more frames to be rendered. For example, the frame timing data may be related to a first time that a first frame of the one or more frames is entered into the queue and a second time that a second frame of the one or more frames is entered into the queue.

[0023] According to some examples, the performative power management by proactive frame rate monitoring further includes the hardware monitor. Unlike the hardware monitor implemented without the frame timeline guardian, the hardware monitor collects hardware statistics, parameters, etc. related to the rendering of the one or more frames and passes postrender timing data to the frame timeline manager. The frame timeline manager incorporates the post-render timing data into the output generated to change a parameter of the device hardware. In some examples, this allows the hardware to more-closely maintain a frame rate. FPS and / or frame time in line with predetermined optimal settings and values.Example Environment

[0024] FIG. 1 illustrates an example environment 100, in which techniques for performative power management by proactive frame rate monitoring may be implemented. A user 102 may use a device 104, such as a mobile gaming device or a smartphone to play a video 106. In aspects, the video 106 consists of multiple still images, or frames, in sequence. In the example where the video 106 is from a video game, such as that pictured, the individual frames may be dynamically generated. In some examples, the frames are pre-stored in a memory of the device 104. In aspects, the frames are processed pre-render and are then rendered to a display of the device 104.

[0025] The device 104, according to some examples, has a target frame rate, or frames- per-second (FPS), for the type of video of the video 106 being rendered. For example, a videogame may have a high value for the target FPS, such as 120 FPS, to give the user 102 a playing experience with relatively few instances of lag. In other examples, the video 106 does not need to have a high FPS in order to give the user 102 a viewing experience meeting expected parameters. For example, the video 106 may be a video conference, where the frames being rendered may be displayed for longer than the 8.33 ms for a video game at 120 FPS, such as for 15 ms.Example Device

[0026] FIG. 2 illustrates an example device 200 for implementing performative power management by proactive frame rate monitoring. The device 200 may be, for example, a smart phone 200-1, a tablet device 200-2, a laptop computer 202-3, a desktop computer 202-4, a smartwatch 202-5, smart glasses 200-6, a gaming device 202-7, AR goggles or an AR headset 200-8, or any other device known to a person of ordinary’ skill in the art where such a device is capable of rendering video. The list of example devices given is meant to be illustrative and not limiting.

[0027] The device 200 includes one or more processors 202 and a memory’ 204. In aspects, the memory' 204 may be one or more computer-readable media. The memory 204 may include instructions 206. such as those for performative power management by proactive frame rate monitoring. The device 200 may also, in some examples, include a display 208. In some examples, the display 208 is not integral to the device 200 but is connected to the device 200, such as by a physical cable or by a wireless connection. The device 200 also includes, according to some examples, a battery 210. Other components common to electronic devices may also be included in the device 200, which are not pictured. The one or more processors 202 may include any7combination of one or more controllers, microcontrollers, processors, microprocessors, hardware processors, hardware processing units, digital-signal-processors, graphics processors, graphics processing units, and the like. The one or more processors 202 may be an integrated processor and memory subsystem (e.g., implemented as a "‘system-on-chip7’), which processes computer-executable instructions to control operations of the device 200.

[0028] The memory’ 204 may be, in aspects, configured as persistent and non-persistent storage of executable instructions (e g., firmware, recovery firmware, software, applications, modules, programs, functions, and the like) and data (e.g., user data, operational data) to support execution of the executable instructions. Examples of the memory 204 include volatile memory' and non-volatile memory, fixed and removable media devices, and any suitable memory device or electronic data storage that maintains executable instructions and supporting data. The memory’ 204 may include various implementations of random-access memory (RAM), read-only memory(ROM), flash memory, and other types of storage memory in various memory device configurations. The memory 204 may exclude propagating signals. The memory 204 may be a solid-state drive (SSD) or a hard disk drive (HDD).

[0029] The device 200 also includes a frame timeline guardian 212. The frame timeline guardian 212 is used to proactively monitor frame-rate and / or frame-time information for prerendered frames. The frame timeline guardian 212, in some examples, is stored on the memory 204 of the device 200. In some examples, the frame timeline guardian 212 is a circuit on the device 200. hi other examples, the frame timeline guardian 212 is a combination of circuitry on the device 200 and the instructions 206 in the memory 204.

[0030] The device 200 further includes a hardware statistics collector 214. The hardware statistics collector 214, in some examples, is stored on the memory7204 of the device 200. In some examples, the hardware statistics collector 214 is a circuit on the device 200. In other examples, the hardware statistics collector 214 is a combination of circuitry on the device 200 and instructions in the memory 204 of the device 200.Example Algorithms

[0031] FIG. 3 illustrates an example of an algorithm 300 for optimizing frame rendering without the use of performative power management by proactive frame rate monitoring. A frame source 302 is selected for rendering. For example, the frame source 302 may be streaming video data from a streaming service for rendering the streaming video data to a device (e.g., smartphone, smart TV) or storing images in memory. The frame source 302 sends a frame stream to a frame pre-renderer 304. The frame pre-renderer 304, in some examples, is a frame buffer used to queue pre-rendered frames. The frame stream is sent from the frame pre-renderer 304 to hardware 306 of the device. The hardware 306 may include one or more processors (e.g., the one or more processors 202), a memory7(e.g., the memory7204), a graphics card, one or more display drivers stored on the memory, or other components known in the art for rendering the frame stream to a display 308.

[0032] The hardware 306 renders the frame stream to a display 308 (e g., the display 208). The rendering of the frame stream, in aspects, displays each frame of the frame stream in sequence. The frame stream has an FPS measure and a frame time measure. The FPS represents the number of frames of the frame stream that are displayed on the display 308 per second. The FPS measure, in some examples, is an average. In some examples, the FPS measure is absolute, such as when measuring the exact number of frames displayed in a second. The frame time measure, in some examples, is an average time each frame in the frame stream is displayed on the display 308 overa predetermined period of time. In some examples, the frame time measure is a measure of the amount of time a single frame is displayed on the display 308.

[0033] In some examples, the FPS and / or the frame time measures do not meet expected parameters. For example, an expected FPS is 60 FPS but the frame stream is being rendered to the display 308 at a rate of 45 FPS. In such an example, unwanted effects such as frame lag, dropped frames, or video judder may occur. A hardware monitor 310 is used to monitor the hardware 306. The hardware monitor 310 monitors the hardware 306 after the frame stream has been sent by the frame pre-renderer 304. In this way, the hardware monitor 310 is a reactive form of monitoring.

[0034] FIG. 4 illustrates an example of an algorithm 400 for optimizing frame rendering using performative power management by proactive frame rate monitoring. A frame source 402 is selected for rendering. For example, the frame source 402 may stream video data from a streaming service for rendering the streaming video data to a device (e.g., smartphone, smart TV) or to store in memory. The frame source 402 sends a frame stream to a frame pre-renderer 404. The frame pre-renderer 404, in some examples, is a frame buffer used to queue pre-rendered frames. Frame timing data is sent from the frame pre-renderer 404 to a frame timeline guardian 406.

[0035] The frame timeline guardian 406 receives the frame timing data. In some examples, the frame timing data is based on a frame entry rate into a queue, such as when the frame pre-renderer 404 is the frame buffer. The frame timing data received by the frame timeline guardian 406 is based on pre-rendered information from the frame stream. The frame timeline guardian 406 compares the frame timing data to one or more threshold values. Consider, for example, the one or more threshold values being a threshold value of 60 FPS. The threshold value of 60 FPS may be a target value for hardware 408 or a display 410. Based on the frame timing data, the frame timeline guardian 406 may, in this example, determine that the output of the frame stream is 45 FPS. The frame timeline guardian may generate an output for the hardware 408 configured to adjust the hardware 408 to increase the frame stream render FPS. For example, the frame timeline guardian may send a command to increase the frequency of one or more processors (e.g., the one or more processors 202) of the hardware 408. The hardware 408 may include one or more processors (e.g., the one or more processors 202), a memory (e.g., the memory 204), a graphics card, one or more display drivers stored on the memory, or other components known in the art for displaying data.

[0036] The frame pre-renderer 404 sends the frame stream to the hardware 408. The hardware 408 renders the frame stream to a display 410 (e.g., the display 208). The rendering of the frame stream, in aspects, displays each frame of the frame stream in sequence. The framestream has an FPS measure and a frame time measure. The FPS represents the number of frames of the frame stream that are displayed on the display 410 per second. The FPS measure, in some examples, is an average. In some examples, the FPS measure is absolute, such as when measuring the exact number of frames displayed in a second. The frame time measure, in some examples, is an average time each frame in the frame stream is displayed on the display 410 over a predetermined period of time. In some examples, the frame time measure is a measure of the amount of time a single frame is displayed on the display 410.

[0037] The frame timeline guardian 406 analyzes the frame timing data. In some examples, the frame timeline guardian 406 compares the frame timing data to the one or more threshold values, such as a target FPS value. Based on the comparison, the frame timeline guardian determines to adjust one or more parameters and / or resources of the hardware 408. For example, the frame timeline guardian 406 may determine that the FPS for rendering the frame stream is below a target value, such as by comparing a predicted FPS of the frame stream based on the frame timing data to the target FPS value. Consider, for example, where the predicted FPS for the frame stream is 45 FPS but the target FPS is 60 FPS. The frame timeline guardian 406 may, in such examples, generate an output intended to increase a clock rate of a processor of the hardware 408, thus increasing the FPS of the rendered frame stream. Such an adjustment may come before the hardware 408 renders the frame steam to the display 410.

[0038] The algorithm 400 may further include a hardware monitor 412. The hardware monitor 412 may, in aspects, allow for fine-tuning of the rendering beyond what has been provided by the frame timeline guardian 406 based on the frame timing data. The frame timeline guardian 406 may. in some examples, receive post-render data from the hardware monitor 412. The frame timeline guardian 406, in some examples, generates a tuning output based on the post-render data. The tuning output, for example, adjusts one or more parameters or resources of the hardware 408. For example, the post-render data may indicate that the FPS of the rendered frame stream is not close enough to the target FPS. In response, in some examples, the frame timeline guardian may respond by generating a command for a multi-core processor to enable an additional core for rendering the frame stream, thus increasing the FPS.

[0039] FIG. 5 illustrates an example buffer queue 500 as a source for pre-rendered frames and frame timing. In some examples, the buffer queue 500 governs the output of a frame stream, the frame stream including a plurality of frames for rendering. The example buffer queue 500 includes two frames being buffered, but in concept this is not limiting. Any number of frames from the frame stream may, in principle, be included in the example buffer queue 500. A frame 1 buffer 502 begins with a first draw start 504. The first draw start 504 initiates a request to dequeue the frame 1 buffer 502. A first dequeue start 506, in some examples, begins the processof making the frame 1 buffer 502 available to queue a first frame of the plurality of frames. A first dequeue end 508 is where the queue of the frame 1 buffer 502 has been cleared and is ready to receive the first frame. Queueing the first frame begins with a first enqueue start 510 and proceeds to a first enqueue end 512. In some examples, the first enqueue end 512 is a completion of the buffering for the first frame and indicates the first frame is ready for rendering to a display (e.g., the display 410).

[0040] Similarly, a second frame of the plurality of frames may be buffered in a frame 2 buffer 514, including a second draw start 516, a second dequeue start 518, a second dequeue end 520, a second enqueue start 522, and a second enqueue end 524. In some examples, a frame timeline guardian (e.g., the frame timeline guardian 406) uses the order of events (504 to 524) as frame timing data. For example, the frame timeline guardian may receive timestamps corresponding with the first enqueue start 510 and the second enqueue start 522. A difference between the timestamp for the first enqueue start 510 and the timestamp for the second enqueue start 522, in some examples, is used as an FPS value or frame time value data.

[0041] In some examples, other points of reference in the buffer queue 500 may be used, such as comparing the first dequeue end 508 and the second dequeue end 520. In examples where the frame timing data is based on timing information from the buffer queue 500, the frame timing information is based on pre-rendered frames. This allows the frame timeline guardian to proactively adjust a hardware of a device used for rendering, which includes adjustments to the hardware to meet one or more parameters related to the rendering of the frame stream.

[0042] FIG. 6 illustrates an example performance increase 600 from employing performative power management by proactive frame rate monitoring. A first plot 602-1 shows time vs. FPS for a frame stream rendered without the use of a frame timeline guardian (e.g., the frame timeline guardian 406). A first curve 604-1 shows the FPS plotted over the time. The first curve 604-1 has a first average (J.) of 42.7 FPS and a first standard deviation (cr) of 1.3 FPS.

[0043] A second plot 602-2 shows time vs. FPS for the frame stream rendered with the frame timeline guardian. A second curve 604-2 shows the FPS plotted over the time. The second curve 604-2 has a second [i of 58.8 FPS and a second cr of 1.5 FPS. The difference between the first / r and the second fi is 16.1 FPS, representing a significant improvement in render performance. In some examples, a developer for an application sets an ideal frame rendering FPS for the application at 60 FPS. The first curve 604-1 shows a suboptimal performance for the ideal frame rendering FPS. The second curve 604-2 shows an acceptable performance for the ideal frame rendering FPS. In this example, the first a and the second <7 have a difference of 0.2 FPS, which is on the order of 10%. In some examples, a difference of 10% between o values is characterized as a small difference, representing comparable stability between methods.

[0044] FIG. 7 illustrates another example performance increase 700 from employing performative power management by proactive frame rate monitoring. A first plot 702-1 shows time vs. FPS for a frame stream rendered with the use of a frame timeline guardian (e.g., the frame timeline guardian 406) but with no hardware monitor (e.g., the hardware monitor 412). A first curve 704-1 shows the FPS plotted over the time (in this example, the first curve 704-1 is the same as the second curve 604-2 of FIG. 6). The first curve 704-1 has a first z of 58.8 FPS and a first cr of 1.5 FPS.

[0045] A second plot 702-2 shows time vs. FPS for the frame stream rendered with the frame timeline guardian and the hardware monitor. A second curve 704-2 shows the FPS plotted over the time. The second curve 704-2 has a second / z of 59.9 FPS and a second cr of 0.4 FPS. The difference between the first / z and the second / z is 1.1 FPS, which, in an example where a target FPS is 60 FPS. represents a positive but small difference (on the order of 2%). The difference between the first cr and the second cr is also 1.1 FPS, which represents a much more significant difference as the first r is 375% of the second cr. In some examples, this represents an increase in stability by using the hardware monitor.

[0046] FIG. 8 illustrates an example resource optimization 800 from employing performative power management by proactive frame rate monitoring. Overuse of resources for rendering, in some examples, results in unwanted effects such as excess battery usage and / or excess heat production. A first plot 802-1 shows time vs. frame time for a frame stream rendered without using a frame timeline guardian (e.g., the frame timeline guardian 406). A first ideal line 804-1 is a plot of an acceptable frame time, and a first curve 806-1 shows the frame time for the rendering of the frames over time. For example, a developer for an application may determine that there is no meaningful difference to an end user when frames are rendered at a frame time less than the acceptable frame time, shown by the first ideal line 804-1 at 17 ms. Any rendering for less frame time may, in this example, use greater resources for no discernable return. A first resource usage gap 808-1 shows a gap between the first ideal line 804-1 and the first curve 806- 1.

[0047] A second plot 802-2 shows time vs. frame time for the frame stream rendered with the frame timeline guardian (e.g., the frame timeline guardian 406). A second ideal line 804-2 is a plot of the acceptable frame time (in this example. 17 ms), and a second curve 806-2 shows the frame time for the rendering of the frames over time. In the example where the developer for the application determines that there is no meaningful difference to an end user when frames are rendered at the frame time less than the acceptable frame time, a second resource usage gap 808- 2 shows a gap between the second ideal line 804-2 and the second curve 806-2. In this example, a difference betw een a first / z of the first resource usage gap 808-1 and a second / z of the secondresource gap 808-2 is on the order of 50%, which, in some examples, reduces a processor power consumption by 23%.Example Methods

[0048] Example methods 900 and 1000 are described with reference to FIGs. 9 and 10, respectively, in accordance with one or more aspects for performative power management by proactive frame rate monitoring. The order in which the method operations are described is not intended to be construed as a limitation, and any number of the described method operations may be combined in any order to implement a method or an alternate method. Generally, any of the components, modules, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry). manual processing, or any combination thereof. Some operations of the example methods may be described in the general context of executable instructions stored on a computer-readable storage memory that is local and / or remote to a computer processing system, and implementations may include software applications, programs, functions, and the like. Alternatively or in addition, any of the functionalities described herein may be performed, at least in part, by one or more hardware logic components, such as, and without limitation, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SoCs), complex programmable logic devices (CPLDs), and the like.

[0049] FIG. 9 illustrates an example method 900 for performative power management by proactive frame rate monitoring. Operation 902 is receiving frame timing data. The frame timing data, in some examples, is based on one or more pre-rendered frames configured to be rendered to a display. In some examples, the frame timing data is based on infonnation from a framerendering queue, the frame-rendering queue including a buffer for the one or more pre-rendered frames. For example, the buffer may include information on frame queueing, such as a first time for a queueing of a first frame and a second time for a queueing of a second frame, the first frame and the second frame being in sequence. In some examples, the buffer stores pointers to the one or more frames and not the one or more frames themselves.

[0050] Operation 904 is comparing the frame timing data to a pre-render threshold value. In some examples, the pre-render threshold value is a frame-rate value, such as a FPS value. For example, the FPS value may be 30 FPS, 60 FPS. 120 FPS, 240 FPS, or another FPS value. In examples where the pre-render threshold value is the FPS value, the frame timing data may also be based on FPS. For example, the frame timing data may be the FPS at which the buffer is queueing frames. For example, the first frame may be queued by the buffer at the first time andthe second frame may be queued by the buffer at the second time, the second time occurring 10 milliseconds (ms) after the first time. In this example, the FPS for the frame timing data is:

[0051] In Eq. 1, (p is the value of a difference between the first time and the second time and ip is the corresponding FPS. In the example where (p is 10 ms, ip is 100 FPS. The pre-render threshold value as the FPS value may be represented by 6. For example, an electronic device implementing the method 900 may have a top rendering frame-rate of 120 FPS. The manufacturer may set 6 as:6 = 120 Eq. 2

[0052] In some examples, the comparison may be represented as a value K. Consider, for example, the comparison being an absolute difference as:19 — i \ = K Eq. 3

[0053] K in Eq. 3 is the absolute difference. Other comparison schemes may be employed. In some examples, K is a ratio as: = K Eq. 4

[0054] In some examples, the pre-render threshold value is a frame time value. The frame time value represents the amount of time a single frame is displayed. The pre-render threshold value as the frame time value may be represented by 8. For example, 8 may be 30 ms, 15 ms, 8 ms, 5ms, or another time value. In examples where the pre-render threshold value is 8. the frame timing data may also be based on frame time. For example, the frame timing data may be the time delta at which the buffer is queueing frames. For example, the first frame may be queued by the buffer at the first time and the second frame may be queued by the buffer at the second time, the second time occurring 10 ms after the first time, giving 3 of 10 ms.

[0055] In some examples, K is a weighted average value. For example, 8 may take on several values, such as by recording n times after the first time in the buffer queue for a frame stream. In such an example, <5 has multiple members and a comparison of differences such as in Eq. 4 may take the form:

[0056] Eqs. 3 through 5 are meant to be illustrative of the types of comparisons that maybe performed, not limiting. Other forms of comparison may also be employed, such as, but not limited to, mean absolute difference, weighted ratio, and correlation matrix calculations. The frame timing data and the pre-render threshold may also take on other forms than those outlined above.

[0057] Operation 906 is determining to adjust one or more resources of the electronic device. The determination is based on the comparison of the frame-rate timing data to the prerender threshold value. For example, 6 may be 120 FPS, i may be 100 FPS, and K is then, in the example where K is an absolute difference (as in Eq. 3). 20 FPS. In some examples, this K value may be above an acceptable threshold deviation value co, such as co = 5 FPS. In this example, there is a determination to adjust one or more resources of the electronic device.

[0058] Operation 908 is generating an adjustment output. The adjustment output is configured to be output to the electronic device and includes one or more adjustment instructions configured to adjust one or more resources of the electronic device. For example, the one or more adjustment instructions include a command to change a behavior of a processor of the electronic device, the buffer, or a memory' of the electronic device. In some examples, the behavior is a speed, such as a processor speed of the processor. In some examples, the one or more adjustment instructions include a command to allocate fewer resources of the electronic device to rendering the frame stream. In some examples, the adjustment output is generated by a frame timeline guardian (e.g., the frame timeline guardian 406).

[0059] For example, 3 is 10 ms and 0 is a minimum acceptable frame time, such as 15 ms. 0 may be set at 15 ms, for example, when the electronic device is configured to save power and it has been determined that a frame-time of 15 ms is optimal for device performance. In such an example, the one or more resources of the electronic device may be processor time and the one or more adjustment instructions may be to allocate less processor time to rendering.

[0060] In some examples, the one or more resources of the electronic device are two or more processor types and the one or more adjustment instructions is a command to switch between a first processor type of the two or more processor types and a second processor type of the two or more processor ty pes. In some examples, the one or more resources of the electronic device are the one or more processors and the one or more adjustment instructions is a command to adjust a frequency of the one or more processors, where the command to adjust the frequency of the one or more processors comprises a command to a dynamic voltage and frequency scaling (DVFS) subsystem of the device. In some examples, the one or more resources of the electronic device is a battery of the device. In some examples, the one or more adjustment instructions is a command to change a state of the device, where the state of the device is one of an idle state, an active state, or a high-active state.

[0061] FIG. 10 illustrates another example method 1000 for performative power management by proactive frame rate monitoring. The method 1000 includes the method 900. Operation 1002 is receiving post-render timing data. The post-render timing data is based on one or more post-rendered frames rendered to the display. For example, the post-render timing datamay be the processor speed of the processor of the electronic device rendering the one or more post-rendered frames to the display. In some examples, the post-render timing data is a type of feedback data (opposed to the frame timing data, which is a type of feedforward data). In some examples, the post-render timing data is based on one or more frame boundaries of the buffer.

[0062] Operation 1004 is comparing the post-render timing data to a post-render threshold value. In some examples, the post-render timing data indicates a frame-rate or an FPS value. In such examples, the post-render threshold value may similarly be based on the FPS value. For example, the post-render timing data may indicate a display rate for the one or more post-rendered frames rendered to the display of 45 FPS. The post-render threshold value may be, for example, 60 FPS. Comparisons, in some examples, are made, such as in Eqs. 3 and 4. Other comparison techniques, as outlined above with the comparisons between cf>, ip, and 0. In other examples, the post-render timing data is related to a frame time. In such examples, the post-render threshold value may similarly be based on the frame time. Examples of comparisons for these examples are similar to those outlined above, including in Eq. 5.

[0063] Operation 1006 is determining to adjust the one or more resources of the electronic device. The determination is based on the comparison of the post-render timing data to the postrender threshold value. For example, the post-render timing data may be based on a hardware monitor (e.g., the hardware monitor 412) measuring parameters of the electronic device. In some examples, the hardware monitor measures an output of the processor. It may be determined, for example, that the output of the processor is above an expected threshold for video rendering. Other hardware or software parameters may be included in the determination of to adjust the one or more resources of the electronic device.

[0064] Operation 1008 is generating a tuning output. The tuning output is configured to be output to the electronic device and includes one or more tuning instructions configured to adjust the one or more resources of the electronic device. In some examples, the tuning output is part of the adjustment output from the frame timeline guardian (e.g., the frame timeline guardian 406). In some examples, the one or more tuning instructions is a command to change a behavior of the processor of the electronic device, the buffer, or the memory of the device. For example, the behavior may be a speed, such as a speed of the processor or a clock rate of the processor. In some examples, the one or more tuning instructions include a command to allocate fewer of the one or more resources of the electronic device to rendering the one or more pre-rendered frames.Additional Examples

[0065] The following are additional examples of the described devices and methods for performative power management by proactive frame rate monitoring.

[0066] Example 1: A method comprising receiving, by one or more processors, frame timing data, the frame timing data based on one or more pre-rendered frames configured to be rendered, comparing, by the one or more processors, the frame timing data to a pre-render threshold value, determining, by the one or more processors, to adjust one or more resources of an electronic device, the determination based on the comparison of the frame timing data to the pre-render threshold value, and generating, by the one or more processors, an adjustment output, the adjustment output configured to be output to the electronic device, and comprising one or more adjustment instructions configured to adjust one or more resources of the electronic device.

[0067] The method may be a method for adjusting display resources. The pre-rendered frames may be configured to be rendered to a display. The generating may be responsive to the determining. The electronic device may be configured to render the frames. The one or more adjustment instructions may cause a parameter of the rendering to be closer to a target parameter value.

[0068] Example 2: The method of example 1, wherein the frame timing data is based on a frame-rendering queue, the frame-rendering queue comprising a buffer for the one or more prerendered frames.

[0069] Example 3: The method of example 2, wherein the buffer for the one or more prerendered frames stores pointers to one or more memory locations of the one or more pre-rendered frames.

[0070] Example 4: The method of any one of the examples 2 or 3, wherein the frame timing data is based on a first time that a first frame of the one or more pre-rendered frames enters the frame-rendering queue and a second time that a second frame of the one or more pre-rendered frames enters the frame-rendering queue, the second time being after the first time. The frame timing data may be based on a difference between the first time and the second time.

[0071] Example 5: The method of any one of the previous examples, wherein the prerender threshold value is a frame-rate value.

[0072] Example 6: The method of example 5, wherein the one or more adjustment instructions comprise a command to change a behavior of a processor, the buffer for the one or more pre-rendered frames, or a memory. The processor may be a processor of the electronic device.

[0073] Example 7: The method of example 6, wherein the behavior is a speed.

[0074] Example 8: The method of any one of examples 5 to 7, wherein the frame-rate value is one of 30 frames-per-second (FPS), 60 FPS, 120 FPS, or 240 FPS.

[0075] Example 9: The method of any one of examples 1 to 4, wherein the pre-render threshold value is a frame-time value.

[0076] Example 10: The method of example 9, wherein the one or more adjustment instructions comprise a command to allocate fewer of the resources of the electronic device to rendering the one or more pre-rendered frames.

[0077] Example 11 : The method of any preceding example, further comprising receiving, by the one or more processors, post-render timing data, the post-render timing data based on one or more post-rendered frames. The one or more post-rendered frames may be frames rendered to the display.

[0078] Example 12: The method of example 11 , further comprising comparing, by the one or more processors, the post-render timing data to a post-render threshold value.

[0079] Example 13: The method of example 12, further comprising determining, by the one or more processors, to adjust the one or more of the resources of the electronic device, the determination based on the comparison of the post-render timing data to the post-render threshold value.

[0080] Example 14: The method of example 13, further comprising generating, by the one or more processors, a tuning output, the tuning output configured to be output to the electronic device, and comprising one or more tuning instructions configured to adjust the one or more resources of the electronic device.

[0081] Example 15: The method of any one of examples 11 to 14, wherein the post-render timing data is further based on one or more frame boundaries of the buffer.

[0082] Example 16: The method of any one of examples 12 to 15, wherein the post-render threshold value is a frame-rate value.

[0083] Example 17: The method of example 16, wherein the one or more tuning instructions comprise a command to change a behavior of a processor, the buffer for the one or more pre-rendered frames, or a memory.

[0084] Example 18: The method of example 17, wherein the behavior is a speed.

[0085] Example 19: The method of any one of examples 16 to 18, wherein the frame-rate value is one of 30 frames-per-second (FPS). 60 FPS, 120 FPS, or 240 FPS.

[0086] Example 20: The method of any one of examples 12 to 15, wherein the post-render threshold value is a frame-time value.

[0087] Example 21 : The method of example 20, wherein the one or more tuning instructions comprise a command to allocate fewer of the resources of the electronic device to rendering the one or more pre-rendered frames.

[0088] Example 22: The method of any preceding example, wherein the one or more resources of the electronic device comprise two or more processor types.

[0089] Example 23: The method of example 22, wherein the one or more adjustment instructions comprise a command to switch between a first processor type of the two or more processor types and a second processor type of the two or more processor types.

[0090] Example 24: The method of any one of examples 1 to 21, wherein the one or more resources of the electronic device comprise the one or more processors.

[0091] Example 25. The method of example 24, wherein the one or more adjustment instructions comprise a command to adjust a frequency of the one or more processors.

[0092] Example 26: The method of example 25, wherein the command to adjust the frequency of the one or more processors comprises a command to a dynamic voltage and frequency scaling (DVFS) subsystem of the device.

[0093] Example 27: The method of any one of examples 1 to 21, wherein the one or more resources of the electronic device comprise a battery' of the device.

[0094] Example 28: The method of example 27, wherein the one or more adjustment instructions comprise a command to change a state of the device.

[0095] Example 29: The method of example 28, wherein the state of the device is one of an idle state, an active state, or a high-active state.

[0096] Example 30: An electronic device comprising one or more processors and a memory containing instructions that, when accessed by the one or more processors, cause the one or more processors to perform any one of the methods of examples 1-29. The electronic device may be configured to render the frames. The electronic device may comprise a frame buffer configured to queue pre-rendered frames.

[0097] Example 31 : The electronic device of claim 30, wherein the electronic device is any one of a smartphone, a desktop computer, a laptop computer, a smart watch, smart glasses, a smart helmet, a virtual-reality (VR) headset, an augmented-reality (AR) device, a smart television (TV), or a streaming device configured to be coupled to a display.

[0098] Example 32: A non-transitory, computer-readable storage medium comprising instructions that, when executed, configure at least one processor of an electronic device to perform any one of the methods of examples 1-29.

[0099] Example 33: A computer program product comprising instructions that, when accessed by one or more processors, cause the one or more processors to execute any one of the methods of examples 1-29.Conclusion

[0100] While the present subj ect matter has been described in detail with respect to various specific example implementations thereof, each example is provided by way of explanation, notlimitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such implementations. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations, and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one implementation can be used with another implementation to yield a still further implementation. Thus, it is intended that the present disclosure cover such alterations, variations, and equivalents.[ooiot] As used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. As an example, “at least one of a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a- a- a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).

[0102] While various embodiments of the disclosure are described in the foregoing description and shown in the drawings, it is to be distinctly understood that this disclosure is not limited thereto but may be variously embodied to practice within the scope of the following claims. From the foregoing description, it will be apparent that various changes may be made without departing from the spirit and scope of the disclosure as defined by the following claims.

Claims

CLAIMS:What is claimed is:

1. A method comprising: receiving, by one or more processors, frame timing data, the frame timing data based on one or more pre-rendered frames configured to be rendered; comparing, by the one or more processors, the frame timing data to a pre-render threshold value; determining, by the one or more processors, to adjust one or more resources of an electronic device, the determination based on the comparison of the frame timing data to the prerender threshold value; and generating, responsive to the determining and by the one or more processors, an adjustment output, the adjustment output: configured to be output to the electronic device; and comprising one or more adjustment instructions configured to adjust one or more resources of the electronic device.

2. The method of claim 1. wherein the frame timing data is based on a framerendering queue, the frame-rendering queue comprising a buffer for the one or more pre-rendered frames.

3. The method of claim 2. wherein the frame timing data is based on a first time that a first frame of the one or more pre-rendered frames enters the frame-rendering queue and a second time that a second frame of the one or more pre-rendered frames enters the frame-rendering queue, the second time being after the first time.

4. The method of any one of the previous claims, wherein the pre-render threshold value is a frame-rate value.

5. The method of any one of the claims 1 to 3, wherein the pre-render threshold value is a frame-time value.

6. The method of any preceding claim, further comprising: receiving, by the one or more processors, post-render timing data, the post-render timing data based on one or more post-rendered frames; comparing, by the one or more processors, the post-render timing data to a post-render threshold value; determining, by the one or more processors, to further adjust the one or more of the resources of the electronic device, the determination based on the comparison of the post-render timing data to the post-render threshold value; and generating, responsive to detennining to further adjust and by the one or more processors, a tuning output, the tuning output: configured to be output to the electronic device; and comprising one or more tuning instructions configured to adjust the one or more resources of the electronic device.

7. The method of any one of the previous claims, wherein the one or more resources of the electronic device comprise two or more processor types.

8. The method of claim 7. wherein the one or more adjustment instructions comprise a command to switch between a first processor type of the two or more processor ty pes and a second processor type of the two or more processor ty pes.

9. The method of any one of claims 1 to 6, wherein the one or more resources of the electronic device comprise the one or more processors.

10. The method of claim 9, wherein the one or more adjustment instructions comprise a command to adjust a frequency of the one or more processors.

11. The method of claim 10, wherein the command to adjust the frequency of the one or more processors comprises a command to a dynamic voltage and frequency scaling (DVFS) subsystem of the device.

12. The method of any one of claims 1 to 6, wherein the one or more resources of the electronic device comprise a battery of the device.

13. The method of claim 12, wherein the one or more adjustment instructions comprise a command to change a state of the batten-.

14. A mobile electronic device comprising: one or more processors; and a memory containing instructions that, when accessed by the one or more processors, cause the one or more processors to perform any one of the methods of claims 1-13.

15. A computer program product comprising instructions that, when accessed by one or more processors, cause the one or more processors to execute any one of the methods of examples of claims 1-13.

Images