Frequency modulation method and electronic equipment

Abstract

A frequency modulation method and an electronic device relate to the technical field of terminals. The method comprises the following steps: the electronic equipment displays a first image; within a first time interval in which the first image is displayed. At the first moment, the electronic equipment starts to wait for the GPU to complete processing of the second image, and the first timer is started; and when the electronic equipment waits for the GPU to complete the processing of the second image and the timing time of the first timer is not reached, the electronic equipment cancels the first timer. And when the timing time of the first timer is reached and the GPU does not finish the processing of the second image, the electronic equipment improves the frequency of the GPU. Wherein the second image and the first image are two continuous frames of images; the timing time comprises a time interval between the first moment and a frequency increasing time point, and the frequency increasing time point is located in the first time interval. In this way, the electronic device can adjust the frequency of the GPU based on the processing progress of the GPU for the current frame image, and frame loss is avoided.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of terminal, and in particular, to a frequency adjusting method and electronic device. BACKGROUND

[0002] In electronic devices such as mobile phones and tablets, a graphics processing unit (GPU) is an important device for completing image processing. The image processing can include image rendering and image synthesis. Generally, the higher the frequency of the GPU, the stronger the computing power provided by the GPU, and the higher the efficiency of image processing. The higher the frequency of the GPU, the weaker the computing power provided by the GPU, and the lower the efficiency of image processing. Therefore, the electronic device can adjust the frequency of the GPU to meet the efficiency requirements of image processing.

[0003] In the prior art, the electronic device can select an appropriate frequency according to the usage rate (GPU_busy) of the GPU. For example, if the usage rate is above 90% for a certain period of time, it indicates that the GPU has very little idle time when running at the current frequency, and therefore the electronic device can increase the frequency of the GPU so that the GPU has more idle time to meet the higher processing requirements.

[0004] However, the electronic device using the above prior art increases the frequency of the GPU only after the usage rate is above 90% for a certain period of time, which has a lag in frequency increase and does not meet the real-time image processing requirements, which may result in frame loss. SUMMARY

[0005] The present application provides a frequency adjusting method and electronic device, which can adjust the frequency of the GPU based on the processing progress of the GPU for the current frame image to ensure no frame loss.

[0006] To achieve the above object, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a frequency modulation method applied to an electronic device, the electronic device comprising a graphics processing unit (GPU). The method comprises: displaying, by the electronic device, a first image (e.g., a (k-1)-th frame image in the following description). In a first time interval during which the first image is displayed, the electronic device can obtain a next frame image, denoted as a second image (e.g., a k-th frame image in the following description), that is, the second image and the first image are two consecutive images. Specifically, at a first time point (e.g., time T1 or time T10 in the following description) in the first time interval, the electronic device starts waiting for the GPU to complete processing of the second image, and starts a first timer (e.g., a timer in the following description). The timing time of the first timer comprises a time interval (e.g., Δt in the following description) between the first time point and a frequency raising time point (e.g., frequency raising time point 2 in the following description), and the frequency raising time point is located in the first time interval. Then, reaching the timing time of the first timer is equivalent to reaching the frequency raising time point.

[0008] When the electronic device waits for the GPU to complete processing of the second image (e.g., image rendering and image synthesis, the same in the following description), and the timing time of the first timer is not reached, it indicates that the GPU completes the image processing before the frequency raising time point is reached, and the frequency of the GPU is sufficient, so that the frequency of the GPU does not need to be raised by the timing time of the timer, and the electronic device can cancel the first timer.

[0009] When the timing time of the first timer is reached, and the GPU has not completed processing of the second image, it indicates that the GPU cannot complete the image processing before the frequency raising time point is reached, and the electronic device can raise the frequency of the GPU, so that the GPU can provide stronger computing power to meet the efficiency requirements of image processing and avoid frame loss.

[0010] In summary, by using the present application, in the process of processing one frame of image, the electronic device can determine whether to raise the frequency of the GPU based on the time point at which the GPU completes the image processing and the timing of the timer. On the one hand, in the scenario where the GPU cannot complete the image processing in time, the electronic device can raise the frequency of the GPU in time when the timing time of the timer is reached, thereby avoiding frame loss. On the other hand, in the scenario where the GPU can complete the image processing in time, the electronic device can cancel the timer, so that the frequency of the GPU is not raised.

[0011] In a possible design of the first aspect, the method further comprises: in the first time interval, when the electronic device waits for the GPU to complete processing of the second image, and the timing time of the first timer is reached, that is, the timing time of the timer is reached first and then the GPU completes the image processing, because the timing time has been reached, the first timer will not continue timing, and the electronic device can not cancel the first timer.

[0012] Or, in the first time interval, in the case that the GPU has completed the processing of the second image when reaching the timing time of the first timer, that is, the GPU completes the image processing first, and then reaches the timing time of the timer, and before the GPU completes the image processing, the first timer has been canceled, then the electronic device will not trigger the frequency of the GPU to be raised.

[0013] In the process of processing a frame of image, the frequency of the GPU can be too low or too high, and if the frequency is too high, it will cause resource waste. Based on this, in the process of processing a frame of image (such as the first time interval described above), a frequency reduction time point is also set, and the frequency reduction time point is located before the frequency raising time point. If the time point at which the GPU completes the image processing (denoted as the second time, such as time T31 in the following) is before the frequency reduction time point, it indicates that the GPU can complete the image processing very efficiently, that is, the frequency of the GPU is too high.

[0014] Therefore, in a possible design manner in the first aspect, after the electronic device cancels the first timer in the case that the electronic device waits for the GPU to complete the processing of the second image and the timing time of the first timer has not been reached, the method further includes: if the second time is before the frequency reduction time point (corresponding to scenario 22 in the following), the electronic device reduces the frequency of the GPU, so as to save resources. Of course, if the second time is after the frequency reduction time point (corresponding to scenario 22 in the following), the electronic device can not reduce the frequency of the GPU.

[0015] In a possible design manner in the first aspect, the method further includes: the electronic device waits for the GPU to complete the processing of the second image by using a waiting function (such as a plane_wait_input_fence), and the waiting function calls a second timer (referred to as a waiting timer in the following) to control the upper limit of the execution time of the waiting function, that is, to control the upper limit of the time for waiting for the GPU to complete the image processing. For example, if the waiting time upper limit is 10 s, the timing time of the second timer can be set to 10 s, the second timer is started when the waiting function starts to be executed, and the waiting is ended when the timing time of the second timer is reached.

[0016] The first timer is not the second timer called by the multiplexed waiting function, that is, the first timer and the second timer are different. In addition, the accuracy of the first timer is higher than that of the second timer, so that the timing of raising the frequency of the GPU can be more accurately controlled, and the frequency of the GPU can be raised too early or too late.

[0017] In a possible design of the first aspect, the first timer has a higher precision than the second timer, including that the first timer is counted in a time unit of 1 millisecond or smaller than 1 millisecond, such as 1 microsecond, 1 nanosecond, etc., and the second timer is counted in jiffies, and one jiffy is longer than 1 millisecond, such as 1 jiffy = 4 milliseconds.

[0018] It can be understood that the counting unit of a timer is the error range of the timer. For example, the first timer is counted in a unit of 1 nanosecond, and the error of the first timer is 1 nanosecond. The second timer is counted in a unit of 1 jiffy, and the error of the second timer is 4 milliseconds.

[0019] In a possible design of the first aspect, the method further includes: at the first time, starting the first timer, and setting the timing time as the time interval (such as Δt). The initial state of the first timer is the first state, which is referred to as an active state in the following.

[0020] In a possible design of the first aspect, the method further includes: when the first timer is not reached at the timing time and the GPU is not completed with the processing of the second image, the electronic device cancels the first timer, including that, when the electronic device is waiting for the GPU to complete the processing of the second image, and the state of the first timer is the first state, it indicates that the first timer has not reached the timing time, and the GPU has completed the image processing, and the electronic device can cancel the first timer. In this way, the electronic device can accurately determine whether the timing time of the first timer is reached by the state of the first timer.

[0021] In a possible design of the first aspect, when the first timer is reached at the timing time and the GPU is not completed with the processing of the second image, the electronic device increases the frequency of the GPU, including that, when the state of the first timer is updated to the second state, the electronic device increases the frequency of the GPU. The state of the first timer being updated to the second state indicates that the timing time of the first timer is reached. If the GPU has completed the image processing before the timing time of the first timer is reached, the electronic device should have canceled the first timer, that is, the state of the first timer can be updated to the second state, which indicates that the GPU has not completed the image processing. It can be seen that the state of the first timer being updated to the second state can express two meanings: one is that the timing time of the first timer is reached, and the other is that the GPU has not completed the image processing. Based on this, when the state of the first timer is updated to the second state, it is not necessary to further detect whether the GPU has completed the image processing, and the frequency of the GPU can be directly increased.

[0022] In a possible design of the first aspect, in the process of the first image, the GPU can perform image rendering on the second image and first image synthesis (e.g., complex image synthesis) on the second image in the first time interval. The hardware compositor HWC in the electronic device performs second image synthesis (e.g., simple hardware-level image synthesis) on the second image. Accordingly, the electronic device starts to wait for the GPU to complete the processing of the second image, including that the electronic device (e.g., a display driver) starts to wait for the GPU to complete the processing of the second image after the HWC completes the second image synthesis, and the processing of the second image by the GPU includes image rendering and first image synthesis on the second image. Subsequently, the electronic device can display the second image after waiting for the GPU to complete the processing of the second image.

[0023] In the second aspect, the present application further provides an electronic device, which includes a display screen, a memory, and one or more processors. The display screen, the memory, and the processors are coupled. The memory is configured to store computer program codes, and the computer program codes include computer instructions. When the computer instructions are executed by the processors, the electronic device performs the method in the first aspect and possible design manners thereof.

[0024] In the third aspect, the present application provides a chip system, which is applied to an electronic device including a display screen and a memory. The chip system includes one or more interface circuits and one or more processors. The interface circuits and the processors are interconnected through lines. The interface circuits are configured to receive signals from the memory of the electronic device and send signals to the processors. The signals include computer instructions stored in the memory. When the processors execute the computer instructions, the electronic device performs the method in the first aspect and possible design manners thereof.

[0025] In the fourth aspect, the present application provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the method in the first aspect and possible design manners thereof.

[0026] In the fifth aspect, the present application provides a computer program product, which, when executed on a computer, causes the computer to perform the method in the first aspect and possible design manners thereof.

[0027] It can be understood that the electronic device in the second aspect, the chip system in the third aspect, the computer storage medium in the fourth aspect, and the computer program product in the fifth aspect can achieve the beneficial effects as described in the first aspect and possible design manners thereof, which will not be described herein. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A schematic diagram of image processing provided for an embodiment of the present application;

[0029] Figure 2 A schematic diagram of a frequency raising time point provided for an embodiment of the present application;

[0030] Figure 3 A frequency raising schematic diagram of scenario 1 provided for an embodiment of the present application;

[0031] Figure 4 A frequency raising schematic diagram of scenario 2 provided for an embodiment of the present application;

[0032] Figure 5 A frequency raising schematic diagram of scenario 2 provided for an embodiment of the present application;

[0033] Figure 6 A flowchart of a frequency raising scheme provided for an embodiment of the present application;

[0034] Figure 7 A flowchart of a frequency raising scheme provided for an embodiment of the present application using a waiting timer;

[0035] Figure 8 A flowchart of a frequency raising scheme provided for an embodiment of the present application using a frequency raising timer;

[0036] Figure 9 A precision difference diagram of a waiting timer and a frequency raising timer provided for an embodiment of the present application;

[0037] Figure 10 A precision difference diagram of a waiting timer and a frequency raising timer provided for an embodiment of the present application;

[0038] Figure 11 A software and hardware architecture diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing the specific embodiments and are not intended to be limiting on the present application. As used in the specification and the appended claims of the present application, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and / or “including,” as used herein, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The term “and / or” used in the description herein refers to three relationships; for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural.

[0040] Reference in the specification to “one embodiment” or “some embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases “in one embodiment”, “in some embodiments”, “in other embodiments”, “in additional embodiments”, and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. The terms “comprise”, “comprising”, “have”, “having”, “include”, “including” and “contain”, “containing” are not intended to be limiting, unless otherwise specifically stated. The term “connect” includes both direct connections and indirect connections, unless otherwise specifically stated. “First”, “second”, “third”, etc. are used only for descriptive purposes and do not connote or imply relative importance or an ordering.

[0041] In the embodiments of the present application, the words “exemplary” and “for example” are used to mean serving as an example, instance, or illustration, in no way limiting. The embodiments or designs described as “exemplary” or “for example” in the embodiments of the present application are not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words “exemplary” and “for example” is intended to present related concepts in a concrete manner.

[0042] The frequency modulation method provided by the embodiments of the present application can be applied to scenarios in which the electronic device needs GPU to process images. In particular, the frequency modulation method can be applied to scenarios in which the GPU needs to continuously process multiple frames of images. For example, in the process of displaying a desktop dynamic effect, the GPU needs to continuously process multiple dynamic effect frames. In the scenario of playing a game, the GPU needs to continuously process multiple frames of game pictures.

[0043] In the above scenario, by using the frequency modulation method provided in the embodiments of the present application, the electronic device can modulate the frequency of the GPU to meet the demand for work efficiency and ensure no frame loss.

[0044] Before the embodiments of the present application are described in detail, a brief introduction is first given to the processing procedure of an image:

[0045] Referring to Figure 1 The processing procedure of an image mainly includes: step 1, image rendering; step 2, image composition; and step 3, display.

[0046] Step 1, image rendering.

[0047] When starting to draw a new image, the application can send a rendering instruction to the GPU. After receiving the rendering instruction, the GPU can perform rendering processing.

[0048] Step 2, image composition.

[0049] Generally, the hardware composer (HWC) and the GPU in the electronic device can both be used for image composition. However, the HWC can only complete some simple composition work, while the GPU can be used for complex composition processing. In a specific implementation manner, the image composition processing service (SurfaceFlinger) in the electronic device can decide whether the GPU needs to participate in image composition. If so, the SurfaceFlinger can distribute a composition task to the GPU, so that the GPU completes the composition work that the HWC cannot complete, thereby realizing image composition. Of course, the simple composition work is still completed by the HWC, which will not be described here in detail.

[0050] It should be noted that during the execution of step 2, the GPU can not have completed image rendering yet, and therefore, the GPU needs to complete image composition after completing image rendering.

[0051] Step 3, display.

[0052] After completing image composition, the HWC can send the composition result to the display driver. On the basis of the composition result, the display driver further combines the processing result of the GPU, i.e., the processing result of the GPU performing image rendering and image composition, to obtain a to-be-displayed image and send it to the display screen for display.

[0053] It should be noted that when the HWC completes image composition, the GPU can not have completed image processing (including image rendering and image composition, the same hereinafter). Therefore, after receiving the composition result of the HWC, the display driver still needs to further wait for the GPU to complete image processing, so as to obtain the to-be-displayed image and send it to the display screen.

[0054] Therefore, in the process of processing a frame of image, the process in which the GPU participates includes image rendering and synthesis, and if the GPU does not complete image processing in time, the display driver cannot obtain the image to be displayed in time and send it to the display screen for display, thereby possibly causing a frame loss phenomenon.

[0055] Meanwhile, the higher the frequency of the GPU is, the stronger the computing power provided is, and the higher the efficiency of image processing is. Therefore, the electronic device can adjust the frequency of the GPU so that the GPU provides computing power that matches the current image processing task, to meet the efficiency (such as frame rate, refresh rate, etc.) requirement of image processing.

[0056] In some conventional technologies, the electronic device can use a frequency adjustment scheme provided by the developer of the GPU, that is, dynamic clock and voltage scaling (DCVS), to adjust the frequency level of the GPU to a corresponding frequency level according to the usage rate (GPU_busy) of the GPU in a period of time. GPU_busy can be understood as the proportion of the working time of the GPU in a period of time to the total time in the period of time. For example, after GPU_busy is above 90% for a certain period of time, the electronic device can increase the frequency level of the GPU.

[0057] Practice shows that the electronic device may have the following problems when using DCVS to adjust the frequency of the GPU:

[0058] Problem 1: The electronic device only increases the frequency level of the GPU when GPU_busy is above a certain value (such as 90%, the same hereinafter) for a certain period of time, and the adjustment of the frequency level is lagging behind, which may still cause frame loss.

[0059] Problem 2: GPU_busy is calculated in a period of time. If the working time of the GPU is less in a period of time, GPU_busy is low; if the working time of the GPU is more, GPU_busy is high. However, the GPU_busy in the historical time period cannot accurately represent the real-time image processing task, and therefore the control of the electronic device on the frequency may be inaccurate, causing a continuous frame loss phenomenon.

[0060] Problem 3: The electronic device uses DCVS to increase the frequency level when GPU_busy is above a certain value. In practice, it is often possible that GPU_busy is close to 90% but does not exceed 90% (which can be referred to as a dangerous level). When GPU_busy is at the dangerous level, the work of the GPU is easily fluctuated due to the influence of the whole machine, such as the power and temperature of the electronic device, thereby causing unstable performance of the electronic device, such as occasional frame loss.

[0061] In addition to the DCVS scheme described above, in some conventional technologies, the developer of the GPU also provides a customized Perflock scheme for the electronic device to adjust the frequency level of the GPU more flexibly, to overcome the problems of the DCVS to some extent. The main idea of Perflock is as follows: the electronic device can add a stub point for a problem scenario that needs to adjust the frequency, and then identify the problem scenario through the stub point, and adjust the frequency level of the GPU for the problem scenario through an interface provided by the developer of the GPU.

[0062] However, Perflock is mainly suitable for some problem scenarios that must adjust the frequency (which can also be referred to as must-be scenarios), and for some non-must-be scenarios, i.e., problem scenarios that may need to adjust the frequency in some cases and may not need to adjust the frequency in other cases, the electronic device using Perflock often causes waste of resources.

[0063] A typical scenario is as follows: when the screen refresh rate or the application frame rate of the electronic device is high, the GPU usually needs to complete more image processing tasks of frames of images in a unit of time, and thus the computing power requirement of the GPU is high, and the GPU needs to work at a high frequency level. Therefore, the electronic device can use Perflock to add a stub point for a scenario in which the screen refresh rate is high (such as reaching 120 hertz (Hz)), i.e., to mark that the frequency level of the GPU needs to be raised when the screen refresh rate reaches 120 Hz. Subsequently, during the image processing of the GPU, if the electronic device detects that the screen refresh rate reaches 120 Hz, the frequency level of the GPU can be raised.

[0064] However, in actual application, when the screen refresh rate of the electronic device reaches 120 Hz, it does not necessarily need to raise the frequency level of the GPU. For example, although the screen refresh rate of the electronic device reaches 120 Hz, the image processing task that needs to be completed by the GPU is not much, for example, the GPU does not need to complete complex composition processing, and thus the GPU can complete the current image processing task in time even if the frequency level of the GPU is not raised, i.e., a non-must-be scenario. If this scenario still uses Perflock to raise the frequency level of the GPU when the screen refresh rate reaches 120 Hz, resource waste can be caused.

[0065] Based on the problems of the DCVS and the Perflock, an embodiment of the present application provides a frequency adjustment method. During the processing of a frame of image, the electronic device can set a timer corresponding to a frequency adjustment time point and monitor whether the GPU completes the image processing. In another scenario (referred to as scenario 1), when the timer reaches the frequency adjustment time point, the electronic device detects that the GPU has not completed the image processing, that is, the GPU cannot complete the image processing in time. The electronic device can increase the frequency of the GPU to improve the image processing efficiency of the GPU. In a scenario (referreded to as scenario 2), when the electronic device monitors that the GPU completes the image processing, the timer has not reached the frequency adjustment time point, that is, the GPU can complete the image processing in time. The electronic device can cancel the timer, that is, not continue to count the frequency adjustment time point, and the processing of the frame of image will not be frequency-adjusted.

[0066] In this way, the electronic device can determine whether to increase the frequency of the GPU based on the time point at which the GPU completes the image processing during the processing of each frame of image and the counting of the timer. On the one hand, the electronic device can increase the frequency of the GPU in time in the above scenario 2 in which the GPU cannot complete the image processing in time. On the other hand, the electronic device can not continue the counting of the timer in the above scenario 1 in which the GPU can complete the image processing in time, so as not to increase the frequency.

[0067] The frequency adjustment time point can be flexibly set by those skilled in the art according to actual conditions, and the present application does not make specific limitations thereto.

[0068] In some embodiments, the frequency adjustment time point can be a time point at which the time length from the next refresh time T2 of the display screen is a fixed time length 1, and is referred to as frequency adjustment time point 1. It can be understood that the display screen needs to display a new frame of image every time it is refreshed. Therefore, the GPU needs to complete the processing of the current frame of image before the next refresh, and the frequency adjustment time point 1 also needs to be before the next refresh time T2. In this way, the electronic device can quickly determine the frequency adjustment time point 1 based on the next refresh time T2 and the fixed time length 1.

[0069] In the embodiment, the mobile phone can start the timer every time it is refreshed, and the timeout of the timer is the time interval between the current refresh and the frequency adjustment time point 1.

[0070] In other embodiments, the frequency adjustment time point is a time point at which the proportion of the time length from the time T1 at which the display driver starts waiting for the GPU to complete the image processing to the time length L from the time T1 to the next refresh time T2 is a preset proportion λ1 (such as 0.7, 0.8, etc.), and is referred to as frequency adjustment time point 1.

[0071] Taking the preset proportion λ1 as 0.7 as an example, in the case of Figure 2In the processing of the kth frame image (at this time, the k-1th frame image is being displayed), the electronic device can determine the frequency boosting time point 2 as time T1+0.7*(T2-T1).

[0072] In this embodiment, the mobile phone can start the timer at time T1 when the display driving starts to wait for the GPU to complete the image processing, and the timing time of the timer is the time interval between time T1 and the frequency boosting time point 2.

[0073] As can be seen, by using this embodiment, the electronic device can determine the matching frequency boosting time point 2 according to the processing progress of the current frame image, so that the frequency gear of the GPU can be adjusted accordingly.

[0074] In addition, it should be emphasized that when the display driving starts to wait for the GPU to complete the image processing, it means that the CPU has basically completed the related processing of image display. Thereafter, whether frame loss occurs mainly depends on the working efficiency of the GPU. Therefore, configuring the frequency boosting time point 2 as a time after the display driving starts to wait for the GPU to complete the image processing can accurately control no frame loss by adjusting the frequency of the GPU under the premise of being affected only by the working efficiency of the GPU.

[0075] In practice, the display driving can wait for the GPU to complete the image processing through a waiting function (such as plane_wait_input_fence). It can be understood that after the GPU completes the image processing, the corresponding fence is released. Correspondingly, after the display driving queries that the fence is released, it can be determined that the GPU has completed the image processing. At this time, the display driving can execute the subsequent display process to drive the display screen to display the current frame image. Based on this, in a specific implementation manner, the electronic device can determine time T1 as the time (denoted as T10) when the display driving starts to execute the waiting function. That is, at time T10 when the display driving starts to execute the waiting function, the electronic device starts the timer, and the timing time of the timer is the time interval At between time T10 when the display driving starts to execute the waiting function and the frequency boosting time point 2.

[0076] At this point, it should be noted that the next refresh time T2 can be determined based on the previous refresh time and the frame rate of the application. For example, the electronic device can calculate the frame length based on the frame rate, and then add the frame length to the time (i.e. the previous refresh time) when the k-1th frame image is refreshed to obtain the time (i.e. the next refresh time T2) when the kth frame image is refreshed.

[0077] The following example illustrates the solution of this application by taking the moment when the display driver starts executing the wait function at time T10, when the electronic device starts a timer and the timer duration is the time interval Δt between time T10 and frequency boosting time point 2. The solution is further illustrated in conjunction with scenarios 1 and 2.

[0078] Scene 1

[0079] by Figure 3 Taking scenario 1 as an example, during the processing of the k-th frame image, the time for the GPU to complete image processing (denoted as T300) is later than the frequency boosting time point 2, that is, the GPU did not complete image processing in time.

[0080] See also Figure 3 The electronic device can start a timer at time T10 when it detects that the display driver has started executing the wait function, and set the timer's timing period to Δt. At time boosting point 2, the electronic device can monitor that the timer has reached its timing period Δt, and that the GPU has not yet completed image processing, indicating that the GPU cannot complete image rendering and image compositing in time. The electronic device can then increase the GPU's frequency (referred to as "frequency boosting" in the diagram, and the same applies to the following diagrams). In this way, the electronic device can promptly increase the GPU's frequency, enabling the GPU to quickly complete the image rendering and image compositing of the k-th frame, avoiding frame drops.

[0081] See also Figure 3 After increasing the GPU frequency at time point 2, the electronic device can wait until the GPU completes image processing at time T30. If the timer duration Δt has already been reached, it indicates that the GPU frequency has been increased, and the electronic device does not need to increase the GPU frequency.

[0082] Scene 2

[0083] by Figure 4 Taking scenario 2 as an example, during the processing of the k-th frame image, the time for the GPU to complete image processing (denoted as T31) is earlier than the frequency boosting time point 2, meaning that the GPU can complete image processing in a timely manner.

[0084] See also Figure 4 The electronic device can start a timer at time T10 when it detects that the display driver has started executing a wait function, and set the timer's duration to Δt. At time T31, the electronic device waits until the GPU completes image processing and monitors that the timer's duration Δt has not yet been reached. This indicates that the GPU can complete image processing before the frequency boost time point 2, so there is no need to trigger the frequency boost through the timer, and the electronic device can cancel the timer. Subsequently, since the timer has been canceled, when the frequency boost time point 2 is reached, the electronic device will not monitor that the timer has reached its duration Δt, and therefore no processing is required.

[0085] In practice, the electronic device can not only increase the frequency of the GPU to meet the efficiency requirement of image processing in the case of insufficient frequency of the GPU, but also decrease the frequency of the GPU to avoid resource waste caused by excessively high frequency of the GPU. Further, for scenario 2, in the case that the GPU can complete image processing in time (for example, before the frequency-increasing time point 2), the electronic device can also compare the order of time T31 and the frequency-decreasing time point (denoted as frequency-decreasing time point 3). If time T31 is located after the frequency-decreasing time point 3 (denoted as sub-scenario 21), it indicates that the efficiency of the GPU in completing image processing is between the frequency-decreasing time point 3 and the frequency-increasing time point, which belongs to the case that neither frequency increase nor frequency decrease is needed, and the electronic device can not perform frequency adjustment. If time T31 is located before the frequency-decreasing time point 3 (denoted as sub-scenario 22), it indicates that the efficiency of the GPU in completing image processing is extremely high, and the electronic device can decrease the frequency of the GPU to save resources. In addition, if time T31 is the same as the frequency-decreasing time point 3, the electronic device can process in the manner of scenario 21 or scenario 22, which is not limited in the embodiments of the present application.

[0086] Similar to the frequency-increasing time point mentioned above, the frequency-increasing time point can also be flexibly set by those skilled in the art according to actual conditions, which is not limited in the embodiments of the present application. In some embodiments, the frequency-decreasing time point 3 can be a time point at a fixed time length 2 from the next refresh time T2, and the fixed time length 2 is greater than the fixed time length 1. In this way, the mobile phone can also quickly determine the frequency-decreasing time point 3 based on the next refresh time T2 and the fixed time length 2. In other embodiments, the frequency-decreasing time point 3 can be a time point in the time length L, and the proportion of the time length from time T1 to the time length L is a preset proportion λ2 (such as 0.5, 0.6, etc.). The preset proportion λ2 is less than the preset proportion λ1. For details, please refer to the related description of the frequency-increasing time point above, which will not be repeated here.

[0087] The following will introduce scenarios 21 and 22, and the corresponding frequency adjustment schemes in combination with the drawings:

[0088] Scenario 21

[0089] Continuing to refer to Figure 4 , time T31 is located after the frequency-decreasing time point 3. Correspondingly, after the electronic device waits for the GPU to complete image processing at time T31 and monitors that the timing time Δt of the timer is not reached, the electronic device can also compare the order of time T31 and the frequency-decreasing time point 3 to determine that time T31 is located after the frequency-decreasing time point 3, and the electronic device can not decrease the frequency.

[0090] Scenario 22

[0091] Different from Figure 4 , please refer to Figure 5, the time T31 is located before the frequency reduction time point 3. Correspondingly, after the electronic device waits for the GPU to complete image processing and monitors that the timing time at of the timer is not reached, the electronic device can also compare the order of the time T31 and the frequency reduction time point 3 to determine that the time T31 is located before the frequency reduction time point 3, and the electronic device can reduce the frequency of the GPU.

[0092] In some embodiments, in order to take into account various scenarios described above, the electronic device can use the following Figure 6 to implement frequency adjustment. Specifically, in Figure 6 , still taking the time T10 when the display driver starts to execute the waiting function, the electronic device starts the timer, and the timing time of the timer is the time interval at between the time T10 when the display driver starts to execute the waiting function and the frequency increasing time point 2. The frequency increasing method includes:

[0093] S601, when detecting that the display driver starts to execute the waiting function, the electronic device calculates the frequency increasing time point 2 and the frequency reduction time point 3 according to the current time (the time T10) and the next refresh time T2.

[0094] For specific implementation of calculating the frequency increasing time point 2 and the frequency reduction time point 3, reference can be made to the related description in the foregoing, which will not be described here again.

[0095] S602, the electronic device starts the timer, and the timing time of the timer is the time interval at between the current time (the time T10) and the frequency increasing time point 2.

[0096] The electronic device starts the timer at the time T10, so that the timer starts timing from the time T10, and the timing time is the time interval at between the time T10 and the frequency increasing time point 2, so that the timer can end timing at the frequency increasing time point 2.

[0097] S603, the electronic device waits for the GPU to complete image processing.

[0098] For example, the display driver in the electronic device can wait for the GPU to complete image processing through the waiting function. Wherein, when the GPU is processing image, the display driver can query that the fence is occupied through the waiting function, so as to determine that the GPU is processing image. When the GPU completes image processing, the display driver can query that the fence is released through the waiting function, so as to determine that the GPU completes image processing.

[0099] The electronic device will perform corresponding processing when the timer reaches the timing time and the GPU completes image processing. Wherein, in the case that the timer reaches the timing time, S604-S607 can be performed; in the case that the GPU completes image processing, S608 can be performed.

[0100] S604, when waiting for the GPU to complete image processing, the electronic device detects whether the timer reaches the timing time. If not, S605 is executed; if yes, S606 is executed.

[0101] When the electronic device waits for the GPU to complete image processing and the timer does not reach the timing time, it indicates that the GPU completes image processing before the frequency raising time point 2, and the frequency of the GPU does not need to be raised during the processing of the frame image, and the electronic device can execute S605 as follows, which can correspond to the case in which the timer is revoked at the time T31 when the GPU completes image processing in the above Figure 4 and Figure 5 .

[0102] When the electronic device waits for the GPU to complete image processing, but the timer has reached the timing time, it indicates that the GPU completes image processing after the frequency raising time point 2, that is, the timer reaches the timing time first. Since the frequency of the GPU has been raised when the timer reaches the timing time, as shown in S608 below, the frequency of the GPU does not need to be raised at this time. Moreover, since the timer has reached the timing time, it will not continue to count, so there is no need to revoke the timer, that is, the electronic device does not need to execute S605.

[0103] S605, the electronic device revokes the timer.

[0104] S606, the electronic device compares whether the current time (time T30 or time T31) is before the frequency lowering time point 3; if yes, S607 is executed; if not, the frequency adjustment of the current frame ends.

[0105] After the electronic device executes S605, or after the determination of no in S604, the electronic device can execute S606 to detect whether the frequency of the GPU needs to be lowered. Specifically, if the current time (i.e., the time when the GPU completes image processing) is before the frequency lowering time point 3, it indicates that the frequency of the GPU needs to be lowered, and the electronic device can execute S608 to avoid resource waste, which can correspond to the case in which the frequency is lowered at the time T31 when the GPU completes image processing in the above Figure 5 . If the current time (i.e., the time when the GPU completes image processing) is not before the frequency lowering time point 3, it indicates that the frequency of the GPU does not need to be lowered, and the electronic device can not lower the frequency of the GPU, which can correspond to the case in which the frequency is not lowered at the time T31 when the GPU completes image processing in the above Figure 4 .

[0106] It should be noted that the determination in S604 is no, that is, the GPU completes the image processing after the frequency boosting time point 2, and the frequency boosting time point 2 is located after the frequency reduction time point 2. In this case, the result obtained in S606 is naturally that the current time (that is, the time when the GPU completes the image processing) is not before the frequency reduction time point 3, and thus the frequency reduction is not needed.

[0107] In S607, the electronic device reduces the frequency of the GPU.

[0108] In S608, the electronic device monitors that the timer reaches the timing time, and the GPU does not complete the image processing, and the electronic device boosts the frequency of the GPU.

[0109] The timer reaches the timing time, which indicates that the frequency boosting time point 2 is reached. At this time, the electronic device does not wait for the GPU to complete the image processing, and the fence is occupied, which indicates that the GPU does not complete the image processing in time. The electronic device can boost the frequency of the GPU to ensure that the frequency is boosted in time. This case can correspond to the frequency boosting in the frequency boosting time point 2 in the above-mentioned scenario 1. Figure 3

[0110] It can be understood that if the GPU completes the image processing when the timer reaches the timing time, the fence is released, and thus the frequency boosting is naturally not needed.

[0111] It should be noted that the execution order of S604-S608 is not limited to that shown in Figure 6 . In practice, the electronic device first monitors that the timer reaches the timing time, and then S604-S607 are executed first, and then S608 is executed after the GPU completes the image processing.

[0112] By using the above-mentioned Figure 6 flow, the frequency boosting in time and the frequency reduction can be achieved, which takes into account the above-mentioned scenario 1 and scenario 2, and further takes into account the scenario 21 and scenario 22 included in the scenario 2.

[0113] In a specific implementation manner, the electronic device can reuse a timer (referred to as a waiting timer) used in a waiting function to implement the frequency boosting timing.

[0114] Specifically, in the waiting function, the display driver can control the upper limit of the time for waiting for the GPU to complete the image processing through the waiting timer. When the timing time of the waiting timer is reached, the display driver exits the waiting. For example, the upper limit of the time is 10s, and the display driver can set the timing time of the waiting timer to 10s. After the timing time of the waiting timer is reached, the display driver exits the waiting.

[0115] In this implementation manner, the electronic device reuses the waiting timer to implement the frequency adjustment flow as shown in Figure 7 . ​

[0116] In one aspect, S602 in the above Figure 6 may be replaced by the following S701:

[0117] S701, the electronic device starts a waiting timer, the timing time of the waiting timer is the time interval At between the current time (time T10) and the frequency boosting time point 2, and setting the timing time of the waiting timer triggers the frequency of the GPU to be boosted.

[0118] The electronic device sets the timing time of the waiting timer to trigger the frequency of the GPU to be boosted, so that the frequency of the GPU can be triggered to be boosted after reaching the frequency boosting time point 2, and timely frequency boosting is achieved, as shown in the following S704, which will not be described in detail here.

[0119] On the other hand, the waiting timer can return different values before and after reaching the timing time. For example, before reaching the timing time, the waiting timer can return a value 1, such as the remaining time; after reaching the timing time, the waiting timer can return a value 2, such as -1. Based on this, the electronic device can obtain the return value of the waiting timer, so as to monitor whether the timing time of the waiting timer is reached.

[0120] Correspondingly, S604 in the above Figure 6 may be replaced by the following S702-S703:

[0121] S702, when waiting for the GPU to complete image processing, the electronic device obtains the return value of the waiting timer.

[0122] S703, the electronic device detects whether the return value is value 1; if yes, S605 is executed; if no, S606 is executed.

[0123] The return value is value 1, which indicates that the timing time of the waiting timer has not been reached. The return value is not value 1, such as value 2, which indicates that the timing time of the waiting timer has been reached.

[0124] And S608 in the above Figure 6 may be replaced by the following S704:

[0125] S704, the electronic device monitors the return value of the waiting timer to be 2, and the frequency of the GPU is boosted.

[0126] If the return value is 2, it indicates that the timing time of the waiting timer is reached. In the foregoing S702 and S703, the electronic device waits for the GPU to complete the image processing, and when the return value is 1, the electronic device can cancel the waiting timer. If the return value of the timer is 2, it indicates that the waiting timer is not canceled because the GPU completes the image processing first, that is, the GPU has not completed the image processing, and therefore, the electronic device does not need to detect whether the GPU completes the image processing, but can directly increase the frequency of the GPU. This also proves that, in the foregoing S701, the timing time of the waiting timer is set to trigger the increase of the frequency of the GPU, and other detection and judgment do not need to be performed.

[0127] The part not described in detail in the foregoing S701-S704 can be referred to the description of the corresponding part in Figure 6 .

[0128] Therefore, it can be seen that, in the implementation manner, the electronic device can increase the frequency of the GPU in time by multiplexing the waiting timer. Figure 7

[0129] In addition, the waiting timer is a low-power timer provided by the Linux kernel of the electronic device, and can perform time counting by taking the number of beats (jiffies) as a timing unit. The specific granularity (that is, the timing unit) is determined when the Linux kernel is compiled. In general, the granularity is 1s timing 250 times, that is, the granularity is 1s / 250 times = 4ms, and the time changes by 4ms when the waiting timer is counted once. The granularity of such counting is obviously larger than the granularity of counting once per ms or even per ns, and the power consumption of counting is smaller, so that the power consumption of the electronic device can be saved.

[0130] In a specific implementation manner, the electronic device can use a separate timer (which can be referred to as a frequency adjustment timer) to implement the frequency increase timing.

[0131] In the implementation manner, the electronic device can still use the waiting timer in the waiting function to control the upper limit of the time for waiting for the GPU to complete the image processing. In addition, the electronic device can also call the frequency adjustment timer outside the waiting function to implement the frequency increase timing. The flow of the electronic device calling the frequency adjustment timer to implement the frequency adjustment is as shown in Figure 8 .

[0132] On the one hand, the S602 in the foregoing Figure 6 may be replaced by the following S801:

[0133] S801, the electronic device starts the frequency adjustment timer, the timing time of the frequency adjustment timer is the time interval At between the current time (time T10) and the frequency increase time point 2, and the timing time of the frequency adjustment timer is set to trigger the increase of the frequency of the GPU.​

[0134] The electronic device sets the timing time of the frequency modulation timer to trigger the frequency of the GPU, so that the frequency of the GPU can be triggered after the frequency boost time point 2 is reached, and the frequency of the GPU is boosted in time, as shown in S804. Details are not described here.

[0135] On the other hand, the electronic device can monitor the state change of the frequency modulation timer. When the frequency modulation timer is started, the initial state is active, and before the timing time is reached, the electronic device keeps the timer active. When the timing time is reached, the electronic device updates the timer to the inactive state. Based on this, the electronic device can query the state of the frequency modulation timer to monitor whether the timing time of the frequency modulation timer is reached.

[0136] Correspondingly, S604 in the above Figure 6 may be replaced by the following S802-S803:

[0137] S802, when waiting for the GPU to complete image processing, the electronic device queries the state of the frequency modulation timer.

[0138] S803, the electronic device detects whether it is in the active state; if yes, S605 is executed; if no, S606 is executed.

[0139] The state of the frequency modulation timer is active, indicating that the timing time of the frequency modulation timer has not been reached. The state of the frequency modulation timer is not active, i.e. inactive, indicating that the timing time of the frequency modulation timer has been reached.

[0140] And S608 in the above Figure 6 may be replaced by the following S804:

[0141] S804, the electronic device monitors the state of the frequency modulation timer to update to the inactive state, and boosts the frequency of the GPU.

[0142] The state of the frequency modulation timer is updated to the inactive state, indicating that the timing time of the frequency modulation timer has been reached. In the foregoing S802 and S803, the electronic device waits for the GPU to complete image processing, and when the state is active, the electronic device can cancel the frequency modulation timer. Then, if the timer is in the inactive state, it indicates that the frequency modulation timer has not been canceled because the GPU has completed image processing, i.e. the GPU has not completed image processing, so the electronic device does not need to detect whether the GPU has completed image processing, but can directly boost the frequency of the GPU. This also proves that the timing time of the frequency modulation timer can be set to trigger the frequency of the GPU in the above S801, without the need to perform other detection and judgment.

[0143] The parts not described in detail in S801-S804 can be referred to the descriptions of the corresponding parts in Figure 6

[0144] Therefore, by using the implementation manner of the present application, the electronic device can improve the frequency of the GPU in time without using the waiting timer. Figure 8

[0145] Further, the frequency adjustment timer is a timer with a granularity smaller than that of the waiting timer, for example, the frequency adjustment timer counts in 1ms or 1ns (nanosecond) instead of 4ms. In this way, compared with the implementation manner shown in Figure 7 , the present application can determine more accurately whether the frequency of the GPU needs to be improved.

[0146] Example one: taking the timing time as 2ms, the granularity of the frequency adjustment timer as 1ms, and the granularity of the waiting timer as 4ms as an example, the accuracy of the frequency improvement timing by using the waiting timer and the frequency adjustment timer is introduced respectively as follows:

[0147] Method one: using the waiting timer.

[0148] The timing time 2ms is converted into the number of beats as 2ms / 4ms=0.5 beats, and the number of beats is rounded up, so that 0.5 beats can be approximated as 1 beat, i.e., the timing time is the number of beats plus 1 by using the waiting timer. Referring to the time axis of the waiting timer in Figure 9 , after the number of beats is updated to X, the electronic device starts the waiting timer, and then after 2ms, the number of beats is updated to X+1, i.e., the number of beats plus 1, reaching the timing time of the waiting timer. If the GPU does not complete the image processing during the process, the frequency of the GPU will be improved when the number of beats of the waiting timer is updated to X+1. In fact, after the waiting timer is started for 2ms, the actual time point of improving the frequency is reached. It can be seen that in example one, the actual frequency improvement time point by using the waiting timer is 1ms later than the actual frequency improvement time point, which increases the risk of frame loss.

[0149] Method two: using the frequency adjustment timer.

[0150] The timing time 2ms is converted into the granularity of the frequency adjustment timer 1ms, and still 2ms is obtained, i.e., the timing time is increased by 2ms by using the frequency adjustment timer. Referring to Figure 9 ​​In the timeline of the frequency modulation timer, the electronic device starts the frequency modulation timer when its timer duration is Ams. After a 2ms interval, the timer duration is updated to (A+2)ms, meaning it increases by 2ms and reaches its set time. If the GPU has not completed image processing during this process, a frequency boost will be triggered when the frequency modulation timer duration updates to A+2. In fact, the actual required frequency boost time is reached 2ms after the frequency modulation timer starts. Therefore, in Example 1, the actual frequency boost time using the frequency modulation timer matches the required frequency boost time.

[0151] Example 1: Taking a timing interval of 3ms, a frequency modulation timer with a granularity of 1ms, and a wait timer with a granularity of 4ms as an example, the following describes how to achieve frequency boosting timing precision using a wait timer and a frequency modulation timer respectively:

[0152] Method 1: Use a waiting timer.

[0153] Converting the 3ms timing period to the number of beats, we get 3ms / 4ms = 0.75 beats. Since beat counts are rounded up, 0.75 beats can be approximated as 1 beat. Therefore, a wait timer can be used, with the timing period being the number of beats plus 1. See also... Figure 10 In the timeline of the wait timer, after the tick count is updated to X, the electronic device starts the wait timer 2ms later. Then, after another 2ms interval, the tick count is updated to X+1, meaning the tick count increments by 1, reaching the wait timer's set timeout. If the GPU hasn't completed image processing during this period, a frequency boost will be triggered when the wait timer's tick count updates to X+1. However, in reality, the actual required frequency boost occurs 3ms after the wait timer starts. Therefore, in Example 2, the actual frequency boost time using the wait timer is 1ms earlier than the required boost time. If the GPU completes image processing exactly within this 1ms window, the frequency boost will be a waste of resources and is clearly unnecessary.

[0154] Method 2: Use a frequency modulation timer.

[0155] Converting the 3ms timing interval to a 1ms granularity for a frequency modulation (FM) timer still results in 3ms. Therefore, using an FM timer increases the timing interval by 3ms. (See also...) Figure 10In the timeline of the frequency modulation timer, the electronic device starts the frequency modulation timer when the time of the frequency modulation timer is Bms, and then the time of the frequency modulation timer is updated to (B+3) ms after an interval of 3 ms, that is, 3 ms is added, and the timing time of the frequency modulation timer is reached. If the GPU does not complete image processing during this process, the frequency of the GPU will be triggered to be raised when the time of the frequency modulation timer is updated to B+3. In fact, 3 ms after starting the frequency modulation timer, the actual time point at which the frequency needs to be raised is reached. As can be seen, in Example Two, the actual frequency raising time point of the frequency modulation timer is consistent with the actual frequency raising time point.

[0156] As can be seen from Example One and Example Two, the frequency modulation timer can raise the frequency of the GPU at a more accurate time, avoiding frame loss caused by late frequency raising, and avoiding resource waste caused by too early frequency raising.

[0157] Of course, in another specific implementation, the electronic device can also modify the granularity of the waiting timer to meet the high-precision requirement. However, it should be noted that the waiting timer is used in a large number of low-precision timing scenarios in the electronic device, such as timing scenarios of automatic screen locking, and the accuracy requirements of these scenarios are not high. If the granularity of the waiting timer is uniformly modified, the timing power consumption of a large number of low-precision timing scenarios will be increased. Therefore, the frequency modulation timer described above can meet the high-precision requirement of the frequency modulation scenario without increasing the power consumption of other low-precision timing scenarios.

[0158] For example, the electronic device in the embodiments of the present application can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), an Internet access device, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) \ virtual reality (VR) device, and the like. The specific form of the electronic device is not specially limited in the embodiments of the present application. In the following, the electronic device will be mainly taken as a mobile phone as an example to illustrate the solutions of the present application.

[0159] In the embodiments of the present application, the software system of the mobile phone can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. In the following embodiments, the software and hardware architecture of the mobile phone will be mainly illustrated by taking an Android system with a layered architecture as an example.

[0160] The layered architecture divides software into several layers, each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, referring toFigure 11 The hardware and software architecture of the mobile phone is arranged from top to bottom as follows: an application layer, an application framework layer, a native layer, a hardware abstraction layer (HAL), a kernel layer, and a hardware layer.

[0161] The application layer can install desktop applications, game applications, video players, chat applications, and the like. These applications need to display images through the display screen of the mobile phone. For example, application icons, game pictures, video frames, and the like need to be displayed through the display screen of the mobile phone. Most of these images need to be processed by the GPU (such as rendering and composition), i.e., belong to the scenario that needs GPU processing of images.

[0162] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the application layer. For example, the application framework layer can include a window manager, a content provider, a resource manager, and the like.

[0163] The native layer provides various services for the upper layer (such as the application framework layer). For example, the native layer includes a SurfaceFlinger image composition processing service, a three-dimensional (3D) graphics processing library (such as OpenGL ES), and the like.

[0164] The SurfaceFlinger can also be used for refresh rate control. It can be understood that when an application in the application framework layer needs to display a new frame of image, the application can request a Vertical Synchronization (vSync) signal through the SurfaceFlinger. The vSync signal can trigger the application to complete rendering of a new frame of image, and then the image is finally sent to the display after being composed. Moreover, after one refresh of the display screen of the electronic device is completed, the vSync signal can be distributed to the application requesting the vSync signal through the SurfaceFlinger, i.e., to the application that needs rendering. That is, the SurfaceFlinger can control the application to start rendering after one refresh of the display screen by distributing the vSync signal from the display screen to the application, i.e., to achieve refresh rate control.

[0165] The hardware abstraction layer encapsulates the underlying hardware driver and provides a general interface for calling the driver to the upper layer, so that the upper layer can call the driver to drive the corresponding hardware to work. The hardware abstraction layer includes a Hardware composer (HWC).

[0166] The HWC can control the frequency adjustment of the GPU. Of course, the HWC can also perform image synthesis and the like, which is not limited in the present application.

[0167] The kernel layer includes drivers for driving hardware, such as display drivers and GPU drivers (kgsl driver).

[0168] The display driver can wait for the GPU to complete image processing and send the image to the display after the GPU completes the image processing. In addition, the display driver can also determine whether the frequency of the GPU needs to be adjusted. For example, the display driver can execute the flow decision in the foregoing embodiment, and send a frequency adjustment message to the upper layer (such as the HWC in the hardware abstraction layer) in the case of determining that the frequency needs to be adjusted. Further, the frequency adjustment message includes a frequency increase message or a frequency decrease message. The frequency increase message indicates to increase the frequency of the GPU, and the frequency decrease message indicates to decrease the frequency of the GPU. Of course, the display driver can also be used to drive the display screen to display the image, which is not limited in the present application. Figures 6-8

[0169] The kernel layer also includes basic capabilities provided by the Linux kernel. Each module in the kernel layer can call these basic capabilities to implement corresponding functions. In some embodiments, the basic capabilities include timers of various precisions, such as high-precision frequency adjustment timers and low-precision waiting timers. The display driver can call these timers according to the needs. For example, the waiting timer is called in the waiting function to control the upper limit of the time for waiting for the GPU to complete image processing. For another example, the frequency adjustment timer is called outside the waiting function to count the frequency.

[0170] The GPU driver is used to drive the GPU to work, such as adjusting the frequency of the GPU.

[0171] The hardware layer includes devices such as display screens and GPUs. The display screen can be used for image display, and the GPU can be used for image rendering and synthesis.

[0172] The following describes the interaction between some modules in the above-mentioned software and hardware architecture in the frequency adjustment method provided by the embodiments of the present application, taking the frequency adjustment timer as an example.

[0173] Continuing to refer to Figure 11 , in step 1, the display driver calls the frequency adjustment timer provided by the Linux kernel when starting to execute the waiting function, and sets the timing time and the frequency increase of the GPU triggered when the timing time is reached. In the case that the timing time is reached but the fence is still occupied, the display driver determines that the frequency of the GPU needs to be increased. In step 2, the display driver can send a frequency increase instruction to the HWC. In step 3, the HWC controls the GPU to increase the frequency based on the frequency increase instruction through the GPU driver. ​

[0174] With reference to Figure 11 , step 1 can further include, in the case that the fence is released but not reaching the timing time, the display driver revoking the frequency modulation timer. Further, step 1 can further include, after the fence is released, comparing the current time and the frequency reduction time point 3 in the order, and determining to reduce the frequency of the GPU in the case that the current time is before the frequency reduction time point 3. Correspondingly, step 2 can further include the display driver sending a frequency reduction instruction to the HWC, and step 3 can further include the HWC controlling the GPU to reduce the frequency based on the frequency reduction instruction.

[0175] The embodiment of the present application further provides an electronic device, which can include a display screen, a memory and one or more processors (such as CPU, GPU, NPU, etc.). The display screen, the memory and the processor are coupled. The memory is used to store computer program codes, which include computer instructions. When the processor executes the computer instructions, the electronic device can execute various functions or steps performed by the device in the above method embodiments.

[0176] The embodiment of the present application further provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a circuit. For example, the interface circuit can be used to receive signals from other devices (such as the memory of the electronic device). For another example, the interface circuit can be used to send signals to other devices (such as the processor). Illustratively, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can execute various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not limited in the embodiment of the present application.

[0177] The embodiment of the present application further provides a computer storage medium, which stores computer instructions. When the computer instructions are run on the electronic device, the electronic device executes the above related method steps to implement the image processing method in the above embodiments.

[0178] The embodiment of the present application further provides a computer program product, which makes the computer execute the above related steps to implement the image processing method in the above embodiments when the computer program product is run on the computer.

[0179] In addition, the embodiment of the present application further provides a device, which can be a chip, a component or a module. The device can include a processor and a memory connected to each other. The memory is used to store computer execution instructions. When the device is running, the processor can execute the computer execution instructions stored in the memory to make the chip execute the image processing method in the above method embodiments.

[0180] The electronic device, the computer storage medium, the computer program product or the chip provided in the embodiment are used for executing the corresponding method provided in the above, and thus the beneficial effects achieved by the electronic device, the computer storage medium, the computer program product or the chip can refer to the beneficial effects of the corresponding method provided in the above, which will not be described here again.

[0181] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0182] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented by other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0183] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place, or can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0184] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0185] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or in other words the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A frequency modulation method characterized by, The method is applied to an electronic device comprising a graphic processing unit (GPU), and the method comprises: The electronic device displays a first image; In a first time interval during which the first image is displayed: At a first time point, the electronic device starts a first timer to wait for the GPU to complete processing of a second image; If the electronic device has not reached a timing time of the first timer when the GPU completes processing of the second image, the electronic device cancels the first timer; If the electronic device reaches the timing time of the first timer before the GPU completes processing of the second image, the electronic device increases a frequency of the GPU; The second image and the first image are two consecutive images; the timing time comprises a time interval between the first time point and a frequency-increasing time point, and the frequency-increasing time point is located in the first time interval.

2. The method of claim 1, wherein, The method further comprises: If the electronic device reaches the timing time of the first timer when the GPU completes processing of the second image, the electronic device does not cancel the first timer; or If the electronic device reaches the timing time of the first timer before the GPU completes processing of the second image, the electronic device does not increase the frequency of the GPU.

3. The method according to claim 1 or 2, characterized in that, After the electronic device cancels the first timer when the electronic device has not reached the timing time of the first timer when the GPU completes processing of the second image, the method further comprises: If a second time point at which the electronic device waits for the GPU to complete processing of the second image is before a frequency-decreasing time point, the electronic device decreases the frequency of the GPU, the frequency-decreasing time point is located in the first time interval, and the frequency-decreasing time point is before the frequency-increasing time point.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: The electronic device waits for the GPU to complete processing of the second image by using a waiting function, and the waiting function calls a second timer to control an upper limit of an execution time of the waiting function; The first timer and the second timer are different, and a precision of the first timer is higher than a precision of the second timer.

5. The method of claim 4, wherein, The precision of the first timer is higher than the precision of the second timer, which comprises that the first timer counts time in units of 1 millisecond or smaller than 1 millisecond, and the second timer counts time in units of beats, and a length of time of one beat is longer than 1 millisecond.

6. The method according to claim 4 or 5, characterized in that, The method further comprises: At the first time point, the electronic device starts the first timer and sets the timing time as the time interval; and an initial state of the first timer is a first state; When the electronic device reaches the timing time of the first timer, the electronic device updates a state of the first timer to a second state; When the electronic device cancels the first timer, the method further comprises: In a case where the state of the first timer is the first state when the electronic device waits for the GPU to complete processing of the second image, the electronic device cancels the first timer.

7. The method of claim 6, wherein, In a case where the state of the first timer is the first state when the electronic device waits for the GPU to complete processing of the second image, the electronic device cancels the first timer. In a case where the state of the first timer is the first state when the electronic device waits for the GPU to complete processing of the second image, the electronic device cancels the first timer.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: In the first time interval, the GPU performs image rendering on the second image and first image composition on the second image, and a hardware compositor (HWC) in the electronic device performs second image composition on the second image; The electronic device starts waiting for the GPU to complete processing of the second image, including: After the HWC completes the second image composition, the electronic device starts waiting for the GPU to complete processing of the second image, and the processing of the second image by the GPU includes image rendering and first image composition on the second image.

9. An electronic device, comprising: including: a display screen, one or more processors, and one or more memories; the one or more processors are coupled with the display screen and the one or more memories; the one or more memories are configured to store computer program code including computer instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 1-8.

10. A computer-readable storage medium comprising instructions, characterized in that, The instructions, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-8.

Images