Frequency modulation method and electronic device

Abstract

A frequency modulation method and an electronic device, which relate to the technical field of terminals, and can adjust the frequency of a GPU on the basis of the processing progress of the GPU for the current image frame, ensuring no frame loss. The method comprises an electronic device displaying a first image. During a first time interval in which the first image is displayed, the electronic device detects whether a first end time for completing the processing of a second image by a GPU exceeds a first time, wherein the second image and the first image are two consecutive image frames, and the first time is within the first time interval. If the first end time exceeds the first time, the electronic device increases the frequency of the GPU. After the GPU has completed the processing of the second image, the electronic device displays the second image.

Description

Frequency modulation method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 22, 2023, with application number 202311240144.X and invention name “A frequency modulation method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of terminal technology, and in particular to a frequency modulation method and electronic device. Background Art

[0003] In electronic devices such as mobile phones and tablets, the Graphics Processing Unit (GPU) is an important component for completing image processing. Image processing can include image rendering and image synthesis. Generally speaking, the higher the GPU frequency, the stronger the computing power it provides and the higher the efficiency of image processing. The higher the GPU frequency, the weaker the computing power provided by the GPU, and accordingly, the lower the image processing efficiency. Therefore, electronic devices can meet the efficiency requirements of image processing by adjusting the GPU frequency.

[0004] In the prior art, electronic devices can select an appropriate frequency based on the GPU usage (GPU_busy). For example, if the usage remains above 90% for a certain period of time, it indicates that the GPU has very little idle time when running at the current frequency. Therefore, the electronic device can increase the GPU frequency to allow the GPU to have more idle time to meet more efficient processing needs.

[0005] However, electronic devices using the above-mentioned existing technology will only increase the GPU frequency after the usage rate has been above 90% for a certain period of time. The frequency increase has a lag and does not meet the processing requirements of real-time image processing, which may cause frame drops.

[0006] Summary of the Invention

[0007] The present application provides a frequency modulation method and electronic device, which can adjust the frequency of a GPU based on the processing progress of the GPU for the current frame image to ensure that no frame is lost.

[0008] To achieve the above objectives, this application adopts the following technical solutions:

[0009] In a first aspect, the present application provides a frequency modulation method, which is applied to an electronic device, the electronic device including a graphics processing unit (GPU). The electronic device displays a first image. During a first time interval for displaying the first image, the electronic device detects whether a first end time at which the GPU completes processing of a second image exceeds a first time, where the second image and the first image are two consecutive frames, and the first time is within the first time interval. If the first end time exceeds the first time, the electronic device increases the frequency of the GPU. After the GPU completes processing the second image, the electronic device displays the second image.

[0010] That is, while the electronic device is displaying the first image (i.e., the first frame of two consecutive frames), the electronic device can process the second image (i.e., the second frame of two consecutive frames). Furthermore, if the GPU does not complete image processing (such as image rendering and synthesis) before the first time during the processing, it indicates that the GPU cannot complete image processing of the second image in time, and the electronic device can increase the GPU frequency to avoid frame drops.

[0011] In this way, the electronic device can increase the GPU frequency if the GPU cannot complete image processing in time during the processing of the second image. On the one hand, the electronic device enables the GPU to process the second image in a timely manner, ensuring that no frames are dropped. On the other hand, the electronic device does not increase the GPU frequency if the GPU can complete image processing in time, thereby avoiding wasting resources.

[0012] In one possible design of the first aspect, the electronic device further includes a display driver and a hardware overlay controller (HWC). The first time is after the second time at which the display driver begins waiting for the GPU to complete processing of the second image. After receiving a synthesis result of the second image from the HWC, the display driver begins waiting for the GPU to complete processing of the second image.

[0013] When the display driver begins waiting for the GPU to complete image processing, it indicates that the CPU has essentially completed the image processing required for display. From then on, whether frame drops occur depends primarily on the GPU's efficiency. Therefore, by configuring the preset time T0 as the time after the display driver begins waiting for the GPU to complete image processing, the GPU frequency can be adjusted to accurately control frame drops, with the only impact being the GPU's efficiency.

[0014] In a possible design of the first aspect, the electronic device detects whether a first end time for the GPU to complete processing the second image exceeds a first time, including: at the first time, the electronic device detects whether a display driver is waiting for the GPU to complete processing the second image. If the display driver is waiting for the GPU to complete processing the second image, the first end time exceeds the first time. If the display driver is not waiting for the GPU to complete processing the second image, the first end time does not exceed the first time.

[0015] In this way, the electronic device can accurately determine whether the GPU has completed image processing of the second image according to whether the display driver is waiting for the GPU to complete processing of the second image.

[0016] In one possible design of the first aspect, the second time includes the time when the display driver begins executing a wait function. The wait function is configured to allow the display driver to wait for the GPU to complete processing of the second image, and the display driver terminates execution of the wait function after the GPU completes processing of the second image.

[0017] In one possible design of the first aspect, the electronic device increasing the GPU frequency includes: the electronic device increasing the minimum frequency of the GPU. It is understood that after the electronic device increases the minimum frequency of the GPU, the actual operating frequency of the GPU also changes to the increased minimum frequency, so that the actual operating frequency is greater than or equal to the minimum frequency, and the GPU can then operate at the frequency level corresponding to the increased minimum frequency.

[0018] Accordingly, the method further includes: during the first time interval of displaying the first image, the electronic device detecting whether the first end time exceeds a third time, and if the first end time does not exceed the third time, the electronic device reducing the GPU frequency. The third time is within the first time interval and is before the first time.

[0019] If the first end time does not exceed the third time, it indicates that the GPU can complete the image processing of the second image in advance, and the electronic device can reduce the frequency of the GPU to avoid resource waste.

[0020] In a possible design method of the first aspect, the electronic device detects whether the first end time of the GPU completing the processing of the second image exceeds the first time, including: if the first end time exceeds the third time, the electronic device detects whether the first end time of the GPU completing the processing of the second image exceeds the first time.

[0021] Because the third time is after the first time, if the first end time does not exceed the third time, then the first end time will not exceed the first time, and there is no need to increase the GPU frequency. In this case, after the electronic device detects that the first end time exceeds the third time, it does not need to further check whether the first end time exceeds the first time. Therefore, the electronic device only further checks whether the first end time exceeds the first time after detecting that the first time exceeds the third time, thereby avoiding invalid detection and saving computing resources.

[0022] In a possible design of the first aspect, the electronic device detects whether a first end time at which the GPU completes processing of the second image exceeds a first time, including: if the time interval between the first time and a fourth time is greater than a duration threshold, the electronic device detects whether the first end time exceeds the first time. The electronic device detects whether the first end time exceeds a third time, including: if the time interval between the third time and the fourth time is greater than a duration threshold, the electronic device detects whether the first end time exceeds the third time. The fourth time is the time at which the electronic device last reduced the GPU frequency before displaying the first image.

[0023] In this way, the electronic device can avoid detecting and adjusting the GPU frequency when it is not effective.

[0024] In one possible design of the first aspect, if the first end time exceeds the third time but does not exceed the first time, this indicates that the GPU operating at the current frequency can ensure timely completion of image processing without wasting resources, i.e., the current frequency is appropriate. Therefore, the electronic device does not need to adjust the GPU frequency. This ensures that the GPU operates at an appropriate frequency.

[0025] In one possible design of the first aspect, the method further includes: if the first end time corresponding to at least a first number of consecutive images exceeds the third time but does not exceed the first time, indicating that the current frequency can maintain the GPU's image processing efficiency within a reasonable range, the electronic device may use dynamic clock and voltage scaling (DCVS) to adjust the GPU frequency. That is, if the GPU's operating efficiency meets the requirements, the electronic device may further adjust the GPU frequency based on the GPU's utilization rate to further ensure that the GPU frequency matches the utilization rate.

[0026] In one possible design of the first aspect, the electronic device increases the GPU frequency, including: the electronic device increasing the actual operating frequency of the GPU. It is understood that after the electronic device increases the actual operating frequency of the GPU, since the minimum frequency of the GPU does not change, the electronic device can use DCVS to control the GPU frequency after the actual operating frequency of the GPU is increased. In other words, the actual value of the GPU frequency may not remain stable after the increase. Specifically, after the electronic device increases the GPU frequency, the method further includes: the electronic device using dynamic clock and voltage regulation (DCVS) to adjust the GPU frequency.

[0027] In a possible design manner of the first aspect, the above method also includes: if the frequency of the GPU is increased continuously for more than a preset number of times, the electronic device uses DCVS to reduce the frequency of the GPU to a first value, and the electronic device increases the minimum frequency of the GPU to a second value, and the frequency gear to which the second value belongs is one gear higher than the frequency gear to which the first value belongs, so that the GPU can operate at a frequency that makes the first end time of completing image processing earlier than the first time.

[0028] In a possible design of the first aspect, the electronic device further includes a display screen. The electronic device displays a first image, including: at a fifth time after the GPU completes processing the first image, the display driver drives the display screen to display the first image. Accordingly, the method further includes: the electronic device obtains the fifth time and the second time. The electronic device adds the refresh cycle of the display screen to the fifth time to obtain a sixth time. The electronic device determines, as the first time point, a time point at which the ratio of the time interval between the second time and the sixth time to the time interval between the second time and the sixth time is a first ratio.

[0029] In a second aspect, the present application further provides an electronic device comprising a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The display screen comprises a first screen and a second screen. When the display screen is in a folded state, the display directions of the first screen and the second screen are opposite to each other, and the first screen and the second screen are two different display areas of the display screen. The memory is used to store computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs the method of the first aspect and its possible design methods.

[0030] In a third aspect, the present application provides a chip system, which is applied to an electronic device including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected by lines; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method as in the first aspect and any possible design method thereof.

[0031] In a fourth aspect, the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect and any possible design thereof.

[0032] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method according to the first aspect and any possible design thereof.

[0033] It can be understood that the beneficial effects that can be achieved by the electronic device of the second aspect, the chip system of the third aspect, the computer storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a diagram of the software and hardware architecture of a mobile phone;

[0035] FIG2 is a schematic diagram of an image processing process;

[0036] Figure 3 is one of the problem diagrams using DCVS frequency modulation;

[0037] Figure 4 is a second schematic diagram of the problem of using DCVS frequency modulation;

[0038] Figure 5 is a third diagram of the problem of using DCVS frequency modulation;

[0039] FIG6 is a schematic diagram of a preset time T0 provided in an embodiment of the present application;

[0040] FIG7 is a schematic diagram showing the principle of determining the next refresh time T2 according to an embodiment of the present application;

[0041] FIG8 is a diagram showing the corresponding relationship between frequency levels and frequency values ​​provided in an embodiment of the present application;

[0042] FIG9 is a schematic diagram of the preset time T3 provided in an embodiment of the present application;

[0043] FIG10 is one of the example diagrams of frequency modulation provided in an embodiment of the present application;

[0044] FIG11 is a second example diagram of frequency modulation provided in an embodiment of the present application;

[0045] FIG12 is a third example diagram of frequency modulation provided in an embodiment of the present application;

[0046] FIG13 is a fourth example diagram of frequency modulation provided in an embodiment of the present application;

[0047] FIG14 is a fifth example diagram of frequency modulation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0049] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0050] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0051] The frequency modulation method provided in the embodiments of the present application can be applied to scenarios in electronic devices where the GPU is required to process images. This is particularly true in scenarios where the GPU is required to continuously process multiple frames of images. For example, when displaying desktop animations, the GPU needs to continuously process multiple animation frames; when playing games, the GPU needs to continuously process multiple frames of game images.

[0052] In the above scenario, by adopting the frequency modulation method provided in the embodiment of the present application, the electronic device can adjust the frequency of the GPU to meet the requirements of work efficiency and ensure that no frames are lost.

[0053] For example, the electronic device in the embodiments of the present application can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook computer, as well as a cellular phone, personal digital assistant (PDA), augmented reality (AR) or virtual reality (VR) device, etc., which supports display functions. The embodiments of the present application do not impose any special restrictions on the specific form of the electronic device. Hereinafter, the present application scheme will be mainly explained by taking the electronic device being a mobile phone as an example.

[0054] In the embodiment of the present application, the software system of the mobile phone can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture or a cloud architecture. The following embodiment will mainly take the Android system with a layered architecture as an example to illustrate the software and hardware architecture of the mobile phone 100.

[0055] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, as shown in Figure 1 , the software and hardware architecture of a mobile phone, from top to bottom, consists of the application layer (applications), application framework layer (framework), native layer (native), hardware abstraction layer (HAL), kernel layer (kernel), and hardware layer.

[0056] The application layer can host applications such as calls, memos, browsers, contacts, cameras, galleries, calendars, desktops, games, videos, and chats, though only a few are shown in the figure. These applications all require the phone's display to display images. For example, the phone's display may display application icons, game images, or video frames on the desktop. These images all require GPU processing (such as rendering and compositing), meaning they require GPU image processing.

[0057] The application framework layer provides an application programming interface (API) and programming framework for the application layer. Exemplarily, the application framework layer may include a window manager, content provider, view system, telephony manager, resource manager, notification manager, etc., only some of which are shown in the figure.

[0058] Furthermore, the application framework layer also includes a rendering thread (RenderThread). RenderThread can receive calls from upper-layer applications and complete image rendering, and then send the rendering results to the bottom layer (such as the local layer) for further processing.

[0059] The native layer provides various services to upper layers (such as the application framework layer). For example, the native layer includes image compositing services (SurfaceFlinger, SF), a three-dimensional (3D) graphics library (e.g., OpenGL ES), and a two-dimensional (2D) graphics engine (e.g., SGL). The figure shows only some of these.

[0060] Among them, SF can be used for refresh rate control. It is understandable that when an application in the application framework layer needs to display a new frame of image, it can request a vertical synchronization (vSync) signal through SF. The vSync signal can trigger the application to complete the rendering of a new frame of image, and then synthesize it and finally send it for display. Moreover, after the display screen of the electronic device is refreshed once, the vSync signal can be distributed through SF to the application that requested the vSync signal, that is, to the application with rendering requirements. In other words, SF can control the application to start rendering after the display screen is refreshed once by distributing the vSync signal from the display screen to the application. That is, refresh rate control is achieved.

[0061] In addition, SF can also be used to control the rendered content, such as layer stitching.

[0062] The Hardware Abstraction Layer (HAL) encapsulates the underlying hardware drivers and provides a general interface for upper layers to call these drivers, allowing them to drive the corresponding hardware. The HAL includes a hardware composer (HWC), which can perform image synthesis on the content processed by the SF.

[0063] It is understandable that SF can call the interface of the graphics library and complete image synthesis through the GPU. Obviously, completing image synthesis through the GPU requires GPU resources, so image synthesis through HWC can reduce the pressure on the GPU.

[0064] The kernel layer includes drivers that drive the hardware, such as display drivers, audio drivers, Bluetooth drivers, etc., of which only some are shown in the figure. Among them, the display driver is used to drive the display screen to display images.

[0065] Furthermore, the kernel layer also includes a GPU management module for executing the frequency modulation method provided in the embodiment of the present application to achieve frequency regulation of the GPU.

[0066] The hardware layer includes devices such as display screens and GPUs. The display screen can be used for image display, while the GPU can be used for image rendering and synthesis.

[0067] To facilitate understanding of the role of the GPU in processing a frame of image, the present embodiment of the present invention further illustrates the process of image rendering and display in an electronic device based on FIG1 and FIG2 :

[0068] 2 , the processing of a frame of image mainly includes: step 1, rendering; step 2, synthesis; and step 3, sending for display.

[0069] Step 1. Render.

[0070] After receiving the vSync signal from the SF, the application can start drawing a new frame of image. Specifically, the application submits a drawing instruction to the rendering thread (RenderThread).

[0071] After receiving a drawing command, the RenderThread processes it and sends it to the GPU. After receiving the processed drawing command, the GPU can perform the rendering process. That is, during the processing of a frame of image, the GPU can be used for image rendering.

[0072] Step 2: Synthesis.

[0073] The RenderThread can also send the resulting image content, such as the rendered image, to the SF after processing the drawing instructions. After receiving the image content, the SF determines whether it needs to be synthesized and displayed in the current frame. If it does, it processes the content, for example, by adjusting the refresh rate. Furthermore, if it doesn't need to be synthesized and displayed, it places it in a waiting queue and determines again whether it needs to be synthesized and displayed in the next frame.

[0074] Typically, HWC can only perform simple compositing tasks, while complex compositing requires the GPU. Therefore, SF can determine whether the current frame's image compositing requires GPU support. If so, SF can call the graphics library interface to use the GPU to complete the compositing tasks that HWC cannot complete. In other words, during the processing of a single frame, the GPU can also be used for image compositing.

[0075] It should be noted that during the execution of step 2, the GPU may not have completed image rendering. Therefore, the GPU needs to complete image synthesis after completing image rendering.

[0076] Step 3: Send and display.

[0077] After completing image synthesis, the HWC sends the synthesis results to the display driver. The display driver combines this synthesis result with the GPU's processing results (i.e., the GPU's image rendering and image synthesis results) to generate the image to be displayed and sends it to the display screen.

[0078] It should be noted that when the HWC completes image synthesis, the GPU may not have completed image rendering and image synthesis. Therefore, after receiving the synthesis result from the HWC, the display driver needs to wait for the GPU to complete image rendering and image synthesis before it can obtain the image to be displayed and send it to the display screen.

[0079] As can be seen, during the processing of a frame of image, the GPU needs to complete image rendering and image synthesis. If the GPU fails to complete these tasks in a timely manner, the display driver will not be able to obtain the image to be displayed and send it to the display screen in time, resulting in frame drop.

[0080] At the same time, the higher the GPU frequency, the greater the computing power it provides, and the more efficient image processing (including image rendering and image synthesis). Therefore, electronic devices can adjust the GPU frequency to make the GPU provide computing power that matches the current image processing task, thereby meeting work efficiency requirements (such as frame rate, refresh rate, etc.).

[0081] Usually, the GPU developer will provide a set of native frequency adjustment schemes for electronic devices equipped with the GPU. TMFor example, Qualcomm TM A native frequency scaling solution, Dynamic Clock and Voltage Scaling (DCVS), is provided. Electronic devices using DCVS can adjust the GPU's frequency level to the appropriate frequency level based on the GPU's usage over a period of time. GPU usage is defined as the percentage of the GPU's operating time over the total duration of that period.

[0082] It should be noted that the GPU frequency is usually expressed in frequency gears. Generally speaking, the higher the frequency gear number, the lower the GPU frequency; the lower the frequency gear number, the higher the GPU frequency. For example, the frequency gear includes 8 gears, 0-7, and the GPU frequency gradually decreases from 0 to 7. For example, gear 0 corresponds to a frequency of 680MHz, gear 6 corresponds to a frequency of 295MHz, and gear 7 corresponds to a frequency of 220MHz.

[0083] One implementation of using DCVS to adjust the GPU's frequency level is as follows: After the GPU's utilization rate remains above 90% for a certain period of time, the electronic device determines that the GPU is not efficient enough to complete the task at its current frequency level, so the electronic device increases the GPU's frequency level. When the GPU's frequency level is increased, the GPU provides greater computing power, so the GPU can complete tasks in a shorter time, and the GPU's utilization rate will decrease accordingly. Of course, if the GPU's utilization rate still cannot be reduced below 90%, the electronic device will continue to increase the GPU's frequency level until it is adjusted to a suitable frequency level that can keep the GPU's utilization rate stable below 90%.

[0084] Electronic devices using DCVS can adjust the GPU frequency to keep the GPU usage below 90%. However, using DCVS in electronic devices may cause the following problems:

[0085] Problem 1: Frame loss occurs.

[0086] Electronic devices using DCVS will only increase the GPU frequency level after the GPU usage rate remains above 90% for a certain period of time, that is, the frequency level adjustment is delayed.

[0087] Referring to Figure 3, before time t1, the GPU's frequency level is controlled at level 7, and the GPU utilization rate can be maintained below 90%, but close to 90%. At time t1, the GPU load (i.e., image processing tasks such as rendering and compositing) suddenly increases, and the GPU utilization rate also increases, for example, exceeding 90%. However, because the requirement for the GPU utilization rate to remain above 90% for a certain period of time is not met within a short period of time, the electronic device does not increase the GPU's frequency level. In other words, the GPU will continue to operate at level 7 for a short period of time (e.g., between time t1 and time t2). During this period, the low frequency may prevent the GPU from efficiently completing tasks, resulting in frame drops, as shown in Figure 3, where two frames are dropped between time t1 and time t2. After a period of time, such as at time t2, the electronic device detects that the GPU utilization rate has remained above 90% for a certain period of time, at which point the frequency level is increased, for example, from level 7 to level 0. Only then can the GPU complete each frame of image processing in a timely manner, without frame drops. However, due to the lag in frequency adjustment, the frame loss between time t1 and time t2 cannot be recovered.

[0088] Problem 2: The frequency level may not be increased even after frame loss, which may lead to continued frame loss.

[0089] GPU utilization is calculated over a period of time. If the GPU is operating for a short period of time, the utilization is low; if the GPU is operating for a long period of time, the utilization is high. However, historical utilization does not accurately reflect the performance of real-time image processing tasks. This can lead to inaccurate frequency control in electronic devices, resulting in continuous frame dropouts.

[0090] Referring to Figure 4, between time t3 and time t4, the frequency gear of the GPU is controlled at gear 7, and the GPU utilization rate can be maintained below 90%, but frame drops occur, such as a total of 3 frames are dropped. Subsequently, at time t4, the electronic device calculates the GPU utilization rate between time t3 and time t4. Since frame drops occur between time t3 and time t4, the calculated utilization rate will not be very high, such as still maintaining below 90%. Therefore, the electronic device will not increase the frequency gear of the GPU. That is, the GPU will still work at gear 7, such as working at gear 7 between time t4 and time t5, and will continue to drop frames, such as a total of 2 frames are dropped.

[0091] Question 3: It can easily lead to unstable performance of electronic equipment.

[0092] Electronic devices using DCVS will only increase the frequency level when GPU utilization reaches above 90%. In practice, it's common to see utilization approaching 90% but not exceeding it (referred to as the "dangerous level"). When utilization reaches the dangerous level, the GPU's performance can fluctuate due to overall system factors, such as the device's battery level and temperature. This can cause unstable performance, such as occasional frame drops.

[0093] As shown in Figure 5, between times t6 and t7, the GPU utilization fluctuated between 85% and 90%. Because the utilization rate did not remain above 90% for a sustained period, the electronic device did not adjust the GPU frequency level, and the GPU remained operating at the same frequency level, such as level 7. However, between times t6 and t7, frame drops occurred intermittently, as shown in Figure 5, where two frames are dropped.

[0094] In addition to the aforementioned DCVS solution, GPU developers also provide electronic device developers with a customized performance locking (Perflock) solution, allowing them to more flexibly adjust the GPU's frequency level, overcoming the aforementioned DCVS issues to some extent. The key idea behind Perflock is that electronic devices can add points for problematic scenarios requiring frequency adjustment. These points can then be used to identify the problematic scenarios and, using an interface provided by the GPU developer, adjust the GPU's frequency level accordingly.

[0095] However, Perflock is mainly applicable to some problem scenarios that inevitably require frequency modulation (also called must-occur scenarios). For some non-must-occur scenarios, that is, problem scenarios that may require frequency modulation in some cases but may not require frequency modulation in other cases, the use of Perflock in electronic devices often results in a waste of resources.

[0096] A typical scenario is as follows: When the screen refresh rate or application frame rate of an electronic device is high, the GPU is usually required to complete image processing tasks for more frames of images per unit time. Therefore, the computing power requirements for the GPU are high, which requires the GPU to work at a higher frequency gear. Therefore, electronic devices can use Perflock to set stake points for scenarios with high screen refresh rates (such as 120 Hz), that is, to mark the need to increase the frequency gear of the GPU when the screen refresh rate reaches 120Hz. Subsequently, during the GPU's image processing process, if the electronic device detects that the screen refresh rate reaches 120Hz, the frequency gear of the GPU can be increased.

[0097] However, in actual applications, when the screen refresh rate of an electronic device reaches 120Hz, it is not necessarily necessary to increase the frequency level of the GPU. For example, although the screen refresh rate of an electronic device reaches 120Hz, there are not many image processing tasks that need to be completed by the GPU. For example, there is no need for the GPU to complete complex synthesis processing. Therefore, even if the frequency level of the GPU is not increased, the GPU can complete the current image processing task in a timely manner, which is a non-essential scenario. In this scenario, if Perflock is still used and the frequency level of the GPU is increased when the screen refresh rate reaches 120Hz, it may lead to a waste of resources.

[0098] Based on the above-mentioned problems with DCVS and Perflock, an embodiment of the present application provides a frequency modulation method that can be used for GPU frequency modulation.

[0099] Using an embodiment of the present application, during the processing of a frame of image, the electronic device (such as the GPU management module in the kernel layer) can detect whether the end time (also referred to as the first end time) of the GPU completing image processing (including rendering and synthesis) exceeds the preset time T0 (also referred to as the first time). If the end time exceeds the preset time T0, it indicates that the GPU cannot complete the processing of the current frame image in time. The electronic device can increase the frequency gear of the GPU to ensure that the GPU completes the processing of the current frame image in time to avoid frame loss.

[0100] This allows the electronic device to increase the GPU frequency if the GPU cannot complete image processing in the current frame in a timely manner. This allows the GPU to process the current frame in a timely manner, ensuring no frame drops. Furthermore, the electronic device will not increase the GPU frequency if the GPU can complete image processing in a timely manner, thus avoiding wasted resources.

[0101] In some embodiments, the preset time T0 can be a fixed time duration 1 before the next refresh of the display screen. It will be appreciated that each time the display screen is refreshed, a new frame of image must be displayed. Therefore, the GPU must complete processing of the current frame before the next refresh, and the preset time T0 must also be before the next refresh. In this way, the electronic device can quickly determine the preset time T0 based on the next refresh time and the fixed time duration 1.

[0102] In other embodiments, considering that the processing progress of different frame images by the electronic device may be different, the preset time T0 may also be a time point that matches the processing progress of the current frame image.

[0103] Specifically, the preset time T0 is the time duration (denoted as duration L) between the time T1 (also referred to as the second time) when the display driver begins waiting for the GPU to complete image processing and the next refresh time T2, at which the ratio of the time duration from T1 to duration L is a preset ratio λ1 (e.g., 0.7, 0.8, etc., also referred to as the first ratio). Regarding the display driver waiting for the GPU to complete image processing, please refer to the description of step 3 in FIG. 2 above and will not be repeated here.

[0104] Taking the preset proportion λ1 as 0.7 as an example, in the process of processing the k-th frame image shown in FIG6 , the electronic device may determine the preset time T0 as the moment T1+0.7*(T2-T1).

[0105] It can be seen that, by adopting this embodiment, the electronic device can determine the matching preset time T0 according to the processing progress of the current frame image, so as to adjust the frequency level of the GPU in a targeted manner.

[0106] It's also important to emphasize that when the display driver begins waiting for the GPU to complete image processing, the CPU has essentially completed the image processing required for display. Whether frame drops occur after this point depends primarily on the GPU's efficiency. Therefore, by configuring the preset time T0 as the time after the display driver begins waiting for the GPU to complete image processing, the GPU frequency can be adjusted to accurately prevent frame drops, while being affected only by the GPU's efficiency.

[0107] The following describes the specific implementation of determining the time T1 and the next refresh time T2:

[0108] First, determine time T1.

[0109] In practice, the display driver can wait for the GPU to complete image processing through a wait function (such as plane_wait_input_fence). It is understood that after the GPU completes image processing, it releases the corresponding fence. Accordingly, when the display driver detects that the fence has been released through the wait function, it can determine that the GPU has completed image processing. At this point, the display driver can execute the subsequent display processing and drive the display screen to display the current frame image. Based on this, the electronic device can determine that time T1 is the time when the display driver begins executing the wait function.

[0110] Second, determine the next refresh time T2.

[0111] While the display screen is displaying the k-1th frame (also called the first image), the electronic device needs to complete the processing of the kth frame (also called the second image), such as rendering, compositing, and displaying. This allows the display screen to refresh and display the kth frame at the next refresh time. Therefore, the electronic device needs to complete image processing one frame in advance.

[0112] The display driver can complete image processing one frame ahead of time, based on the display's hardware interrupt (TE, which can be understood as the refresh period). As shown in Figure 7, between TE0 and TE1, the display displays frame 0, and the display driver processes frame 1. Between TE1 and TE2, the display displays frame 1, and the display driver processes frame 2. Between TE(n-1) and TEn, the display displays frame n-1, and the display driver processes frame n. The time interval during which the display displays one frame is referred to as the first time interval.

[0113] In some embodiments, the electronic device may use the time when the display driver completes processing the previous frame of the image as the approximate start time of the current TE period (also referred to as the fifth time). Taking the current TE period as TE1 to TE2 shown in Figure 7 as an example, the time when the display driver completes processing the first frame of the image is Tlast. The electronic device may determine Tlast as the start time of the TE1 to TE2 period, i.e., TE1.

[0114] It is understandable that if the first frame of the image is dropped, it is simply a delayed response, which can also be understood as poor hand tracking, and will not significantly affect user experience. Therefore, the electronic device can use the solution of this application to adjust the GPU frequency for the second frame of the image and subsequent images. Then, the time when the display driver completes processing the first frame of the image is the first Tlast, such as TE1 in Figure 7.

[0115] In practice, after the display driver completes processing of a frame of image, it can indicate the completion of processing of a frame of image by using an end function (such as sde_kms_complete_commit). Based on this, the electronic device can use the time when the display driver executes the end function as the start time of the current TE cycle.

[0116] It can be understood that after each refresh, the display screen will send a vSync signal to trigger the processing of the next frame of image, that is, a refresh cycle is the time interval between two vSync signals, which can be recorded as △vSync (also called refresh cycle). For example, if the refresh rate is 120Hz, then △vSync = 8.3333ms; if the refresh rate is 90Hz, then △vSync = 11.1111ms; if the refresh rate is 60Hz, then △vSync = 16.6666ms. Then, the electronic device adds △vSync to the moment when the display driver finishes processing the previous frame of image, and the end moment of the current TE cycle (also called the sixth time) can be obtained, which is the next refresh moment T2. Still taking the current TE cycle as TE1 to TE2 shown in Figure 7 as an example, the moment when the display driver finishes processing the first frame of image is Tlast, then the electronic device can determine the next refresh moment T2 = Tlast + △vSync.

[0117] Of course, the electronic device can also record the latest refresh time and add △vSync to the latest refresh time to obtain the next refresh time T2.

[0118] After determining the time T1 and the next refresh time T2, the electronic device can determine the preset time T0 based on the time T1 and the next refresh time T2. Subsequently, the electronic device can detect whether the end time of the GPU completing image processing exceeds the preset time T0.

[0119] After time T1: If the display driver is still executing the wait function, it indicates that the display driver is still waiting for the GPU to complete image processing. In other words, the GPU has not yet completed image processing. Conversely, if the display driver has finished executing the wait function, it indicates that the display driver has waited until the GPU completes image processing. In other words, the GPU has completed image processing. Based on this, after time T1, the electronic device can detect whether the GPU has completed image processing by detecting whether the display driver is still executing the wait function.

[0120] In one specific implementation, the electronic device may detect whether the display driver is executing a wait function after a preset time T0 is reached. If the display driver is still executing the wait function at the preset time T0, the electronic device may determine that the GPU has completed image processing at a time exceeding the preset time T0. If the display driver has finished executing the wait function at the preset time T0, the electronic device may determine that the GPU has completed image processing at a time within the preset time T0.

[0121] In practice, after detecting that the display driver has started executing the wait function, the electronic device calculates the time difference between the preset time T0 and the current time (such as time T1), and sets a timer with a timing duration equal to the time difference. Subsequently, the electronic device can detect whether the display driver is executing the wait function when the timer expires. The end of the timer is the preset time, so the electronic device detects whether the display driver is executing the wait function when the timer expires, thereby detecting whether the display driver is executing the wait function after the preset time T0 is reached.

[0122] After determining that the end time of the GPU completing image processing exceeds the preset time T0, the electronic device can increase the frequency gear of the GPU. Exemplarily, the electronic device can increase the frequency gear of the GPU to the highest gear to ensure that the GPU can complete the image processing of the current frame before the next refresh time T2, thereby ensuring that the current frame is not lost. Of course, the electronic device can also increase the frequency gear of the GPU by a preset number of gears, such as increasing 3 frequency gears. The embodiment of the present application does not specifically limit this. hereinafter, the explanation is mainly based on the example of increasing to the highest gear.

[0123] Increasing the GPU frequency level includes increasing the minimum frequency (also called the minimum frequency) or the actual frequency (also called the actual operating frequency). It is understood that the actual value of the GPU frequency must be greater than or equal to the minimum value, that is, the GPU must operate at or above the minimum frequency.

[0124] After increasing the minimum value, the actual value of the GPU frequency will also change to the increased minimum value, so as to ensure that the actual value is greater than or equal to the minimum value. The GPU can then operate at the frequency gear corresponding to the increased minimum value.

[0125] Taking the correspondence between frequency levels and frequency values ​​shown in Figure 8 as an example, if before the increase, the minimum value of the frequency is f7 and the actual value is also f7, that is, the minimum value and the actual value are both in level 7; the electronic device increases the minimum value to f0, that is, the minimum value is increased to level 0; accordingly, the actual value of the GPU frequency will also increase to f7, that is, the actual value is also increased to level 0.

[0126] After increasing the actual value, since the minimum value of the GPU frequency does not change, the electronic device can use DCVS to control the GPU frequency after the actual value of the GPU frequency is increased. In other words, the actual value of the GPU frequency may not remain stable after the increase.

[0127] Using the relationship between frequency levels and values ​​shown in Figure 8 as an example, if the minimum frequency value before the increase is f7 and the actual value is also f7, meaning both the minimum and actual values ​​are in level 7, then the electronic device increases the actual value to f0, meaning the actual value is increased to level 0. However, since the minimum value still corresponds to level 7 and GPU utilization is gradually decreasing, the electronic device can use DCVS to gradually lower the actual value to f1, f2, and so on. It should be noted that when using DCVS, the electronic device also controls the actual value by adjusting the minimum value.

[0128] The following will explain how to increase the minimum value and the actual value respectively:

[0129] Method 1: Increase the minimum value.

[0130] After increasing the minimum GPU frequency, the GPU can operate at the frequency level corresponding to the increased minimum value for a long period of time, such as operating at level 0. However, operating the GPU at a higher frequency level for a long period of time may result in resource waste. Therefore, after increasing the minimum value, during subsequent frame image processing, if the electronic device detects that the GPU can efficiently complete image processing, it can reduce the GPU frequency level to avoid resource waste.

[0131] Similarly, lowering the GPU frequency level includes: lowering the minimum value.

[0132] After the minimum value is lowered, the actual value of the GPU frequency will also change to the lowered minimum value, and the GPU can then operate at the frequency gear corresponding to the lowered minimum value. Taking the correspondence between frequency gears and frequency values ​​shown in Figure 8 as an example, if the minimum value of the frequency before the reduction is f0 and the actual value is also f0, that is, the minimum value and the actual value are both in gear 7; the electronic device reduces the minimum value to f1, that is, the minimum value is reduced to gear 1; subsequently, the actual value of the GPU frequency will also be reduced to f1, that is, the actual value is also reduced to gear 1.

[0133] However, reducing the GPU frequency level usually does not include reducing the actual value. The reason is as follows: if the actual value and the minimum value are the same, reducing the actual value will cause the actual value to be lower than the minimum value, which does not meet the requirement that the actual value of the GPU frequency should be greater than or equal to the minimum value.

[0134] In some embodiments, after increasing the minimum value of the GPU frequency, the electronic device may detect whether the end time of the GPU completing image processing exceeds a preset time T3 (also referred to as a third time) during subsequent image processing. If the end time does not exceed the preset time T3, indicating that the GPU can efficiently complete image processing, the electronic device may lower the minimum value, and the actual value of the subsequent GPU frequency will gradually decrease to the minimum value, thereby saving resources while the GPU can efficiently complete image processing.

[0135] Similar to the preset time T0, the preset time T3 may be a time point at which the time from the next refresh time T2 is a fixed time 2, where the fixed time 2 is greater than the fixed time 1. In this way, the electronic device can also quickly determine the preset time T3 based on the next refresh time T2 and the fixed time 2.

[0136] Alternatively, the preset time T3 may be a time point in the duration L at which the ratio of the time length from the moment T1 to the duration L is a preset ratio λ2 (eg, 0.5, 0.6, etc.), and the preset ratio λ2 is less than the preset ratio λ1.

[0137] Taking the preset proportion λ2 of 0.5 as an example, during the processing of the second frame image shown in Figure 9 (such as at the preset time T0), the electronic device increases the minimum value of the GPU frequency to the frequency value of level 0; subsequently, during the processing of the third frame image, the electronic device can determine the preset time T3 as the moment T1+0.5*(T2-T1).

[0138] Regarding the determination of the time T1 and the next refresh time T2, reference may be made to the above description of the determination of the preset time T0, which will not be repeated here.

[0139] Similarly, after time T1, the electronic device can detect whether the GPU has completed image processing by detecting whether the display driver is executing a wait function. In one specific implementation, the electronic device can detect whether the display driver is executing a wait function after reaching a preset time T3. If the display driver is still executing the wait function at the preset time T3, the electronic device can determine that the GPU has completed image processing for a time exceeding the preset time T3; if the display driver has finished executing the wait function at the preset time T3, the electronic device can determine that the GPU has completed image processing for a time not exceeding the preset time T3.

[0140] 9 , at the preset time T3 , the display driver has finished executing the waiting function, and the electronic device can determine that the end time of the GPU completing the image processing does not exceed the preset time T3 .

[0141] In the case that the end time of the GPU completing the image processing does not exceed the preset time T3, the electronic device can reduce the minimum value of the GPU frequency to the frequency value of a frequency gear one level lower than the current frequency gear. Still taking the processing process of the third frame image shown in Figure 9 as an example, the end time of the GPU completing the image processing does not exceed the preset time T3, and the electronic device can reduce the minimum value of the GPU frequency to the frequency value of gear 1. Alternatively, the electronic device can also reduce the minimum value to the frequency value of a frequency gear that is a preset number (such as 2, 3, etc.) levels lower than the current frequency gear. In the following, the example of reducing the minimum value to the frequency value of a frequency gear that is one level lower than the current frequency gear is mainly used for explanation.

[0142] After lowering the minimum value of the GPU frequency, the GPU can operate at the frequency gear corresponding to the lowered minimum value for a long time, such as operating at gear 1 for a long time. During the processing of subsequent frame images, the electronic device can continue to detect whether the frequency gear is appropriate and adjust the minimum value if it is detected to be inappropriate. Among them, the inappropriate frequency gear includes that the end time of the GPU completing image processing does not exceed the preset time T3 (which will result in resource waste) or the end time of the GPU completing image processing exceeds the preset time T0 (which will result in frame loss).

[0143] As can be seen, in the first approach, during the processing of each image frame, the electronic device can detect whether the time it takes for the GPU to complete image processing exceeds the preset time T3. If it does not exceed the preset time T3, the minimum value of the GPU frequency is reduced, such as to a frequency value of a lower frequency gear. Alternatively, the electronic device can detect whether the time it takes for the GPU to complete image processing exceeds the preset time T0. If it does exceed the preset time T0, the minimum value of the GPU frequency is increased, such as to a frequency value of the highest gear.

[0144] Furthermore, since the preset proportion λ1 is greater than the preset proportion λ2, this indicates that during the processing of a single image frame, the preset time T0 is closer to the next refresh time T2, while the preset time T3 is closer to time T1. That is, from front to back, the sequence is: time T1, preset time T3, preset time T0, and next refresh time T2. Therefore, during the processing of a single image frame, if the electronic device detects that the GPU has completed image processing within the preset time T3, it can omit the step of detecting whether the GPU has completed image processing beyond the preset time T0, thereby avoiding invalid detection and conserving computing resources.

[0145] At this point, it's important to note that after an electronic device lowers the minimum value of the GPU frequency, even if the actual value of the GPU frequency slowly drops to the minimum value, it still satisfies the requirement that the actual value is greater than or equal to the minimum value. Furthermore, the slow drop also ensures that the actual value of the GPU frequency doesn't suddenly change significantly. Therefore, after lowering the minimum value of the GPU frequency, it takes a while for the actual value to drop to equal the minimum value. In other words, only after a delay does the GPU actually operate at the reduced minimum value, i.e., only after the delay does the setting take effect. For example, it takes 100ms for the setting to take effect.

[0146] Referring to Figure 10, before TE0, the actual value and minimum value of the GPU frequency are both the frequency value f7 of gear 7. The end time of the GPU completing image processing between TE0-TE1 exceeds the preset time T0 between TE0-TE1, and the electronic device increases the minimum value of the GPU frequency to the frequency value f0 of gear 0, and the actual value of the GPU frequency value will also quickly (such as at the preset time T0) increase to the frequency value f0. Subsequently, at the preset time T3 between TE1-TE2, the electronic device reduces the minimum value of the GPU frequency to the frequency value f1 of gear 1, but the actual value of the GPU frequency is not reduced to the frequency value f1 until time T4. Subsequently, at the preset time T3 between TE2-TE3, the electronic device further reduces the minimum value of the GPU frequency to the frequency value f2 of gear 2, but the actual value of the GPU frequency is not reduced to the frequency value f2 until time T5.

[0147] That is to say, within the delay duration (also called duration threshold), such as between the preset time T3 and moment T4 between TE1-TE2 and between the preset time T3 and moment T5 between TE2-TE3 in Figure 10, the actual value of the GPU frequency has not actually dropped to the same as the minimum value, and only after the delay duration is reached, it is actually dropped to the same as the minimum value.

[0148] In the example of Figure 10 above, the delay time is relatively short. The electronic device reduces the minimum value of the GPU frequency during the processing of the current frame image, and in the first half of the processing of the next frame image, the actual value of the GPU frequency will be reduced to the same as the minimum value. For example, the electronic device reduces the minimum value to the frequency value f1 at the preset time T3 between TE1-TE2 in Figure 10, and then at the moment T4 in the first half of the stage between TE2-TE3, the actual value of the GPU frequency will also be reduced to the frequency value f1. In this case, the electronic device detects whether the actual value after taking effect is appropriate at the preset time T0 or the preset time T3 during the processing of each frame image.

[0149] In practice, the delay may be shorter or longer. If the delay is longer, the preset time T0 or T3 during the processing of the next frame or frames after the minimum value is lowered may fall within the delay, meaning the actual value has not yet fallen to the minimum value. Therefore, during the preset time T0 or T3 within the delay, it is impossible to determine whether the actual value after the effectivity is appropriate.

[0150] Based on this, in some embodiments, after the electronic device reduces the minimum value of the GPU frequency (which may also be referred to as the fourth time), it will detect whether the frequency gear is appropriate after the delay time is reached. That is, within the delay time, even if the preset time T0 or the preset time T3 is reached, it will not detect whether the frequency gear is appropriate, let alone adjust the frequency gear. In a specific implementation method, after the electronic device reduces the minimum value of the GPU frequency, it can set a timer with a timing duration of the delay duration. Before the timer expires, when the preset time T0 or the preset time T3 is reached, the electronic device does not detect whether the frequency gear is appropriate; after the timer expires, when the preset time T0 or the preset time T3 is reached, the electronic device detects whether the frequency gear is appropriate. In this way, the electronic device can avoid detecting and adjusting the frequency gear when it is not effective.

[0151] For example, during the preset time T3 between TE2 and TE3 shown in FIG11 , the minimum value of the GPU frequency of the electronic device is reduced to the frequency value of level 2. However, during the delay period (e.g., denoted as <Δt), the actual value does not actually reduce to the frequency value of level 2. For example, if the time interval between the preset time T3 between TE3 and TE4 and the preset time T3 between TE2 and TE3 shown in FIG11 is <Δt, this indicates that during the preset time T3 between TE3 and TE4, the actual value of the GPU frequency has not yet reduced to the frequency value of level 2. Therefore, even if the end time of the GPU completing image processing between TE3 and TE4 does not exceed the preset time T3 between TE3 and TE4, as shown in FIG11 , this does not mean that the actual value of the GPU frequency can efficiently complete image processing at the frequency value of level 2. Therefore, the electronic device does not need to detect whether the end time of the GPU completing image processing exceeds the preset time T3 between TE3 and TE4, and will not further reduce the minimum value, thereby avoiding incorrect frequency level adjustment.

[0152] Continuing with FIG11 , the time interval between the preset time T3 between TE4 and TE5 and the preset time T3 between TE2 and TE3 shown in FIG11 exceeds the delay duration (e.g., represented as >Δt), indicating that at the preset time T3 between TE4 and TE5, the actual value of the GPU frequency has been reduced to the frequency value of level 2. Therefore, at the preset time T3 between TE4 and TE5, the electronic device can detect whether the end time of the GPU completing image processing exceeds the preset time T3 between TE4 and TE5. Furthermore, if, as shown in FIG11 , the end time of the GPU completing image processing does not exceed the preset time T3 between TE4 and TE5, the electronic device can continue to reduce the minimum value of the GPU frequency, such as to the frequency value of level 3.

[0153] During the processing of any image frame, if the electronic device detects that the GPU completes image processing after a predetermined time T3 but not before a predetermined time T0, the GPU operating at the current frequency level can ensure timely completion of image processing without wasting resources. In other words, the current frequency level is appropriate. In this case, the electronic device does not need to increase or decrease the minimum value.

[0154] Furthermore, if the electronic device detects that the GPU completes image processing for a period exceeding the preset time T3 and within the preset time T0 during the processing of y consecutive image frames (y ≥ 1, where y is an integer and may also be referred to as a first number), the current frequency level can maintain the GPU's image processing efficiency within a reasonable range. Subsequently, the electronic device may employ DCVS to assist in adjusting the GPU frequency.

[0155] Taking y=3 as an example, after the electronic device reduces the minimum value of the GPU frequency to the frequency value f2 shown in Figure 12, within three consecutive TE cycles of TE3-TE4, TE4-TE5, and TE5-TE6, that is, during the processing of three consecutive frames of images, the electronic device detects that the end time of the GPU completing image processing exceeds the preset time T3 and does not exceed the preset time T0. In this case, the electronic device can use DCVS frequency modulation after TE6.

[0156] Of course, if the electronic device subsequently detects that the end time of the GPU completing image processing exceeds the preset time T0, the above method 1 can be repeated again until a suitable frequency gear is found.

[0157] Method 2: Increase the actual value.

[0158] Unlike Method 1, after increasing the actual value of the GPU frequency, the actual value will automatically change under DCVS control, such as decreasing, without the electronic device further reducing the minimum value based on the GPU's completion time for image processing, thereby reducing the actual value. Therefore, in Method 2, the focus is on increasing the actual value.

[0159] In some embodiments, after the electronic device detects that the GPU has completed image processing for a period exceeding a preset time T0, it increases the actual value of the GPU frequency to a maximum frequency value, such as frequency value f0. Subsequently, the electronic device may use DCVS to control the actual value of the GPU frequency. After the electronic device again detects that the GPU has completed image processing for a period exceeding the preset time T0, it may again increase the actual value of the GPU frequency to a maximum frequency value.

[0160] This ensures that the GPU can complete image processing in a timely manner and avoid frame drops. It also ensures that the GPU frequency level matches the usage rate.

[0161] For example, the electronic device detects that the end time of the GPU completing image processing exceeds the preset time T0 in the corresponding TE cycle at the preset time T0 between TEx1-TEx2, TEx3-TEx4, and TEx5-TEx6 shown in Figure 13. Therefore, the electronic device can increase the actual value of the frequency to the highest frequency value, such as f0, at the preset time T0 between TEx1-TEx2, TEx3-TEx4, and TEx5-TEx6. Between each two increases in the actual value to the highest frequency value, the electronic device can use DCVS to adjust the actual value of the GPU frequency. For example, as the utilization rate of the GPU gradually decreases, the electronic device can gradually lower the actual value using DCVS.

[0162] It should be noted that, in the example shown in FIG13 , the electronic device uses DCVS to gradually reduce the actual value from the frequency value f0 to the frequency value f7.

[0163] However, in actual implementation, electronic devices can adjust based on actual usage and are not limited to Figure 13. Referring to Figure 14, after the electronic device increases the actual frequency value to f0 during the preset time T0 between TEx1-TEx2, TEx3-TEx4, TEx5-TEx6, and TEx7-TEx8, the electronic device uses DCVS to gradually adjust the actual value from f0 to f6, f5, f5, and f5, respectively. In other words, the frequency is not always adjusted down to f7.

[0164] Furthermore, in this embodiment, if after the actual value is increased to f0 for z consecutive times (also referred to as a preset number of times), the electronic device uses DCVS to adjust the actual value from the frequency value f0 to the same frequency value (also referred to as the first value), the electronic device can increase the minimum value to a frequency value that is one frequency level higher than the same frequency value (also referred to as the second value), so that the GPU can operate at a frequency that completes the image processing earlier than the preset time T0.

[0165] Taking z = 3 as an example, as shown in Figure 14, after the electronic devices TEx3-TEx4, TEx5-TEx6, and TEx7-TEx8 increase their actual values ​​to f0 three times in a row during the preset time T0, the electronic devices use DCVS to adjust the actual values ​​from frequency f0 to frequency f5, i.e., frequency f5 is the same frequency value as above. Therefore, the electronic devices can increase the minimum value to a frequency value one frequency step higher than frequency f5, such as f4.

[0166] An embodiment of the present application further provides an electronic device, which may include: a display screen, a memory, and one or more processors (such as a CPU, GPU, NPU, etc.). The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform the various functions or steps performed by the device in the above method embodiment.

[0167] An embodiment of the present application further provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as a processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiment of the present application.

[0168] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0169] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0170] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the image processing method in the above-mentioned method embodiments.

[0171] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0172] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0175] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A frequency modulation method, characterized in that: Applied to an electronic device, the electronic device includes a graphics processor GPU, and the method includes: The electronic device displays a first image; In a first time interval when the first image is displayed: the electronic device detects whether a first end time when the GPU completes processing of a second image exceeds a first time, the second image and the first image are two consecutive frames of images, and the first time is within the first time interval; if the first end time exceeds the first time, the electronic device increases the frequency of the GPU; After the GPU completes processing of the second image, the electronic device displays the second image.

2. The method according to claim 1, characterized in that: The electronic device also includes a display driver and a hardware overlay HWC. The first time is after the second time when the display driver starts waiting for the GPU to complete processing of the second image. After receiving the synthesis result of the second image from the HWC, the display driver starts waiting for the GPU to complete processing of the second image.

3. The method according to claim 2, characterized in that The electronic device detects whether a first end time at which the GPU completes processing of the second image exceeds a first time, including: At the first time, the electronic device detects whether the display driver is waiting for the GPU to complete processing of the second image; If the display driver is waiting for the GPU to complete processing of the second image, the first end time exceeds the first time; If the display driver is not waiting for the GPU to complete processing of the second image, the first end time does not exceed the first time.

4. The method according to claim 2 or 3, characterized in that: The second time includes: the time when the display driver starts to execute the waiting function; The waiting function is used by the display driver to wait for the GPU to complete processing of the second image. After the GPU completes processing of the second image, the display driver ends executing the waiting function.

5. The method according to any one of claims 1 to 4, characterized in that The electronic device increases the frequency of the GPU, including: The electronic device increases the minimum frequency of the GPU; The method further comprises: In a first time interval during which the first image is displayed: the electronic device detects whether the first end time exceeds a third time, and if the first end time does not exceed the third time, the electronic device reduces the frequency of the GPU; The third time is within the first time interval, and the third time is before the first time.

6. The method according to claim 5, characterized in that The electronic device detects whether a first end time at which the GPU completes processing of the second image exceeds a first time, including: If the first end time exceeds the third time, the electronic device detects whether the first end time at which the GPU completes processing of the second image exceeds the first time.

7. The method according to claim 5 or 6, characterized in that: The electronic device detects whether a first end time at which the GPU completes processing of the second image exceeds a first time, including: If the time interval between the first time and the fourth time is greater than the time threshold, the electronic device detects whether the first end time exceeds the first time; The electronic device detects whether the first end time exceeds a third time, including: If the time interval between the third time and the fourth time is greater than the time threshold, the electronic device detects whether the first end time exceeds the third time; The fourth time is the time when the electronic device last reduced the frequency of the GPU before displaying the first image.

8. The method according to any one of claims 5 to 7, characterized in that: If the first end time exceeds the third time but does not exceed the first time, the electronic device does not adjust the frequency of the GPU.

9. The method according to claim 8, characterized in that The method further comprises: If the first end time corresponding to at least a first number of consecutive images exceeds the third time but does not exceed the first time, the electronic device uses dynamic clock and voltage regulation DCVS to adjust the frequency of the GPU.

10. The method according to any one of claims 1 to 4, characterized in that The electronic device increases the frequency of the GPU, including: The electronic device increases the actual operating frequency of the GPU; After the electronic device increases the frequency of the GPU, the method further includes: The electronic device adjusts the frequency of the GPU using dynamic clock and voltage scaling DCVS.

11. The method according to claim 10, characterized in that The method further comprises: If the frequency of the GPU is increased for more than a preset number of consecutive times, the electronic device uses the DCVS to reduce the frequency of the GPU to a first value, and the electronic device increases the minimum frequency of the GPU to a second value, and the frequency gear to which the second value belongs is one gear higher than the frequency gear to which the first value belongs.

12. The method according to any one of claims 2 to 4, characterized in that: The electronic device also includes a display screen; The electronic device displays a first image, including: At a fifth time after the GPU completes processing of the first image, the display driver drives the display screen to display the first image; The method further comprises: The electronic device acquires the fifth time and the second time; The electronic device adds the fifth time to the refresh period of the display screen to obtain a sixth time; The electronic device determines, as the first time point, a time point between the second time and the sixth time at which a ratio of the time interval with the second time to the time interval between the second time and the sixth time is a first ratio.

13. An electronic device, characterized in that: include: A display screen, one or more processors, and one or more memories; the one or more processors are coupled to the display screen and the one or more memories; the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method described in any one of claims 1-12.

14. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 12.

Images