Frequency modulation method and electronic equipment

Abstract

The invention discloses a frequency modulation method and electronic equipment, relates to the technical field of terminals, and can adjust the frequency of a GPU for the processing progress of a current frame image based on the GPU to ensure no frame loss. The method comprises the following steps: the electronic equipment displays a first image; in a first time interval for displaying the first image, the electronic device detects whether first end time for the GPU to complete processing of the second image exceeds first time or not, the second image and the first image are two continuous frames of images, and the first time is in the first time interval. And if the first end time exceeds the first time, the electronic equipment improves the frequency of the GPU. And after the GPU completes the processing of the second image, the electronic equipment displays the second image.

Description

[0001] This application is a divisional application. The application number of the original application is 202311240144.X, the original application date is September 22, 2023, and the entire content of the original application is incorporated herein by reference. Technical Field

[0002] Embodiments of this application relate to the technical field of terminals, and in particular, to a frequency modulation method and an electronic device. Background Art

[0003] In electronic devices such as mobile phones and tablets, a Graphics Processing Unit (GPU) is an important device for completing image processing. Among them, image processing may include image rendering and image synthesis. Usually, the higher the frequency of the GPU, the stronger the computing power provided, and the higher the efficiency of image processing. The higher the frequency of the GPU, the weaker the computing power provided by the GPU, and correspondingly, the lower the image processing efficiency. Therefore, an electronic device can adjust the frequency of the GPU to meet the efficiency requirements of image processing.

[0004] In the prior art, an electronic device can select an appropriate frequency according to the usage rate of the GPU (GPU_busy). Exemplarily, if the usage rate exceeds 90% and lasts for a certain period of time, it indicates that when the GPU runs at the current frequency, the idle time is extremely short. Therefore, the electronic device can increase the frequency of the GPU so that the GPU can have more idle time to meet more efficient processing requirements.

[0005] However, when the electronic device adopts the above prior art, it increases the frequency of the GPU only after the usage rate exceeds 90% and lasts for a certain period of time. The frequency increase has hysteresis and does not meet the processing requirements of real-time image processing, which may cause frame drops. Summary of the Invention

[0006] This application provides a frequency modulation method and an electronic device, which can adjust the frequency of the GPU based on the processing progress of the GPU for the current frame image to ensure no frame drops.

[0007] To achieve the above object, this application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a frequency modulation method applied to an electronic device, where the electronic device includes a Graphics Processing Unit (GPU). The electronic device displays a first image. During a first time interval when the first image is being displayed: the electronic device detects whether a first end time when the GPU finishes processing a second image exceeds a first time. The second image and the first image are two consecutive frames of images, and the first time is within the first time interval. If the first end time exceeds the first time, the electronic device increases the frequency of the GPU. After the GPU finishes processing the second image, the electronic device displays the second image.

[0009] That is to say, when the electronic device is displaying the first image (i.e., the previous frame of the two consecutive frames of images), the electronic device can process the second image (i.e., the next frame of the two consecutive frames of images). And if during the processing, the GPU does not finish image processing (such as image rendering and composition) before the first time, it indicates that the GPU cannot complete the image processing of the second image in time, and the electronic device can increase the frequency of the GPU to avoid frame drops.

[0010] In this way, the electronic device can increase the frequency of the GPU based on the situation that the GPU cannot complete image processing in time during the processing of the second image. On the one hand, the electronic device enables the GPU to process the second image in time to ensure no frame drops; on the other hand, the electronic device does not increase the frequency of the GPU when the GPU can complete image processing in time, thus not causing resource waste.

[0011] In a possible design of the first aspect, the electronic device further includes a display driver and a Hardware Composer (HWC). The first time is after a second time when the display driver starts waiting for the GPU to finish processing the second image. The display driver starts waiting for the GPU to finish processing the second image after receiving the composition result of the second image from the HWC.

[0012] When the display driver starts waiting for the GPU to finish image processing, it indicates that the CPU has basically completed the relevant processing of sending the image for display. After that, whether frame drops occur mainly depends on the working efficiency of the GPU. Therefore, by configuring a preset time T0 as the time after the display driver starts waiting for the GPU to finish image processing, it is possible to accurately control no frame drops by adjusting the frequency of the GPU on the premise of being only affected by the working efficiency of the GPU.

[0013] In a possible design of the first aspect, the electronic device detects whether a first end time when the GPU finishes processing the second image exceeds a first time, including: at the first time, the electronic device detects whether the display driver is waiting for the GPU to finish processing the second image. If the display driver is waiting for the GPU to finish processing the second image, the first end time exceeds the first time. If the display driver is not waiting for the GPU to finish processing the second image, the first end time does not exceed the first time.

[0014] In this way, the electronic device can accurately determine whether the GPU has completed processing the second image according to whether the display driver is waiting for the GPU to finish processing the second image.

[0015] In a possible design of the first aspect, the second time includes: the time when the display driver starts to execute a waiting function. The waiting function is used for the display driver to wait for the GPU to finish processing the second image, and after the GPU finishes processing the second image, the display driver ends the execution of the waiting function.

[0016] In a possible design of the first aspect, the electronic device increases the frequency of the GPU, including: the electronic device increases the minimum frequency of the GPU. It can be understood that after the electronic device increases the minimum frequency of the GPU, the actual working frequency of the GPU will also change to the increased minimum frequency, so that the actual working frequency is greater than or equal to the minimum frequency, and then the GPU can work at the frequency gear corresponding to the increased minimum frequency.

[0017] Correspondingly, the above method further includes: within a first time interval for displaying the first image: the electronic device detects whether a first end time exceeds a third time. If the first end time does not exceed the third time, the electronic device decreases the frequency of the GPU. The third time is within the first time interval and before the first time.

[0018] Among them, if the first end time does not exceed the third time, it indicates that the GPU can complete processing the second image ahead of time, and the electronic device can decrease the frequency of the GPU to avoid resource waste.

[0019] In a possible design of the first aspect, the electronic device detects whether a first end time when the GPU finishes processing the second image exceeds a first time, including: if the first end time exceeds the third time, the electronic device detects whether the first end time when the GPU finishes processing the second image exceeds the first time.

[0020] Since the third time is after the first time, if the first end time does not exceed the third time, then the first end time will not exceed the first time, and there is no need to increase the frequency of the GPU. In this case, after the electronic device detects that the first end time exceeds the third time, it does not need to further detect whether the first end time exceeds the first time. Therefore, the electronic device further detects whether the first end time exceeds the first time only when it detects that the first time exceeds the third time, thereby avoiding ineffective detection and saving computing resources.

[0021] In a possible design of the first aspect, the electronic device detects whether the first end time when the GPU finishes processing the second image exceeds the first time, including: if the time interval between the first time and the fourth time is greater than the duration threshold, the electronic device detects whether the first end time exceeds the first time. The electronic device detects whether the first end time exceeds the third time, including: if the time interval between the third time and the fourth time is greater than the duration threshold, the electronic device detects whether the first end time exceeds the third time. Wherein, the fourth time is the time when the electronic device last reduced the frequency of the GPU before displaying the first image.

[0022] In this way, the electronic device can avoid detecting and adjusting the frequency of the GPU when it is not effective.

[0023] In a possible design of the first aspect, if the first end time exceeds the third time but does not exceed the first time, it indicates that the GPU working at the current frequency can not only ensure timely completion of image processing but also avoid resource waste, that is, the current frequency is appropriate. Therefore, the electronic device can not adjust the frequency of the GPU. In this way, it can ensure that the GPU works at an appropriate frequency.

[0024] In a possible design of the first aspect, the above method further includes: if the first end times corresponding to at least the first number of consecutive images exceed the third time but do not exceed the first time, it indicates that the current frequency can maintain the image processing efficiency of the GPU within a reasonable range all the time, and the electronic device can use dynamic clock and voltage scaling (DCVS) to adjust the frequency of the GPU. That is, the electronic device can further adjust the frequency of the GPU according to the usage rate of the GPU when the working efficiency of the GPU meets the requirements, so as to further ensure the matching of the frequency of the GPU and the usage rate.

[0025] In a possible design of the first aspect, the above-mentioned electronic device increases the frequency of the GPU, including: the electronic device increases the actual working frequency of the GPU. It can be understood that after the electronic device increases the actual working frequency of the GPU, since the minimum frequency of the GPU does not change, after the actual working frequency of the GPU is increased, the electronic device can use DCVS to control the frequency of the GPU. That is to say, the actual value of the frequency of the GPU may not remain stable after the increase. Specifically, after the electronic device increases the frequency of the GPU, the above method further includes: the electronic device uses dynamic clock and voltage scaling (DCVS) to adjust the frequency of the GPU.

[0026] In a possible design of the first aspect, the above method further includes: if the frequency of the GPU is increased continuously for more than a preset number of times, the electronic device uses DCVS to reduce the frequency of the GPU to a first value, and the electronic device increases the minimum frequency of the GPU to a second value, and the frequency range to which the second value belongs is one level higher than the frequency range to which the first value belongs, so that the GPU can work at a frequency that makes the first end time for completing image processing earlier than the first time.

[0027] In a possible design of the first aspect, the above-mentioned electronic device further includes a display screen. The electronic device displays a first image, including: at a fifth time after the GPU completes the processing of the first image, the display driver drives the display screen to display the first image. Correspondingly, the above method further includes: the electronic device obtains the fifth time and the second time. The electronic device adds the refresh period of the display screen to the fifth time to obtain a sixth time. The electronic device determines a first time point at which the time interval between the second time and the sixth time and the time interval between the second time and the sixth time accounts for a first ratio.

[0028] In a second aspect, the present application further provides an electronic device, which includes a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The display screen includes a first screen and a second screen. When the display screen is in a folded state, the display directions of the first screen and the second screen are opposite, and the first screen and the second screen are two different display areas of the display screen. The memory is used to store computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device is caused to execute the methods in the above-mentioned first aspect and its possible design manners.

[0029] In a third aspect, the present application provides a chip system, which is applied to an electronic device including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are configured to receive signals from the memory of the electronic device and send the signals to the processors, and the signals include computer instructions stored in the memory; when the processors execute the computer instructions, the electronic device executes the method according to the first aspect and any possible design manner thereof.

[0030] In a fourth aspect, the present application provides a computer storage medium, which includes computer instructions. When the computer instructions run on an electronic device, the electronic device is caused to execute the method according to the first aspect and any possible design manner thereof.

[0031] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a computer, the computer is caused to execute the method according to the first aspect and any possible design manner thereof.

[0032] It can be understood that for the beneficial effects that can be achieved by the electronic device in the second aspect, the chip system in the third aspect, the computer storage medium in the fourth aspect, and the computer program product in the fifth aspect provided above, reference may be made to the beneficial effects in the first aspect and any possible design manner thereof, which will not be elaborated herein. Description of the Drawings

[0033] Figure 1 It is a software and hardware architecture diagram of a mobile phone;

[0034] Figure 2 It is a schematic diagram of an image processing process;

[0035] Figure 3 It is one of the schematic diagrams of problems with DCVS frequency modulation;

[0036] Figure 4 It is another schematic diagram of problems with DCVS frequency modulation;

[0037] Figure 5 It is yet another schematic diagram of problems with DCVS frequency modulation;

[0038] Figure 6 It is a schematic diagram of the preset time T0 provided by the embodiment of the present application;

[0039] Figure 7 It is a schematic diagram of the principle for determining the next refresh time T2 provided by the embodiment of the present application;

[0040] Figure 8 It is a corresponding relationship diagram of frequency levels and frequency values provided by the embodiment of the present application;

[0041] Figure 9 Schematic diagram of preset time T3 provided by an embodiment of the present application;

[0042] Figure 10 One of the example diagrams of frequency modulation provided by an embodiment of the present application;

[0043] Figure 11 Another example diagram of frequency modulation provided by an embodiment of the present application;

[0044] Figure 12 Yet another example diagram of frequency modulation provided by an embodiment of the present application;

[0045] Figure 13 Still another example diagram of frequency modulation provided by an embodiment of the present application;

[0046] Figure 14 The fifth example diagram of frequency modulation provided by an embodiment of the present application. Detailed implementation manners

[0047] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "the", "above-mentioned", "this" and "such" are also intended to include the forms such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of the present application, "at least one" and "one or more" mean one or more than two (including two). The term "and / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship.

[0048] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants mean "including but not limited to", unless otherwise specifically emphasized. The term "connection" includes direct connection and indirect connection, unless otherwise stated. "First" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features.

[0049] In the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0050] The frequency modulation method provided by the embodiments of the present application can be applied to scenarios where a GPU needs to process image processing in an electronic device. Especially in scenarios where the GPU needs to continuously process multiple frames of images. For example, during the display of desktop animations, the GPU needs to continuously process to obtain multiple animation frames; in the scenario of playing games, the GPU needs to continuously process multiple frames of game screens.

[0051] In the above scenarios, by adopting the frequency modulation method provided by the embodiments of the present application, the electronic device can adjust the frequency of the GPU to meet the requirements of work efficiency and ensure no frame loss.

[0052] Exemplarily, the electronic device in the embodiments of the present application can be a mobile phone, a tablet computer, a desktop type, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device, etc. that support display functions. The embodiments of the present application do not impose special restrictions on the specific form of the electronic device. In the following text, the electronic device being a mobile phone will be mainly used as an example to illustrate the solution of the present application.

[0053] In the embodiments of this application, the software system of a mobile phone may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the following embodiments, the Android system with a layered architecture will be mainly used as an example to exemplarily illustrate the software and hardware architectures of the mobile phone 100.

[0054] The layered architecture divides software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, referring to Figure 1 , the software and hardware architectures of the mobile phone from top to bottom are the application layer (applications), the application framework layer (framework), the native layer (Native), the hardware abstraction layer (HAL), the kernel layer (kernel), and the hardware layer.

[0055] Among them, applications such as calls, memos, browsers, contacts, cameras, galleries, calendars, desktops, games, videos, and chats can be installed in the application layer, and only some are shown in the figure. These applications all need to display images through the display screen of the mobile phone. For example, it is necessary to display application icons, game screens, video frames, etc. on the desktop through the display screen of the mobile phone. These images all need to be processed (such as rendered and synthesized) by the GPU, that is, they belong to the scenarios that require the GPU to process images.

[0056] The application framework layer provides application programming interfaces (application programming interfaces, APIs) and programming frameworks for the application layer. Exemplarily, the application framework layer may include a window manager, a content provider, a view system, a telephone manager, a resource manager, a notification manager, etc., and only some of them are shown in the figure.

[0057] Furthermore, the application framework layer also includes a RenderThread. The RenderThread can receive calls from upper-layer applications, complete image drawing, and then send the drawing results to the lower layer (such as the native layer) for further processing.

[0058] The native layer provides various services for the upper layer (such as the application framework layer). Exemplarily, the native layer includes an image composition processing service (SurfaceFlinger, SF), a three-dimensional (3D) graphics processing library (for example: OpenGL ES), a two-dimensional (2D) graphics engine (for example: SGL), etc., and only some of them are shown in the figure.

[0059] Among them, SF can be used for refresh rate control. It can be understood that when an application in the application framework layer needs to display a new frame of image, it can request a vertical synchronization (vSync) signal through SF. The vSync signal can trigger the application to complete the rendering of a new frame of image, and then after synthesis, it is finally sent for display. Moreover, after a refresh of the display screen of the electronic device is completed, the vSync signal can be distributed to the application that requests the vSync signal through SF, that is, distributed to the application with rendering requirements. That is to say, SF can control the application to start rendering after the display screen is refreshed once by distributing the vSync signal from the display screen to the application. That is, to achieve refresh rate control.

[0060] In addition, SF can also be used to manage the content obtained by rendering, such as performing layer splicing.

[0061] The hardware abstraction layer encapsulates the underlying hardware drivers and provides a general interface for calling the drivers to the upper layer, so that the upper layer can call the drivers to drive the corresponding hardware to work. The hardware abstraction layer includes a hardware composer (Hwcomposer, HWC). HWC can perform image synthesis on the content processed by SF.

[0062] It can be understood that SF can call the interface of the graphics library to complete image synthesis through the GPU. Obviously, completing image synthesis through the GPU requires consuming GPU resources. Therefore, the pressure on the GPU can be reduced through the image synthesis of HWC.

[0063] The kernel layer includes drivers for driving hardware work, such as display drivers, audio drivers, Bluetooth drivers, etc. Only some of them are shown in the figure. Among them, the display driver is used to drive the display screen to display images.

[0064] Furthermore, the kernel layer also includes a GPU management module for executing the frequency modulation method provided by the embodiments of the present application to achieve frequency regulation of the GPU.

[0065] The hardware layer includes devices such as a display screen and a GPU. Among them, the display screen can be used for image display, and the GPU can be used for image rendering and synthesis.

[0066] To facilitate understanding of the role of the GPU in the processing process of a frame of image, the embodiments of the present application further combine Figure 1 on this basis Figure 2 to exemplarily introduce the process of image drawing and sending for display in the electronic device:

[0067] See Figure 2 , the processing process of a frame of image mainly includes: Step 1, rendering; Step 2, synthesis; and Step 3, sending for display.

[0068] Step 1, Rendering.

[0069] After the application receives the vSync signal from the SF, it can start drawing a new frame of the image. Specifically, the application submits drawing instructions to the RenderThread.

[0070] After receiving the drawing instructions, the RenderThread processes the drawing instructions and then sends them to the GPU. After receiving the processed drawing instructions, the GPU can perform rendering processing. That is, during the processing of a frame of the image, the GPU can be used for image rendering.

[0071] Step 2, Composition.

[0072] The RenderThread can also send the image content obtained after processing the drawing instructions, such as the drawn image, to the SF. After receiving the image content, the SF determines whether the image content needs to be composited and displayed in this frame, and processes the content that needs to be composited and displayed. For example, it completes the processing following the refresh rate. And the content that does not need to be composited and displayed is put into the waiting queue and judged again in the next frame whether it needs to be composited and displayed.

[0073] Normally, the HWC can only complete some simple composition work, while complex composition work needs to be completed by the GPU. Therefore, the SF can judge whether the image composition of the current frame requires the support of the GPU. If so, the SF can call the interface of the graphics library to complete the composition work that the HWC cannot complete through the GPU. That is, during the processing of a frame of the image, the GPU can also be used for image composition.

[0074] It should be noted that during the execution of Step 2, the GPU may not have completed the image rendering yet. Therefore, the GPU needs to complete the image composition after completing the image rendering.

[0075] Step 3, Display.

[0076] After the HWC completes the image composition, it can send the composition result to the display driver. Based on this composition result, the display driver further combines the processing result of the GPU, that is, the processing result of the GPU performing image rendering and image composition, to obtain the image to be displayed and send it to the display screen.

[0077] It should be noted that when the HWC completes the image composition, the GPU may not have completed the image rendering and image composition yet. Therefore, after receiving the composition result of the HWC, the display driver still needs to further wait for the GPU to complete the image rendering and image composition before it can obtain the image to be displayed and send it to the display screen.

[0078] It can be seen that during the processing of one frame of an image, the GPU needs to complete image rendering and image composition. Moreover, if the GPU fails to complete the tasks in a timely manner, the display driver cannot obtain the image to be displayed in a timely manner and send it to the display screen for display, resulting in frame loss.

[0079] Meanwhile, the higher the frequency of the GPU, the stronger the computing power it provides, and the higher the efficiency of image processing (including image rendering and image composition). Therefore, the electronic device can adjust the frequency of the GPU to make the GPU provide computing power matching the current image processing task to meet the requirements of work efficiency (such as frame rate, refresh rate, etc.).

[0080] Generally, the GPU developer will provide a set of native frequency modulation schemes for electronic devices equipped with this GPU. Taking the developer as Qualcomm TM as an example, Qualcomm TM provides a native frequency modulation scheme - Dynamic Clock and Voltage Scaling (DCVS). When an electronic device adopts DCVS, it can adjust the frequency level of the GPU to the corresponding frequency level according to the usage rate of the GPU within a period of time. Among them, the usage rate of the GPU can be understood as the proportion of the working duration of the GPU in the total duration of this period of time within a period of time.

[0081] It should be noted here that the frequency of the GPU is usually represented by frequency levels. Generally, the higher the number of the frequency level, the lower the frequency of the GPU; the lower the number of the frequency level, the higher the frequency of the GPU. Exemplarily, the frequency levels include 8 levels from 0 to 7. From level 0 to level 7, the frequency of the GPU gradually decreases. For example, the frequency corresponding to level 0 is 680 MHz, the frequency corresponding to level 6 is 295 MHz, and the frequency corresponding to level 7 is 220 MHz.

[0082] An implementation of adjusting the frequency level of the GPU using DCVS is as follows: After the usage rate of the GPU remains above 90% for a certain period of time, the electronic device determines that the GPU is not sufficient to complete tasks efficiently when operating at the current frequency level. Therefore, the electronic device increases the frequency level of the GPU. After the frequency level of the GPU is increased, the computing power provided by the GPU is stronger. Therefore, the GPU can complete tasks in a shorter time, and the usage rate of the GPU will decrease accordingly. Of course, if the usage rate of the GPU still cannot be reduced below 90%, the electronic device will continue to increase the frequency level of the GPU until it is adjusted to a suitable frequency level that can keep the usage rate of the GPU stable below 90%.

[0083] The electronic device adopts DCVS, which can adjust the frequency level of the GPU to control the GPU utilization rate below 90%. However, when the electronic device adopts DCVS, the following problems may occur:

[0084] Problem 1: Frame drops may occur.

[0085] When the electronic device adopts DCVS, the frequency level of the GPU will only be increased after the GPU utilization rate has been above 90% for a certain period of time, that is, the adjustment of the frequency level lags.

[0086] See Figure 3 , before time t1, the frequency level of the GPU is controlled at level 7, and the GPU utilization rate can be maintained below 90% but close to 90%. At time t1, the load of the GPU (i.e., image processing tasks, such as rendering, synthesis, etc.) suddenly increases, and the GPU utilization rate also increases accordingly, such as exceeding 90%. However, since the condition that the GPU utilization rate is above 90% for a certain period of time is not met within a short period, the electronic device will not increase the frequency level of the GPU. That is, the GPU will still work at level 7 within a short period (such as between time t1 and time t2). During this period, the GPU may not be able to complete tasks efficiently due to too low a frequency, resulting in frame drops, such as Figure 3 2 frames are dropped between time t1 and time t2 in . After a period of time, such as at time t2, the electronic device only detects that the GPU utilization rate has been above 90% for a certain period of time, and then the frequency level is increased, such as from level 7 to level 0. After that, the GPU can complete the processing of each frame of the image in time and no frame drops will occur. However, due to the lag in the adjustment of the frequency level, the frame drops between time t1 and time t2 cannot be recovered.

[0087] Problem 2: The frequency level may still not be increased after frame drops, which may further lead to continuous frame drops.

[0088] The GPU utilization rate is calculated over a period of time. If the working time of the GPU is short within a period of time, the utilization rate is low; if the working time of the GPU is long, the utilization rate is high. However, the utilization rate in the historical time period cannot accurately represent the situation of real-time image processing tasks, so it may lead to inaccurate frequency control by the electronic device and continuous frame drop phenomena.

[0089] See Figure 4, between time t3 and time t4, the frequency gear of the GPU is controlled at gear 7, and the utilization rate of the GPU can be maintained below 90%. However, there is a frame drop phenomenon, such as a total of 3 frames dropped. Subsequently, at time t4, the electronic device calculates the utilization rate of the GPU between time t3 and time t4. Since there is frame drop between time t3 and time t4, the calculated utilization rate will not be very high, such as still maintained below 90%. Then, the electronic device will not increase the frequency gear of the GPU. That is, the GPU will still work at gear 7, such as working at gear 7 between time t4 and time t5, and thus will continue to drop frames, such as a total of 2 frames dropped.

[0090] Problem 3: It is easy to cause the performance of the electronic device to be unstable.

[0091] The electronic device adopts DCVS and will only increase the frequency gear when the utilization rate of the GPU reaches more than 90%. In practice, it often may occur that the utilization rate is close to 90% but does not exceed 90% (which can be simply referred to as the dangerous level). And when the utilization rate is at the dangerous level, affected by the whole machine, such as the power and temperature of the electronic device, etc., the operation of the GPU is very likely to fluctuate, thus causing the performance of the electronic device to be unstable, such as occasional frame drops.

[0092] See Figure 5 , between time t6 and time t7, the utilization rate of the GPU has been fluctuating between 85% and 90%. Since it does not meet the condition that the utilization rate is above 90% for a certain period of time, the electronic device has not adjusted the frequency gear of the GPU, and the GPU has been working at the same frequency gear, such as gear 7. However, between time t6 and time t7, there will be occasional frame drops, such as Figure 5 2 frames dropped in

[0093] In addition to the above DCVS solution, the GPU developer also provides a customized performance lock (Perflock) solution for the electronic device developer to more flexibly adjust the frequency gear of the GPU and overcome the problems existing in the above DCVS to a certain extent. Among them, the main idea of Perflock is as follows: The electronic device can add stubs for the problem scenarios that need frequency modulation, then identify the problem scenarios through the stubs, and use the interfaces provided by the GPU developer to adjust the frequency gear of the GPU for the problem scenarios.

[0094] However, Perflock is mainly applicable to some problem scenarios that necessarily require frequency modulation (which can also be called must-occur scenarios), while for some non-must-occur scenarios, that is: in some cases, frequency modulation may be required, and in other cases, frequency modulation may not be required. When the electronic device adopts Perflock, it often causes waste of resources.

[0095] A typical scenario is as follows: When the screen refresh rate or application frame rate of an electronic device is relatively high, usually the GPU needs to complete more frame image processing tasks per unit time. Therefore, the computing power requirement for the GPU is relatively high, and the GPU needs to work at a higher frequency gear. Therefore, the electronic device can adopt Perflock to set a break point for the scenario where the screen refresh rate is relatively high (such as reaching 120 Hertz (Hz)), that is, mark that the frequency gear of the GPU needs to be increased when the screen refresh rate reaches 120 Hz. Subsequently, during the process of the GPU performing image processing, if the electronic device detects that the screen refresh rate reaches 120 Hz, it can increase the frequency gear of the GPU.

[0096] However, in actual applications, when the screen refresh rate of the electronic device reaches 120 Hz, it is not necessarily necessary to increase the frequency gear of the GPU. Exemplarily, although the screen refresh rate of the electronic device reaches 120 Hz, the image processing tasks that need to be completed by the GPU are not many. For example, there is no need for the GPU to complete complex synthesis processing. Therefore, even if the frequency gear of the GPU is not increased, the GPU may still be able to complete the current image processing tasks in a timely manner, which is a non-essential scenario. In such a scenario, if Perflock is still adopted to increase the frequency gear of the GPU when the screen refresh rate reaches 120 Hz, it may lead to resource waste.

[0097] Based on the problems existing in the above-mentioned DCVS and Perflock, the embodiments of the present application provide a frequency modulation method, which can be used for GPU frequency modulation.

[0098] By adopting the embodiments of the present application, during the processing of a frame of image, the electronic device (such as the GPU management module in the kernel layer) can detect whether the end time (which can also be called the first end time) of the GPU completing image processing (including rendering and synthesis) exceeds a preset time T0 (which can also be called the first time). If the end time exceeds the preset time T0, it indicates that the GPU cannot complete the processing of the current frame image in a timely manner. The electronic device can increase the frequency gear of the GPU to ensure that the GPU can complete the processing of the current frame image in a timely manner and avoid frame loss.

[0099] In this way, the electronic device can increase the frequency of the GPU based on the situation that the GPU cannot complete image processing in a timely manner in the current frame. On the one hand, the electronic device enables the GPU to process and obtain the current frame image in a timely manner, ensuring no frame loss; on the other hand, the electronic device will not increase the frequency of the GPU when the GPU can complete image processing in a timely manner, thus not causing resource waste.

[0100] In some embodiments, the preset time T0 can be a time point at which the duration until the next refresh moment of the display screen is a fixed duration 1. It can be understood that every time the display screen is refreshed, a new frame of image needs to be displayed. Therefore, the GPU needs to complete the processing of the current frame image before the next refresh, and the preset time T0 also needs to be before the next refresh moment. In this way, the electronic device can quickly determine the preset time T0 based on the next refresh moment and the fixed duration 1.

[0101] In other embodiments, considering that the processing progress of different frame images by the electronic device may vary, the preset time T0 can also be a time point that matches the processing progress of the current frame image.

[0102] Specifically, the preset time T0 is a time point in the time length (denoted as duration L) from the moment T1 (which can also be called the second time) when the display driver starts waiting for the GPU to complete image processing to the next refresh moment T2, and the ratio of the time length from the moment T1 to the duration L is a preset ratio λ1 (such as 0.7, 0.8, etc., which can also be called the first ratio). Regarding the content of the display driver waiting for the GPU to complete image processing, reference can be made to the description of step 3 in the foregoing Figure 2 and will not be elaborated here.

[0103] Taking the preset ratio λ1 as 0.7 as an example, during the processing of the kth frame image shown in Figure 6 the electronic device can determine the preset time T0 as the moment T1 + 0.7 * (T2 - T1).

[0104] It can be seen that by adopting this embodiment, the electronic device can determine a matching preset time T0 for the processing progress of the current frame image, so as to adjust the frequency gear of the GPU targeted. In the following text, this embodiment will be mainly used to illustrate the solution of the present application.

[0105] In addition, it should be emphasized that: when the display driver starts waiting for the GPU to complete image processing, it indicates that the CPU has basically completed the relevant processing of image sending and display. After that, whether frames are dropped mainly depends on the working efficiency of the GPU. Therefore, configuring the preset time T0 as the time after the display driver starts waiting for the GPU to complete image processing can, on the premise of being only affected by the working efficiency of the GPU, accurately control no frame drop by adjusting the frequency of the GPU.

[0106] The following will separately introduce the specific implementations of determining the moment T1 and the next refresh moment T2:

[0107] First, determine the moment T1.

[0108] In practice, the display driver can wait for the GPU to complete image processing through a waiting function (such as plane_wait_input_fence). It can be understood that after the GPU completes image processing, it will release the corresponding fence. Correspondingly, after the display driver queries through the waiting function that the fence has been released, it can determine that the GPU has completed image processing. At this time, the display driver can execute the subsequent display process to drive the display screen to display the current frame image. Based on this, the electronic device can determine that time T1 is the time when the display driver starts to execute the waiting function.

[0109] Second, determine the next refresh time T2.

[0110] During the process of the display screen displaying the (k - 1)-th frame image (which can also be called the first image), the electronic device needs to complete the processing of the k-th frame image (which can also be called the second image), such as rendering, composition, and display sending. In this way, at the next refresh time, the display screen can refresh and display the k-th frame image. Therefore, the electronic device needs to complete image processing one frame in advance.

[0111] Among them, the display driver can complete image processing one frame in advance according to the hardware interruption (TE, which can be understood as the refresh period) of the display screen. As Figure 7 shown, between TE0 and TE1, the display screen displays the 0-th frame image, and the display driver processes the 1-st frame image; between TE1 and TE2, the display screen displays the 1-st frame image, and the display driver processes the 2-nd frame image... between TE(n - 1) and TEn, the display screen displays the (n - 1)-th frame image, and the display driver processes the n-th frame image. Among them, the time interval for the display screen to display one frame can be called the first time interval.

[0112] In some embodiments, the electronic device can approximately use the time when the display driver finishes processing the previous frame image as the start time of the current TE cycle (which can also be called the fifth time). Taking the TE1 to TE2 shown in Figure 7 as an example, if the time when the display driver finishes processing the 1-st frame image is Tlast, then the electronic device can determine Tlast as the start time of TE1 to TE2, that is, TE1.

[0113] It can be understood that if the 1-st frame image is dropped, it is only a matter of untimely response, which can also be understood as not being very responsive, and it will not have too much impact on the user's use. Therefore, for the 2-nd frame image and subsequent images, the electronic device can adopt the solution of this application to adjust the frequency of the GPU. Then, the time when the display driver finishes processing the 1-st frame image is the first Tlast, such as Figure 7 TE1 in

[0114] In practice, after the display driver finishes processing a frame of image, it can indicate the completion of the processing of a frame of image through an end function (such as sde_kms_complete_commit). Based on this, the electronic device can use the moment when the display driver executes the end function as the start moment of the current TE cycle.

[0115] It can be understood that after each refresh of the display screen, a vSync signal will be sent to trigger the processing of the next frame of image. That is, a refresh cycle is the time interval between two vSync signals, which can be denoted as △vSync (also called the refresh cycle). For example, if the refresh rate is 120Hz, then △vSync = 8.3333ms; if the refresh rate is 90Hz, then △vSync = 11.1111ms; if the refresh rate is 60Hz, then △vSync = 16.6666ms. Then, if the electronic device adds △vSync to the moment when the display driver finishes processing the previous frame of image, it can obtain the end moment of the current TE cycle (also called the sixth time), that is, the next refresh moment T2. Still taking the TE1 to TE2 shown in the current TE cycle as an example, if the moment when the display driver finishes processing the first frame of image is Tlast, then the electronic device can determine that the next refresh moment T2 = Tlast + △vSync. Figure 7 Shown as TE1 to TE2, if the moment when the display driver finishes processing the first frame of image is Tlast, then the electronic device can determine that the next refresh moment T2 = Tlast + △vSync.

[0116] Of course, the electronic device can also record the latest refresh moment and add △vSync to the latest refresh moment to obtain the next refresh moment T2.

[0117] After the electronic device determines the moment T1 and the next refresh moment T2, it can determine the preset time T0 according to the moment T1 and the next refresh moment T2. Subsequently, the electronic device can detect whether the end time when the GPU finishes image processing exceeds the preset time T0.

[0118] After the moment T1: If the display driver is still executing the wait function, it indicates that the display driver is still waiting for the GPU to complete image processing. That is, the GPU has not completed image processing. On the contrary, if the display driver has ended the execution of the wait function, it indicates that the display driver has waited for the GPU to complete image processing. That is, the GPU has completed image processing. Based on this, after the moment T1, the electronic device can detect whether the GPU has completed image processing by detecting whether the display driver is still executing the wait function.

[0119] In a specific implementation, the electronic device can detect whether the display driver is executing a wait function after reaching the preset time T0. If the display driver is still executing the wait function at the preset time T0, the electronic device can determine that the end time for the GPU to complete image processing exceeds the preset time T0; if the display driver has ended the execution of the wait function at the preset time T0, the electronic device can determine that the end time for the GPU to complete image processing does not exceed the preset time T0.

[0120] In practice, after the electronic device detects that the display driver starts to execute the wait function, it calculates the time difference between the preset time T0 and the current moment (such as moment T1), and sets a timer with the timing duration being this time difference. Subsequently, the electronic device can detect whether the display driver is executing the wait function when the timer expires. Here, the expiration of the timer is the preset time. Therefore, by detecting whether the display driver is executing the wait function when the timer expires, the electronic device can detect whether the display driver is executing the wait function after reaching the preset time T0.

[0121] After determining that the end time for the GPU to complete image processing exceeds the preset time T0, the electronic device can increase the frequency level of the GPU. Exemplarily, the electronic device can increase the frequency level of the GPU to the highest level to ensure that the GPU can complete the image processing of the current frame before the next refresh moment T2, thereby ensuring that the current frame is not dropped. Of course, the electronic device can also increase the frequency level of the GPU by a preset number of levels, such as increasing 3 frequency levels. The embodiments of the present application do not make specific limitations in this regard. In the following text, mainly taking increasing to the highest level as an example for illustration.

[0122] Among them, increasing the frequency level of the GPU includes: increasing the minimum value of the frequency (which can also be called the minimum frequency) or increasing the actual value of the frequency (which can also be called the actual operating frequency). It can be understood that the actual value of the frequency of the GPU needs to be greater than or equal to the minimum value, that is, the GPU needs to operate at the minimum value of the frequency and above.

[0123] After increasing the minimum value, the actual value of the frequency of the GPU will also change to the increased minimum value, so as to ensure that the actual value is greater than or equal to the minimum value. Then the GPU can operate at the frequency level corresponding to the increased minimum value.

[0124] Taking Figure 8 the corresponding relationship between the frequency levels and frequency values shown as an example, if before the increase, the minimum value of the frequency is f7 and the actual value is also f7, that is, both the minimum value and the actual value are in the 7th level; the electronic device increases the minimum value to f0, that is, the minimum value is increased to the 0th level; accordingly, the actual value of the frequency of the GPU will also increase to f7, that is, the actual value is also increased to the 0th level.

[0125] After increasing the actual value, since the minimum value of the GPU frequency remains unchanged, after the actual value of the GPU frequency is increased, the electronic device can use DCVS to control the GPU frequency. That is to say, the actual value of the GPU frequency may not remain stable after the increase.

[0126] Still taking Figure 8 the correspondence between the frequency levels and frequency values shown as an example, if before the increase, the minimum value of the frequency is f7 and the actual value is also f7, that is, both the minimum value and the actual value are in level 7; the electronic device increases the actual value to f0, that is, the actual value is increased to level 0; however, since the minimum value still corresponds to level 7 and the GPU utilization rate is gradually decreasing, the electronic device can use DCVS to gradually lower the actual value to f1, f2... f7. It should be noted that when the electronic device uses DCVS, it also controls the change of the actual value by adjusting the minimum value.

[0127] The following will separately illustrate the methods of increasing the minimum value and increasing the actual value:

[0128] Method 1, increasing the minimum value.

[0129] After increasing the minimum value of the GPU frequency, the GPU can work long-term at the frequency level corresponding to the increased minimum value, such as working long-term at level 0. However, the GPU working long-term at a relatively high frequency level may lead to resource waste. Based on this, after increasing the minimum value, during the processing of subsequent frame images, when the electronic device detects that the GPU can efficiently complete image processing, it can also lower the GPU frequency level to avoid resource waste.

[0130] Similarly, lowering the GPU frequency level includes: lowering the minimum value.

[0131] After lowering the minimum value, the actual value of the GPU frequency will also change to the lowered minimum value, and then the GPU can work at the frequency level corresponding to the lowered minimum value. Taking Figure 8 the correspondence between the frequency levels and frequency values shown as an example, if before the decrease, the minimum value of the frequency is f0 and the actual value is also f0, that is, both the minimum value and the actual value are in level 7; the electronic device lowers the minimum value to f1, that is, the minimum value is lowered to level 1; subsequently, the actual value of the GPU frequency will also be lowered to f1, that is, the actual value is also lowered to level 1.

[0132] However, lowering the GPU frequency level generally does not include lowering the actual value. The reason is as follows: when the actual value and the minimum value are the same, if the actual value is lowered, it will cause the actual value to be lower than the minimum value, which does not meet the requirement that the actual value of the GPU frequency should be greater than or equal to the minimum value.

[0133] In some embodiments, after increasing the minimum value of the GPU frequency, the electronic device can, during the processing of subsequent images, detect whether the end time for the GPU to complete image processing exceeds a preset time T3 (which can also be referred to as the third time). If the end time does not exceed the preset time T3, it indicates that the GPU can efficiently complete image processing. The electronic device can then decrease the minimum value, and the actual value of the subsequent GPU frequency will gradually decrease to the minimum value, thereby saving resources when the GPU can efficiently complete image processing.

[0134] Similar to the preset time T0: The preset time T3 can be a time point at a fixed duration 2 from the next refresh moment T2, where the fixed duration 2 is greater than the fixed duration 1. In this way, the electronic device can also quickly determine the preset time T3 based on the next refresh moment T2 and the fixed duration 2.

[0135] Alternatively, the preset time T3 can be a time point in the duration L where the ratio of the time length from the moment T1 to the duration L is a preset ratio λ2 (such as 0.5, 0.6, etc.), and the preset ratio λ2 is less than the preset ratio λ1.

[0136] Taking the preset ratio λ2 as 0.5 as an example, during Figure 9 the processing of the second frame image shown (such as at the preset time T0), the electronic device increases the minimum value of the GPU frequency to the frequency value of gear 0; subsequently, during the processing of the third frame image, the electronic device can determine the preset time T3 as the moment T1 + 0.5 * (T2 - T1).

[0137] Regarding the determination of the moment T1 and the next refresh moment T2, reference can be made to the relevant description of determining the preset time T0 in the foregoing, which will not be elaborated here.

[0138] Similarly, after the moment T1, the electronic device can detect whether the GPU has completed image processing by detecting whether the display driver is executing a wait function. In a specific implementation, the electronic device can, after reaching the preset time T3, detect whether the display driver is executing a wait function. If the display driver is still executing the wait function at the preset time T3, the electronic device can determine that the end time for the GPU to complete image processing exceeds the preset time T3; if the display driver has ended the execution of the wait function at the preset time T3, the electronic device can determine that the end time for the GPU to complete image processing does not exceed the preset time T3.

[0139] Continuing to refer to Figure 9 , at the preset time T3, the display driver has ended the execution of the wait function, and the electronic device can determine that the end time for the GPU to complete image processing does not exceed the preset time T3.

[0140] When the end time of the GPU to complete image processing does not exceed the preset time T3, the electronic device can reduce the minimum value of the GPU frequency to the frequency value of the frequency gear one level lower than the current frequency gear. Still taking Figure 9 the processing process of the 3rd frame image shown as an example, when the end time of the GPU to complete image processing does not exceed the preset time T3, the electronic device can reduce the minimum value of the GPU frequency to the frequency value of the 1st gear. Alternatively, the electronic device can also reduce the minimum value to the frequency value of the frequency gear at a preset number (such as 2, 3, etc.) of levels lower than the current frequency gear. In the following text, it is mainly illustrated by taking the minimum value being reduced to the frequency value of the frequency gear one level lower than the current frequency gear as an example.

[0141] After reducing the minimum value of the GPU frequency, the GPU can work at the frequency gear corresponding to the reduced minimum value for a long time, such as working at the 1st gear for a long time. During the processing of subsequent frame images, the electronic device can continue to detect whether the frequency gear is appropriate and adjust the minimum value when it detects that it is inappropriate. Among them, the frequency gear being inappropriate includes that the end time of the GPU to complete image processing does not exceed the preset time T3 (which will cause waste of resources) or the end time of the GPU to complete image processing exceeds the preset time T0 (which will cause frame drops).

[0142] It can be seen that by adopting Method 1, during the processing of each frame image, the electronic device can detect whether the end time of the GPU to complete image processing exceeds the preset time T3. If it does not exceed the preset time T3, the minimum value of the GPU frequency is reduced, such as being reduced to the frequency value of the lower-level frequency gear. And, the electronic device can detect whether the end time of the GPU to complete image processing exceeds the preset time T0. If it exceeds the preset time T0, the minimum value of the GPU frequency is increased, such as being increased to the frequency value of the highest gear.

[0143] In addition, since the preset ratio λ1 is greater than the preset ratio λ2, it means that during the processing of a frame image, the preset time T0 is closer to the next refresh time T2, and the preset time T3 is closer to the time T1. That is, from front to back, they are: time T1, preset time T3, preset time T0, and the next refresh time T2. Then, during the processing of a frame image, when the electronic device detects that the end time of the GPU to complete image processing does not exceed the preset time T3, it can omit the step of detecting whether the end time of the GPU to complete image processing exceeds the preset time T0, thereby avoiding ineffective detection and saving computing resources.

[0144] Up to this point, it should be noted that: after the electronic device reduces the minimum value of the GPU frequency, even if the actual value of the GPU frequency slowly drops to the minimum value, it still satisfies that the actual value is greater than or equal to the minimum value. Moreover, the slow drop can also ensure that the actual value of the GPU frequency does not suddenly change significantly. Therefore, after reducing the minimum value of the GPU frequency, the actual value needs to be delayed for a period of time before it drops to be equal to the minimum value. That is to say, after a delay of a period of time, the GPU will truly work at the reduced minimum value, that is, it will take effect. For example, it takes effect after a delay of 100 ms.

[0145] See Figure 10 , before TE0, both the actual value and the minimum value of the GPU frequency are the frequency value f7 of the 7th gear. If the end time for the GPU to complete image processing between TE0 and TE1 exceeds the preset time T0 between TE0 and TE1, the electronic device raises the minimum value of the GPU frequency to the frequency value f0 of the 0th gear, then the actual value of the GPU frequency will also quickly (such as within the preset time T0) rise to the frequency value f0. Subsequently, at the preset time T3 between TE1 and TE2, the electronic device reduces the minimum value of the GPU frequency to the frequency value f1 of the 1st gear, but the actual value of the GPU frequency drops to the frequency value f1 at time T4. Subsequently, at the preset time T3 between TE2 and TE3, the electronic device further reduces the minimum value of the GPU frequency to the frequency value f2 of the 2nd gear, but the actual value of the GPU frequency drops to the frequency value f2 at time T5.

[0146] That is to say, within the delay duration (which can also be called the duration threshold), such as Figure 10 the preset time T3 between TE1 and TE2 and time T4, the preset time T3 between TE2 and TE3 and time T5 in , the actual value of the GPU frequency does not truly drop to be consistent with the minimum value, and it is only after reaching the delay duration that it truly drops to be consistent with the minimum value.

[0147] In the above Figure 10 example, the delay durations are all relatively short. When the electronic device reduces the minimum value of the GPU frequency during the processing of the current frame image, in the first half of the processing of the next frame image, the actual value of the GPU frequency will drop to be consistent with the minimum value. For example, the electronic device reduces the minimum value to the frequency value f1 at the preset time T3 between TE1 and TE2 in Figure 10 , and subsequently at time T4 in the first half of the period between TE2 and TE3, the actual value of the GPU frequency will also drop to the frequency value f1. In this case, what the electronic device detects at the preset time T0 or the preset time T3 during the processing of each frame image is whether the actual value after taking effect is appropriate.

[0148] In practice, the delay duration may be shorter or longer. Among them, if the delay duration is longer, the preset time T0 or the preset time T3 during the processing of the next frame or more frames of images after reducing the minimum value may be within the delay duration, that is: the actual value has not yet been reduced to be consistent with the minimum value. Then, during the delay duration, the preset time T0 or the preset time T3 cannot detect whether the actual value after taking effect is appropriate.

[0149] Based on this, in some embodiments, after the electronic device reduces the minimum value of the GPU frequency (which can also be referred to as the fourth time), it will detect whether the frequency gear is appropriate only after the delay duration arrives. That is, within the delay duration, even if the preset time T0 or the preset time T3 is reached, the electronic device will not detect whether the frequency gear is appropriate, let alone adjust the frequency gear. In a specific implementation manner, after the electronic device reduces the minimum value of the GPU frequency, it can set a timer with a timing duration equal to the delay duration. Before the timing of the timer ends, when the preset time T0 or the preset time T3 is reached, the electronic device does not detect whether the frequency gear is appropriate; after the timing of the timer ends, when the preset time T0 or the preset time T3 is reached, the electronic device detects whether the frequency gear is appropriate. In this way, the electronic device can avoid detecting and adjusting the frequency gear when it has not taken effect.

[0150] Exemplarily, at Figure 11 the preset time T3 between TE2 and TE3 as shown, the minimum value of the frequency of the GPU of the electronic device is reduced to the frequency value of the second gear, but within the delay duration (denoted as <△t), the actual value has not really been reduced to the frequency value of the second gear. For example, Figure 11 if the time interval between the preset time T3 between TE3 and TE4 and the preset time T3 between TE2 and TE3 as shown is <△t, it indicates that at the preset time T3 between TE3 and TE4, the actual value of the GPU frequency has not been reduced to the frequency value of the second gear. Then, even if as Figure 11 shown, between TE3 and TE4, the end time of the GPU to complete image processing does not exceed the preset time T3 between TE3 and TE4, it does not mean that the actual value of the GPU frequency can efficiently complete image processing at the frequency value of the second gear. Therefore, the electronic device can not detect whether the end time of the GPU to complete image processing exceeds the preset time T3 between TE3 and TE4, and will not further lower the minimum value, thus avoiding incorrect adjustment of the frequency gear.

[0151] Continue to refer to Figure 11 , Figure 11If the time interval between the preset time T3 between TE4 and TE5 and the preset time T3 between TE2 and TE3 exceeds the delay duration (denoted as > Δt), it indicates that at the preset time T3 between TE4 and TE5, the actual value of the GPU's frequency has been reduced to the frequency value of gear 2. Therefore, at the preset time T3 between TE4 and TE5, the electronic device can detect whether the end time for the GPU to complete image processing exceeds the preset time T3 between TE4 and TE5. And if, as Figure 11 shown, the end time for the GPU to complete image processing does not exceed the preset time T3 between TE4 and TE5, the electronic device can continue to reduce the minimum value of the GPU's frequency, such as reducing it to the frequency value of gear 3.

[0152] During the processing of any frame of image, if the electronic device detects that the end time for the GPU to complete image processing exceeds the preset time T3 and does not exceed the preset time T0, it indicates that the GPU working at the current frequency gear can not only ensure timely completion of image processing but also avoid resource waste. That is, the current frequency gear is appropriate. In this case, the electronic device neither needs to increase the minimum value nor needs to reduce the minimum value.

[0153] Furthermore, if during the processing of consecutive y (y ≥ 1, y is an integer, also referred to as the first quantity) frames of images, the electronic device detects that the end time for the GPU to complete image processing exceeds the preset time T3 and does not exceed the preset time T0, it indicates that the current frequency gear can maintain the GPU's image processing efficiency within a reasonable range. Subsequently, the electronic device can use DCVS to assist in adjusting the GPU's frequency.

[0154] Taking y = 3 as an example, after the electronic device reduces the minimum value of the GPU's frequency to Figure 12 the shown frequency value f2, during the consecutive 3 TE cycles of TE3 - TE4, TE4 - TE5, and TE5 - TE6, that is, during the processing of consecutive 3 frames of images, the electronic device detects that the end time for the GPU to complete image processing exceeds the preset time T3 and does not exceed the preset time T0. Then, after TE6, the electronic device can use DCVS for frequency modulation.

[0155] Of course, if the electronic device subsequently detects again that the end time for the GPU to complete image processing exceeds the preset time T0, it can repeat the above method 1 again until an appropriate frequency gear is found.

[0156] Method 2: Increase the actual value.

[0157] Different from Method 1, after the actual value of the GPU frequency is increased, the actual value will automatically change under the control of DCVS, such as decreasing, without the electronic device further reducing the minimum value based on the end time of the GPU to complete image processing, thereby reducing the actual value. Therefore, in Method 2, only the increase of the actual value needs to be mainly concerned about.

[0158] In some embodiments, after the electronic device detects that the end time of the GPU to complete image processing exceeds the preset time T0, the actual value of the GPU frequency is increased to the frequency value of the highest gear, such as the frequency value f0. Subsequently, the electronic device can use DCVS to control the actual value of the GPU frequency. After the electronic device detects again that the end time of the GPU to complete image processing exceeds the preset time T0, the actual value of the GPU frequency can be increased again to the frequency value of the highest gear.

[0159] In this way, on the one hand, it can ensure that the GPU completes image processing in time and avoid frame loss; on the other hand, it can ensure that the frequency gear of the GPU matches the usage rate.

[0160] Exemplarily, the electronic device Figure 13 detects that the end time of the GPU to complete image processing exceeds the preset time T0 corresponding to the preset time T0 between TEx1 - TEx2, TEx3 - TEx4, and TEx5 - TEx6 shown. Therefore, the electronic device can increase the actual value of the frequency to the frequency value of the highest gear, such as f0, at the preset time T0 between TEx1 - TEx2, TEx3 - TEx4, and TEx5 - TEx6 respectively. And between every two times of increasing the actual value to the frequency value of the highest gear, the electronic device can use DCVS to adjust the actual value of the GPU frequency. For example, as the usage rate of the GPU gradually decreases, the electronic device can gradually lower the actual value using DCVS.

[0161] It should be noted that Figure 13 in the example shown, when the electronic device uses DCVS, it gradually lowers the actual value from the frequency value f0 to the frequency value f7.

[0162] However, in actual implementation, the electronic device can adjust according to the actual situation of the usage rate, not limited to Figure 13 . Referring to Figure 14 , after the electronic device increases the actual value of the frequency to f0 at the preset time T0 between TEx1 - TEx2, TEx3 - TEx4, TEx5 - TEx6, and TEx7 - TEx8, when the electronic device uses DCVS, it can gradually lower the actual value from the frequency value f0 to the frequency value f6, the frequency value f5, the frequency value f5, and the frequency value f5 respectively. That is to say, it does not lower to the frequency value f7 every time.

[0163] Further, in this embodiment, if after continuously increasing the actual value to f0 for z times (which can also be referred to as the preset number of times), the electronic device uses DCVS to lower the actual value from the frequency value f0 to the same frequency value (which can also be referred to as the first value), the electronic device can increase the minimum value to a frequency value that is one frequency step higher than the same frequency value (which can also be referred to as the second value), so that the GPU can operate at a frequency at which the end time for completing image processing is earlier than the preset time T0.

[0164] Taking z = 3 as an example, as Figure 14 shown, after continuously increasing the actual value to f0 for 3 consecutive times of the preset time T0 between TEx3 - TEx4, TEx5 - TEx6, and TEx7 - TEx8 of the electronic device, the electronic device uses DCVS to lower the actual value from the frequency value f0 to the frequency value f5, that is, the frequency value f5 is the above-mentioned same frequency value. Therefore, the electronic device can increase the minimum value to a frequency value that is one frequency step higher than the frequency value f5, such as f4.

[0165] An embodiment of the present application further provides an electronic device, which may include: a display screen, a memory, and one or more processors (such as a CPU, a GPU, an NPU, etc.). The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can perform each function or step executed by the device in the above method embodiment.

[0166] An embodiment of the present application further provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a line. For example, the interface circuit can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as a processor). Exemplarily, the interface circuit can read the instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform each step in the above embodiment. Of course, the chip system may also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0167] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions run on an electronic device, the electronic device performs the above-related method steps to implement the image processing method in the above embodiment.

[0168] This embodiment further provides a computer program product, which when running on a computer, enables the computer to perform the above-related steps to implement the image processing method in the above embodiment.

[0169] In addition, an embodiment of the present application further provides a device, which may specifically be a chip, a component or a module. The device may include a processor and a memory connected to each other. The memory is used to store computer-executable instructions. When the device runs, the processor may execute the computer-executable instructions stored in the memory, so that the chip executes the image processing method in each of the above method embodiments.

[0170] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be elaborated here.

[0171] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0172] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the module or unit is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces. The indirect coupling or communication connection of the device or unit may be in an electrical, mechanical or other form.

[0173] The unit described as a separated component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional unit in each embodiment of the present application may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the embodiments of the present application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A frequency modulation method, applied to an electronic device, the electronic device including a Graphics Processing Unit (GPU), Characterized in that, It includes: Display a first image within a first time interval; Determine whether a first time interval exceeds a first preset time interval, wherein the first time interval is within the first time interval, the first time interval is the time interval between a first moment and a second moment, the first moment is the moment when a wait function starts to be executed, and the second moment is the moment when the wait function ends to be executed; In the case where the first time interval exceeds the first preset time interval, increase the frequency of the GPU, and control the GPU to perform image processing based on the increased frequency to obtain a second image; Display the second image within a second time interval, wherein the second time interval and the first time interval are adjacent time intervals, and the first image and the second image are two consecutive frames of images displayed continuously.

2. The method according to claim 1, Characterized in that, The wait function is plane_wait_input_fence.

3. The method according to claim 1 or 2, Characterized in that, Increasing the frequency of the GPU includes: Increasing the minimum frequency of the GPU from a first gear to a second gear, wherein the frequency value corresponding to the second gear is greater than the frequency value corresponding to the first gear.

4. The method according to claim 1 or 2, Characterized in that, Increasing the frequency of the GPU includes: Increasing the actual working frequency of the GPU from a first frequency to a second frequency, and the second frequency is greater than the first frequency.

5. The method according to claim 4, Characterized in that, After increasing the actual working frequency of the GPU from the first frequency to the second frequency, the method further includes: Adjusting the frequency of the GPU based on the Dynamic Voltage and Frequency Scaling (DVFS).

6. The method according to any one of claims 1-5, Characterized in that, The first preset time interval is the product of a first difference and a preset ratio, the first difference is the difference between a third moment and the first moment, and the third moment is the moment when the second image starts to be refreshed and displayed.

7. The method according to claim 6, Characterized in that, The preset ratio is greater than 0.5 and less than 1.

8. The method according to any one of claims 1-7, Characterized in that, Before determining whether the first time interval exceeds the first preset time interval, the method further includes: Detecting whether the first time interval exceeds a second preset time interval, wherein the second time interval is within the first time interval, and the second preset time interval is less than the first preset time interval; In the case where the first time interval does not exceed the second preset time interval, decrease the frequency of the GPU, and control the GPU to perform image processing based on the increased frequency to obtain the second image.

9. The method according to claim 8, Characterized in that, Detecting whether the first time interval exceeds the first preset time interval includes: When the first time interval exceeds the second time interval, detect whether the first time interval exceeds the first time interval; The method further includes: When the first time interval exceeds the second preset time interval but does not exceed the first preset time interval, do not adjust the frequency of the GPU.

10. The method according to any one of claims 1-9, wherein, The first image and the second image are two consecutive animation frames during the display of the desktop animation.

11. The method according to any one of claims 1-9, wherein, The first image and the second image are two consecutive game screen image frames when running a game application.

12. An electronic device, wherein, includes: A display screen, one or more processors, and one or more memories; the one or more processors are coupled to the display screen and the one or more memories; the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the method according to any one of claims 1-11.

13. A computer-readable storage medium, including instructions, wherein, When the instructions run on an electronic device, the electronic device executes the method according to any one of claims 1-11.

Images