Frequency modulation method and electronic equipment

Abstract

The invention discloses a frequency modulation method and electronic equipment, relates to the technical field of terminals, and can adjust the frequency of a GPU for the processing progress of a current frame image based on the GPU to ensure no frame loss. The method comprises the following steps: the electronic equipment displays a first image; in a first time interval for displaying the first image, the electronic device detects whether first end time for the GPU to complete processing of the second image exceeds first time or not, the second image and the first image are two continuous frames of images, and the first time is in the first time interval. And if the first end time exceeds the first time, the electronic equipment improves the frequency of the GPU. And after the GPU completes the processing of the second image, the electronic equipment displays the second image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to a frequency modulation method and electronic device. Background Art

[0002] In electronic devices such as mobile phones and tablets, the graphics processor (Graphics Processing Unit, GPU) is an important device for completing image processing. Among them, image processing can include image rendering and image synthesis. Generally speaking, the higher the frequency of the GPU, the stronger the computing power it provides and the higher the efficiency of image processing. The higher the frequency of the GPU, the weaker the computing power provided by the GPU, and correspondingly, the lower the image processing efficiency. Therefore, electronic devices can meet the efficiency requirements of image processing by adjusting the frequency of the GPU.

[0003] In the prior art, the electronic device can select a suitable frequency according to the GPU usage rate (GPU_busy). For example, if the usage rate is above 90% for a certain period of time, it means that the GPU has very little idle time when running at the current frequency, so the electronic device can increase the GPU frequency so that the GPU can have more idle time to meet more efficient processing needs.

[0004] However, electronic devices using the above-mentioned existing technology will only increase the GPU frequency after the usage rate has been above 90% for a certain period of time. The frequency increase has a lag and does not meet the processing requirements of real-time image processing, which may cause frame drops. Summary of the invention

[0005] The present application provides a frequency modulation method and an electronic device, which can adjust the frequency of a GPU based on the processing progress of the GPU for the current frame image to ensure that no frame is lost.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a frequency modulation method, which is applied to an electronic device, and the electronic device includes a graphics processor (GPU). The electronic device displays a first image. In a first time interval for displaying the first image: the electronic device detects whether a first end time when the GPU completes processing of a second image exceeds a first time, the second image and the first image are two consecutive frames of images, and the first time is within the first time interval. If the first end time exceeds the first time, the electronic device increases the frequency of the GPU. After the GPU completes processing of the second image, the electronic device displays the second image.

[0008] That is, when the electronic device displays the first image (i.e., the first frame of the two consecutive frames), the electronic device can process the second image (i.e., the second frame of the two consecutive frames). And, if during the processing, the GPU does not complete the image processing (such as image rendering and synthesis) before the first time, it means that the GPU cannot complete the image processing of the second image in time, and the electronic device can increase the frequency of the GPU to avoid frame loss.

[0009] In this way, the electronic device can increase the frequency of the GPU based on the situation that the GPU cannot complete the image processing in time during the processing of the second image. On the one hand, the electronic device enables the GPU to process the second image in time to ensure that no frame is lost; on the other hand, the electronic device will not increase the frequency of the GPU when the GPU can complete the image processing in time, thereby avoiding waste of resources.

[0010] In a possible design of the first aspect, the electronic device further includes a display driver and a hardware overlay HWC. The first time is after the second time when the display driver starts waiting for the GPU to complete processing of the second image, and the display driver starts waiting for the GPU to complete processing of the second image after receiving the synthesis result of the second image from the HWC.

[0011] When the display driver starts waiting for the GPU to complete image processing, it indicates that the CPU has basically completed the relevant processing of sending the image to the display. After that, whether frame loss occurs mainly depends on the working efficiency of the GPU. Therefore, by configuring the preset time T0 as the time after the display driver starts waiting for the GPU to complete image processing, the GPU frequency can be adjusted to accurately control frame loss without being affected only by the working efficiency of the GPU.

[0012] In a possible design of the first aspect, the electronic device detects whether the first end time of the GPU completing the processing of the second image exceeds the first time, including: at the first time, the electronic device detects whether the display driver is waiting for the GPU to complete the processing of the second image. If the display driver is waiting for the GPU to complete the processing of the second image, the first end time exceeds the first time. If the display driver is not waiting for the GPU to complete the processing of the second image, the first end time does not exceed the first time.

[0013] In this way, the electronic device can accurately determine whether the GPU has completed the image processing of the second image according to whether the display driver is waiting for the GPU to complete the processing of the second image.

[0014] In a possible design of the first aspect, the second time includes: the time when the display driver starts to execute the waiting function. The waiting function is used for the display driver to wait for the GPU to complete processing of the second image, and after the GPU completes processing of the second image, the display driver ends executing the waiting function.

[0015] In a possible design of the first aspect, the electronic device increases the frequency of the GPU, including: the electronic device increases the minimum frequency of the GPU. It is understandable that after the electronic device increases the minimum frequency of the GPU, the actual operating frequency of the GPU will also change to the minimum frequency after the increase, so that the actual operating frequency is greater than or equal to the minimum frequency, and then the GPU can operate at the frequency gear corresponding to the minimum frequency after the increase.

[0016] Accordingly, the method further includes: in the first time interval of displaying the first image: the electronic device detects whether the first end time exceeds the third time, and if the first end time does not exceed the third time, the electronic device reduces the frequency of the GPU. The third time is within the first time interval and is before the first time.

[0017] If the first end time does not exceed the third time, it indicates that the GPU can complete the image processing of the second image in advance, and the electronic device can reduce the frequency of the GPU to avoid waste of resources.

[0018] In a possible design of the first aspect, the electronic device detects whether the first end time when the GPU completes processing of the second image exceeds the first time, including: if the first end time exceeds the third time, the electronic device detects whether the first end time when the GPU completes processing of the second image exceeds the first time.

[0019] Since the third time is after the first time, if the first end time does not exceed the third time, the first end time will not exceed the first time, and there is no need to increase the GPU frequency. In this case, after the electronic device detects that the first end time exceeds the third time, it does not need to further detect whether the first end time exceeds the first time. Therefore, the electronic device further detects whether the first end time exceeds the first time only when it detects that the first time exceeds the third time, thereby avoiding invalid detection and saving computing resources.

[0020] In a possible design of the first aspect, the electronic device detects whether the first end time when the GPU completes processing of the second image exceeds the first time, including: if the time interval between the first time and the fourth time is greater than the duration threshold, the electronic device detects whether the first end time exceeds the first time. The electronic device detects whether the first end time exceeds the third time, including: if the time interval between the third time and the fourth time is greater than the duration threshold, the electronic device detects whether the first end time exceeds the third time. The fourth time is the time when the electronic device last reduced the frequency of the GPU before displaying the first image.

[0021] In this way, the electronic device can avoid detecting and adjusting the frequency of the GPU when it is not effective.

[0022] In a possible design of the first aspect, if the first end time exceeds the third time but does not exceed the first time, it indicates that the GPU working at the current frequency can ensure timely completion of image processing and does not cause resource waste, that is, the current frequency is appropriate. Therefore, the electronic device does not need to adjust the frequency of the GPU. In this way, the GPU can be guaranteed to work at an appropriate frequency.

[0023] In a possible design of the first aspect, the method further includes: if the first end time corresponding to at least the first number of consecutive images exceeds the third time but does not exceed the first time, it indicates that the current frequency can maintain the image processing efficiency of the GPU within a reasonable range, and the electronic device can use dynamic clock and voltage regulation DCVS to adjust the frequency of the GPU. That is, the electronic device can further adjust the frequency of the GPU according to the utilization rate of the GPU when the working efficiency of the GPU meets the requirements, so as to further ensure that the frequency of the GPU matches the utilization rate.

[0024] In a possible design of the first aspect, the electronic device increases the frequency of the GPU, including: the electronic device increases the actual operating frequency of the GPU. It is understandable that after the electronic device increases the actual operating frequency of the GPU, since the minimum frequency of the GPU has not changed, the electronic device can use DCVS to control the frequency of the GPU after the actual operating frequency of the GPU is increased. In other words, the actual value of the frequency of the GPU may not remain stable after the increase. Specifically, after the electronic device increases the frequency of the GPU, the method further includes: the electronic device uses dynamic clock and voltage regulation DCVS to adjust the frequency of the GPU.

[0025] In a possible design of the first aspect, the method further includes: if the frequency of the GPU is increased for more than a preset number of consecutive times, the electronic device uses DCVS to reduce the frequency of the GPU to a first value, and the electronic device increases the minimum frequency of the GPU to a second value, and the frequency gear to which the second value belongs is one gear higher than the frequency gear to which the first value belongs, so that the GPU can operate at a frequency that makes the first end time of completing image processing earlier than the first time.

[0026] In a possible design of the first aspect, the electronic device further includes a display screen. The electronic device displays a first image, including: at a fifth time after the GPU completes processing of the first image, the display driver drives the display screen to display the first image. Correspondingly, the method further includes: the electronic device acquires the fifth time and the second time. The electronic device adds the refresh cycle of the display screen to the fifth time to obtain a sixth time. The electronic device determines the time point between the second time and the sixth time, at which the ratio of the time interval of the second time to the time interval between the second time and the sixth time is a first ratio, as the first time point.

[0027] In a second aspect, the present application further provides an electronic device, the electronic device comprising a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The display screen comprises a first screen and a second screen, and when the display screen is in a folded state, the display directions of the first screen and the second screen are opposite to each other, and the first screen and the second screen are two different display areas of the display screen. The memory is used to store computer program code, and the computer program code comprises computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method in the above-mentioned first aspect and its possible design methods.

[0028] In a third aspect, the present application provides a chip system, which is applied to an electronic device including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method as in the first aspect and any possible design method thereof.

[0029] In a fourth aspect, the present application provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect and any possible design thereof.

[0030] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method of the first aspect and any possible design thereof.

[0031] It can be understood that the beneficial effects that can be achieved by the electronic device of the second aspect, the chip system of the third aspect, the computer storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The software and hardware architecture diagram of a mobile phone;

[0033] Figure 2 is a schematic diagram of an image processing process;

[0034] Figure 3 This is one of the problem diagrams for using DCVS frequency modulation;

[0035] Figure 4 This is the second diagram of the problem of using DCVS frequency modulation;

[0036] Figure 5 This is the third diagram of the problem of using DCVS frequency modulation;

[0037] Figure 6 A schematic diagram of the preset time T0 provided in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of the principle of determining the next refresh time T2 provided in an embodiment of the present application;

[0039] Figure 8 A corresponding relationship diagram between frequency gears and frequency values ​​provided in an embodiment of the present application;

[0040] Fig. 9 A schematic diagram of the preset time T3 provided in an embodiment of the present application;

[0041] Fig.10 One of the example diagrams of frequency modulation provided in the embodiment of the present application;

[0042] Fig.11 The second example diagram of frequency modulation provided in the embodiment of the present application;

[0043] Fig.12 The third example diagram of frequency modulation provided in the embodiment of the present application;

[0044] Fig.13 The fourth example diagram of frequency modulation provided in the embodiment of the present application;

[0045] Fig.14 This is the fifth example diagram of frequency modulation provided in the embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in a "or" relationship.

[0047] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0048] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0049] The frequency modulation method provided in the embodiment of the present application can be applied to scenarios in which the GPU is required to process images in electronic devices. In particular, scenarios in which the GPU is required to continuously process multiple frames of images. For example, in the process of displaying desktop animations, the GPU needs to continuously process multiple animation frames; in the scenario of playing games, the GPU needs to continuously process multiple frames of game images.

[0050] In the above scenario, by using the frequency modulation method provided in the embodiment of the present application, the electronic device can adjust the frequency of the GPU to meet the needs of work efficiency and ensure that no frames are lost.

[0051] Exemplarily, the electronic device in the embodiment of the present application can be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) and a virtual reality (VR) device, etc., which support display functions. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device. Hereinafter, the present application scheme will be mainly explained by taking the electronic device as an example of a mobile phone.

[0052] In the embodiment of the present application, the software system of the mobile phone can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture or a cloud architecture. The following embodiment will mainly take the Android system of the layered architecture as an example to illustrate the software and hardware architecture of the mobile phone 100.

[0053] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, see Figure 1 The software and hardware architecture of a mobile phone is composed of the application layer (applications), application framework layer (framework), native layer (Native), hardware abstraction layer (HAL), kernel layer (kernel) and hardware layer from top to bottom.

[0054] Among them, the application layer can install applications such as calls, memos, browsers, contacts, cameras, galleries, calendars, desktops, games, videos, chats, etc., only some of which are shown in the figure. These applications all need to display images through the mobile phone's display. For example, the application icons, game screens, video frames, etc. on the desktop need to be displayed through the mobile phone's display. These images need to be obtained through GPU processing (such as rendering and synthesis), that is, they belong to the scene that requires GPU to process images.

[0055] The application framework layer provides an application programming interface (API) and a programming framework for the application layer. Exemplarily, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc., only some of which are shown in the figure.

[0056] Furthermore, the application framework layer also includes a rendering thread (RenderThread). RenderThread can receive calls from upper-layer applications and complete image drawing, and then send the drawing results to the bottom layer (such as the local layer) for further processing.

[0057] The local layer provides various services for the upper layer (such as the application framework layer). Exemplarily, the local layer includes image synthesis processing service (SurfaceFlinger, SF), three-dimensional (3D) graphics processing library (such as: OpenGL ES), two-dimensional (2D) graphics engine (such as: SGL), etc., only some of which are shown in the figure.

[0058] Among them, SF can be used for refresh rate control. It is understandable that when an application in the application framework layer needs to display a new frame of image, it can request a vertical synchronization (Vertical Synchronization, vSync) signal through SF. The vSync signal can trigger the application to complete the rendering of a new frame of image, and then synthesize it and finally send it to the display. In addition, after a refresh of the electronic device's display screen is completed, the vSync signal can be distributed to the application requesting the vSync signal through SF, that is, to the application with rendering requirements. In other words, SF can distribute the vSync signal from the display screen to the application, so as to control the application to start rendering after the display screen is refreshed once. That is, refresh rate control is achieved.

[0059] In addition, SF can also be used to control the rendered content, such as layer stitching.

[0060] The hardware abstraction layer encapsulates the underlying hardware driver and provides a general interface for the upper layer to call the driver so that the upper layer can call the driver to drive the corresponding hardware work. The hardware abstraction layer includes a hardware composer (HWC). HWC can perform image synthesis on the content processed by SF.

[0061] It is understandable that SF can call the interface of the graphics library to complete image synthesis through the GPU. Obviously, completing image synthesis through the GPU requires GPU resources, so the pressure on the GPU can be reduced through image synthesis in HWC.

[0062] The kernel layer includes drivers for driving hardware operations, such as display drivers, audio drivers, Bluetooth drivers, etc., only some of which are shown in the figure. Among them, the display driver is used to drive the display screen to display images.

[0063] Furthermore, the kernel layer also includes a GPU management module for executing the frequency modulation method provided in the embodiment of the present application to achieve frequency regulation of the GPU.

[0064] The hardware layer includes devices such as display screens and GPUs. The display screen can be used for image display, and the GPU can be used for image rendering and synthesis.

[0065] In order to facilitate understanding of the role of the GPU in the processing of a frame of image, the present application embodiment is described in Figure 1 Based on this, further combining Figure 2 The process of image drawing and display in electronic equipment is introduced as follows:

[0066] See also Figure 2 The processing process of a frame of image mainly includes: step 1, rendering; step 2, synthesis; and step 3, sending to display.

[0067] Step 1: Rendering.

[0068] After receiving the vSync signal from the SF, the application can start drawing a new frame of image. Specifically, the application submits a drawing instruction to the rendering thread (RenderThread).

[0069] After receiving the drawing instruction, RenderThread processes the drawing instruction and then sends it to the GPU. After receiving the processed drawing instruction, the GPU can perform rendering processing. That is, during the processing of a frame of image, the GPU can be used for image rendering.

[0070] Step 2: Synthesis.

[0071] RenderThread can also send the image content obtained after processing the drawing instruction to SF, such as the drawn image. After receiving the image content, SF determines whether the image content needs to be synthesized and displayed in this frame, and processes the content that needs to be synthesized and displayed, for example, completing the processing of following the refresh rate. In addition, the content that does not need to be synthesized and displayed is placed in the waiting queue, and it is determined again whether it needs to be synthesized and displayed in the next frame.

[0072] Normally, HWC can only complete some simple synthesis work, while complex synthesis work needs GPU to complete. Therefore, SF can determine whether the image synthesis of the current frame requires GPU support. If necessary, SF can call the interface of the graphics library to complete the synthesis work that HWC cannot complete through GPU. That is, during the processing of a frame of image, GPU can also be used for image synthesis.

[0073] It should be noted that during the execution of step 2, the GPU may not have completed image rendering. Therefore, the GPU needs to complete image synthesis after completing image rendering.

[0074] Step 3, send and display.

[0075] After completing the image synthesis, the HWC can send the synthesis result to the display driver. Based on the synthesis result, the display driver further combines the processing result of the GPU, that is, the processing result of the GPU performing image rendering and image synthesis, obtains the image to be displayed and sends it to the display screen.

[0076] It should be noted that when the HWC completes image synthesis, the GPU may not have completed image rendering and image synthesis. Therefore, after receiving the synthesis result from the HWC, the display driver needs to further wait for the GPU to complete image rendering and image synthesis before it can obtain the image to be displayed and send it to the display screen.

[0077] It can be seen that in the process of processing a frame of image, the GPU needs to complete image rendering and image synthesis. Moreover, if the GPU fails to complete the process in time, the display driver will not be able to obtain the image to be displayed and send it to the display screen in time, which will cause frame loss.

[0078] At the same time, the higher the GPU frequency, the stronger the computing power it provides, and the higher the efficiency of image processing (including image rendering and image synthesis). Therefore, electronic devices can adjust the GPU frequency so that the GPU provides computing power that matches the current image processing task to meet the requirements of work efficiency (such as frame rate, refresh rate, etc.).

[0079] Usually, the GPU developer will provide a native frequency adjustment scheme for the electronic devices equipped with the GPU. TM For example, Qualcomm TM Provides a native frequency adjustment solution - Dynamic Clock and Voltage Scaling (DCVS). Electronic devices using DCVS can adjust the GPU frequency to the corresponding frequency level according to the GPU usage rate over a period of time. The GPU usage rate can be understood as the ratio of the GPU working time to the total time of the period.

[0080] It should be noted that the frequency of the GPU is usually expressed in frequency gears. Generally speaking, the higher the frequency gear number, the lower the GPU frequency; the lower the frequency gear number, the higher the GPU frequency. Exemplarily, the frequency gear includes 8 gears from 0 to 7, and the frequency of the GPU gradually decreases from 0 to 7. For example, the frequency corresponding to gear 0 is 680MHz, the frequency corresponding to gear 6 is 295MHz, and the frequency corresponding to gear 7 is 220MHz.

[0081] An implementation of using DCVS to adjust the frequency level of a GPU is as follows: After the GPU usage rate is above 90% for a certain period of time, the electronic device determines that the GPU is not efficient enough to complete the task at the current frequency level, so the electronic device increases the frequency level of the GPU. When the frequency level of the GPU is increased, the computing power provided by the GPU is stronger, so the GPU can complete the task in a shorter time, and the GPU usage rate will be reduced accordingly. Of course, if the GPU usage rate still cannot be reduced to below 90%, the electronic device will continue to increase the GPU frequency level until it is adjusted to a suitable frequency level that can keep the GPU usage rate stable below 90%.

[0082] Electronic devices using DCVS can adjust the GPU frequency to control the GPU usage rate to be maintained below 90%. However, electronic devices using DCVS may have the following problems:

[0083] Problem 1: Frame loss occurs.

[0084] Electronic devices use DCVS, and the GPU frequency level will only be increased after the GPU usage rate is above 90% for a certain period of time, that is, the frequency level adjustment is delayed.

[0085] See also Figure 3 Before time t1, the frequency gear of the GPU is controlled at gear 7, and the utilization rate of the GPU can be maintained below 90% but close to 90%. At time t1, the load of the GPU (i.e., image processing tasks, such as rendering, synthesis, etc.) suddenly increases, and the utilization rate of the GPU also increases accordingly, such as exceeding 90%. However, since the condition that the GPU utilization rate is above 90% for a certain period of time is not met in a short period of time, the electronic device will not increase the frequency gear of the GPU. That is, the GPU will still work at gear 7 for a short period of time (such as between time t1 and time t2). During this period of time, the GPU may not be able to complete the task efficiently due to the low frequency, resulting in frame drops, such as Figure 3Two frames are lost between time t1 and time t2. After a period of time, such as time t2, the electronic device detects that the GPU usage rate is above 90% for a certain period of time, and then increases the frequency level, such as from level 7 to level 0, so that the GPU can complete the processing of each frame of the image in time without frame loss. However, due to the lag in adjusting the frequency level, the frame loss between time t1 and time t2 cannot be recovered.

[0086] Problem 2: The frequency level may not be increased after frame loss, which may lead to continuous frame loss.

[0087] The GPU usage rate is calculated based on a period of time. If the GPU works less during a period of time, the usage rate is low; if the GPU works more, the usage rate is high. However, the usage rate of the historical time period cannot accurately represent the situation of real-time image processing tasks, so it may cause the electronic device to control the frequency inaccurately, resulting in continuous frame loss.

[0088] See also Figure 4 , between time t3 and time t4, the frequency gear of the GPU is controlled at gear 7, and the utilization rate of the GPU can be maintained below 90%, but there is a frame loss phenomenon, such as a total of 3 frames are lost. Subsequently, at time t4, the electronic device calculates the utilization rate of the GPU between time t3 and time t4. Since frame losses occur between time t3 and time t4, the calculated utilization rate will not be very high, such as still maintaining below 90%. Then, the electronic device will not increase the frequency gear of the GPU. That is, the GPU will still work at gear 7, such as working at gear 7 from time t4 to time t5, and will continue to lose frames, such as a total of 2 frames are lost.

[0089] Question 3: It can easily lead to unstable performance of electronic equipment.

[0090] Electronic devices use DCVS, and the frequency level will be increased only when the GPU usage rate reaches 90% or above. In practice, it is often the case that the usage rate is close to 90% but does not exceed 90% (which can be referred to as the dangerous level). When the usage rate is at the dangerous level, the GPU's operation is very likely to fluctuate due to the influence of the entire machine, such as the power and temperature of the electronic device, resulting in unstable performance of the electronic device, such as occasional frame drops.

[0091] See also Figure 5 , between time t6 and time t7, the GPU usage rate fluctuates between 85% and 90%. Since the usage rate does not meet the requirement of being above 90% for a certain period of time, the electronic device has not adjusted the GPU frequency level, and the GPU has been working at the same frequency level, such as level 7. However, between time t6 and time t7, frame drops occur from time to time, such as Figure 5 2 frames were lost.

[0092] In addition to the above DCVS solution, GPU developers also provide electronic device developers with a customized performance locking (Perflock) solution to allow electronic device developers to more flexibly adjust the GPU frequency level, overcoming the above DCVS problems to a certain extent. The main idea of ​​Perflock is as follows: electronic devices can add points for problem scenarios that require frequency adjustment, then identify the problem scenarios through the points, and use the interface provided by the GPU developer to adjust the GPU frequency level for the problem scenarios.

[0093] However, Perflock is mainly applicable to some problem scenarios that inevitably require frequency modulation (also called must-occur scenarios). For some non-must-occur scenarios, that is, problem scenarios that may require frequency modulation in some cases but may not require frequency modulation in other cases, the use of Perflock in electronic devices often results in a waste of resources.

[0094] A typical scenario is as follows: When the screen refresh rate or application frame rate of an electronic device is high, the GPU is usually required to complete image processing tasks for more frames of images per unit time, so the GPU's computing power requirements are high, which requires the GPU to work at a higher frequency. Therefore, electronic devices can use Perflock to set stake points for scenes with high screen refresh rates (such as 120 Hz), that is, to mark the need to increase the GPU's frequency level when the screen refresh rate reaches 120Hz. Later, during the GPU's image processing process, if the electronic device detects that the screen refresh rate reaches 120Hz, the GPU's frequency level can be increased.

[0095] However, in actual applications, when the screen refresh rate of an electronic device reaches 120Hz, it is not necessarily necessary to increase the frequency level of the GPU. For example, although the screen refresh rate of an electronic device reaches 120Hz, there are not many image processing tasks that require the GPU to complete. For example, the GPU is not required to complete complex synthesis processing. Therefore, even if the frequency level of the GPU is not increased, the GPU can complete the current image processing task in a timely manner, which is a non-essential scenario. In this scenario, if Perflock is still used and the frequency level of the GPU is increased when the screen refresh rate reaches 120Hz, it may lead to a waste of resources.

[0096] Based on the above problems of DCVS and Perflock, an embodiment of the present application provides a frequency modulation method that can be used for GPU frequency modulation.

[0097] Using an embodiment of the present application, during the processing of a frame of image, the electronic device (such as a GPU management module in the kernel layer) can detect whether the end time (also referred to as the first end time) of the GPU completing image processing (including rendering and synthesis) exceeds the preset time T0 (also referred to as the first time). If the end time exceeds the preset time T0, it means that the GPU cannot complete the processing of the current frame image in time. The electronic device can increase the frequency gear of the GPU to ensure that the GPU completes the processing of the current frame image in time to avoid frame loss.

[0098] In this way, the electronic device can increase the GPU frequency based on the situation that the GPU cannot complete image processing in time in the current frame. On the one hand, the electronic device enables the GPU to process the current frame image in time to ensure that no frame is lost; on the other hand, the electronic device will not increase the GPU frequency when the GPU can complete image processing in time, thereby avoiding waste of resources.

[0099] In some embodiments, the preset time T0 may be a time point at which the time from the next refresh time of the display screen is a fixed time length 1. It is understandable that each time the display screen is refreshed, a new frame image needs to be displayed. Therefore, the GPU needs to complete the processing of the current frame image before the next refresh, and the preset time T0 also needs to be before the next refresh time. In this way, the electronic device can quickly determine the preset time T0 based on the next refresh time and the fixed time length 1.

[0100] In other embodiments, considering that the processing progress of different frame images may be different for the electronic device, the preset time T0 may also be a time point that matches the processing progress of the current frame image.

[0101] Specifically, the preset time T0 is the time length (recorded as time length L) from the moment T1 (also referred to as the second time) when the display driver starts to wait for the GPU to complete image processing to the next refresh moment T2, and the time length from the moment T1 to the time length L is the time point at which the ratio of the preset ratio λ1 (such as 0.7, 0.8, etc., also referred to as the first ratio) is . For the content about the display driver waiting for the GPU to complete image processing, please refer to the above Figure 2 The description of step 3 in the previous section will not be repeated here.

[0102] Taking the preset proportion λ1 as 0.7 as an example, Figure 6 During the processing of the k-th frame image, the electronic device may determine that the preset time T0 is the moment T1+0.7*(T2-T1).

[0103] It can be seen that, by adopting this embodiment, the electronic device can determine the matching preset time T0 according to the processing progress of the current frame image, so as to adjust the frequency level of the GPU in a targeted manner.

[0104] In addition, one point that needs to be emphasized is that when the display driver starts waiting for the GPU to complete image processing, it indicates that the CPU has basically completed the relevant processing of sending the image to the display. After that, whether frame loss occurs mainly depends on the working efficiency of the GPU. Therefore, by configuring the preset time T0 as the time after the display driver starts waiting for the GPU to complete image processing, the GPU frequency can be adjusted to accurately control frame loss without being affected only by the working efficiency of the GPU.

[0105] The following describes the specific implementation of determining the time T1 and the next refresh time T2:

[0106] First, determine time T1.

[0107] In practice, the display driver can wait for the GPU to complete image processing through a waiting function (such as plane_wait_input_fence). It is understandable that after the GPU completes image processing, it will release the corresponding fence. Correspondingly, after the display driver queries the fence through the waiting function and finds that it has been released, it can be determined that the GPU has completed image processing. At this time, the display driver can execute the subsequent display sending process and drive the display screen to display the current frame image. Based on this, the electronic device can determine that time T1 is the time when the display driver starts to execute the waiting function.

[0108] Second, determine the next refresh time T2.

[0109] When the display screen displays the k-1th frame image (also called the first image), the electronic device needs to complete the processing of the kth frame image (also called the second image), such as rendering, synthesis and display. In this way, the display screen can refresh and display the kth frame image at the next refresh time. Therefore, the electronic device needs to complete the image processing one frame in advance.

[0110] Among them, the display driver can complete image processing one frame in advance according to the hardware interrupt (TE, which can be understood as the refresh cycle) of the display screen. Figure 7 As shown, between TE0 and TE1, the display screen displays the 0th frame image, and the display driver processes the 1st frame image; between TE1 and TE2, the display screen displays the 1st frame image, and the display driver processes the 2nd frame image... Between TE(n-1) and TEn, the display screen displays the n-1th frame image, and the display driver processes the nth frame image. The time interval in which the display screen displays one frame can be called the first time interval.

[0111] In some embodiments, the electronic device can display the time when the driver finishes processing the previous frame of the image as the approximate start time of the current TE cycle (also referred to as the fifth time). Figure 7 Taking TE1 to TE2 as an example, the time when the display driver finishes processing the first frame of the image is Tlast, and the electronic device can determine Tlast as the starting time from TE1 to TE2, that is, TE1.

[0112] It is understandable that if the first frame of the image is lost, it is just a delayed response, which can also be understood as a weak hand tracking, and will not cause too much impact on the user's use. Therefore, the electronic device can use the solution of the present application to adjust the GPU frequency for the second frame of the image and subsequent images. Then, the moment when the display driver finishes processing the first frame of the image is the first Tlast, such as Figure 7 TE1 in is the first Tlast.

[0113] In practice, after the display driver completes processing of a frame of image, it can indicate the completion of processing of a frame of image through an end function (such as sde_kms_complete_commit). Based on this, the electronic device can use the time when the display driver executes the end function as the start time of the current TE cycle.

[0114] It is understandable that after each refresh, the display will send a vSync signal to trigger the processing of the next frame of the image, that is, a refresh cycle is the time interval between two vSync signals, which can be recorded as △vSync (also called a refresh cycle). For example, if the refresh rate is 120Hz, then △vSync = 8.3333ms; if the refresh rate is 90Hz, then △vSync = 11.1111ms; if the refresh rate is 60Hz, then △vSync = 16.6666ms. Then, the electronic device will add △vSync to the moment when the display driver finishes processing the previous frame of the image, and the end moment of the current TE cycle (also called the sixth time) can be obtained, which is the next refresh moment T2. Still assuming that the current TE cycle is Figure 7 Taking TE1 to TE2 as an example, the time when the display driver finishes processing the first frame of the image is Tlast, and the electronic device can determine the next refresh time T2 = Tlast + △vSync.

[0115] Of course, the electronic device can also record the latest refresh time and add △vSync to the latest refresh time to obtain the next refresh time T2.

[0116] After determining the time T1 and the next refresh time T2, the electronic device can determine the preset time T0 according to the time T1 and the next refresh time T2. Subsequently, the electronic device can detect whether the end time of the GPU completing the image processing exceeds the preset time T0.

[0117] After time T1: If the display driver is still executing the wait function, it indicates that the display driver is still waiting for the GPU to complete image processing. That is, the GPU has not yet completed image processing. On the contrary, if the display driver has finished executing the wait function, it indicates that the display driver has waited until the GPU completes image processing. That is, the GPU has completed image processing. Based on this, after time T1, the electronic device can detect whether the GPU has completed image processing by detecting whether the display driver is still executing the wait function.

[0118] In a specific implementation, the electronic device can detect whether the display driver is executing the waiting function after reaching the preset time T0. If the display driver is still executing the waiting function at the preset time T0, the electronic device can determine that the end time of the GPU completing the image processing exceeds the preset time T0; if the display driver has finished executing the waiting function at the preset time T0, the electronic device can determine that the end time of the GPU completing the image processing does not exceed the preset time T0.

[0119] In practice, after the electronic device detects that the display driver starts to execute the waiting function, it calculates the time difference between the preset time T0 and the current moment (such as moment T1), and sets the timer with the timing duration as the time difference. Subsequently, the electronic device can detect whether the display driver is executing the waiting function when the timer ends. The end of the timing is the preset time, so the electronic device detects whether the display driver is executing the waiting function when the timing ends, and can detect whether the display driver is executing the waiting function after reaching the preset time T0.

[0120] After determining that the end time of the GPU completing image processing exceeds the preset time T0, the electronic device can increase the frequency gear of the GPU. Exemplarily, the electronic device can increase the frequency gear of the GPU to the highest gear to ensure that the GPU can complete the image processing of the current frame before the next refresh time T2, thereby ensuring that the current frame is not lost. Of course, the electronic device can also increase the frequency gear of the GPU by a preset number of gears, such as increasing 3 frequency gears. The embodiments of the present application do not specifically limit this. In the following, the increase to the highest gear is mainly used as an example to illustrate.

[0121] Among them, increasing the frequency gear of the GPU includes: increasing the minimum value of the frequency (also called the minimum frequency) or increasing the actual value of the frequency (also called the actual working frequency). It can be understood that the actual value of the GPU frequency needs to be greater than or equal to the minimum value, that is, the GPU needs to work at or above the minimum value of the frequency.

[0122] After the minimum value is increased, the actual value of the GPU frequency will also change to the minimum value after the increase, so as to ensure that the actual value is greater than or equal to the minimum value. Then the GPU can work at the frequency level corresponding to the minimum value after the increase.

[0123] by Figure 8 Taking the correspondence between the frequency gear and the frequency value shown in the figure as an example, if before the increase, the minimum value of the frequency is f7, and the actual value is also f7, that is, the minimum value and the actual value are both in the 7th gear; the electronic device increases the minimum value to f0, that is, the minimum value is increased to the 0th gear; accordingly, the actual value of the GPU frequency will also be increased to f7, that is, the actual value is also increased to the 0th gear.

[0124] After the actual value is increased, since the minimum value of the GPU frequency does not change, the electronic device can use DCVS to control the GPU frequency after the actual value of the GPU frequency is increased. In other words, the actual value of the GPU frequency may not remain stable after it is increased.

[0125] Still Figure 8 Take the correspondence between the frequency gear and the frequency value shown in the figure as an example. If the minimum value of the frequency is f7 before the increase, the actual value is also f7, that is, the minimum value and the actual value are both in the 7th gear; the electronic device increases the actual value to f0, that is, the actual value is increased to 0th gear; however, since the minimum value still corresponds to the 7th gear and the GPU usage rate is gradually decreasing, the electronic device can use DCVS to gradually lower the actual value to f1, f2...f7. It should be noted that the electronic device uses DCVS to control the change of the actual value by adjusting the minimum value.

[0126] The following will explain how to increase the minimum value and the actual value respectively:

[0127] Method 1: Increase the minimum value.

[0128] After increasing the minimum value of the GPU frequency, the GPU can work at the frequency level corresponding to the increased minimum value for a long time, such as working at level 0 for a long time. However, the GPU working at a higher frequency level for a long time may lead to resource waste. Based on this, after increasing the minimum value, in the subsequent frame image processing process, the electronic device can also reduce the GPU frequency level when detecting that the GPU can efficiently complete the image processing, thereby avoiding resource waste.

[0129] Likewise, lowering the GPU frequency level includes: lowering the minimum value.

[0130] After the minimum value is lowered, the actual value of the GPU frequency will also change to the minimum value after the reduction, and then the GPU can work at the frequency level corresponding to the minimum value after the reduction. Figure 8 Taking the correspondence between the frequency gear and the frequency value shown in the figure as an example, if before the reduction, the minimum value of the frequency is f0, the actual value is also f0, that is, the minimum value and the actual value are both in the 7th gear; the electronic device reduces the minimum value to f1, that is, the minimum value is reduced to the 1st gear; subsequently, the actual value of the GPU frequency will also be reduced to f1, that is, the actual value is also reduced to the 1st gear.

[0131] However, reducing the frequency level of the GPU usually does not include reducing the actual value. The reason is as follows: when the actual value and the minimum value are consistent, if the actual value is reduced, it will cause the actual value to be lower than the minimum value, which does not meet the requirement that the actual value of the GPU frequency should be greater than or equal to the minimum value.

[0132] In some embodiments, after increasing the minimum value of the GPU frequency, the electronic device can detect whether the end time of the GPU completing the image processing exceeds the preset time T3 (also referred to as the third time) during the subsequent image processing. If the end time does not exceed the preset time T3, it means that the GPU can efficiently complete the image processing, and the electronic device can reduce the minimum value, and the actual value of the subsequent GPU frequency will gradually decrease to the minimum value, thereby saving resources when the GPU can efficiently complete the image processing.

[0133] Similar to the preset time T0, the preset time T3 may be a time point at which the time from the next refresh time T2 is the fixed time 2, and the fixed time 2 is greater than the fixed time 1. In this way, the electronic device can also quickly determine the preset time T3 based on the next refresh time T2 and the fixed time 2.

[0134] Alternatively, the preset time T3 may be a time point in the duration L where the ratio of the time length from the moment T1 to the duration L is a preset ratio λ2 (eg, 0.5, 0.6, etc.), and the preset ratio λ2 is less than the preset ratio λ1.

[0135] Taking the preset proportion λ2 as 0.5 as an example, Fig. 9 During the processing of the second frame image shown (such as at the preset time T0), the electronic device increases the minimum value of the GPU frequency to the frequency value of level 0; subsequently, during the processing of the third frame image, the electronic device can determine the preset time T3 as the moment T1+0.5*(T2-T1).

[0136] Among them, regarding the determination of the time T1 and the next refresh time T2, reference may be made to the related description of determining the preset time T0 in the previous text, which will not be repeated here.

[0137] Similarly, after time T1, the electronic device can detect whether the GPU has completed image processing by detecting whether the display driver is executing the waiting function. In a specific implementation, the electronic device can detect whether the display driver is executing the waiting function after reaching the preset time T3. If the display driver is still executing the waiting function at the preset time T3, the electronic device can determine that the end time of the GPU completing image processing exceeds the preset time T3; if the display driver has finished executing the waiting function at the preset time T3, the electronic device can determine that the end time of the GPU completing image processing does not exceed the preset time T3.

[0138] Continue to see Fig. 9 At the preset time T3, the display driver has finished executing the waiting function, and the electronic device can determine that the end time of the GPU completing the image processing does not exceed the preset time T3.

[0139] If the end time of the GPU completing the image processing does not exceed the preset time T3, the electronic device can reduce the minimum value of the GPU frequency to a frequency value of a frequency level lower than the current frequency level. Fig. 9 Taking the processing process of the third frame of the image as an example, the end time of the GPU completing the image processing does not exceed the preset time T3, and the electronic device can reduce the minimum value of the GPU frequency to the frequency value of the 1st gear. Alternatively, the electronic device can also reduce the minimum value to the frequency value of the frequency gear that is a preset number (such as 2, 3, etc.) lower than the current frequency gear. In the following, the frequency value of the frequency gear that is reduced to a frequency gear that is one level lower than the current frequency gear is mainly described as an example.

[0140] After lowering the minimum value of the GPU frequency, the GPU can work for a long time at the frequency gear corresponding to the lowered minimum value, such as working at gear 1 for a long time. During the processing of subsequent frame images, the electronic device can continue to detect whether the frequency gear is appropriate and adjust the minimum value if it is detected to be inappropriate. Among them, the inappropriate frequency gear includes that the end time of the GPU completing the image processing does not exceed the preset time T3 (resulting in resource waste) or the end time of the GPU completing the image processing exceeds the preset time T0 (resulting in frame loss).

[0141] It can be seen that, in the process of processing each frame of image by adopting the first method, the electronic device can detect whether the end time of the GPU completing the image processing exceeds the preset time T3, and if it does not exceed the preset time T3, the minimum value of the GPU frequency is reduced, such as reducing it to the frequency value of the lower frequency gear. Also, the electronic device can detect whether the end time of the GPU completing the image processing exceeds the preset time T0, and if it exceeds the preset time T0, the minimum value of the GPU frequency is increased, such as increasing it to the frequency value of the highest gear.

[0142] In addition, since the preset proportion λ1 is greater than the preset proportion λ2, it means that during the processing of a frame of image, the preset time T0 is closer to the next refresh time T2, while the preset time T3 is closer to the time T1. That is, from front to back, they are: time T1, preset time T3, preset time T0, and next refresh time T2. Then, during the processing of a frame of image, when the electronic device detects that the end time of the GPU completing the image processing does not exceed the preset time T3, it can omit the step of detecting whether the end time of the GPU completing the image processing exceeds the preset time T0, thereby avoiding invalid detection and saving computing resources.

[0143] At this point, it should be noted that after the electronic device reduces the minimum value of the GPU frequency, even if the actual value of the GPU frequency slowly drops to the minimum value, it still satisfies the requirement that the actual value is greater than or equal to the minimum value. Moreover, the slow drop can also ensure that the actual value of the GPU frequency will not suddenly change drastically. Therefore, after reducing the minimum value of the GPU frequency, the actual value needs to be delayed for a period of time before it is reduced to be equal to the minimum value. In other words, after a delay of a period of time, the GPU will actually work at the reduced minimum value, that is, it will take effect. For example, it will take effect after a delay of 100ms.

[0144] See also Fig.10 , before TE0, the actual value and minimum value of the GPU frequency are both the frequency value f7 of gear 7. The end time of the GPU completing image processing between TE0-TE1 exceeds the preset time T0 between TE0-TE1, and the electronic device increases the minimum value of the GPU frequency to the frequency value f0 of gear 0, and the actual value of the GPU frequency value will also quickly (such as at the preset time T0) increase to the frequency value f0. Subsequently, at the preset time T3 between TE1-TE2, the electronic device reduces the minimum value of the GPU frequency to the frequency value f1 of gear 1, but the actual value of the GPU frequency is not reduced to the frequency value f1 until time T4. Subsequently, at the preset time T3 between TE2-TE3, the electronic device further reduces the minimum value of the GPU frequency to the frequency value f2 of gear 2, but the actual value of the GPU frequency is not reduced to the frequency value f2 until time T5.

[0145] That is to say, within the delay duration (also called duration threshold), such as Fig.10 Between the preset time T3 and moment T4 between TE1-TE2 and between the preset time T3 and moment T5 between TE2-TE3, the actual value of the GPU frequency has not actually dropped to the same as the minimum value, and only after the delay duration is reached, it is actually dropped to the same as the minimum value.

[0146] Above Fig.10 In the example, the delay time is short. When the electronic device processes the current frame image, the GPU frequency is reduced to the minimum value. In the first half of the next frame image processing, the actual value of the GPU frequency is reduced to the same as the minimum value. Fig.10 The preset time T3 between TE1 and TE2 is reduced to the minimum value to the frequency value f1, and then at the moment T4 in the first half of the stage between TE2 and TE3, the actual value of the GPU frequency is also reduced to the frequency value f1. In this case, the electronic device detects whether the actual value after taking effect is appropriate during the preset time T0 or the preset time T3 in the process of processing each frame of the image.

[0147] In practice, the delay time may be shorter or longer. If the delay time is longer, the preset time T0 or preset time T3 in the processing of the next frame or more frames of images after the minimum value is reduced may be within the delay time, that is, the actual value has not yet been reduced to the same as the minimum value. Then, the preset time T0 or preset time T3 within the delay time cannot detect whether the actual value after taking effect is appropriate.

[0148] Based on this, in some embodiments, after the electronic device reduces the minimum value of the GPU frequency (also referred to as the fourth time), it will detect whether the frequency gear is appropriate after the delay time is reached. That is, within the delay time, even if the preset time T0 or the preset time T3 is reached, it will not detect whether the frequency gear is appropriate, let alone adjust the frequency gear. In a specific implementation method, after the electronic device reduces the minimum value of the GPU frequency, it can set a timer with a timing duration of the delay duration. Before the timing of the timer ends, when the preset time T0 or the preset time T3 is reached, the electronic device does not detect whether the frequency gear is appropriate; after the timing of the timer ends, when the preset time T0 or the preset time T3 is reached, the electronic device detects whether the frequency gear is appropriate. In this way, the electronic device can avoid detecting and adjusting the frequency gear when it is not effective.

[0149] For example, in Fig.11In the preset time T3 between TE2 and TE3, the minimum value of the GPU frequency of the electronic device is reduced to the frequency value of level 2, but within the delay time (such as <△t), the actual value is not actually reduced to the frequency value of level 2. For example, Fig.11 The time interval between the preset time T3 between TE3-TE4 and the preset time T3 between TE2-TE3 is less than △t, which means that during the preset time T3 between TE3-TE4, the actual value of the GPU frequency has not yet dropped to the frequency value of the second gear. Fig.11 As shown, between TE3 and TE4, the end time of the GPU completing image processing does not exceed the preset time T3 between TE3 and TE4, nor does it mean that the actual value of the GPU frequency can efficiently complete image processing under the frequency value of level 2. Therefore, the electronic device may not detect whether the end time of the GPU completing image processing exceeds the preset time T3 between TE3 and TE4, and will not further lower the minimum value, thereby avoiding incorrect frequency adjustment.

[0150] Continue to see Fig.11 , Fig.11 If the time interval between the preset time T3 between TE4-TE5 and the preset time T3 between TE2-TE3 exceeds the delay time (e.g., denoted as >△t), it means that at the preset time T3 between TE4-TE5, the actual value of the GPU frequency has been reduced to the frequency value of the second gear. Therefore, at the preset time T3 between TE4-TE5, the electronic device can detect whether the end time of the GPU completing the image processing exceeds the preset time T3 between TE4-TE5. And, if Fig.11 As shown, if the end time of the GPU completing the image processing does not exceed the preset time T3 between TE4 and TE5, the electronic device can continue to reduce the minimum value of the GPU frequency, such as reducing it to the frequency value of level 3.

[0151] During the processing of any frame of image, if the electronic device detects that the end time of the GPU completing the image processing exceeds the preset time T3 and does not exceed the preset time T0, it indicates that the GPU working at the current frequency gear can ensure that the image processing is completed in time and does not cause resource waste. That is, the current frequency gear is appropriate. In this case, the electronic device does not need to increase the minimum value or reduce the minimum value.

[0152] Furthermore, if during the processing of y consecutive (y≥1, y is an integer, also referred to as the first number) frames of images, the electronic device detects that the end time of the GPU completing the image processing exceeds the preset time T3, and does not exceed the preset time T0, it indicates that the current frequency gear can maintain the GPU's image processing efficiency within a reasonable range. Subsequently, the electronic device can use DCVS to assist in adjusting the GPU frequency.

[0153] Taking y=3 as an example, the electronic device reduces the minimum value of the GPU frequency to Fig.12 After the frequency value f2 shown, within three consecutive TE cycles of TE3-TE4, TE4-TE5 and TE5-TE6, that is, during the processing of three consecutive frames of images, the electronic device detects that the end time of the GPU completing image processing exceeds the preset time T3 and does not exceed the preset time T0, then the electronic device can use DCVS frequency modulation after TE6.

[0154] Of course, if the electronic device subsequently detects that the end time of the GPU completing image processing exceeds the preset time T0, the above method 1 can be repeated again until a suitable frequency gear is found.

[0155] Method 2: Increase the actual value.

[0156] Unlike the first method, after the actual value of the GPU frequency is increased, the actual value will automatically change under the control of DCVS, such as decreasing, without the need for the electronic device to further reduce the minimum value based on the end time of the GPU completing image processing, thereby reducing the actual value. Therefore, in the second method, the main focus is on the increase of the actual value.

[0157] In some embodiments, after the electronic device detects that the end time of the GPU completing image processing exceeds the preset time T0, the actual value of the GPU frequency is increased to the highest frequency value, such as the frequency value f0. Subsequently, the electronic device can use DCVS to control the actual value of the GPU frequency. After the electronic device detects again that the end time of the GPU completing image processing exceeds the preset time T0, the actual value of the GPU frequency can be increased again to the highest frequency value.

[0158] In this way, on the one hand, it can ensure that the GPU completes image processing in time to avoid frame loss; on the other hand, it can ensure that the GPU frequency level matches the usage rate.

[0159] For example, the electronic device Fig.13The preset time T0 between TEx1-TEx2, TEx3-TEx4 and TEx5-TEx6 shown respectively detects that the end time of the GPU completing the image processing exceeds the preset time T0 in the corresponding TE cycle. Therefore, the electronic device can increase the actual value of the frequency to the highest-speed frequency value, such as f0, at the preset time T0 between TEx1-TEx2, TEx3-TEx4 and TEx5-TEx6. Between each two increases in the actual value to the highest-speed frequency value, the electronic device can use DCVS to adjust the actual value of the GPU frequency. For example, as the utilization rate of the GPU gradually decreases, the electronic device can gradually lower the actual value using DCVS.

[0160] It should be noted that in Fig.13 In the example shown, the electronic devices use DCVS to gradually reduce the actual value from the frequency value f0 to the frequency value f7.

[0161] However, in actual implementation, electronic equipment can be adjusted according to the actual usage rate, not based on Fig.13 See Fig.14 After the electronic device increases the actual value of the frequency to f0 during the preset time T0 between TEx1-TEx2, TEx3-TEx4, TEx5-TEx6 and TEx7-TEx8, the electronic device uses DCVS to gradually reduce the actual value from the frequency value f0 to the frequency value f6, the frequency value f5, the frequency value f5 and the frequency value f5. In other words, the frequency value f7 is not always reduced.

[0162] Furthermore, in this embodiment, if the electronic device uses DCVS to lower the actual value from the frequency value f0 to the same frequency value (also referred to as the first value) after the actual value is increased to f0 for z consecutive times (also referred to as a preset number of times), the electronic device may increase the minimum value to a frequency value that is one frequency level higher than the same frequency value (also referred to as the second value), so that the GPU can operate at a frequency that completes image processing earlier than the preset time T0.

[0163] Take z=3 as an example, Fig.14 As shown, after the preset time T0 between the electronic devices TEx3-TEx4, TEx5-TEx6 and TEx7-TEx8 increases the actual value to f0 for three consecutive times, the electronic devices use DCVS to adjust the actual value from the frequency value f0 to the frequency value f5, that is, the frequency value f5 is the same frequency value as above. Therefore, the electronic device can increase the minimum value to a frequency value that is one frequency gear higher than the frequency value f5, such as f4.

[0164] The present application also provides an electronic device, which may include: a display screen, a memory, and one or more processors (such as a CPU, a GPU, an NPU, etc.). The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device may perform each function or step performed by the device in the above method embodiment.

[0165] The embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected by lines. For example, the interface circuit can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as a processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

[0166] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0167] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0168] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the image processing method in the above-mentioned method embodiments.

[0169] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0170] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0171] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0172] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0173] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0174] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A frequency modulation method, characterized in that: Applied to an electronic device, the electronic device includes a graphics processor GPU, and the method includes: The electronic device displays a first image; In a first time interval when the first image is displayed: the electronic device detects whether a first end time when the GPU completes processing of a second image exceeds a first time, the second image and the first image are two consecutive frames of images, and the first time is within the first time interval; if the first end time exceeds the first time, the electronic device increases the frequency of the GPU; After the GPU completes processing of the second image, the electronic device displays the second image.

2. The method according to claim 1, characterized in that The electronic device also includes a display driver and a hardware overlay HWC. The first time is after the second time when the display driver starts waiting for the GPU to complete processing of the second image. After receiving the synthesis result of the second image from the HWC, the display driver starts waiting for the GPU to complete processing of the second image.

3. The method according to claim 2, characterized in that The electronic device detects whether a first end time at which the GPU completes processing of the second image exceeds a first time, including: At the first time, the electronic device detects whether the display driver is waiting for the GPU to complete processing of the second image; If the display driver is waiting for the GPU to complete processing of the second image, the first end time exceeds the first time; If the display driver is not waiting for the GPU to complete processing of the second image, the first end time does not exceed the first time.

4. The method according to claim 2 or 3, characterized in that: The second time includes: the time when the display driver starts to execute the waiting function; The waiting function is used by the display driver to wait for the GPU to complete processing of the second image. After the GPU completes processing of the second image, the display driver ends executing the waiting function.

5. The method according to any one of claims 1 to 4, characterized in that The electronic device increases the frequency of the GPU, including: The electronic device increases the minimum frequency of the GPU; The method further comprises: In a first time interval during which the first image is displayed: the electronic device detects whether the first end time exceeds a third time, and if the first end time does not exceed the third time, the electronic device reduces the frequency of the GPU; The third time is within the first time interval, and the third time is before the first time.

6. The method according to claim 5, characterized in that The electronic device detects whether a first end time at which the GPU completes processing of the second image exceeds a first time, including: If the first end time exceeds the third time, the electronic device detects whether the first end time at which the GPU completes processing of the second image exceeds the first time.

7. The method according to claim 5 or 6, characterized in that: The electronic device detects whether a first end time at which the GPU completes processing of the second image exceeds a first time, including: If the time interval between the first time and the fourth time is greater than the time threshold, the electronic device detects whether the first end time exceeds the first time; The electronic device detects whether the first end time exceeds a third time, including: If the time interval between the third time and the fourth time is greater than the time threshold, the electronic device detects whether the first end time exceeds the third time; The fourth time is the time when the electronic device last reduced the frequency of the GPU before displaying the first image.

8. The method according to any one of claims 5 to 7, characterized in that: If the first end time exceeds the third time but does not exceed the first time, the electronic device does not adjust the frequency of the GPU.

9. The method according to claim 8, characterized in that The method further comprises: If the first end time corresponding to at least a first number of consecutive images exceeds the third time but does not exceed the first time, the electronic device uses dynamic clock and voltage regulation DCVS to adjust the frequency of the GPU.

10. The method according to any one of claims 1 to 4, characterized in that The electronic device increases the frequency of the GPU, including: The electronic device increases the actual operating frequency of the GPU; After the electronic device increases the frequency of the GPU, the method further includes: The electronic device adjusts the frequency of the GPU using dynamic clock and voltage scaling DCVS.

11. The method according to claim 10, characterized in that The method further comprises: If the frequency of the GPU is increased for more than a preset number of consecutive times, the electronic device uses the DCVS to reduce the frequency of the GPU to a first value, and the electronic device increases the minimum frequency of the GPU to a second value, and the frequency gear to which the second value belongs is one gear higher than the frequency gear to which the first value belongs.

12. The method according to any one of claims 2 to 4, characterized in that: The electronic device further comprises a display screen; The electronic device displays a first image, including: At a fifth time after the GPU completes processing of the first image, the display driver drives the display screen to display the first image; The method further comprises: The electronic device acquires the fifth time and the second time; The electronic device adds the fifth time to the refresh period of the display screen to obtain a sixth time; The electronic device determines, as the first time point, a time point between the second time and the sixth time at which a ratio of the time interval with the second time to the time interval between the second time and the sixth time is a first ratio.

13. An electronic device, characterized in that: include: A display screen, one or more processors, and one or more memories; the one or more processors are coupled to the display screen and the one or more memories; the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method described in any one of claims 1-12.

14. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 12.

Images