Frequency adjustment method and electronic device

Abstract

The present application relates to the technical field of computers. Provided are a frequency adjustment method and an electronic device, which make the frequency of a GPU meet GPU performance requirements or functional requirements, thereby ensuring the usage experience of a user. In the method, a load that matches a scenario corresponding to the currently rendered image can be determined from among a predicted load and a historical load, and the frequency of the GPU is then determined on the basis of the load, wherein the predicted load can be obtained on the basis of rendering information of the currently rendered image frame. A scenario of an image frame can be determined on the basis of an application state or a system state. The greater the load matching the scenario, the higher the determined GPU frequency, and thus a GPU can process more loads, thereby improving performance, reducing stuttering during image frame display, and improving the usage experience of a user; and the smaller the load matching the scenario, the lower the determined GPU frequency, and thus loads to be processed by the GPU are minimized, thereby reducing the power consumption of the GPU.

Description

Frequency modulation method and electronic device

[0001] The present application claims priority from the Chinese patent application No. 202410767858.4 filed on June 13, 2024, and entitled "Frequency modulation method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of computer, and in particular, to a frequency modulation method and an electronic device. BACKGROUND

[0003] A graphics processing unit (GPU) is a microprocessor that can perform image rendering. The higher the frequency of the GPU, the more tasks (or instructions) it can handle in a unit of time, and the better the performance. The lower the frequency of the GPU, the lower the power consumption.

[0004] Currently, the different requirements (such as performance requirements, power consumption requirements, etc.) for the GPU in the image rendering process can be met by adjusting the frequency of the GPU. For example, in the case of small GPU load (such as tasks), the frequency of the GPU can be reduced to reduce power consumption; and in the case of large GPU load, the frequency of the GPU can be increased to obtain better performance.

[0005] However, in the image rendering process, the load of the GPU will change. In order to meet the requirements for the GPU, the frequency of the GPU needs to be adjusted according to the change of the load, and the period of frequency adjustment is relatively long. In the process before reaching the frequency modulation time, problems such as under-adjustment and over-adjustment are likely to occur, which will further affect the performance or power consumption of the GPU, cannot meet the requirements for the GPU in the image rendering process, and will also affect the user experience. SUMMARY

[0006] The present application provides a frequency modulation method and an electronic device, which can more accurately adjust the frequency of the GPU, so that the frequency of the GPU meets the performance requirements or functional requirements of the GPU, and ensures the user experience.

[0007] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a frequency adjusting method is provided. In the method, first, drawing information of a first image frame to be drawn by a GPU, historical load of a second image frame drawn by the GPU, and a scene corresponding to the first image frame are obtained. The scene corresponding to the first image frame is determined according to an application state or a system state, and the second image frame includes at least one image frame before the first image frame. Then, a predicted load of the first image frame is determined according to the drawing information, and a first load matching the scene corresponding to the first image frame is obtained from the historical load and the predicted load. Finally, a first frequency of the GPU is obtained according to the first load, and the first frequency is used for drawing the first image frame by the GPU.

[0009] In the above method, the predicted load obtained according to the drawing information is as close as possible to the load when the GPU is drawing, and the historical load is also as close as possible to the load when the GPU is drawing. Both the predicted load and the historical load are relatively accurate. In addition, the scene of the image frame is determined according to the application state or the system state. In different application states or system states, the performance and power consumption that the GPU can achieve are different. The scene of the image frame can reflect the performance requirement and power consumption requirement of the GPU when drawing the image frame. The scene of the image frame is used to determine the load that matches the performance requirement of the GPU, for example, a larger load, or the load that matches the power consumption requirement of the GPU, for example, a smaller load, from the predicted load and the historical load.

[0010] If the determined load matches the performance requirement of the GPU, the first frequency determined according to the load can make the GPU process more load to improve the performance and reduce the frame freezing problem when the image frame is displayed. If the determined load matches the power consumption requirement of the GPU, the first frequency determined according to the load can make the GPU process less load as much as possible to reduce the power consumption of the GPU.

[0011] In an implementation form of the first aspect, the scene corresponding to the first image frame includes a scene level. Different scene levels are used to indicate the power consumption requirement of the GPU when drawing the first image frame. The above obtaining the first load matching the scene corresponding to the first image frame from the historical load and the predicted load further includes: if the scene corresponding to the first image frame is a first scene, determining the larger load from the historical load and the predicted load as the first load, wherein the scene level of the first scene is greater than a preset level; if the scene corresponding to the first image frame is a second scene, determining the smaller load from the historical load and the predicted load as the first load, wherein the scene level of the second scene is less than or equal to the preset level.

[0012] In this implementation, the load matched to the scene corresponding to the first image frame can be determined from the predicted load and the historical load, and the determined load is more accurate. Then, the corresponding GPU frequency can also be more accurately determined according to the load, so that the GPU frequency can meet the demand for processing a larger load or a smaller load in the current scene, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0013] In addition, if the scene corresponding to the first image frame is the first scene, the larger load is selected, which can also be regarded as predicting that a burst load occurs when the GPU renders the first image frame; if the scene corresponding to the first image frame is the second scene, the smaller load is selected, which can also be regarded as predicting that a reduced load occurs when the GPU renders the first image frame. Then, the above method provided by the present application can also be regarded as adjusting the frequency for the burst load and the reduced load, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0014] In an implementation of the first aspect, the scene corresponding to the first image frame includes a scene level; different scene levels are used to indicate the power consumption demand of the GPU when rendering the first image frame; and the first load matched to the scene corresponding to the first image frame in the historical load and the predicted load is further determined as follows: when the difference load between the historical load and the predicted load is greater than or equal to a preset value, if the scene corresponding to the first image frame is the first scene, the larger load of the historical load and the predicted load is determined as the first load, where the scene level of the first scene is greater than a preset level; and if the scene corresponding to the first image frame is the second scene, the smaller load of the historical load and the predicted load is determined as the first load, where the scene level of the second scene is less than or equal to the preset level.

[0015] In this implementation, if the difference load between the historical load and the predicted load is large, the larger load or the smaller load needs to be selected from the historical load and the predicted load as the first load, where the larger load can more match the performance demand, and the smaller load can more match the power consumption demand, and then the corresponding GPU frequency can also be more accurately determined according to the load, so that the GPU frequency can meet the demand for processing a larger load or a smaller load in the current scene, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0016] In an implementation of the first aspect, the first load matched to the scene corresponding to the first image frame in the historical load and the predicted load is further determined as follows: when the difference load between the historical load and the predicted load is less than a preset value, the first load is determined according to an average load between the historical load and the predicted load.

[0017] In this implementation, since the difference between the historical load and the predicted load is small, it is not accurate to select either one as the first load, and thus the average load is selected as the first load, and the first load obtained in this way is also relatively accurate, and the corresponding GPU frequency can be more accurately determined according to the load, so that the GPU frequency can meet the demand of processing a larger load or a smaller load in the current scene, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0018] In an implementation of the first aspect, after obtaining the first frequency of the GPU according to the first load, the method further includes: obtaining a preset frequency adjustment period of the first image frame according to the scene corresponding to the first image frame; and during rendering of the first image frame by the GPU according to the first frequency, obtaining at least one frequency adjustment time point in the rendering of the first image frame according to the preset frequency adjustment period. At each frequency adjustment time point, a current load of the GPU from the start of rendering of the first image frame to the frequency adjustment time point is obtained, and a second frequency of the GPU is obtained according to the current load. The second frequency is used for rendering of the first image frame by the GPU from the frequency adjustment time point to a next frequency adjustment time point.

[0019] In this implementation, if the current load is large, the second frequency is also high, and if the current load is small, the second frequency is also low, so that the second frequency can process the current load, and fine adjustment of the frequency during rendering of the first image frame is realized, so that the GPU can be further adjusted in frequency on the basis of the first frequency, and the frequency of the GPU is more accurate.

[0020] In addition, the above implementation can also be regarded as shortening the frequency adjustment period of the GPU. The shorter the frequency adjustment period, the faster the frequency adjustment response of the GPU to the load, and thus the efficiency of the frequency adjustment of the GPU is improved.

[0021] In an implementation of the first aspect, the obtaining of the second frequency of the GPU according to the current load further includes: obtaining a second load that matches the scene corresponding to the first image frame from the first load and the current load, and obtaining the second frequency of the GPU according to the second load.

[0022] In this implementation, the load that matches the scene corresponding to the current rendering of the first image frame can be determined from the relatively accurate first load and the current load, and the determined load is more accurate. The corresponding GPU frequency can be more accurately determined according to the load.

[0023] In an implementation form of the first aspect, the second load, which matches the scene corresponding to the first image frame, is determined from the first load and the current load, further comprising: if the scene corresponding to the first image frame is the first scene, determining the second load as the larger one of the first load and the current load; and if the scene corresponding to the first image frame is the second scene, determining the second load as the smaller one of the first load and the current load.

[0024] This implementation form can also determine a more accurate second load, and thus the corresponding GPU frequency can also be determined more accurately based on the load, so that the GPU can further adjust the frequency more finely during rendering of the first image frame.

[0025] In an implementation form of the first aspect, the method further comprises: obtaining a predicted completion time of the GPU rendering the first image frame, wherein the predicted completion time is determined according to the first frequency and the predicted load; and if the GPU does not complete rendering of the first image frame at the predicted completion time, obtaining a third frequency of the GPU, wherein the third frequency is used for the GPU to render the first image frame after the predicted completion time, and the third frequency is greater than the frequency corresponding to the predicted completion time.

[0026] In this implementation form, if the GPU does not complete rendering of the first image frame at the predicted completion time, it means that the frequency of the GPU at the current predicted completion time (which can be the first frequency or the second frequency) is too low, resulting in too long a time for rendering the first image frame. In this case, a third frequency with a higher frequency can be obtained, so that the GPU can complete rendering of the first image frame as soon as possible and improve the rendering efficiency.

[0027] In an implementation form of the first aspect, the rendering information corresponding to the first image frame comprises one or more of a number of blur layers, a number of graph layers, a number of rendering instructions, an animation effect, a rendering graph type, and a rendering graph size.

[0028] It can be understood that the rendering information can reflect the load of the GPU to some extent.

[0029] In an implementation form of the first aspect, the first scene comprises at least one of an application startup scene, a picture clicking scene, a page switching scene, a music playing scene, and a video playing scene; the second scene comprises at least one of the application startup scene, the picture clicking scene, the page switching scene, the music playing scene, and the video playing scene; and the first scene is different from the second scene.

[0030] It can be understood that the requirements for the GPU are different in different scenarios. For example, in an application startup scenario, the GPU needs to have high performance, i.e., a high GPU frequency, to handle a large load, so as to ensure that the application startup process has as few stalls as possible. For example, in a page switching scenario, the GPU needs to have low power consumption, i.e., a low GPU frequency, to handle a small load.

[0031] In a second aspect, a frequency modulation method for drawing information is provided, and is applied to an electronic device. In the method, before the electronic device draws a first image frame, the electronic device obtains a first load corresponding to the first image frame according to drawing information of the first image frame. Then, the electronic device obtains a frequency of a GPU according to the first load. The GPU is configured to draw the first image frame in a vertical synchronization period according to the frequency. The electronic device detects a drawing progress of the first image frame by the GPU in the vertical synchronization period, and increases the frequency of the GPU in the vertical synchronization period according to the drawing progress. The GPU is configured to continue drawing the first image frame in the vertical synchronization period according to the increased frequency.

[0032] In the method, the electronic device can detect the drawing progress of the first image frame by the GPU, and increase the frequency of the GPU in the vertical synchronization period according to the drawing progress. In this way, the GPU can complete the drawing of the first image frame quickly in the subsequent period by using a higher frequency, and the performance and drawing efficiency of the GPU are improved.

[0033] In an implementation form of the second aspect, when the electronic device detects the drawing progress of the first image frame by the GPU, the electronic device further includes: detecting the drawing progress of the first image frame by the GPU based on a predicted time at which the GPU completes the drawing of the first image frame. The drawing progress indicates that the GPU does not complete the drawing of the first image frame.

[0034] In this implementation form, when the GPU does not complete the drawing of the first image frame at the predicted time, the electronic device increases the frequency of the GPU in the vertical synchronization period. In this way, the drawing of the first image frame by the GPU is accelerated, and the drawing efficiency is improved.

[0035] In an implementation form of the second aspect, when the electronic device detects the drawing progress of the first image frame by the GPU, the electronic device further includes: detecting the drawing progress of the first image frame by the GPU once every preset frequency modulation period in the vertical synchronization period. The preset frequency modulation period is less than or equal to half of the vertical synchronization period.

[0036] In this implementation form, the electronic device can finely adjust the frequency of the GPU in the vertical synchronization period, so that the frequency of the GPU when drawing the first image frame is more accurate, or more in line with the performance requirement or the power consumption requirement, thereby improving the user experience.

[0037] It can be understood that the frequency adjustment method in the second aspect can also be used to adjust the frequency of the central processing unit (CPU) when drawing an image frame.

[0038] In a third aspect, a frequency adjustment apparatus is provided, comprising:

[0039] a load prediction module configured to obtain drawing information of a first image frame to be drawn by a processor.

[0040] a frequency adjustment module configured to obtain a historical load of the processor when drawing a second image frame.

[0041] a scene identification module configured to obtain a scene corresponding to the first image frame; the scene corresponding to the first image frame is determined according to an application state or a system state; the second image frame comprises at least one image frame before the first image frame.

[0042] The load prediction module is further configured to determine a predicted load of the first image frame according to the drawing information.

[0043] The frequency adjustment module is further configured to obtain a first load matched with the scene corresponding to the first image frame from the historical load and the predicted load. The frequency adjustment module is configured to obtain a first frequency of the processor according to the first load; the first frequency is used for the processor to draw the first image frame.

[0044] In an implementation form of the third aspect, the scene corresponding to the first image frame comprises a scene level; different scene levels are used to indicate power consumption requirements of the processor when drawing the first image frame; the frequency adjustment module is further configured to, if the scene corresponding to the first image frame is a first scene, determine a larger load from the historical load and the predicted load as the first load; the scene level of the first scene is greater than a preset level; if the scene corresponding to the first image frame is a second scene, determine a smaller load from the historical load and the predicted load as the first load; the scene level of the second scene is less than or equal to the preset level.

[0045] In an implementation form of the third aspect, the frequency adjustment module is further configured to, when a difference load between the historical load and the predicted load is greater than or equal to a preset value, if the scene corresponding to the first image frame is a first scene, determine a larger load from the historical load and the predicted load as the first load; the scene level of the first scene is greater than a preset level; if the scene corresponding to the first image frame is a second scene, determine a smaller load from the historical load and the predicted load as the first load; the scene level of the second scene is less than or equal to the preset level.

[0046] In an implementation form of the third aspect, the frequency adjustment module is further configured to, when a difference load between the historical load and the predicted load is less than a preset value, determine the first load according to an average load between the historical load and the predicted load.

[0047] In an implementation form of the third aspect, after obtaining the first frequency of the processor according to the first load, the frequency adjusting module is further configured to obtain a preset frequency adjusting period of the first image frame according to the scene corresponding to the first image frame; during the process of rendering the first image frame by the GPU according to the first frequency, obtain at least one frequency adjusting time point in the process of rendering the first image frame according to the preset frequency adjusting period; at each frequency adjusting time point, obtain a current load of the processor corresponding to a period from the start of rendering the first image frame to the frequency adjusting time point; obtain a second frequency of the processor according to the current load; and the second frequency is used for rendering the first image frame by the processor between the frequency adjusting time point and a next frequency adjusting time point.

[0048] In an implementation form of the third aspect, the frequency adjusting module is further configured to obtain a second load matched with the scene corresponding to the first image frame from the first load and the current load; and obtain a second frequency of the processor according to the second load.

[0049] In an implementation form of the third aspect, the frequency adjusting module is further configured to: if the scene corresponding to the first image frame is a first scene, determine a larger load from the first load and the current load as the second load; or if the scene corresponding to the first image frame is a second scene, determine a smaller load from the first load and the current load as the second load.

[0050] In an implementation form of the third aspect, the frequency predicting module is further configured to obtain a predicted completion time point of rendering the first image frame by the processor; the predicted completion time point is determined according to the first frequency and the predicted load; and the frequency adjusting module is further configured to: if the processor does not complete the rendering of the first image frame at the predicted completion time point, obtain a third frequency of the processor; the third frequency is used for rendering the first image frame by the processor after the predicted completion time point; and the third frequency is greater than a frequency of the processor corresponding to the predicted completion time point.

[0051] The processor in the third aspect can be a central processing unit (CPU), a GPU, or a processor integrating a CPU and a GPU.

[0052] The fourth aspect provides an electronic device including the frequency adjusting apparatus in the third aspect and any implementation form thereof.

[0053] The fifth aspect provides an electronic device including a memory and one or more processors; the memory is coupled to the processor; and the memory stores computer program codes including computer instructions, which, when executed by the processor, cause the electronic device to perform the frequency adjusting method in the first aspect and any implementation form thereof, or perform the frequency adjusting method based on rendering information in the second aspect and any implementation form thereof.

[0054] In a sixth aspect, a computer-readable storage medium is provided, including computer instructions, when the computer instructions are executed on an electronic device, causing the electronic device to perform the frequency modulation method as described in the first aspect and any possible implementation thereof, or perform the frequency modulation method based on drawing information as described in the second aspect and any possible implementation thereof.

[0055] In a seventh aspect, a computer program product is provided, when the computer program product is executed on a computer, causing the computer to perform the frequency modulation method as described in the first aspect and any possible implementation thereof, or perform the frequency modulation method based on drawing information as described in the second aspect and any possible implementation thereof.

[0056] The frequency modulation device provided in the third aspect, the electronic device provided in the fourth aspect, the electronic device provided in the fifth aspect, the computer-readable storage medium provided in the sixth aspect, and the computer program product provided in the seventh aspect can achieve the beneficial effects as described in the first aspect and any possible implementation thereof, or the beneficial effects as described in the second aspect and any possible implementation thereof, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0057] FIG. 1 shows a schematic diagram of GPU frequency modulation provided by an embodiment of the present application;

[0058] FIG. 2 shows a schematic diagram of GPU frequency modulation provided by an embodiment of the present application;

[0059] FIG. 3 shows a schematic diagram of GPU frequency modulation provided by an embodiment of the present application;

[0060] FIG. 4 shows a schematic diagram of an electronic device provided by an embodiment of the present application;

[0061] FIG. 5 shows a schematic diagram of an electronic device provided by an embodiment of the present application;

[0062] FIG. 6 shows a schematic diagram of an electronic device provided by an embodiment of the present application;

[0063] FIG. 7 shows a flowchart of a frequency modulation method provided by an embodiment of the present application;

[0064] FIG. 8 shows a schematic diagram of a scene corresponding to an image frame provided by an embodiment of the present application;

[0065] FIG. 9 shows a schematic diagram of GPU frequency modulation provided by an embodiment of the present application;

[0066] FIG. 10 shows a schematic diagram of GPU frequency modulation provided by an embodiment of the present application;

[0067] FIG. 11 shows a schematic diagram of GPU frequency modulation provided by an embodiment of the present application;

[0068] FIG. 12 shows a schematic diagram seven of GPU frequency adjustment provided by an embodiment of the present application;

[0069] FIG. 13 shows a schematic diagram eight of GPU frequency adjustment provided by an embodiment of the present application;

[0070] FIG. 14 shows a schematic diagram four of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the present application, unless otherwise specified, “ / ” represents an “or” relationship between the objects before and after the “ / ”, for example, A / B can represent A or B; “and / or” in the present application is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In the description of the present application, unless otherwise specified, “multiple” means two or more than two. “At least one of the following (one)” or similar expressions means any combination of these items, including any combination of single item (one) or multiple items (one). For example, at least one of a, b, or c, can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same function and role are distinguished by using “first”, “second”, etc. The person skilled in the art can understand that “first”, “second”, etc. do not limit the quantity and execution order, and “first”, “second”, etc. also do not mean necessarily different. At the same time, in the embodiments of the present application, the words “exemplary” or “for example” are used to represent as an example, illustration or description. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of “exemplary” or “for example” and the like is intended to present the relevant concept in a specific manner, for understanding.

[0072] In addition, the business scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that as new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0073] The image rendering process includes a central processing unit (CPU) rendering stage and a GPU rendering stage, that is, the final displayed image is generated by the CPU and the GPU in cooperation. The CPU is responsible for scene management, logical processing, and rendering instruction generation, and the GPU is responsible for actual image rendering and parallel computing according to the rendering instruction.

[0074] For the GPU, the frequency of the GPU affects the performance and power consumption of the GPU in the image rendering process. The higher the frequency of the GPU, the more tasks (or instructions) can be processed in a unit of time, and the better the performance, but the power consumption of the GPU will also increase accordingly. The lower the frequency of the GPU, the lower the power consumption, but the number of tasks that can be processed in a unit of time will decrease, and the performance will be poor. The tasks processed by the GPU can be referred to as GPU load. In the image rendering process, the GPU load is different, and the requirements for the GPU are also different. For example, in the case of small GPU load, the power consumption of the GPU can be required to be low, and in the case of large GPU load, the performance of the GPU can be required to be better.

[0075] Currently, different requirements for the GPU in the image rendering process can be met by adjusting the frequency of the GPU. For example, in the case of small GPU load, the frequency of the GPU can be reduced to reduce power consumption, and in the case of large GPU load, the frequency of the GPU can be increased to obtain better performance.

[0076] However, since the current GPU frequency adjustment period is long, it takes a long time, such as tens of milliseconds or even hundreds of milliseconds, to adjust the frequency of the GPU according to the load during the GPU frequency adjustment process, which makes the response speed of the GPU relatively slow.

[0077] For example, referring to FIG. 1, the frequency of the GPU before t3 is 500 MHz, and after t3, if the frequency is to be adjusted according to the load obtained at t3 (for example, to 750 MHz), it may take t6 to complete the adjustment.

[0078] In addition, during the frequency adjustment process, the load of the GPU changes, for example, there is a burst load (that is, a sudden increase in tasks), and the frequency needs to be adjusted according to the changing load when the time indicated by the frequency adjustment period arrives. However, since the frequency adjustment period is relatively long, during the process of not reaching the frequency adjustment time, the current frequency of the GPU cannot meet the requirements of the current load. For example, the current load is relatively large, but the current frequency of the GPU is relatively small (the frequency adjustment period has not arrived, and the frequency has not been adjusted), which causes the problem of frequency under-adjustment. The occurrence of this problem will affect the performance of the GPU and cannot meet the performance requirements of the GPU in the image rendering process.

[0079] For example, referring to FIG. 2, the load of the GPU increases at time t2, and because the frequency adjustment period of the GPU is long and the response is slow, when the frequency is adjusted for the load after time t2, the frequency of the GPU can be adjusted to the target frequency (e.g., from 300 MHz to 500 MHz) at time t6. During the period between time t2 and time t6, the frequency of the GPU is always low and cannot meet the processing requirements of the load after time t2, which results in frequency under-adjustment. Further, because of the frequency under-adjustment, the GPU cannot process too much load and the performance of the GPU is poor, which results in the final displayed image being stuck and affects the user experience.

[0080] Alternatively, when adjusting the frequency, the target frequency of the GPU can be set according to the user operation scenario corresponding to the image frame currently drawn by the GPU, for example, a scenario of starting an application, a scenario of sliding an interface, a scenario of clicking a picture, and the like. However, if the target frequency set for the user scenario is too high and the current image frame drawn by the GPU does not need a high frequency, frequency over-adjustment is likely to occur when adjusting the frequency of the GPU, which affects the power consumption of the GPU and cannot meet the power consumption requirements of the GPU in the image rendering process.

[0081] For example, the target frequency set by the GPU for the current image frame is 500 MHz, but the current image frame drawn by the GPU can only need 300 MHz, which is a case of frequency over-adjustment. Because of the frequency over-adjustment, the GPU is always in a high power consumption state.

[0082] Alternatively, referring to FIG. 3, the load of the GPU decreases at time t2, and needs to be adjusted to a low frequency (e.g., 300 MHz) at the time (e.g., time t6) indicated by the frequency adjustment period. However, because the frequency adjustment period is long, during the period before the frequency adjustment time (e.g., between time t2 and time t6), the frequency of the GPU is always at a high level (e.g., 500 MHz), which exceeds the processing requirements of the load after time t2, which also results in frequency over-adjustment. Further, because of the frequency over-adjustment, the power consumption of the GPU is high.

[0083] It can be understood that in the manner of setting the GPU target frequency for the user operation scene corresponding to the image frame, if the same target frequency is used in one user operation scene, but the load of the GPU also changes in one user operation scene, the frequency required by the GPU also changes, and the target frequency may not meet the requirements of the GPU or exceed the requirements of the GPU, so that the above-mentioned frequency under-regulation or frequency over-regulation problems are likely to occur, thereby affecting the performance or power consumption of the GPU, and making it difficult for the GPU to meet the performance requirements or power consumption requirements when drawing the image frame.

[0084] In addition, when adjusting the frequency, the load when the GPU draws the previous image frame can also be collected, and the load of the next frame is predicted by using an algorithm, so that the frequency of the GPU when drawing the next image frame is adjusted according to the load of the next frame. This manner is applicable to scenes with little load change, but is not applicable to scenes with burst load. If the burst load occurs when the current image frame is drawn, the frequency under-regulation problem occurs, affecting the performance or power consumption of the GPU, and it also takes tens or even hundreds of milliseconds to increase the frequency of the GPU.

[0085] Based on the above, an embodiment of the present application provides a frequency adjustment method. The method can obtain drawing information of a first image frame to be drawn by a GPU, historical load when the GPU draws a second image frame, and a scene corresponding to the first image frame. The scene corresponding to the first image frame is determined according to an application state or a system state, and the second image frame includes at least one image frame before the first image frame. Then, a preset load of the first image frame is determined according to the drawing information, a first load matched with the scene corresponding to the first image frame is obtained from the historical load and a predicted load, and a first frequency of the GPU is obtained according to the first load, wherein the first frequency is used for the GPU to draw the first image frame.

[0086] In the above method, the drawing information can reflect the load amount when the GPU draws, so that the predicted load obtained according to the drawing information is as close as possible to the load when the GPU draws; the historical load can also be as close as possible to the load when the GPU draws. Both the predicted load and the historical load can be regarded as loads that are close to the current situation when the GPU draws and are relatively accurate.

[0087] In addition, the scene of the image frame is determined according to the application state or the system state, and the performance, power consumption, etc. that the GPU can reach are different in different application states or system states. The scene of the image frame can reflect the performance requirements, power consumption requirements, etc. when the GPU draws the image frame, and the scene of the image frame is used to determine the load that can match the performance requirements of the GPU, for example, a larger load, or the load that matches the power consumption requirements of the GPU, for example, a smaller load, from the predicted load and the historical load.

[0088] If the determined load matches the performance requirement of the GPU, the first frequency determined according to the load can make the GPU process more load to improve performance and reduce the frame freezing problem during image frame display. If the determined load matches the power consumption requirement of the GPU, the first frequency determined according to the load can make the GPU process less load to reduce the power consumption of the GPU.

[0089] It can be understood that the above method introduces the scene of the image frame to determine the frequency of the GPU, so that the frequency of the GPU can match the performance requirement or power consumption requirement of the GPU embodied by the scene, meet the requirement of the GPU in the image rendering process, and improve the user experience.

[0090] The above method can be applied to an electronic device. Referring to FIG. 4, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195.

[0091] It can be understood that the structure of the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than the illustration, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software or a combination of software and hardware.

[0092] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP) (not shown in the figure), a modem processor (not shown in the figure), a GPU, an image signal processor (ISP) (not shown in the figure), a controller (not shown in the figure), a memory (not shown in the figure), a video codec (not shown in the figure), a digital signal processor (DSP) (not shown in the figure), a baseband processor (not shown in the figure), and / or a neural-network processing unit (NPU) (not shown in the figure), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0093] Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0094] The processor 110 can also be provided with a memory for storing instructions and data.

[0095] In some embodiments, the processor 110 can include one or more interfaces. The interface can include an inter-integrated circuit (I2C) interface (not shown in the figure), an inter-integrated circuit sound (I2S) interface (not shown in the figure), a pulse code modulation (PCM) interface (not shown in the figure), a universal asynchronous receiver / transmitter (UART) interface (not shown in the figure), a mobile industry processor interface (MIPI) (not shown in the figure), a general-purpose input / output (GPIO) interface (not shown in the figure), a subscriber identity module (SIM) interface (not shown in the figure), and / or a universal serial bus (USB) interface (not shown in the figure), etc.

[0096] The charging management module 140 is configured to receive charging input from a charger. The charger can be a wireless charger or a wired charger. The charging management module 140 charges the battery 142 and also supplies power to the electronic device through the power management module 141.

[0097] The power management module 141 is configured to connect the battery 142 and the charging management module 140 to the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160, etc. The power management module 141 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc.

[0098] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0099] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals.

[0100] The mobile communication module 150 can provide a solution including 2G / 3G / 4G / 5G wireless communication applied to the electronic device 100. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves radiated through the antenna 1.

[0101] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs sound signals through the audio device or displays images or videos through the display screen 194.

[0102] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100. The wireless communication module 160 receives an electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and radiate it as an electromagnetic wave via the antenna 2.

[0103] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-CDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0104] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0105] In some embodiments, the processor 110 can also implement the operation of image rendering. Further, the operation of image rendering can include a processor rendering stage and a GPU rendering stage. In the processor rendering stage, the processor determines the image frame to be rendered, and can perform scene management, logical processing on the image frame to be rendered, and generate rendering instructions for the image frame, and send the rendering instructions to the GPU. In the GPU rendering stage, the GPU completes the rendering operation of the image frame according to the rendering instructions. The image frame rendered by the GPU can be sent to the display screen 194 by the CPU or the GPU for display.

[0106] The rendering instructions described above can include rendering information of the image frame. For example, the rendering information can include the number of blur layers of the image frame, the number of image layers of the image frame, the number of rendering instructions (or the number of rendering instructions), the animation effect of the image frame, the type of drawing graphics in the image frame, the size of drawing graphics in the image frame, the vertex information of drawing graphics in the image frame, the color information of drawing graphics in the image frame, the rendering material information, the scene information of the image frame, etc.

[0107] It can be understood that during image rendering, the rendering is usually performed frame by frame. Each frame (image frame) is calculated and generated according to the rendering information described above. In addition, "frame" is the basic unit of animation and video, which represents a group of static images displayed at a specific time point, and thus the image frame in the embodiments of the present application includes at least one image.

[0108] In addition, the processor 110 can also be referred to as a CPU.

[0109] In other embodiments, in order to enable the GPU to meet the performance requirement or the power consumption requirement during the image rendering process, the processor can also predict the load of the GPU when the GPU is currently rendering the image frame based on the scene of the current image frame rendered by the GPU, and determine the target frequency of the GPU according to the load, so that the GPU renders the image frame according to the target frequency.

[0110] The scene of the image frame can include an application startup scene, a picture click scene, a page switching scene, a music playing scene, a video playing scene, a music playing scene, a control click scene, a content click scene, etc. In addition, the scene of the image frame includes a scene level, and different scene levels can indicate the performance requirement, the power consumption requirement, etc. of the GPU when rendering the image frame. For example, the scene level of the image frame includes five levels, wherein levels 1, 2 and 3 indicate that the GPU should have low power consumption when rendering the image frame, and levels 4 and 5 indicate that the GPU should have good performance when rendering the image frame.

[0111] The scene of the image frame can include a first scene and a second scene based on a scene level of the image frame. The scene level of the first scene is greater than a preset level, that is, the performance of the GPU is required to be better in the first scene, and the scene level of the second scene is less than or equal to the preset level, that is, the power consumption of the GPU is required to be lower in the second scene.

[0112] For example, the application startup scene requires a good GPU performance, that is, a high GPU frequency, to handle a large load, so as to ensure that the application startup process as little as possible appears a stall and the like; for another example, the page switching scene requires a low GPU power consumption, that is, a low GPU frequency, to handle a small load. It can be seen that the scene of the image frame can reflect the performance requirement or the power consumption requirement required by the GPU, so that the predicted load of the GPU when drawing the image frame can be more matched to the required performance requirement or power consumption requirement, and the GPU can meet the performance requirement or power consumption requirement when drawing the image frame at the determined target frequency.

[0113] It can be understood that the scene level of the image frame can be pre-configured for each image frame, or can be configured by default by the processor 110 after obtaining the scene of the image frame. In addition, the scene level of the same image frame can be the same or different for different electronic devices.

[0114] Taking that the image frame currently drawn by the processor is a first image frame as an example. The processor obtains a predicted load of the GPU when drawing the first image frame according to the drawing information corresponding to the first image frame, and the processor also obtains a historical load of the GPU when drawing a second image frame, wherein the second image frame includes at least one image before the first image frame. The processor obtains a first load matched to the scene corresponding to the first image frame from the historical load and the predicted load, and obtains a first frequency of the GPU according to the first load. Then, the GPU draws the first image frame according to the first frequency.

[0115] For example, the drawing information used to obtain the predicted load can include one or more of the number of blur layers, the number of graph layers, the number of drawing instructions, an animation effect, a drawing graph type, and a drawing graph size.

[0116] The drawing information can reflect the load of the GPU when drawing, so the predicted load obtained according to the drawing information can be as close as possible to the load of the GPU when drawing, and therefore the predicted load can be used as the load of the GPU when drawing the first image frame. The historical load has a certain reference significance, and the historical load can also be as close as possible to the load of the GPU when drawing, and therefore the historical load can also be used as the load of the GPU when drawing the first image frame. The predicted load and the historical load in the embodiments of the present application can be regarded as relatively accurate loads.

[0117] Then, the processor determines a load that can match the scene corresponding to the first image frame from the predicted load and the historical load, such as a larger load that matches the GPU performance requirement or a smaller load that matches the GPU power consumption requirement, according to the scene of the first image frame. Thus, the first frequency determined based on the load can also meet the performance requirement or the power consumption requirement, such as a higher frequency corresponding to a larger load to meet the performance requirement, a lower frequency corresponding to a smaller load to meet the power consumption requirement, and the like.

[0118] In some other embodiments, during the process in which the GPU renders the first image frame according to the first frequency, the processor 110 further acquires a preset frequency adjustment period of the first image frame according to the scene corresponding to the first image frame. The processor 110 can acquire at least one frequency adjustment time point of the GPU during the process of rendering the first image frame according to the preset frequency adjustment period, and the processor 110 acquires a current load corresponding to the GPU between the start of rendering the first image frame and the current frequency adjustment time point at each frequency adjustment time point. Then, the processor acquires a second frequency of the GPU according to the current load, and the GPU renders the first image frame using the second frequency after the current frequency adjustment time point and before the next frequency adjustment time point. The time length between the two frequency adjustment time points represents a frequency adjustment period.

[0119] If the current load is larger, such as greater than the first load, it can be determined that the GPU currently has more tasks to process, and then the GPU frequency can be appropriately increased, that is, the second frequency is greater than the first frequency. If the current load is smaller, such as less than the first load, it can be determined that the GPU currently has less tasks to process, and then the GPU frequency can be appropriately reduced, that is, the second frequency is less than the first frequency.

[0120] Alternatively, the processor acquires a second load that matches the scene corresponding to the first image frame from the first load and the current load, and acquires a second frequency of the GPU according to the second load.

[0121] The above preset frequency adjustment period is less than or equal to half of the vertical synchronization period of the GPU. Further, the preset frequency adjustment period can be used to adjust the GPU more finely, or in other words, intra-frame frequency adjustment. Since the preset frequency adjustment period is shorter, the adjustment of the GPU frequency is more rapid, and the response of the processor 110 to the frequency adjustment operation is also faster. Based on the more rapid frequency adjustment response and the multiple frequency adjustments within a frame, the frequency of the GPU during the rendering of the first image frame can be more accurate, or more in line with the performance requirement or the power consumption requirement, thereby improving the user experience.

[0122] In some other embodiments, the processor 110 further acquires a predicted completion time point of the GPU for rendering the first image frame at the first frequency according to the determined first frequency, determines whether the GPU completes the rendering of the first image frame at the predicted completion time point, and acquires a third frequency of the GPU if the rendering is not completed.

[0123] It can be understood that if the GPU does not complete the rendering of the first image frame at the predicted completion time, it means that the first frequency or the adjusted second frequency can be set too low, and the GPU cannot complete the rendering task on time. In this case, the frequency of the GPU can be increased again to improve the processing performance of the GPU. Therefore, the third frequency is greater than the frequency corresponding to the predicted completion time of the GPU, that is, the third frequency is greater than the first frequency or the second frequency.

[0124] In the above embodiment, the processor 110 can obtain the predicted load according to the rendering information, and determine the first load matched with the scene corresponding to the first image frame according to the historical load and the predicted load, and determine the frequency of the GPU according to the first load.

[0125] In other embodiments, the processor 110 can also directly obtain the first load of the GPU according to the rendering information, and then determine the frequency of the GPU according to the first load. For example, before the GPU renders the first image frame, the processor 110 obtains the first load corresponding to the first image frame according to the rendering information of the first image frame, and obtains the frequency of the GPU according to the first load. The GPU renders the first image frame according to the obtained frequency within a vertical synchronization period. Within the vertical synchronization period, the processor 110 detects the rendering progress of the GPU on the first image frame, and increases the frequency of the GPU within the vertical synchronization period according to the rendering progress, and the GPU continues to render the first image frame within the vertical synchronization period according to the increased frequency. In this way, the GPU can use a higher frequency to quickly complete the rendering of the first image frame in the future, so as to reduce the possibility of display lag.

[0126] It can be understood that the GPU can render according to the rendering instruction sent by the processor 110. The rendering progress represents the completion of the rendering instruction by the GPU, or shows the completion of the rendering of the first image frame.

[0127] In some embodiments, when detecting the rendering progress, the processor 110 detects the rendering progress of the GPU on the first image frame based on the predicted time at which the GPU completes the rendering of the first image frame (i.e., the predicted completion time), so as to increase the frequency of the GPU within a vertical synchronization period according to the rendering progress of the GPU at the predicted completion time. The rendering progress indicates that the GPU does not complete the rendering of the first image frame. That is, if the GPU does not complete the rendering of the first image frame at the predicted completion time, the frequency of the GPU within the vertical synchronization period is increased.

[0128] In some embodiments, the processor 110 detects the rendering progress of the first image frame once every preset frequency adjustment period within one vertical synchronization period when detecting the rendering progress, so as to increase or decrease the frequency of the GPU in this vertical synchronization period according to the rendering progress each time. That is, the electronic device can make more fine adjustment to the frequency of the GPU within one vertical synchronization period.

[0129] The display screen 194 is configured to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0130] In some embodiments, the display screen 194 can display a function page such as an application page, a desktop, a negative one screen page, a settings page, etc. of the electronic device 100. The function page can display pictures, videos, controls, components, etc.

[0131] The user can operate the function page displayed by the display screen 194, such as switching pages, pulling down the page to display the negative one screen page, clicking an icon to start an application, clicking a picture, clicking a control to play music, etc. When the user operates, the display screen 194 displays the effects of the user operation accordingly, such as the display effect of page switching, the display effect of application starting, the display effect of a picture being clicked, the display effect of playing music, the display effect of playing a video, etc. The above-mentioned GPU performs image rendering for finally displaying these effects on the display screen 194. It can be understood that the scene of the above-mentioned image frame is related to the user operation.

[0132] Alternatively, the electronic device 100 can also automatically trigger the above-mentioned operation when certain conditions are met. In this case, the GPU can also perform image rendering, and the display screen 194 can also display the corresponding effects, and the scene of the image frame is related to the operation automatically triggered by the electronic device 100. The application embodiments do not make specific limitations on how to trigger the above-mentioned operation.

[0133] The electronic device 100 can realize the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc. The ISP is configured to process the data fed back by the camera 193. The camera 193 is configured to capture still images or videos. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0134] The digital signal processor is configured to process digital signals, which can be digital image signals and other digital signals. The video codec is configured to compress or decompress digital videos. The electronic device 100 can support one or more video codecs.

[0135] The NPU is a neural-network (NN) computing processor, which can quickly process input information by drawing on the structure of a biological neural network, such as the transmission mode between human brain neurons, and can also continuously self-learn.

[0136] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to realize the expansion of the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function.

[0137] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121.

[0138] The electronic device 100 can realize audio functions through the audio module 170 and the application processor, etc. For example, music playing, recording, etc.

[0139] The audio module 170 is used to convert digital audio information into analog audio signals and output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals.

[0140] The key 190 includes a power-on key, a volume key, etc. The key 190 can be a mechanical key. It can also be a touch key.

[0141] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback.

[0142] The indicator 192 can be an indicator light, which can be used to indicate the charging state, the power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0143] The SIM card interface 195 is used to connect a SIM card.

[0144] When the GPU frequency adjustment method in the embodiments of the present application is implemented based on the electronic device shown in FIG. 4, the scene of the image frame is introduced to determine the frequency of the GPU, so that the frequency of the GPU can match the performance or power consumption of the GPU when drawing the image frame, meet the demand of the GPU in the image rendering process, and also improve the user experience.

[0145] In some possible application scenarios, the software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. Embodiments of this application take a mobile operating system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100. The layered architecture divides software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. For example, referring to FIG. 5, the layered architecture of the electronic device 100 divides the Android system into four layers, from top to bottom, an application layer, an application framework layer, a system service layer, and a kernel layer.

[0146] The application layer can include a series of application packages.

[0147] As shown in FIG. 5, the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, and the like.

[0148] In some embodiments, when an application is running, the electronic device 100 can display a page corresponding to the application. The user can operate on the page corresponding to the application. Alternatively, the user can directly operate on the desktop of the electronic device 100. In response to the operation of the user, the electronic device 100 can perform image rendering to present a display effect of an image matching the operation of the user.

[0149] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0150] As shown in FIG. 5, the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0151] The window manager is used to manage windows programs. The content provider is used to store and retrieve data and make the data accessible to the application programs. The view system includes visual controls, such as controls to display text, controls to display pictures, etc. The telephony manager is used to provide the communication function of the electronic device 100. For example, the management of the call status (including call connection, call hang-up, etc.). The resource manager provides various resources for the application programs, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables the application programs to display notification information in the status bar, which can be used to convey the type of message, which can automatically disappear after a short stay, without user interaction. The notification manager can also be a notification that appears in the form of a chart or a scroll bar text in the top status bar of the system, such as the notification of the application program running in the background, and can also be a notification that appears in the form of a dialogue window on the screen. For example, prompting text information in the status bar, issuing a prompt sound, the electronic device vibrating, the indicator light flashing, etc.

[0152] The system service layer can include a plurality of functional modules. For example: a surface manager, media libraries, a three-dimensional graphics processing library (for example: OpenGL ES), a 2D graphics engine (for example: SGL, Vulkan), etc.

[0153] The surface manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for a plurality of application programs.

[0154] The media library supports a plurality of commonly used audio, video format playback and recording, and static image files, etc.

[0155] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0156] The 2D graphics engine is a drawing engine for 2D drawing.

[0157] In some embodiments, the system service layer can also include a load prediction module (or a rendering module) and a scene recognition module.

[0158] The load prediction module can obtain the predicted load of the GPU when rendering the image frame according to the rendering information of the image frame. The scene recognition module can obtain the scene of the image frame and obtain the corresponding preset frequency modulation period according to the scene of the image frame. Different scenes correspond to different preset frequency modulation periods, and the correspondence between the scene and the preset frequency modulation period is saved in the configuration file corresponding to the GPU or in the form of a configuration parameter. In addition, the scene recognition module can configure the correspondence between the scene and the preset frequency modulation period and save the correspondence in the scene recognition module, or the scene recognition module can obtain the correspondence between the scene and the preset frequency modulation period from the storage space of the electronic device 100. The setting and obtaining of the correspondence are not limited in the embodiments of the present application.

[0159] Then, the load prediction module sends the predicted load to the kernel layer for processing, and the scene recognition module sends the scene of the image frame and the preset frequency modulation period to the kernel layer for processing.

[0160] In some other embodiments, the load prediction module can obtain the predicted load based on a load prediction model, wherein the input of the load prediction model is one or more of the rendering information of the image frame, such as the number of blur layers, the number of image layers, the number of rendering instructions, animation effects, rendering graphic types, and rendering graphic sizes; and the output of the load prediction model is the predicted load of the GPU when rendering the current image frame.

[0161] It can be understood that the load prediction model is a relatively accurate prediction model obtained by machine learning and repeated training based on the rendering information and historical load when a large number of historical image frames are processed, and the embodiments of the present application do not make specific limitations.

[0162] In some other embodiments, the processor rendering stage in the image rendering process can correspond to the processing process of the load prediction module, the scene recognition module, and the frequency modulation module. The rendering information included in the rendering instruction generated by the processor rendering stage can be generated by at least one application, that is, the application can perform scene management, logical processing, and self-generate the rendering instruction of the image frame, and the application sends the rendering instruction to the load prediction module to enable the load prediction module to obtain the rendering information in the rendering instruction.

[0163] Alternatively, the rendering instruction can also be an instruction generated by the load prediction module in the processing process, that is, the load prediction module can perform scene management, logical processing, and self-generate the rendering instruction of the image frame, so that the load prediction module can directly obtain the rendering information. In the embodiments of the present application, how the load prediction module obtains the rendering information is not limited.

[0164] In other embodiments, the load prediction module and the scene identification module described above can implement the functions described above based on the three-dimensional graphics processing library and the 2D graphics engine shown in FIG. 5, etc.

[0165] The kernel layer is a layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers, and sensor drivers.

[0166] In some embodiments, the kernel layer can further include a frequency tuning module and a frequency driver.

[0167] The kernel layer receives the predicted load, receives the preset frequency tuning period, receives the scene of the image frame, and can further obtain a historical load of the GPU when drawing a historical image frame. Further, the frequency tuning module obtains a first load matched with the scene of the current image frame to be drawn from the predicted load and the historical load, and obtains a first frequency of the GPU according to the first load. The frequency tuning module controls the frequency driver of the GPU according to the first frequency, so as to adjust the frequency of the GPU to the first frequency when the GPU starts to draw the image frame.

[0168] The frequency tuning module further determines at least one frequency tuning time point of the GPU in the process of drawing the image frame according to the preset frequency tuning period, and obtains a current load of the GPU corresponding to a period from the start of drawing the first image frame to the frequency tuning time point at each frequency tuning time point, and obtains a second frequency of the GPU according to the current load. The frequency tuning module controls the frequency driver of the GPU according to the second frequency, so as to adjust the frequency of the GPU to the second frequency at the frequency tuning time point.

[0169] In other embodiments, after the frequency tuning module obtains the first frequency, the frequency tuning module further determines a predicted completion time point of the GPU drawing the image frame according to the first frequency. In addition, the load prediction module sets a timer according to the predicted completion time point, and determines whether the GPU completes the drawing of the image frame when the predicted completion time point arrives. If the GPU does not complete the drawing of the image frame at the predicted completion time point, the load prediction module sends an instruction to the frequency tuning module again, so that the frequency tuning module adjusts the frequency of the GPU to a third frequency at the predicted completion time point, for example, to improve the performance of the GPU, so that the GPU can complete the drawing of the image frame as soon as possible.

[0170] In some embodiments, the load prediction module, the scene identification module, and the frequency tuning module described above can not be specific functional modules, but can be services providing corresponding functions, such as a load prediction service, a scene identification service, and a frequency tuning service. The specific forms capable of implementing the functions described above are not limited in the embodiments of the present application.

[0171] Exemplarily, the specific roles of the load prediction module, the scene identification module and the frequency adjustment module described above can also be seen from FIG. 6. The implementation processes of the various modules can be referred to the functions of the various modules in the above embodiments, which will not be repeated here. In addition, the scene identification module can obtain the state information of the application or the system, and identify the scene of the image frame to be drawn according to the state information of the application or the system.

[0172] When the frequency adjustment method in the embodiments of the present application is implemented based on the electronic device shown in FIG. 5, the scene of the image frame can also be introduced to determine the frequency of the GPU, so that the frequency of the GPU can match the performance or power consumption of the GPU when drawing the image frame embodied by the scene, meet the demand for the GPU in the image rendering process, and improve the user experience.

[0173] In addition, the load prediction module, the scene identification module, the frequency adjustment module and the frequency driving shown in FIG. 5 can also be used for frequency adjustment of the CPU. The specific adjustment method can be referred to the foregoing frequency adjustment of the GPU. After the electronic device implements the frequency adjustment method, the frequency of the CPU can also match the performance or power consumption of the CPU when drawing the image frame embodied by the scene, meet the demand for the CPU in the image rendering process (such as performance demand or power consumption demand), and improve the user experience.

[0174] The electronic device in the above embodiments can be a mobile phone, a tablet computer, a notebook computer, a television, a smart watch or the like, which has a display function and has a GPU.

[0175] The frequency adjustment method in the embodiments of the present application will be described below with the electronic device being a mobile phone as an example.

[0176] Referring to FIG. 7, the frequency adjustment method can include the following steps S701-S704.

[0177] S701, the mobile phone obtains the drawing information of a first image frame to be drawn by the GPU, the historical load when the GPU draws a second image frame, and the scene corresponding to the first image frame.

[0178] Exemplarily, the drawing information can include one or more of the number of blur layers, the number of graph layers, the number of drawing instructions, animation effects, drawing graph types and drawing graph sizes.

[0179] The second image frame includes at least one image frame before the first image frame, that is, the second image frame represents a historical image frame. The historical load represents a load of the GPU when drawing the historical image frame. For example, if the second image frame includes one image frame, the historical load is the load of the GPU when drawing the image frame; if the second image frame includes two or more image frames, the historical load is the average load of the GPU when drawing the image frames, for example, the load of the GPU when drawing the image frame 1 is L1, the load of the GPU when drawing the image frame 2 is L2, and the load of the GPU when drawing the image frame 3 is L3, and the historical load can be the average of L1, L2, and L3.

[0180] It can be understood that the historical load is the load of the GPU when drawing image frames in the past period of time, the historical load has a certain reference significance for the prediction of the current load of the GPU, and the historical load can also be close to the load when the GPU draws. Therefore, the historical load can also be regarded as a relatively accurate load.

[0181] The scene corresponding to the first image frame is determined according to the application state and the system state on the mobile phone. In addition, the scene corresponding to the first image frame can reflect the performance requirement or power consumption requirement of the GPU when drawing the first image frame. For example, the scene corresponding to the first image frame can include at least one of an application startup scene, a picture click scene, a page switching scene, a music playing scene, and a video playing scene. Different scenes can have different requirements for the GPU, for example, some scenes can require high performance of the GPU (that is, have performance requirements), some scenes can require low power consumption of the GPU (that is, have power consumption requirements), and the like.

[0182] For example, the application startup scene can be seen from (a) of FIG. 8, the picture click scene can be seen from (b) of FIG. 8, and the page switching scene can be seen from (c) of FIG. 8.

[0183] S702, the mobile phone determines the predicted load of the first image frame according to the drawing information.

[0184] In some embodiments, the mobile phone can also obtain the predicted load by using the load prediction model and the drawing information corresponding to the first image frame. The load prediction model can be a model trained offline, and a large number of drawing information and historical loads of the GPU when drawing historical image frames are used for training. The trained prediction load model takes the drawing information corresponding to the image frame as input and takes the predicted load as output. Since the drawing information can reflect the task amount of the GPU when drawing the first image frame to a certain extent, that is, the load, the predicted load obtained based on the drawing information will also be more accurate and will be close to the real load of the GPU when drawing the first image frame.

[0185] S703, the mobile phone acquires a first load corresponding to the scene of the first image frame from the historical load and the predicted load.

[0186] In some embodiments, the first load can also be regarded as a predicted load further acquired from the historical load and the predicted load, and S703 can also be regarded as an operation of further predicting the load.

[0187] In some embodiments, the scene corresponding to the first image frame can include a scene level. The scene with a scene level greater than a preset level is a first scene, and the scene with a scene level less than or equal to the preset level is a second scene. The first scene or the second scene according to different scenes can reflect the performance requirement or the power consumption requirement of the GPU when drawing the first image frame, and therefore, the mobile phone can acquire the load matching the performance requirement or the power consumption requirement from the historical load and the predicted load.

[0188] For example, if the scene corresponding to the first image frame is the first scene, the acquired first load needs to match the performance requirement of the GPU, which requires the GPU to process more loads or predicts that the GPU needs to process more loads in the current scene, and accordingly, a load with a larger load needs to be selected as the first load from the historical load and the predicted load. If the scene corresponding to the first image frame is the second scene, the acquired first load needs to match the power consumption requirement of the GPU, which requires the GPU to process as few loads as possible or predicts that the GPU needs to process as few loads as possible in the current scene, and accordingly, a load with a smaller load needs to be selected as the first load from the historical load and the predicted load.

[0189] This way of determining the first load can determine the load matching the scene corresponding to the drawing of the first image frame from the more accurate predicted load and the historical load, and the determined load is more accurate. Then, the corresponding GPU frequency can also be more accurately determined according to the load, so that the GPU frequency can meet the requirement of processing more loads or fewer loads in the current scene, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0190] In addition, if the scene corresponding to the first image frame is the first scene, the larger load is selected, which can also be regarded as predicting that a burst load occurs when the GPU draws the first image frame. If the scene corresponding to the first image frame is the second scene, the smaller load is selected, which can also be regarded as predicting that a reduced load occurs when the GPU draws the first image frame. It can be seen that the above way of determining the first load can also be regarded as predicting the load according to different load changes, so that the first load can match the current load change. This way can also be regarded as adjusting the frequency for the burst load and the reduced load, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0191] In some embodiments, the phone determines a difference degree between the historical load and the predicted load, and when the difference degree is large, the phone obtains a first load matching the scene from the historical load and the predicted load. When the difference degree is small, the phone obtains an average load of the historical load and the predicted load, and determines the average load as the first load.

[0192] The difference degree between the historical load and the predicted load can be determined according to a difference load between the historical load and the predicted load. The difference load between the historical load and the predicted load can be represented as an absolute value of a difference between the historical load and the predicted load. The larger the difference load, the greater the difference degree between the historical load and the predicted load. The smaller the difference load, the smaller the difference degree between the historical load and the predicted load.

[0193] For example, when the difference load between the historical load and the predicted load is greater than or equal to a preset value, it indicates that the difference load between the historical load and the predicted load is large. In this case, if the scene corresponding to the first image frame is the first scene, the phone determines a larger load from the historical load and the predicted load as the first load. If the scene corresponding to the first image frame is the second scene, the phone determines a smaller load from the historical load and the predicted load as the first load.

[0194] It can be seen that if the difference load between the historical load and the predicted load is large, a larger load or a smaller load needs to be selected from the historical load and the predicted load as the first load. The larger load can better match the performance requirement, and the smaller load can better match the power consumption requirement. Therefore, the corresponding GPU frequency can be more accurately determined according to the load, so that the GPU frequency can meet the requirement of processing a larger load or a smaller load in the current scene, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0195] In addition, when the difference load between the historical load and the predicted load is less than the preset value, it indicates that the difference load between the historical load and the predicted load is small. In this case, the phone determines the first load according to an average load between the historical load and the predicted load.

[0196] Since the difference load is small, it may not be accurate to select which one as the first load. In this case, the average load is selected as the first load, and the first load obtained is also relatively accurate. Therefore, the corresponding GPU frequency can be more accurately determined according to the load, so that the GPU frequency can meet the requirement of processing a larger load or a smaller load in the current scene, thereby improving the performance of the GPU or reducing the power consumption of the GPU.

[0197] S704, the mobile phone obtains a first frequency of the GPU according to the first load.

[0198] The first frequency is used for the GPU to draw the first image frame, that is, the GPU draws the first image frame according to the first frequency, so as to finally display the content and display effect of the first image frame on the screen of the mobile phone.

[0199] In the above scheme S701-S704, if the determined first load matches the performance requirement, the first frequency determined according to the first load can make the GPU process more load, so as to improve the performance and reduce the frame freezing problem during the display of the first image frame; if the determined first load matches the power consumption requirement, the first frequency determined according to the first load can make the GPU process less load as much as possible, so as to reduce the power consumption of the GPU. In the above scheme, the scene of the first image frame is introduced to determine the first frequency of the GPU, so that the first frequency of the GPU can match the performance requirement or power consumption requirement embodied by the scene, meet the requirement of the GPU in the image rendering process, and improve the user experience.

[0200] In some embodiments, after the mobile phone obtains the first frequency of the GPU, the frequency of the GPU can be adjusted, or after the mobile phone obtains the first frequency of the GPU, the frequency of the GPU can be adjusted when the GPU starts to draw the first image frame. Then, the GPU can use the first frequency when starting to draw the first image frame.

[0201] In other embodiments, after the mobile phone obtains the first frequency of the GPU, the first frequency can also be sent to the GPU, and the GPU can adjust the frequency according to the first frequency, or after the mobile phone obtains the first frequency of the GPU, the first frequency can be sent to the GPU when the GPU starts to draw the first image frame, and the GPU can adjust the frequency according to the first frequency. Thus, the GPU can use the first frequency when starting to draw the first image frame.

[0202] In some embodiments, after obtaining the first frequency, the mobile phone also obtains a preset frequency adjustment period of the first image frame according to the scene corresponding to the first image frame. Different scenes can correspond to different preset frequency adjustment periods, such as 1ms, 2ms, 3ms, 4ms or 5ms, etc. In addition, the preset frequency adjustment period is less than the frequency adjustment period of the GPU itself or the vertical synchronization period (such as tens of milliseconds, hundreds of milliseconds, etc.).

[0203] Afterwards, the mobile phone acquires at least one frequency adjustment time in the process of drawing the first image frame by the GPU according to the first frequency according to a preset frequency adjustment period, and at each frequency adjustment time, the mobile phone acquires a current load corresponding to the GPU between starting to draw the first image frame and the frequency adjustment time, and acquires the second frequency of the GPU according to the current load. The second frequency is used for the GPU to draw the first image frame between the frequency adjustment time and the next frequency adjustment time, that is, the GPU draws the first image frame according to the second frequency between the current frequency adjustment time and the next frequency adjustment time.

[0204] If the current load is large, the second frequency is also high, and if the current load is small, the second frequency is also low, so that the second frequency can process the current load, realize fine adjustment of the frequency in the process of drawing the first image frame, and make further frequency adjustment of the GPU on the basis of the first frequency, so that the frequency of the GPU is more accurate. It can be understood that the second frequency can be greater than the first frequency or less than the first frequency.

[0205] For example, referring to FIG. 9, the solid line therein represents the process of drawing the first image frame by the GPU according to the first frequency, in which process, T1, T2, T3, etc. are taken as various frequency adjustment times, and TO represents the time of starting to draw the first image frame. The mobile phone acquires the current load between TO and T1 at T1, and acquires the second frequency according to the load, and then the GPU draws the first image frame according to the second frequency at T1-T2. Alternatively, the mobile phone acquires the current load between TO and T2 at T2, and acquires the second frequency according to the load, and then the GPU draws the first image frame according to the second frequency at T2-T3. The dashed line in FIG. 9 represents the process of drawing the first image frame by the GPU according to the second frequency.

[0206] It can be understood that the second frequency can be greater than the first frequency, or less than the first frequency, or equal to the first frequency, and the second frequency can be adjusted according to the current load.

[0207] In addition, the above scheme not only realizes fine adjustment of the frequency in the process of drawing the first image frame, but also shortens the frequency adjustment period of the GPU, and the shorter the frequency adjustment period, the faster the frequency adjustment response of the GPU to the load, thereby improving the efficiency of the frequency adjustment of the GPU.

[0208] In some embodiments, when the mobile phone acquires the second frequency of the GPU according to the current load, it can also acquire a second load corresponding to a scene matched with the first image frame from the first load and the current load, and acquire the second frequency of the GPU according to the second load.

[0209] The second load corresponding to the scene of the first image frame is obtained from the first load and the current load. Details can refer to the above-mentioned embodiments.

[0210] The second load can be regarded as a predicted load obtained from the first load and the current load. The first load is a predicted load that matches the current scene, and the current load also has a certain reference significance for predicting the load at the current frequency adjustment time. The current load is also relatively accurate.

[0211] In this way, the load corresponding to the scene of the first image frame is determined from the first load and the current load, and the determined load is more accurate. Then, the corresponding GPU frequency can also be more accurately determined according to the load.

[0212] In some embodiments, if the scene corresponding to the first image frame is the first scene, the phone determines the larger load of the first load and the current load as the second load; if the scene corresponding to the first image frame is the second scene, the phone determines the smaller load of the first load and the current load as the second load. In this way, the determined second load is more accurate, and the corresponding GPU frequency can also be more accurately determined according to the load, so that the GPU can further adjust the frequency more finely during the rendering of the first image frame.

[0213] In some embodiments, the phone can also obtain a predicted completion time of the GPU rendering the first image frame according to the first frequency and the predicted load. For example, the phone can determine the predicted completion time of the GPU rendering the first image frame according to the first frequency, then determine the time when the GPU starts to render the first image frame, and add the time length of rendering the first image frame to obtain the predicted completion time.

[0214] Then, the phone can detect whether the GPU completes the rendering of the first image frame at the predicted completion time. If the GPU does not complete the rendering of the first image frame at the predicted completion time, it means that the frequency of the GPU at the predicted completion time (which can be the first frequency or the second frequency) is too low, or the frequency of the GPU before the predicted completion time (which can be the first frequency or the second frequency) is too low, resulting in too long time for rendering the first image frame. In this case, in order to complete the rendering of the first image frame as soon as possible and improve the rendering efficiency, the phone can obtain a third frequency with a higher frequency. Then, the GPU renders the first image frame according to the third frequency after the predicted completion time.

[0215] For example, refer to FIG. 10, in which the solid line represents the process of drawing the first image frame by the GPU according to the first frequency, in which T4 represents the predicted completion time. If the GPU does not complete drawing at T4, the mobile phone can obtain the third frequency, and the GPU draws the first image frame according to the third frequency after T4 until the first image frame is completed. The dashed line in FIG. 10 represents the process of drawing the first image frame by the GPU according to the third frequency after T4.

[0216] It can be understood that the third frequency is greater than the frequency corresponding to the predicted completion time of the GPU, that is, the third frequency is greater than the first frequency, or the third frequency is greater than the second frequency.

[0217] The above process can be understood as the predicted completion time being after the last frequency adjustment time when the GPU draws the first image frame.

[0218] In other embodiments, the predicted completion time can also be between two frequency adjustment times when the GPU draws the first image frame. In this case, the third frequency can be obtained in the manner described in the above embodiments, and the GPU draws the first image frame according to the third frequency after the predicted completion time until the drawing is completed, without obtaining other frequencies at subsequent frequency adjustment times. Alternatively, the GPU draws the first image frame according to the third frequency from the predicted completion time to the next frequency adjustment time, and the mobile phone obtains the second frequency again at the next frequency adjustment time, so that the GPU can draw the first image frame again according to the second frequency. Regardless of the position of the predicted completion time, after the frequency of the GPU is adjusted to the third frequency, the drawing process of the first image frame can be accelerated to a certain extent, and the drawing efficiency can be improved.

[0219] In some embodiments, if the GPU does not complete the drawing of the first image frame at the predicted completion time, it can also be considered that the initial predicted first frequency is too low. In this case, the mobile phone can send feedback information, such as a predicted frequency that is too low, to the load prediction model, so as to control the load prediction model to dynamically correct and improve the accuracy of the frequency prediction next time.

[0220] In some embodiments, the frequency adjustment method in the above embodiments can also be used to adjust the frequency of the CPU, so that the CPU can meet the performance requirement or the power consumption requirement when drawing an image frame, improve the user experience, and improve the efficiency of the CPU when drawing an image frame.

[0221] In some embodiments, the above frequency adjustment method can also be a frequency adjustment method based on drawing information. When the electronic device is a mobile phone, the above frequency adjustment method based on drawing information can include the following steps S801-S804.

[0222] S801, before drawing the first image frame, the mobile phone acquires a first load corresponding to the first image frame according to drawing information of the first image frame.

[0223] S802, the mobile phone acquires a frequency of the processor according to the first load, wherein the processor is configured to draw the first image frame according to the frequency within a vertical synchronization period.

[0224] S803, the mobile phone detects drawing progress of the first image frame by the processor within the vertical synchronization period.

[0225] S804, the mobile phone improves the frequency of the processor within the vertical synchronization period according to the drawing progress, wherein the processor is configured to continue drawing the first image frame according to the improved frequency within the vertical synchronization period.

[0226] In some embodiments, the processor can be a CPU, a GPU, or a processor integrating a CPU and a GPU. That is, the frequency of the CPU can be adjusted alone, the frequency of the GPU can be adjusted alone, or the frequency of the CPU and the frequency of the GPU can be adjusted.

[0227] In the above-mentioned S801-S804, the mobile phone can detect the drawing progress of the first image frame and improve the frequency of the processor within the vertical synchronization period according to the drawing progress. In this way, the processor can complete the drawing of the first image frame quickly in the subsequent period with a higher frequency, thereby improving the performance of the processor and the drawing efficiency of the processor.

[0228] In some embodiments, the mobile phone can also detect the drawing progress of the first image frame by the processor based on a predicted time (i.e., a predicted completion time) at which the processor is predicted to complete the drawing of the first image frame, and improve the frequency of the processor within the vertical synchronization period according to the drawing progress. That is, if the processor does not complete the drawing of the first image frame at the predicted completion time, the mobile phone improves the frequency of the processor within the vertical synchronization period. In this way, the drawing of the first image frame by the processor is accelerated, and the drawing efficiency is improved.

[0229] In some embodiments, the mobile phone can also detect the drawing progress of the first image frame by the processor every preset frequency adjustment period within the vertical synchronization period, and determine whether to improve or reduce the frequency of the processor within the vertical synchronization period according to the drawing progress each time. That is, the mobile phone can adjust the frequency of the processor more finely within the vertical synchronization period, so that the frequency of the processor when drawing the first image frame is more accurate or more in line with performance requirements or power consumption requirements, thereby improving the user experience.

[0230] The frequency matching method based on the rendering information provided by the embodiments of the present application not only makes the frequency of the processor match the performance requirement or the power consumption requirement embodied by the scene, but also timely responds to the burst load during the rendering of the image frame.

[0231] Without the method in the embodiments of the present application, for example, as shown in (a) of FIG. 11, after the last image frame 1 is rendered, if the load corresponding to the current image frame 2 increases when the GPU renders the image frame 2, the GPU may take a long time to adjust the frequency, for example, 22 ms, so that the rendering of the image frame 2 may take a long time, resulting in a low image frame rate of the GPU, for example, 117.9 fps. The long frequency adjustment time makes the GPU unable to timely respond to the burst load.

[0232] After the method in the embodiments of the present application is used, if the load increases when the GPU renders the image frame 2, a load matching the scene of the image frame 2 can be obtained from the historical load before the image frame 2 and the predicted load determined for the image frame 2, wherein the scene corresponding to the image frame 2 can predict the current load increase to a certain extent, and the predicted load can also predict the current load increase to a certain extent, so that a more accurate frequency is determined, and the GPU can render the image frame 2 at the frequency, and the frequency can be adjusted more finely multiple times during the rendering of the image frame 2, and the frequency can be increased after the prediction is completed to speed up the rendering of the image frame 2. For example, as shown in (b) of FIG. 11, the frequency adjustment time of the GPU is reduced (for example, from the first frequency to the third frequency), for example, to 9 ms, so that the time for the GPU to complete the rendering of the image frame 2 is greatly reduced, and the image frame rate of the GPU is improved, for example, to 119.2 ms. Not only is the frequency adjustment period reduced, but also the frequency matches the scene corresponding to the image frame 2, so that the GPU can timely respond to the burst load.

[0233] On the other hand, the frequency adjustment method provided by the embodiments of the present application can also timely respond to the frequent changes of the load during the rendering of the image frame.

[0234] Without using the method in the embodiments of the present application, for example, as shown in (a) of FIG. 12, if the load frequently changes during the process of drawing an image frame by the GPU, for example, corresponding to high load when drawing image frame 1, corresponding to low load when drawing image frame 2, corresponding to high load when drawing image frame 3, and corresponding to low load when drawing image frame 4. Because the frequency adjustment period of the GPU is long, the frequency may be adjusted to be high only after the drawing of image frame 1 is completed, so that the low frequency is still used when drawing image frame 1, and the high frequency is used when drawing image frame 2, but image frame 2 corresponds to low load and does not need high frequency. Therefore, the GPU cannot respond to the frequent change of the load in time.

[0235] After using the method in the embodiments of the present application, if the above-mentioned frequent change of the load exists, because the GPU can perform more fine frequency adjustment multiple times during the process of drawing an image frame, that is, the frequency adjustment period is reduced, the frequency of the GPU may be adjusted to be high before the end of drawing of image frame 1, and the frequency of the GPU may be adjusted to be low before the end of drawing of image frame 2, for example, as shown in (b) of FIG. 12, so that the GPU can respond to the frequent change of the load in time.

[0236] On the other hand, the above-mentioned frequency adjustment method provided by the embodiments of the present application can also respond to the situation that the load becomes low in the later stage of drawing an image frame in time.

[0237] Without using the method in the embodiments of the present application, for example, as shown in (a) of FIG. 13, if the load becomes small in the later stage of drawing image frame 2 by the GPU, because the frequency adjustment period of the GPU is long, the frequency may be adjusted to be low only after the drawing of image frame 2 is completed, so that the high frequency is still used in the later stage of drawing image frame 2. Therefore, the GPU cannot respond to the situation that the load becomes low in the later stage of drawing in time.

[0238] After using the method in the embodiments of the present application, if the above-mentioned situation that the load becomes small in the later stage of drawing image frame 2 exists, because the GPU can perform more fine frequency adjustment multiple times during the process of drawing an image frame, that is, the frequency adjustment period is reduced, the frequency of the GPU may be adjusted to be low in the later stage of drawing image frame 2, for example, as shown in (b) of FIG. 13, so that the GPU can respond to the situation that the load becomes low in the later stage of drawing in time.

[0239] In the above-mentioned embodiments, the frequency adjustment method is taken as an example of being implemented on a mobile phone, and in some other embodiments, the above-mentioned frequency adjustment method can also be applied to electronic devices with display function and with GPU, such as tablet computers, notebook computers, televisions, smart watches, etc. The specific implementation process can refer to the content of the above-mentioned embodiments, and will not be described here.

[0240] In some scenarios, multiple embodiments of the present application can be combined, and the combined scenario can be implemented. Optionally, some operations in the flow of each method embodiment are optionally combined, and / or the order of some operations is optionally changed. Also, the execution order between the steps of each flow is only exemplary, and does not constitute a limitation on the execution order between the steps, and other execution orders between the steps can also be used. The described execution order is not intended to indicate that these operations can only be executed in this order. A person of ordinary skill in the art will think of various ways to reorder the operations described in the embodiments of the present application. In addition, it should be pointed out that the process details involved in a certain embodiment of the present application are also applicable in a similar manner to other embodiments, or different embodiments can be combined for use.

[0241] In addition, some steps in the method embodiments can be equivalently replaced by other possible steps. Alternatively, some steps in the method embodiments can be optional and can be deleted in some use scenarios. Alternatively, other possible steps can be added to the method embodiments.

[0242] Also, each method embodiment can be implemented individually or in combination.

[0243] It can be understood that, in order to realize the above functions, the foregoing electronic device contains hardware and / or software modules corresponding to each function. The algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in hardware or a combination of hardware and computer software. Whether a certain function is executed in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered beyond the scope of the present application.

[0244] The present embodiment can divide the functional modules of the electronic device according to the above method examples, for example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware. It should be noted that the division of modules in the present embodiment is illustrative, and is only a logical functional division. When actually implemented, there can be another division manner.

[0245] The present embodiment provides a frequency modulation device, which can include a load prediction module, a scene identification module, a frequency modulation module, and the like as shown in FIG. 6.

[0246] The load prediction module is configured to obtain drawing information of a first image frame to be drawn by a GPU.

[0247] The frequency adjusting module is configured to obtain a historical load when the GPU draws the second image frame.

[0248] The scene recognition module is configured to obtain a scene corresponding to the first image frame, and the scene corresponding to the first image frame is determined according to an application state or a system state. The second image frame includes at least one image frame before the first image frame.

[0249] The load prediction module is further configured to determine a predicted load of the first image frame according to the drawing information.

[0250] The frequency adjusting module is further configured to obtain a first load in the historical load and the predicted load that matches the scene corresponding to the first image frame. The frequency adjusting module is further configured to obtain a first frequency of the GPU according to the first load, and the first frequency is used for drawing the first image frame by the GPU.

[0251] In some other embodiments, the functions of the load prediction module, the scene recognition module and the frequency adjusting module can refer to the foregoing content, and details are not described herein.

[0252] In some other embodiments, the frequency adjusting device can also be used to adjust the frequency of the CPU.

[0253] Embodiments of the present application further provide an electronic device, which can include the frequency adjusting device.

[0254] Embodiments of the present application further provide an electronic device, as shown in FIG. 14, which can include one or more processors 1401, a memory 1402 and a communication interface 1403.

[0255] The memory 1402 and the communication interface 1403 are coupled with the processor 1401. For example, the memory 1402 and the communication interface 1403 can be coupled together with the processor 1401 through a bus 1404.

[0256] The communication interface 1403 is configured to perform data transmission with other devices. The memory 1402 stores computer program code. The computer program code includes computer instructions, which, when executed by the processor 1401, cause the electronic device to perform the frequency adjusting method in the embodiments of the present application.

[0257] The processor 1401 can be a processor or controller, for example, a CPU, a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure. The processor can also be a combination of implementing computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0258] The bus 1404 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 1404 can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in FIG. 14, but it does not mean that there is only one bus or only one type of bus.

[0259] The embodiments of the present application also provide a computer readable storage medium, which includes computer instructions, when the computer instructions are run on an electronic device, the electronic device executes the related method steps in the above method embodiments.

[0260] The embodiments of the present application also provide a computer program product, when the computer program product is run on a computer, the computer executes the related method steps in the above method embodiments.

[0261] The frequency modulation device, the electronic device, the computer readable storage medium, or the computer program product provided by the present application are all used to execute the corresponding method provided above, so the beneficial effects they can achieve can refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.

[0262] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0263] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the modules or units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0264] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or a plurality of physical units, that is, can be located in one place, or can be distributed to a plurality of different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0265] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0266] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for making a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage program codes.

[0267] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A frequency modulation method characterized by, The method comprises the following steps: obtaining drawing information of a first image frame to be drawn by a graphics processing unit (GPU), a historical load when the GPU draws a second image frame, and a scene corresponding to the first image frame; the scene corresponding to the first image frame is determined according to an application state or a system state; the second image frame comprises at least one image frame before the first image frame; determining a predicted load of the first image frame according to the drawing information; obtaining a first load matched with the scene corresponding to the first image frame from the historical load and the predicted load; obtaining a first frequency of the GPU according to the first load; the first frequency is used for the GPU to draw the first image frame.

2. The method of claim 1, wherein, The scene corresponding to the first image frame comprises scene levels; different scene levels are used to indicate power consumption requirements of the GPU when drawing the first image frame. The step of obtaining the first load matched with the scene corresponding to the first image frame from the historical load and the predicted load comprises the following steps: if the scene corresponding to the first image frame is a first scene, determining the greater load from the historical load and the predicted load as the first load; the scene level of the first scene is greater than a preset level; if the scene corresponding to the first image frame is a second scene, determining the smaller load from the historical load and the predicted load as the first load; the scene level of the second scene is less than or equal to the preset level.

3. The method of claim 1, wherein, The scene corresponding to the first image frame comprises scene levels; different scene levels are used to indicate power consumption requirements of the GPU when drawing the first image frame. The step of obtaining the first load matched with the scene corresponding to the first image frame from the historical load and the predicted load comprises the following steps: when a difference load between the historical load and the predicted load is greater than or equal to a preset value, if the scene corresponding to the first image frame is a first scene, determining the greater load from the historical load and the predicted load as the first load; the scene level of the first scene is greater than a preset level; if the scene corresponding to the first image frame is a second scene, determining the smaller load from the historical load and the predicted load as the first load; the scene level of the second scene is less than or equal to the preset level.

4. The method of claim 3, wherein, The step of obtaining the first load matched with the scene corresponding to the first image frame from the historical load and the predicted load comprises the following steps: when the difference load between the historical load and the predicted load is less than a preset value, determining the first load according to an average load between the historical load and the predicted load.

5. The method according to any one of claims 1 to 4, characterized in that, After the step of obtaining the first frequency of the GPU according to the first load, the method further comprises the following steps: obtaining a preset frequency modulation period of the first image frame according to the scene corresponding to the first image frame; during the process that the GPU draws the first image frame according to the first frequency, obtaining at least one frequency modulation time point in the process of drawing the first image frame according to the preset frequency modulation period. At each frequency adjustment moment, a current load of the GPU between a start of drawing the first image frame and the frequency adjustment moment is obtained; a second frequency of the GPU is obtained according to the current load; the second frequency is used for the GPU to draw the first image frame between the frequency adjustment moment and a next frequency adjustment moment.

6. The method of claim 5, wherein, The second frequency of the GPU obtained according to the current load comprises: a second load matched with a scene corresponding to the first image frame is obtained from the first load and the current load; a second frequency of the GPU is obtained according to the second load.

7. The method of claim 6, wherein, The second load matched with the scene corresponding to the first image frame is obtained from the first load and the current load, and comprises: if the scene corresponding to the first image frame is a first scene, a larger load of the first load and the current load is determined as the second load; if the scene corresponding to the first image frame is a second scene, a smaller load of the first load and the current load is determined as the second load.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: a predicted completion moment of the GPU drawing the first image frame is obtained; the predicted completion moment is determined according to the first frequency and the predicted load; if the GPU does not complete drawing of the first image frame at the predicted completion moment, a third frequency of the GPU is obtained; the third frequency is used for the GPU to draw the first image frame after the predicted completion moment; the third frequency is greater than a frequency corresponding to the GPU at the predicted completion moment.

9. The method according to any one of claims 1 to 8, characterized in that, The drawing information of the first image frame comprises one or more of a blur layer number, a graph layer number, a drawing instruction number, an animation effect, a drawing graph type, and a drawing graph size.

10. The method of claim 2 or 3, wherein, The first scene comprises at least one of an application startup scene, a picture click scene, a page switching scene, a music playing scene, and a video playing scene; the second scene comprises at least one of the application startup scene, the picture click scene, the page switching scene, the music playing scene, and the video playing scene; the first scene is different from the second scene.

11. A frequency modulation method based on drawing information, characterized by, The method applied to an electronic device, and comprises: before drawing a first image frame, a first load corresponding to the first image frame is obtained according to drawing information of the first image frame; a frequency of a processor is obtained according to the first load; the processor is used to draw the first image frame according to the frequency in a vertical synchronization period; drawing progress of the processor drawing the first image frame is detected in the vertical synchronization period; the frequency of the processor in the vertical synchronization period is increased according to the drawing progress; the processor is used to continue drawing the first image frame in the vertical synchronization period according to the increased frequency.

12. The method of claim 11, wherein, The drawing progress of the processor drawing the first image frame is detected based on a predicted moment at which the processor completes drawing of the first image frame; wherein the drawing progress indicates that the processor does not complete drawing of the first image frame. ​ 13. The method of claim 11, wherein, The detection processor detects the rendering progress of the first image frame, including: In the vertical synchronization period, the rendering progress of the first image frame is detected once every preset frequency modulation period, and the preset frequency modulation period is less than or equal to half of the vertical synchronization period.

14. An electronic device, comprising: The electronic device comprises a memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, and the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the frequency modulation method as claimed in any one of claims 1-10, or executes the frequency modulation method based on the rendering information as claimed in any one of claims 11-13.

15. A computer-readable storage medium, characterized in that, The computer instructions, when executed on an electronic device, cause the electronic device to execute the frequency modulation method as claimed in any one of claims 1-10, or execute the frequency modulation method based on the rendering information as claimed in any one of claims 11-13.

16. A computer program product, characterised in that, The computer program product, when executed on a computer, causes the computer to execute the frequency modulation method as claimed in any one of claims 1-10, or execute the frequency modulation method based on the rendering information as claimed in any one of claims 11-13.

Images