Frequency control method and apparatus, device, chip, and program product

Abstract

The present application relates to the technical field of computers, and discloses a frequency control method and apparatus, a device, a chip, and a program product. The method comprises: acquiring a rendering parameter of a current image frame, wherein the rendering parameter is a rendering-related parameter generated before a rendering apparatus renders the current image frame; predicting a load of the current image frame on the basis of the rendering parameter of the current image frame and historical data, wherein the historical data comprises loads and rendering parameters respectively corresponding to a plurality of historical image frames, and the load refers to computational load information of rendering an image by the rendering apparatus; and controlling the rendering apparatus to render the current image frame at a target frequency, wherein the target frequency matches the load of the current image frame. In the method, the accuracy of the predicted load can be improved by means of the rendering parameter of the current image frame, thereby improving the accuracy of controlling the frequency on the basis of the load, so that the method can cope with a load burst scenario, and the matching degree between the operating frequency of the rendering apparatus and the load is improved.

Description

Frequency control method, device, apparatus, chip and program product

[0001] The present application claims priority to the Chinese patent application No. 202411233400.7, filed on September 3, 2024, and entitled "Frequency control method, device, apparatus, chip and program product", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of computers, and in particular to a frequency control method, device, apparatus, chip and program product. BACKGROUND

[0003] In an electronic device such as a smart phone, a rendering device is responsible for image rendering and other graphics data processing, so that the electronic device can display images through a display device. Since the rendering content and visual effect of each frame of image displayed by the electronic device are different, the load of the rendering device processing each frame of image is different. In the case that the operating frequency of the rendering device does not match the load, the problem that performance and power consumption are difficult to be optimized simultaneously is prone to occur. For example, when the operating frequency of the rendering device is too low, the image rendering cannot be completed within a specified time, and the performance problem of frame loss occurs; when the operating frequency of the rendering device is too high, although the display performance requirement can be met, the higher the operating frequency, the greater the power consumption, so that the operating frequency that is too high exists power consumption waste.

[0004] Therefore, how to control the operating frequency of the rendering device so that the operating frequency matches the load is a problem to be solved. SUMMARY

[0005] The present application provides a frequency control method, device, apparatus, chip and program product for controlling the operating frequency of a rendering device.

[0006] In a first aspect, a frequency control method is provided, which includes: obtaining a rendering parameter of a current frame of image, the rendering parameter being a parameter related to rendering generated before a rendering device renders the current frame of image; predicting a load of the current frame of image based on the rendering parameter of the current frame of image and historical data, the historical data including loads and rendering parameters corresponding to a plurality of historical frames of image respectively, the load being a calculation amount information of the rendering device rendering the image; controlling the rendering device to render the current frame of image at a target frequency, the target frequency matching the load of the current frame of image.

[0007] The method combines the rendering parameter of the current frame image and the historical data to predict the load of the current frame image, not only considers the change trend of the rendering parameter and the load presented in the historical data, but also considers the rendering parameter of the current frame image, which can reflect the rendering complexity of the current frame image. Therefore, the accuracy of the predicted load can be improved by the rendering parameter of the current frame image, and the accuracy of the frequency control according to the load is improved, so that the method can cope with the load burst scene, and the matching degree of the running frequency of the rendering device and the load is improved.

[0008] In a possible implementation, the rendering parameter includes at least one of a related parameter of interprocess communication (IPC) data transmission or a related parameter of a rendering tree. The rendering parameter can effectively reflect the rendering complexity of the current frame image, and the way of obtaining the rendering parameter is relatively flexible.

[0009] In a possible implementation, the way of predicting the load of the current frame image based on the rendering parameter of the current frame image and the historical data can include: performing feature extraction on the rendering parameter of the current frame image and the historical data to obtain a first feature vector; determining a first load corresponding to the first feature vector based on a mapping relationship between the feature vector and the load, and taking the first load as the load of the current frame image. In this way, the amount of prediction calculation can be reduced by extracting the feature vector, and the prediction efficiency can be improved by the mapping relationship determined in advance.

[0010] In a possible implementation, the way of determining the first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load can include: calling a machine learning model to determine the first load corresponding to the first feature vector, the machine learning model containing the mapping relationship, and the machine learning model being obtained based on a data set. The data set includes feature vectors and loads corresponding to a plurality of historical frame images respectively. The prediction accuracy can be improved by training the machine learning model.

[0011] In a possible implementation, the target frequency matches the load of the current frame image, that is, the completion time of rendering the current frame image by the rendering device according to the target frequency is less than or equal to a maximum allowed time, and the maximum allowed time is a rendering time that meets the display performance requirement. In this way, the running frequency of the GPU can meet the display performance requirement.

[0012] In a possible implementation, the maximum allowed time is determined based on a display time interval between the current frame image and a previous frame image, or determined based on the display time interval and a rendering delay time of a historical frame before the current frame image, or determined based on the display time interval and a rendering remaining time of the historical frame before the current frame image. The rendering device is caused to complete rendering of the current frame image at the target frequency within a time less than or equal to the display time interval of the adjacent frame image, so as to ensure that the display of the current frame image is not stuck, and the continuity of the multiple frame images is ensured in consideration of the display delay time of the historical frame.

[0013] In a possible implementation, the target frequency is a frequency corresponding to power consumption satisfying the power consumption requirement in the selectable frequencies, and the power consumption corresponding to any frequency refers to power consumption generated by the rendering device when rendering the current frame image at the frequency. The selectable frequencies include multiple frequencies matching the load of the current frame image. In this way, the target frequency satisfies the display performance requirement and the power consumption requirement, and a balance between the display performance and the power consumption is achieved.

[0014] In a second aspect, a frequency control device is provided, configured to execute the method in the first aspect or any possible implementation of the first aspect. Specifically, the frequency control device includes modules configured to execute the method in the first aspect or any possible implementation of the first aspect. In a possible implementation, the frequency control device includes: a data acquisition module configured to acquire a rendering parameter of a current frame image, the rendering parameter being a parameter related to rendering generated before the rendering device renders the current frame image; a load prediction module configured to predict a load of the current frame image based on the rendering parameter of the current frame image and historical data, the historical data including loads and rendering parameters corresponding to multiple historical frame images respectively, and the load referring to calculation amount information of the rendering device when rendering the image; and a frequency control module configured to control the rendering device to render the current frame image at a target frequency, the target frequency matching the load of the current frame image.

[0015] In a possible implementation, the rendering parameter includes at least one of a related parameter of IPC data transmission or a related parameter of a rendering tree.

[0016] In a possible implementation, the load prediction module is configured to perform feature extraction on the rendering parameter of the current frame image and the historical data to obtain a first feature vector, determine a first load corresponding to the first feature vector based on a mapping relationship between the feature vector and the load, and take the first load as the load of the current frame image.

[0017] In a possible implementation, the load prediction module is configured to invoke a machine learning model to determine the first load corresponding to the first feature vector, the machine learning model comprises a mapping relationship, and the machine learning model is obtained based on a dataset, the dataset comprising feature vectors and loads corresponding to a plurality of historical frame images respectively.

[0018] In a possible implementation, the target frequency is matched with the load of the current frame image, that is, the rendering device completes rendering of the current frame image at the target frequency in a time less than or equal to a maximum allowed time, and the maximum allowed time is a rendering time that meets a display performance requirement.

[0019] In a possible implementation, the maximum allowed time is determined based on a display time interval between the current frame image and a previous frame image, or the maximum allowed time is determined based on the display time interval and a rendering delay time of a historical frame before the current frame image, or the maximum allowed time is determined based on the display time interval and a rendering remaining time of the historical frame before the current frame image.

[0020] In a possible implementation, the target frequency is a frequency corresponding to power consumption that meets a power consumption requirement in optional frequencies, the power consumption corresponding to any frequency is power consumption generated by the rendering device when rendering the current frame image at the any frequency, and the optional frequencies comprise a plurality of frequencies matched with the load of the current frame image.

[0021] In a third aspect, a visual device is provided, comprising a rendering device, and the visual device is configured to execute the frequency control method in the first aspect or any of the first aspect to control a running frequency of the rendering device.

[0022] In a fourth aspect, a terminal device is provided, comprising a processor and a memory, the memory stores at least one program instruction or code, the at least one program instruction or code is loaded and executed by the processor to enable the terminal device to implement the frequency control method in the first aspect or any of the first aspect.

[0023] Optionally, the processor is one or more, and the memory is one or more.

[0024] Optionally, the memory can be integrated with the processor, or the memory and the processor are separately arranged.

[0025] In the implementation process, the memory can be a non-transitory memory, for example, a read only memory (ROM), which can be integrated on the same chip with the processor, or can be arranged on different chips respectively, and the type of the memory and the arrangement manner of the memory and the processor are not limited in the present application.

[0026] In a fifth aspect, a computer readable storage medium is provided, and the storage medium stores at least one instruction, which is loaded and executed by a processor to enable a computer to implement the frequency control method in the first aspect or any possible implementation of the first aspect.

[0027] In a sixth aspect, a computer program (product) is provided, and the computer program (product) includes computer program code, which, when executed by a computer, causes the computer to perform the frequency control method in the first aspect or any possible implementation of the first aspect.

[0028] In a seventh aspect, a chip is provided, and the chip includes a processor configured to invoke and execute instructions stored in a memory, so that a device installed with the chip performs the frequency control method in the first aspect or any possible implementation of the first aspect.

[0029] In an eighth aspect, another chip is provided, and the chip includes an input interface, an output interface, a processor and a memory, and the input interface, the output interface, the processor and the memory are connected through internal connection paths. The processor is configured to execute code in the memory, and when the code is executed, the processor is configured to perform the frequency control method in the first aspect or any possible implementation of the first aspect.

[0030] It should be understood that the beneficial effects achieved by the second aspect to the eighth aspect and the corresponding possible implementations of the present application can be referred to the technical effects of the first aspect and the corresponding possible implementations, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0031] FIG. 1 is a schematic diagram of an implementation environment of a frequency control method provided by an embodiment of the present application;

[0032] FIG. 2 is a flowchart of a frequency control method provided by an embodiment of the present application;

[0033] FIG. 3 is a schematic diagram of a frame rendering process provided by an embodiment of the present application;

[0034] FIG. 4 is a whole flowchart of GPU frequency adjustment provided by an embodiment of the present application;

[0035] FIG. 5 is a structural schematic diagram of a frequency control apparatus provided by an embodiment of the present application;

[0036] FIG. 6 is a structural schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the drawings.

[0038] With the rapid development of computer technology, in order to balance the display performance of the screen and the running power consumption of the rendering device, the electronic device such as a smart phone needs to flexibly control the running frequency of the rendering device according to the load of the rendering image, so as to match the running frequency with the load. In the embodiments of the present application, the rendering device refers to any device with rendering function, for example, a graphics processing unit (GPU), a digital signal processing unit (DSP) or other chips with image processing function.

[0039] Taking the GPU as an example, in the related technology one, the GPU load of the historical time period is calculated according to the ratio between the working time and the idle time of the GPU in the historical time period; the GPU load of the current time period is predicted through the GPU load of the historical time period; and then the running frequency of the GPU in the current time period is controlled according to the predicted GPU load. However, since the load of the current time period is only predicted according to the load of the historical time period, when facing the scene with burst load, the predicted GPU load is inaccurate, and thus the matching degree of the running frequency of the GPU with the load is low.

[0040] In the related technology two, the power parameters (such as the number of call of drawing function and the number of instructions) required by the current frame image are predicted according to the power parameters of the historical frame image, the GPU load is predicted according to the predicted power parameters, and finally the running frequency of the GPU for processing the current frame is controlled according to the predicted GPU load. However, since there are two predictions, the accuracy is difficult to guarantee, and since the prediction is also only based on the power parameters of the historical frame image, it is also difficult to cope with the burst load scene, that is, the accuracy of the predicted GPU load is low, and thus the matching degree of the running frequency of the GPU with the load is low.

[0041] The embodiments of the present application provide a frequency control method, which can improve the matching degree of the running frequency of the rendering device with the load. Referring to FIG. 1, FIG. 1 is a schematic diagram of an implementation environment of a frequency control method provided by the embodiments of the present application. As shown in FIG. 1, the implementation environment includes an electronic device 101. The electronic device 101 includes a rendering device, which is responsible for image rendering and other graphic data processing. The type of the electronic device 101 is not limited in the embodiments of the present application, which can be a terminal device or other visual device other than the terminal.

[0042] Optionally, the terminal device can be any electronic product that can interact with a user through one or more of a keyboard, a touchpad, a touch screen, a remote controller, voice interaction, a handwriting device, and the like, such as a personal computer (PC), a smart phone, a camera, a personal digital assistant (PDA), a wearable device, a smart watch, a pocket PC (PPC), a tablet computer, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a terminal in self-driving, a terminal in a smart grid, a terminal in transportation safety, a terminal in a smart city, a terminal in a smart home, an Internet of Things terminal, and the like.

[0043] Those skilled in the art should understand that the electronic device 101 described above is only an example, and other existing or future electronic devices can also be applicable to the present application and should be included in the protection scope of the present application, and are hereby included by reference.

[0044] Referring to FIG. 2, FIG. 2 is a flowchart of a frequency control method according to an embodiment of the present application. The method is described by taking an electronic device as an example. The electronic device can be the electronic device 101 shown in FIG. 1, for example. As shown in FIG. 2, the frequency control method includes, but is not limited to, the following steps 201-203.

[0045] In step 201, a rendering parameter of a current frame image is obtained. The rendering parameter is a parameter related to rendering generated before a rendering device renders the current frame image.

[0046] In the embodiments of the present application, the rendering parameter refers to a parameter related to rendering that can be obtained before the rendering device renders the current frame image. Optionally, the rendering parameter includes at least one of a related parameter of IPC data transmission or a related parameter of a render tree, or further includes at least one of a related parameter of a resource such as a memory state, a related parameter of preliminary rendering before rendering processing, or a rendering related parameter generated by the rendering device before starting to render the current frame image. For example, taking the rendering device as a GPU, the IPC data transmission can refer to data transmission from a central processing unit (CPU) process to a GPU process. The render tree is a data structure in the rendering process, which is used for layout and drawing of a display page, that is, the GPU renders according to the structure and content of the render tree. Specifically, the render tree contains all element nodes that need to be displayed, and each element node contains corresponding style information and geometric information.

[0047] For example, the rendering parameter includes, but is not limited to, at least one of a number of times of IPC data transmission, a data size of each IPC data transmission, a number of element nodes to be rendered, a cache size of the element nodes, a number of times of blur, a blur area, a number of times of shadow, a shadow area, a number of times of highlight, a highlight area, a dirty area size, a texture creation time consumption, or a number of pixels actually rendered by the GPU. The blur is used to simulate a dynamic blur effect in a photography technique, and the shadow is used to enhance the depth of a scene to enhance the real sense and artistic sense of a visual effect. The dirty area refers to an area that has been edited, updated or affected by adding or deleting elements, and only the dirty area part that needs to be redrawn can be drawn when each frame image is drawn, so as to save rendering resources. Optionally, the rendering parameter is generated by the electronic device in the process of executing system code, and therefore, the rendering data can be obtained from the system code of the electronic device.

[0048] Referring to FIG. 3, FIG. 3 is a schematic diagram of a frame rendering process provided in an embodiment of the present application. As shown in FIG. 3, taking a GPU as an example of the rendering device, the electronic device implements frame rendering through cooperation of the CPU and the GPU. For each frame image (frame 1, frame 2, frame 3) to be rendered, the CPU is first used for processing, for example, the CPU is used for measurement, layout, animation or drawing, etc. Subsequently, the CPU synchronizes the processed data to the GPU. Finally, the GPU renders each frame image according to the processed data. For each frame image, before the CPU processing stage or the GPU rendering stage, the relevant parameters of the GPU rendering the frame image, i.e. the rendering parameters in the embodiment of the present application, can be obtained. That is, in the rendering process of a frame image, the CPU is responsible for layout calculation and drawing instruction generation, etc. Before the GPU starts rendering, the rendering parameters of the current frame image can be obtained from the CPU side. Exemplarily, the rendering parameters of the current frame image are obtained from the system code executed on the CPU side.

[0049] In step 202, the load of the current frame image is predicted based on the rendering parameters of the current frame image and historical data, the historical data including loads and rendering parameters respectively corresponding to a plurality of historical frame images, and the load referring to the calculation amount information of the rendering device rendering the image.

[0050] The historical frame image refers to a frame image that has been rendered before the current frame image, for example, a plurality of frame images that are adjacent to the current frame image and are continuously rendered before the current frame image. In the embodiment of the present application, the calculation amount information of the rendering device rendering the image can be measured by the number of clock cycles, the running time or the number of executed instructions of the rendering device running a program. Exemplarily, taking the number of clock cycles as an example of the load, the load of any historical frame image can be 1 million clock cycles. Alternatively, the historical data can be recorded in real time during the process of the rendering device rendering the historical frame image, and the obtaining method of the rendering parameters can be described in the related description of step 201, which will not be repeated here.

[0051] After obtaining the rendering parameters respectively corresponding to the current frame image and the plurality of historical frame images, and the loads respectively corresponding to the plurality of historical frame images, the load of the current frame image can be predicted according to the change rule of the rendering parameters from the plurality of historical frame images to the current frame image, and the change rule of the loads between the plurality of historical frame images. In a possible implementation manner, the method of predicting the load of the current frame image based on the rendering parameters of the current frame image and the historical data includes but is not limited to the following steps 2021 and 2022.

[0052] In step 2021, feature extraction is performed on the rendering parameters of the current frame image and the historical data to obtain a first feature vector.

[0053] Optionally, the methods for feature extraction of the rendering parameters and historical data of the current frame image include, but are not limited to: performing temporal feature extraction on the first time-series data and the second time-series data, and / or performing frequency domain feature extraction on the first time-series data and the second time-series data. The first time-series data includes the rendering parameters corresponding to the current frame image and multiple historical frame images, respectively, and the second time-series data includes the payloads corresponding to multiple historical frame images, respectively. For example, taking the rendering parameters including the number of IPC data transmissions C and the number of blurrings M as an example, the first time-series data includes {C1, C2, C3, ..., C...} n- 1,C n} and {M1,M2,M3,…,M n-1 M n The second time series data includes {L1, L2, L3, ..., L}. n-1}, where L represents the load, n represents the current frame image, and (1,2,…,n-1) represents the n-1 historical frame images preceding the current frame image.

[0054] The time-domain features extracted through time-domain feature extraction include, but are not limited to, at least one of the following: mean, maximum, minimum, standard deviation, variance, peak period, peak-to-peak value, root mean square value, skewness coefficient, or kurtosis coefficient. The frequency features extracted through frequency feature extraction include, but are not limited to, at least one of the following: centroid frequency, mean square frequency, root mean square frequency, frequency variance, or frequency standard deviation. At least one of the extracted time-domain features or frequency features is fused to obtain a first feature vector. The fusion method may include merging, concatenating, or adding.

[0055] In one possible implementation, in addition to obtaining the first feature vector according to step 2021, the first feature vector can also be obtained by extracting features from the rendering parameters corresponding to the current frame image and multiple historical frame images respectively, while the loads corresponding to the multiple historical frame images are used to determine the following mapping relationship between the feature vector and the load.

[0056] Step 2022: Determine the first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load, and use the first load as the load of the current frame image.

[0057] In the embodiment of the present application, the mapping relationship between the feature vector and the load can represent the relationship between the change rule of the feature vector and the change rule of the load, and then after the first feature vector is determined, the first feature vector can be mapped to the corresponding first load according to the mapping relationship. The mapping relationship can be obtained by summarizing and analyzing the change rule between the feature vector and the load in the data set, or can be obtained by training a machine learning model through the data set. The data set includes the feature vectors and the loads corresponding to the plurality of frames of images collected on the electronic device, and the feature vector of any frame of image is obtained by performing feature extraction on the rendering parameters of any frame of image and the historical data corresponding to any frame of image. The feature extraction is the same as the feature extraction in step 2021, and will not be repeated here.

[0058] In the case of training a machine learning model, the mapping relationship between the feature vector and the load is included in the machine learning model, that is, the input of the machine learning model is the feature vector, and the output is the load. Therefore, the method for determining the first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load can include calling the machine learning model to determine the first load corresponding to the first feature vector. For example, the model structure of the machine learning model can be a bidirectional encoder representation from transformers (BERT) model, a convolutional neural networks (CNN) model, a long short term memory (LSTM) model, a bidirectional LSTM model, or a gated recurrent unit (GRU) model, etc., and the embodiment of the present application does not limit this.

[0059] Alternatively, the load in the embodiment of the present application can be a numerical value of the calculation amount of the rendering device for rendering the current frame of image, that is, a load value, for example, 1 million clock cycles; or the load in the embodiment of the present application can also be a range of the calculation amount of the rendering device for rendering the current frame of image, that is, a load interval, for example, between 1 million clock cycles and 2 million clock cycles. For the case of calling the machine learning model to determine the first load corresponding to the first feature vector, if the first load output by the machine learning model is a load value, the type of the machine learning model can be a regression model; if the first load output by the machine learning model is a load interval, the type of the machine learning model can be a classification model, and different load intervals are divided into different categories at this time.

[0060] Thus, by the steps 2021 and 2022, the load of the current frame image can be accurately predicted based on the rendering parameters of the current frame image and the historical data. Since the data used for prediction not only includes the historical data, i.e., the rendering parameters and the load of the historical frame images, but also includes the rendering parameters of the current frame image, the method can learn the trend of the load from the historical data and learn the rules of the burst load in combination with the historical data and the rendering data of the current frame image, thereby realizing accurate load prediction.

[0061] The embodiments of the present application do not limit the load prediction manner. In addition to the prediction manner shown in the steps 2021 and 2022, a prediction model can be directly called to predict the load of the current frame image based on the rendering parameters of the current frame image and the historical data. The input of the prediction model is the rendering parameters corresponding to the current frame image and the plurality of historical frame images respectively and the load corresponding to the plurality of historical frame images respectively, and the output is the load of the current frame image. The prediction model is trained in advance by training data, and the training data includes training samples and training labels corresponding to a plurality of frame images respectively. Taking the nth frame image as an example, the training sample is the rendering parameters corresponding to the first frame image to the nth frame image respectively and the load corresponding to the first frame image to the (n-1)th frame image respectively, and the training label is the load of the nth frame image.

[0062] In step 203, the rendering device is controlled to render the current frame image at a target frequency, and the target frequency matches the load of the current frame image.

[0063] After the load of the current frame image is predicted, the target frequency matching the current frame image can be determined, and then the rendering device is controlled to render the current frame image at the target frequency, so that the running frequency of the rendering device matches the load. In the embodiments of the present application, the target frequency matching the load of the current frame image can mean that the completion time of the rendering device rendering the current frame image at the target frequency is less than or equal to the maximum allowed time, and the maximum allowed time is the rendering time meeting the display performance requirement.

[0064] In a possible implementation, the maximum allowed time is determined based on a display time interval between the current frame image and the previous frame image, i.e., the current frame image and the previous frame image are displayed with the display time interval, which can meet the display requirement. Alternatively, the display time interval between the current frame image and the previous frame image can be determined based on the screen refresh rate of the electronic device. For example, taking the current screen refresh rate of 120 frames per second as an example, the display time interval between adjacent frame images is within 1000 / 120=8.3 milliseconds, i.e., the completion time of the rendering device rendering each frame image needs to be controlled within 8.3 milliseconds, and the maximum allowed time can be 8.3 milliseconds.

[0065] Optionally, the maximum allowed time can also be determined based on the display time interval and the rendering delay time of the historical frame before the current frame image, or based on the display time interval and the rendering remaining time of the historical frame before the current frame image. In this case, the rendering delay time refers to the time when the rendering completion time exceeds the display time interval, and the rendering remaining time refers to the time when the rendering completion time is earlier than the display time interval, considering the case of the rendering delay or the rendering advance of the historical frame image before the current frame. The rendered image is cached in the electronic device, and the electronic device reads and displays a frame of image every display time interval, so as to ensure that the rendering of the current frame image is completed before the current frame image is read. If the rendering of the current frame image is not completed when the current frame image is read, the current frame image will be lost to cause display lag; if the rendering of the current frame image is completed too early, the cache time will be too long, and the cache space will be wasted.

[0066] Taking the case that the screen refresh rate is kept at 120 frames per second as an example, if the rendering device delays 1 millisecond compared with 8.3 milliseconds of the last frame image, that is, the rendering delay time is 1 millisecond, the rendering device needs to control the completion time of the current frame image within 8.3-1=7.3 milliseconds, so as to ensure the display performance requirements of the last frame image and the current frame image. At this time, the maximum allowed time can be 7.3 milliseconds. If the rendering device is 1 millisecond ahead of 8.3 milliseconds of the last frame image, that is, the rendering remaining time is 1 millisecond, the rendering device can control the completion time of the current frame image within 8.3+1=9.3 milliseconds, so as to avoid that the rendering speed is too fast to cause too many cached images. At this time, the maximum allowed time can be 9.3 milliseconds.

[0067] After the maximum allowed time is determined, whether the completion time of the current frame image rendered by the rendering device according to different running frequencies is less than the maximum allowed time can be used to determine the target frequency. Optionally, the minimum running frequency, that is, the lower limit value of the target frequency, can be determined according to the load and the maximum allowed time, the optional range of the target frequency can be determined according to the lower limit value of the target frequency, and a target frequency is selected in the optional range. In the embodiment of the present application, the lower limit value of the target frequency f require can be calculated by the following formula (1), where require_time represents the maximum allowed time, load_cycles represents the predicted value when the predicted load of the current frame image is the predicted value, and load_cycles represents the upper limit value of the predicted interval when the predicted load of the current frame image is the predicted interval. Therefore, the optional range of the target frequency is greater than f require .

[0068] In a possible implementation, the target frequency is a frequency corresponding to the power consumption meeting the power consumption requirement in the optional frequencies, and the power consumption requirement can be the lowest power consumption. The power consumption corresponding to any frequency refers to the power consumption of the rendering device for rendering the current frame image at any frequency. The optional frequencies include a plurality of frequencies matching the load of the current frame image, for example, the optional frequencies can be determined based on the optional range of the target frequency. For example, the optional frequencies include a plurality of frequencies greater than f require . Thus, the frequency with the lowest power consumption can be selected from the range of the optional frequencies, so that the running frequency of the rendering device for rendering the current frame image can match the load and reduce the energy consumption for rendering the current frame image, achieving a balance between performance and power consumption.

[0069] wherein the power consumption of the rendering device can be determined according to the running power and the running time of the rendering device, and the running power of the rendering device refers to the energy consumption of the rendering device per unit time. The running power corresponding to different running frequencies is different. For example, the running frequency is positively correlated with the running power, that is, the greater the running frequency, the greater the running power, and the smaller the running frequency, the smaller the running power. The manner of obtaining the running power of the rendering device is not limited, for example, the running power can be obtained from the power table of the hardware of the rendering device or manually input. Alternatively, the power consumption e require of the rendering device for rendering the current frame image at the target frequency can be calculated by the following formula (2), wherein p require represents the running power of the rendering device corresponding to the target frequency, and t require represents the completion time of the rendering device for rendering the current frame image at the target frequency. e require = p require* t require formula (2)

[0070] Alternatively, one of the plurality of frequencies can be a frequency value or a combination of at least two frequency values. For example, the plurality of frequencies include a plurality of frequency values, and the optional frequencies can include f1 and f2, that is, f1≥f require and f2≥f require . The completion time of the rendering device for rendering the current frame image at f1 is t1, that is, t1≤require_time, the running power corresponding to f1 is p1, and the power consumption of the rendering device for rendering the current frame image at f1 is t1*p1=e1; the completion time of the rendering device for rendering the current frame image at f2 is t2, that is, t2≤require_time, the running power corresponding to f2 is p2, and the power consumption of the rendering device for rendering the current frame image at f2 is t2*p2=e2. In the case of e1

[0071] Taking a combination of at least two frequency values in a plurality of frequencies as an example, the optional frequency can include a combination of f1 and f2 and f3. For the combination of f1 and f2, the rendering device renders the current frame image in t1+t2, wherein the rendering device renders the current frame image according to f1 in t1, the rendering device renders the current frame image according to f2 in t2, t1+t2≤require_time, and f1*t1+f 2* t2≥load_cycles, the power consumption of the rendering device for rendering the current frame image according to the combination of f1 and f2 is t 1* p1+t 2* p2=e3; the power consumption of the rendering device for rendering the current frame image according to f3 is e4. In the case of e3

[0072] Optionally, in the process of determining the target frequency based on the optional frequency, in addition to considering the corresponding power consumption, the running temperature of the electronic device and the like can also be considered. For example, the target frequency is the frequency in the optional frequency corresponding to the running temperature less than the temperature threshold, and the running temperature corresponding to any frequency refers to the temperature when the rendering device runs according to the frequency, and the temperature threshold can be flexibly adjusted according to the application scenario. Alternatively, the target frequency is the frequency in the optional frequency corresponding to the power consumption satisfying the power consumption requirement and the running temperature less than the temperature threshold.

[0073] Therefore, after the target frequency is determined, the rendering device can be controlled to render the current frame image according to the target frequency. Illustratively, the target frequency is sent to the rendering device driver to drive the rendering device to run according to the target frequency, and the rendering device runs is used to render the current frame image. Therefore, for different frame images, the rendering device can flexibly adjust the running frequency of the rendering device by predicting the load of different frame images, that is, the running frequency of the rendering device for rendering different frame images can be different, so that the current running frequency of the rendering device matches the current load. That is, the frequency is adjusted frame by frame, and the target frequency is accurately supplied.

[0074] Illustratively, taking the rendering device as a GPU as an example, the rendering parameter is obtained through the CPU side, and the rendering parameter is referred to as CPU parameter data; the load is obtained through the GPU side, and the load is referred to as GPU load data. Taking the electronic device executing the GPU frequency modulation process as an example, referring to FIG. 4, FIG. 4 is a whole process diagram of a GPU frequency modulation provided by an embodiment of the present application, and the GPU frequency modulation process includes but is not limited to the following S1-S5.

[0075] S1, record the historical data, and the historical data includes the CPU parameter data and the GPU load data for rendering each frame image.

[0076] S2, obtain CPU parameter data of the current frame image, and combine the historical data to form feature data, and the feature data corresponds to the first feature vector.

[0077] S3, calculate the GPU load prediction value of the current frame image based on the feature data.

[0078] S4, determine the target frequency according to the GPU load prediction value of the current frame image and the display performance requirement of the current frame image.

[0079] S5, adjust the GPU to render the current frame image at the target frequency.

[0080] In summary, the method provided by the embodiments of the present application combines the rendering parameters of the current frame image and the historical data to predict the load of the current frame image, not only considers the change trend of the rendering parameters and the load in the historical data, but also considers the rendering parameters of the current frame image, which can reflect the rendering complexity of the current frame image. Therefore, the rendering parameters of the current frame image can improve the accuracy of the predicted load, and further improve the accuracy of the frequency control according to the load, so that the method can cope with the load burst scene and improve the matching degree of the operating frequency and the load of the rendering device.

[0081] The above introduces the frequency control method of the embodiments of the present application. Corresponding to the above method, the embodiments of the present application also provide a frequency control device. FIG. 5 is a structural schematic diagram of a frequency control device provided by an embodiment of the present application. The device is applied to an electronic device, which can be the electronic device shown in FIG. 1. Based on the following multiple modules shown in FIG. 5, the frequency control device shown in FIG. 5 can perform all or part of the operations performed by the electronic device, for example, perform all or part of the operations in the method shown in FIG. 2. It should be understood that the device can include more additional modules than the shown modules or omit part of the shown modules, and the embodiments of the present application do not limit this. As shown in FIG. 5, the device includes a data acquisition module 501, a load prediction module 502, a frequency decision module 503 and a frequency control module 504.

[0082] The data acquisition module 501 is configured to obtain the rendering parameters of the current frame image, and the rendering parameters are rendering-related parameters generated before the rendering device renders the current frame image. The load prediction module 502 is configured to predict the load of the current frame image based on the rendering parameters of the current frame image and the historical data, and the historical data includes the loads and the rendering parameters corresponding to a plurality of historical frame images respectively, and the load refers to the calculation amount information of the rendering device rendering the image. The frequency control module 504 is configured to control the rendering device to render the current frame image at the target frequency, and the target frequency matches the load of the current frame image. Optionally, the device further includes the frequency decision module 503, which is configured to determine the target frequency matching the load of the current frame image.

[0083] Exemplarily, the data collection module 501 is also responsible for collecting historical data of rendering parameters on the CPU side and load on the rendering device side, and an output of the data collection module 501 is connected with the load prediction module 502. Optionally, the data collection module 501 collects the rendering parameters or the load from the system code of the electronic device. The load prediction module 502 receives the rendering parameters and the historical data of the current frame image output by the data collection module 501, assembles the rendering parameters and the historical data of the current frame image into feature data, and predicts the load of the current frame image based on the feature data. An output of the load prediction module 502 is connected with the frequency decision module 503. The load prediction module 502 can be located in a user CPU process or a kernel management process, and completes feature processing of data and load prediction before the rendering device starts a task.

[0084] The frequency decision module 503 receives the prediction result, i.e., the load of the current frame image, output by the load prediction module 502, determines the running frequency of the rendering device, i.e., the target frequency, according to the load of the current frame image and the display performance requirement, and an output of the frequency decision module 503 is connected with the frequency control module 504. The frequency decision module 503 can be located in the kernel management process of the electronic device. The frequency control module 504 receives the target frequency output by the frequency decision module 503, controls the rendering device hardware to run at the target frequency, i.e., to render the current frame image, by calling an interface of the rendering device driver.

[0085] In a possible implementation, the load prediction module 502 is configured to perform feature extraction on the rendering parameters and the historical data of the current frame image to obtain a first feature vector, determine a first load corresponding to the first feature vector based on a mapping relationship between the feature vector and the load, and take the first load as the load of the current frame image.

[0086] In a possible implementation, the load prediction module 502 is configured to call a machine learning model to determine the first load corresponding to the first feature vector, the machine learning model contains the mapping relationship, and the machine learning model is obtained based on a data set. The data set includes feature vectors and loads corresponding to a plurality of historical frame images, respectively.

[0087] In a possible implementation, the rendering parameters include at least one of a related parameter of IPC data transmission or a related parameter of a rendering tree.

[0088] In a possible implementation, the target frequency matches the load of the current frame image, which means that a completion time of rendering the current frame image by the rendering device at the target frequency is less than or equal to a maximum allowed time, and the maximum allowed time is a rendering time that meets the display performance requirement.

[0089] In a possible implementation, the maximum allowed time is determined based on a display time interval between the current frame image and a previous frame image, or determined based on the display time interval and a rendering delay time of a historical frame before the current frame image, or determined based on the display time interval and a rendering remaining time of the historical frame before the current frame image.

[0090] In a possible implementation, the target frequency is a frequency corresponding to power consumption satisfying the power consumption requirement in the selectable frequencies, and the power consumption corresponding to any frequency refers to power consumption generated by the rendering device when rendering the current frame image at the frequency, and the selectable frequencies include multiple frequencies matching the load of the current frame image.

[0091] To sum up, the device provided in the embodiments of the present application predicts the load of the current frame image in combination with the rendering parameter of the current frame image and the historical data, not only considers the change trend of the rendering parameter and the load presented in the historical data, but also considers the rendering parameter of the current frame image, which can reflect the rendering complexity of the current frame image. Therefore, the rendering parameter of the current frame image can improve the accuracy of the predicted load, and further improve the accuracy of the frequency control according to the load, so that the method can cope with the load burst scenario and improve the matching degree of the operating frequency of the rendering device and the load.

[0092] It should be understood that the device provided in the above Figure 5, when realizing its functions, is only exemplified by the above division of functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. The data acquisition module 501, the load prediction module 502, the frequency adjustment decision module 503 and the frequency control module 504 can be flexibly located in a CPU process, a GPU process, a kernel process and the like, and the embodiments of the present application do not limit this. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process and beneficial effects are detailed in the method embodiments, which will not be described here.

[0093] Referring to Figure 6, Figure 6 shows a structural schematic diagram of a terminal device 2000 provided in an example embodiment of the present application. The terminal device 2000 shown in Figure 6 is used to execute the operations involved in the frequency control method shown in Figure 2. The terminal device 2000 can be realized by a general bus architecture.

[0094] As shown in Figure 6, the terminal device 2000 includes at least one processor 2001, a memory 2003 and at least one communication interface 2004.

[0095] The processor 2001 is, for example, a general-purpose central processing unit (CPU), a digital signal processor (DSP), a network processer (NP), a graphics processing unit (GPU), a neural-network processing units (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits used to implement a set of instructions. The GPU corresponds to the rendering device in the above embodiments.

[0096] For example, the processor 2001 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or other programmable logic device, transistor logic, a hardware component, or any combination thereof. The PLD is, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can implement or execute various logical blocks, modules, and circuits described in conjunction with the disclosure of the embodiments of the application. The processor can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0097] Optionally, the terminal device 2000 also includes a bus. The bus is used to transmit information between the components of the terminal device 2000. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in FIG. 6, but it does not mean that there is only one bus or only one type of bus.

[0098] The memory 2003 is, for example, a read-only memory (ROM) or other type of static storage device that can store static information and instructions that are not altered, for example, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions for execution by the processor 2001, for example, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disk storage, for example, a compact disc, for example, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, or the like, a magnetic disk storage medium, or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory 2003 is, for example, independent and connected to the processor 2001 through the bus. The memory 2003 can also be integrated with the processor 2001.

[0099] The communication interface 2004 uses any transceiver-like mechanism for communicating with other devices or a communication network, which can be an Ethernet, a radio access network (RAN), a wireless local area networks (WLAN), or the like. The communication interface 2004 can include a wired communication interface and can also include a wireless communication interface. Specifically, the communication interface 2004 can be an Ethernet interface, a Fast Ethernet (FE) interface, a Gigabit Ethernet (GE) interface, an Asynchronous Transfer Mode (ATM) interface, a wireless local area networks (WLAN) interface, a cellular network communication interface, or a combination thereof. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. In the embodiments of the present application, the communication interface 2004 can be used for the terminal device 2000 to communicate with other devices.

[0100] In particular implementations, as one example, the processor 2001 can include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 6. Each of these processors can be a single-core CPU or a multi-core CPU. A processor, as used herein, can refer to one or more devices, circuits, and / or processing cores for processing data, such as computer program instructions.

[0101] In particular implementations, as one example, the terminal device 2000 can include multiple processors, such as the processor 2001 and the processor 2005 shown in FIG. 6. Each of these processors can be a single-core CPU or a multi-core CPU. A processor, as used herein, can refer to one or more devices, circuits, and / or processing cores for processing data, such as computer program instructions.

[0102] In particular implementations, as one example, the terminal device 2000 can also include an output device and an input device. The output device is in communication with the processor 2001 and can display information in various ways. For example, the output device can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, a projector, or the like. The input device is in communication with the processor 2001 and can receive user input in various ways. For example, the input device can be a mouse, a keyboard, a touch screen device, a sensor device, or the like.

[0103] In some embodiments, the memory 2003 is configured to store program code 2010 for performing the solutions of the present application, and the processor 2001 can execute the program code 2010 stored in the memory 2003. That is, the terminal device 2000 can implement the frequency control method provided by the method embodiments through the processor 2001 and the program code 2010 in the memory 2003. The program code 2010 can include one or more software modules. Alternatively, the processor 2001 itself can also store program codes or instructions for performing the solutions of the present application.

[0104] In particular embodiments, the terminal device 2000 of the embodiments of the present application can correspond to the terminal device in each of the above method embodiments, and the processor 2001 in the terminal device 2000 reads instructions in the memory 2003, so that the terminal device 2000 shown in FIG. 6 can perform all or part of the operations performed by the terminal device.

[0105] Specifically, the processor 2001 is configured to acquire a rendering parameter of a current frame image, the rendering parameter being a rendering-related parameter generated before the GPU renders the current frame image; predict a load of the current frame image based on the rendering parameter of the current frame image and historical data, the historical data including loads and rendering parameters corresponding to a plurality of historical frame images respectively, the load being a calculation amount of the GPU rendering the image; and control the GPU to render the current frame image at a target frequency, the target frequency being matched with the load of the current frame image.

[0106] Other optional embodiments will not be described here for brevity.

[0107] The terminal device 2000 can also correspond to the frequency control apparatus shown in FIG. 5, and each functional module in the frequency control apparatus is implemented by software of the terminal device 2000. In other words, the functional modules included in the frequency control apparatus are generated after the processor 2001 of the terminal device 2000 reads the program code 2010 stored in the storage 2003.

[0108] The steps of the frequency control method shown in FIG. 2 are completed by the integrated logic circuit of the hardware or the instructions in the form of software in the processor of the terminal device 2000. The steps of the method disclosed in the embodiments of the present application can be directly embodied as execution completed by the hardware processor, or executed by the combination of the hardware and software modules in the processor. The software module can be located in the storage medium mature in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the storage, and the processor reads the information in the storage, and combines the hardware to complete the steps of the above method. To avoid repetition, the details will not be described here.

[0109] It should be understood that the above processor can be a CPU, and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It should be noted that the processor can be a processor supporting advanced RISC machines (ARM) architecture.

[0110] Further, in an alternative embodiment, the aforementioned memory can include read-only memory and random access memory, and provide the processor with instructions and data. The memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0111] The memory can be volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of illustration, and not limitation, nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which acts as external cache. By way of illustration and not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM) are available.

[0112] The embodiment of the present application further provides a visual device, the visual device comprising a rendering device, and the visual device is used for executing the frequency control method in any of the above to control the operation frequency of the rendering device.

[0113] The embodiment of the present application further provides a computer readable storage medium, at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to make a computer implement the frequency control method in any of the above.

[0114] The embodiment of the present application further provides a computer program (product), when the computer program is executed by a computer, can make the processor or the computer execute the corresponding steps and / or processes in the above method embodiments.

[0115] The embodiment of the present application further provides a chip, comprising a processor, which is configured to call and run instructions stored in a memory, so that a device installed with the chip performs the frequency control method according to any one of the above.

[0116] The embodiment of the present application further provides another chip, comprising an input interface, an output interface, a processor and a memory, which are connected through internal connection paths, and the processor is configured to execute codes in the memory, and when the codes are executed, the processor is configured to perform the frequency control method according to any one of the above.

[0117] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD) or semiconductor media (such as solid state disk) and the like.

[0118] Those skilled in the art can realize that, in combination with the method steps and modules described in the embodiments disclosed herein, the embodiments can be implemented in software, hardware, firmware or any combination thereof. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of the embodiments have been described in the above description. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0119] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program instructing relevant hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0120] When implemented using software, the functions can be implemented using one or more computer program products. The computer program product can include one or more computer program instructions. As an example, the methods of embodiments of the present application can be described in the context of computer readable medium including machine-executable instructions, such as program modules, executed by devices in the target real or virtual processor. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., which perform specific tasks or implement specific abstract data types. In various embodiments, the functions of the program modules can be combined or divided among the described program modules. Machine-executable instructions for program modules can be executed within a local or distributed device. In a distributed device, program modules can be located in both local and remote storage media.

[0121] The computer program code for implementing the methods of embodiments of the present application can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that the program codes cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented when the computer or other programmable data processing apparatus executes the program codes. The program codes can be executed entirely on the computer, partially on the computer, as a separate software package, partially on the computer and partially on a remote computer, or entirely on a remote computer or server.

[0122] In the context of embodiments of the present application, computer program code or related data can be carried by any appropriate carrier to enable the device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, etc.

[0123] Examples of signals can include electrical, optical, radio, sound or other forms of propagated signals, such as carrier waves, infrared signals, etc.

[0124] A machine-readable medium can be any tangible medium that includes or stores programs for execution by an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), and a digital versatile disc (DVD), or any suitable combination of the foregoing.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and module can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0126] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the module is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, and can also be electrical, mechanical or other form of connection.

[0127] The module described as a separate component can or can not be physically separated, and the component displayed as a module can or can not be a physical module, that is, it can be located in one place, or it can be distributed to a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0128] In addition, the functional modules in each embodiment of the present application can be integrated into a processing module, or each module can exist physically, or two or more modules can be integrated into one module. The above integrated module can be realized in the form of hardware or in the form of a software functional module.

[0129] The integrated module, if implemented in the form of a software function module and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a terminal device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0130] The terms "first", "second", and the like in the present application are used to distinguish between items or similar items having substantially the same function and action. It should be understood that there is no logical or chronological dependency between "first", "second", and "nth", and the number and execution order are not limited. It should also be understood that although the following description uses the terms first, second, and the like to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. The first image and the second image can both be images, and in some cases, can be separate and distinct images.

[0131] It should also be understood that in various embodiments of the present application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] The term "at least one" in the present application means one or more, and the term "multiple" in the present application means two or more, for example, multiple second messages refer to two or more second messages. The terms "system" and "network" are often used interchangeably in this document.

[0133] It should be understood that the terms used in the description of various described examples herein are only for the purpose of describing specific examples and are not intended to be limiting. As used in the description of various described examples and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0134] It should also be understood that, as used in this specification, the terms "comprises", "comprising", "includes", "including", "with" or "comprising", specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0135] It should also be understood that the terms "comprises", "comprising", "includes", "including", "with" or "comprising", specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0136] It should also be understood that the terms "if' and "when" can be construed to mean "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be construed to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]", depending on the context.

[0137] It should be understood that a determination of B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0138] It should also be understood that the terms "one embodiment", "an embodiment", "one possible implementation", as used in the specification are intended to mean that a particular feature, structure, or characteristic described in connection with the embodiment or implementation is included in at least one implementation of the application. Thus, appearances of the phrases "in one embodiment" or "in an embodiment", "one possible implementation", as well as variations thereof, in various places throughout the specification are not necessarily intended to refer to the same embodiment or implementation, unless otherwise indicated. Furthermore, it is appreciated that the particular features, structures, or characteristics can be combined in any suitable manner in one or more implementations.

[0139] The above description is only optional embodiments of the application and is not used to limit the application. Any modification, equivalent replacement, improvement, etc. made within the principles of the application shall be included in the protection scope of the application.

Claims

1. A frequency control method characterized by, The method comprises: obtaining rendering parameters of a current frame image, the rendering parameters being rendering-related parameters generated before a rendering device renders the current frame image; predicting a load of the current frame image based on the rendering parameters of the current frame image and historical data, the historical data comprising loads and rendering parameters corresponding to a plurality of historical frame images respectively, the load being computational load information of the rendering device rendering an image; controlling the rendering device to render the current frame image at a target frequency, the target frequency matching the load of the current frame image.

2. The method of claim 1, wherein, The rendering parameters comprise at least one of inter-process communication (IPC) data transmission-related parameters or rendering tree-related parameters.

3. The method according to claim 1 or 2, characterized in that, The prediction of the load of the current frame image based on the rendering parameters of the current frame image and the historical data comprises: performing feature extraction on the rendering parameters of the current frame image and the historical data to obtain a first feature vector; determining a first load corresponding to the first feature vector based on a mapping relationship between feature vectors and loads, and taking the first load as the load of the current frame image.

4. The method of claim 3, wherein, The determination of the first load corresponding to the first feature vector based on the mapping relationship between feature vectors and loads comprises: calling a machine learning model to determine the first load corresponding to the first feature vector, the machine learning model containing the mapping relationship, and the machine learning model being obtained based on a data set, the data set comprising feature vectors and loads corresponding to a plurality of historical frame images respectively.

5. The method according to any of claims 1 to 4, characterized in that, The target frequency matching the load of the current frame image means that the rendering device completes the rendering of the current frame image at the target frequency in a time less than or equal to a maximum allowable time, the maximum allowable time being a rendering time meeting a display performance requirement.

6. The method of claim 5, wherein, The maximum allowable time is determined based on a display time interval between the current frame image and a previous frame image, or the maximum allowable time is determined based on the display time interval and a rendering delay time of a historical frame before the current frame image, or the maximum allowable time is determined based on the display time interval and a rendering remaining time of the historical frame before the current frame image.

7. The method according to any of claims 1 to 6, characterized in that, The target frequency is a frequency corresponding to power consumption meeting a power consumption requirement in optional frequencies, the power consumption corresponding to any frequency being power consumption generated by the rendering device when rendering the current frame image at the any frequency, and the optional frequencies comprising a plurality of frequencies matching the load of the current frame image.

8. A frequency control device, characterized by The device comprises: a data acquisition module configured to obtain rendering parameters of a current frame image, the rendering parameters being rendering-related parameters generated before a rendering device renders the current frame image; a load prediction module configured to predict a load of the current frame image based on the rendering parameters of the current frame image and historical data, the historical data comprising loads and rendering parameters corresponding to a plurality of historical frame images respectively, the load being computational load information of the rendering device rendering an image; a frequency control module configured to control the rendering device to render the current frame image at a target frequency, the target frequency matching the load of the current frame image.

9. The apparatus of claim 8, wherein, The rendering parameter comprises at least one of an inter-process communication (IPC) data transmission related parameter or a rendering tree related parameter.

10. The apparatus of claim 8 or 9, wherein, The load prediction module is configured to perform feature extraction on the rendering parameter of the current frame image and the historical data to obtain a first feature vector. The first load corresponding to the first feature vector is determined based on a mapping relationship between the feature vector and the load, and the first load is taken as the load of the current frame image.

11. The apparatus of claim 10, wherein, The load prediction module is configured to call a machine learning model to determine the first load corresponding to the first feature vector, the machine learning model comprises the mapping relationship, and the machine learning model is obtained based on a data set, the data set comprising feature vectors and loads corresponding to a plurality of historical frame images respectively.

12. The apparatus of any of claims 8-11, wherein, The target frequency matches the load of the current frame image, which means that the completion time of rendering the current frame image by the rendering device according to the target frequency is less than or equal to a maximum allowed time, and the maximum allowed time is a rendering time meeting a display performance requirement.

13. The apparatus of claim 12, wherein, The maximum allowed time is determined based on a display time interval between the current frame image and a previous frame image, or the maximum allowed time is determined based on the display time interval and a rendering delay time of a historical frame before the current frame image, or the maximum allowed time is determined based on the display time interval and a rendering remaining time of the historical frame before the current frame image.

14. The apparatus of any of claims 8-13, wherein, The target frequency is a frequency corresponding to power consumption meeting a power consumption requirement in optional frequencies, the power consumption corresponding to any frequency is power consumption generated when the rendering device renders the current frame image according to the any frequency, and the optional frequencies comprise a plurality of frequencies matching the load of the current frame image.

15. A visualization device, characterized by The visualization device comprises a rendering device, and the visualization device is configured to perform the frequency control method in any one of claims 1-7 to control the operating frequency of the rendering device.

16. A terminal device, comprising: The terminal device comprises a processor coupled with a memory, and the memory stores at least one program instruction or code, which is loaded and executed by the processor to enable the terminal device to implement the frequency control method in any one of claims 1-7.

17. A chip, characterized by The chip comprises a processor configured to call and run instructions stored in a memory to enable a device installed with the chip to perform the frequency control method in any one of claims 1-7.

18. A computer program product, characterised in that, The computer program product comprises computer program code loaded and executed by a computer to enable the computer to implement the frequency control method in any one of claims 1-7.

Images