Frequency control method, device, equipment, chip and program product

Abstract

The invention discloses a frequency control method and device, equipment, a chip and a program product, and relates to the technical field of computers. The method comprises the following steps: acquiring rendering parameters of a current frame image, wherein the rendering parameters are parameters which are generated before the current frame image is rendered by a rendering device and are related to rendering; the load of the current frame image is predicted based on the rendering parameters of the current frame image and historical data, the historical data comprises loads and rendering parameters corresponding to a plurality of historical frame images, and the loads refer to calculated amount information of the rendering device for rendering the image; and controlling the rendering device to render the current frame image according to the target frequency, wherein the target frequency is matched with the load of the current frame image. According to the method, the accuracy of the predicted load can be improved through the rendering parameters of the current frame image, and then the accuracy of controlling the frequency according to the load is improved, so that the method can cope with a load emergency scene, and the matching degree of the operation frequency of the rendering device and the load is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to frequency control methods, devices, equipment, chips, and software products. Background Technology

[0002] In smartphones and other electronic devices, the rendering unit is responsible for processing graphics data such as image rendering, enabling the device to display images. Because the content and visual effects of each frame displayed by the electronic device differ, the workload on the rendering unit varies for each frame. When the operating frequency of the rendering unit is mismatched with the workload, it becomes difficult to simultaneously optimize performance and power consumption. For example, if the rendering unit's operating frequency is too low, it may not be able to complete image rendering within a specified time, resulting in frame dropping. Conversely, while the rendering unit's operating frequency may meet display performance requirements, higher frequencies consume more power, leading to wasted energy.

[0003] Therefore, how to control the operating frequency of the rendering device to match the load is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a frequency control method, apparatus, device, chip, and program product for controlling the operating frequency of a rendering device.

[0005] In a first aspect, a frequency control method is provided, the method comprising: acquiring rendering parameters of the current frame image, the rendering parameters being rendering-related parameters generated by the rendering device before rendering the current frame image; predicting the load of the current frame image based on the rendering parameters of the current frame image and historical data, the historical data including the load and rendering parameters corresponding to multiple historical frame images respectively, the load referring to the computational load information of the rendering device rendering the image; and controlling the rendering device to render the current frame image at a target frequency, the target frequency being matched with the load of the current frame image.

[0006] This method combines the rendering parameters of the current frame image with historical data to predict the load of the current frame image. It considers not only the changing trends of rendering parameters and load presented in historical data, but also the rendering parameters of the current frame image, which reflect the rendering complexity of the current frame image. Therefore, by using the rendering parameters of the current frame image, the accuracy of the predicted load can be improved, thereby improving the accuracy of frequency control based on load. This allows the method to cope with sudden load surges and improves the matching degree between the operating frequency of the rendering device and the load.

[0007] In one possible implementation, the rendering parameters include at least one of the parameters related to interprocess communication (IPC) data transmission or the parameters related to the rendering tree. This allows the rendering parameters to effectively reflect the rendering complexity of the current frame image and makes the acquisition of the rendering parameters more flexible.

[0008] In one possible implementation, predicting the load of the current frame image based on its rendering parameters and historical data may include: extracting features from the rendering parameters and historical data to obtain a first feature vector; determining a first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load; and using the first load as the load of the current frame image. Thus, extracting feature vectors reduces the computational load of prediction, and pre-determining the mapping relationship improves prediction efficiency.

[0009] In one possible implementation, determining the first load corresponding to the first feature vector based on the mapping relationship between feature vectors and loads may include calling a machine learning model to determine the first load corresponding to the first feature vector. The machine learning model contains the mapping relationship and is trained on a dataset that includes feature vectors and loads corresponding to multiple historical frame images. Training the machine learning model can improve prediction accuracy.

[0010] In one possible implementation, matching the target frequency to the load of the current frame image means that the rendering time of the rendering device at the target frequency is less than or equal to the maximum allowed time, which is the rendering time that meets the display performance requirements. This ensures that the GPU's operating frequency meets the display performance requirements.

[0011] In one possible implementation, the maximum allowable time is determined based on the display time interval between the current frame and the previous frame; or, the maximum allowable time is determined based on the display time interval and the rendering delay time of historical frames preceding the current frame; or, the maximum allowable time is determined based on the display time interval and the remaining rendering time of historical frames preceding the current frame. This ensures that the rendering device can complete the rendering of the current frame at the target frequency in a time less than or equal to the display time interval between adjacent frames, guaranteeing smooth display of the current frame and ensuring the continuity of multiple frames while considering the display delay time of historical frames.

[0012] In one possible implementation, the target frequency is the frequency among the selectable frequencies that meets the power consumption requirements. The power consumption corresponding to any frequency refers to the power consumption generated by the rendering device when rendering the current frame image at any frequency. The selectable frequencies include multiple frequencies that match the load of the current frame image. Thus, the target frequency satisfies both display performance requirements and power consumption requirements, achieving a balance between display performance and power consumption.

[0013] In a second aspect, a frequency control device is provided for executing the method in the first aspect or any possible implementation thereof. Specifically, the frequency control device includes modules for executing the method in the first aspect or any possible implementation thereof. In one possible implementation, the frequency control device includes: a data acquisition module for acquiring rendering parameters of the current frame image, the rendering parameters being rendering-related parameters generated before the rendering device renders the current frame image; a load prediction module for predicting the load of the current frame image based on the rendering parameters of the current frame image and historical data, the historical data including the load and rendering parameters corresponding to multiple historical frame images, the load referring to the computational load information of the rendering device rendering the image; and a frequency control module for controlling the rendering device to render the current frame image at a target frequency, the target frequency being matched with the load of the current frame image.

[0014] In one possible implementation, the rendering parameters include at least one of the parameters related to IPC data transmission or the parameters related to the rendering tree.

[0015] In one possible implementation, the load prediction module is used to extract features from the rendering parameters and historical data of the current frame image to obtain a first feature vector; determine the first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load, and use the first load as the load of the current frame image.

[0016] In one possible implementation, the load prediction module is used to call a machine learning model to determine the first load corresponding to the first feature vector. The machine learning model contains a mapping relationship and is trained on a dataset, which includes feature vectors and loads corresponding to multiple historical frame images.

[0017] In one possible implementation, matching the target frequency with the load of the current frame image means that the rendering device completes rendering the current frame image at the target frequency in a time less than or equal to the maximum allowable time, where the maximum allowable time is the rendering time that meets the display performance requirements.

[0018] In one possible implementation, the maximum allowed time is determined based on the display time interval between the current frame image and the previous frame image; or, the maximum allowed time is determined based on the display time interval and the rendering delay time of historical frames preceding the current frame image; or, the maximum allowed time is determined based on the display time interval and the remaining rendering time of historical frames preceding the current frame image.

[0019] In one possible implementation, the target frequency is the frequency among the selectable frequencies that meets the power consumption requirements. The power consumption corresponding to any frequency refers to the power consumption generated by the rendering device when rendering the current frame image at any frequency. The selectable frequencies include a variety of frequencies that match the load of the current frame image.

[0020] Thirdly, a visualization device is provided, comprising: a rendering apparatus, the visualization device being configured to execute the frequency control method described in the first aspect or any one of the first aspects to control the operating frequency of the rendering apparatus.

[0021] Fourthly, a terminal device is provided, comprising: a processor coupled to a memory, the memory storing at least one program instruction or code, the at least one program instruction or code being loaded and executed by the processor to enable the terminal device to implement the frequency control method as described in the first aspect or any one of the first aspects.

[0022] Optionally, the processor may be one or more, and the memory may be one or more.

[0023] Optionally, the memory may be integrated with the processor, or the memory may be separated from the processor.

[0024] In the specific implementation process, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. This application does not limit the type of memory or the way the memory and processor are set.

[0025] Fifthly, a computer-readable storage medium is provided, the storage medium storing at least one instruction, the instruction being loaded and executed by a processor to enable a computer to implement the frequency control method described in the first aspect or any possible implementation thereof.

[0026] In a sixth aspect, a computer program (product) is provided, the computer program (product) comprising: computer program code, which, when executed by a computer, causes the computer to perform the frequency control method described in the first aspect or any possible implementation thereof.

[0027] In a seventh aspect, a chip is provided, including a processor for retrieving and executing instructions stored in a memory, causing a device on which the chip is mounted to perform the frequency control method described in the first aspect or any possible implementation thereof.

[0028] Eighthly, another chip is provided, comprising: an input interface, an output interface, a processor, and a memory, wherein the input interface, the output interface, the processor, and the memory are connected via an internal connection path, and the processor is configured to execute code in the memory, wherein when the code is executed, the processor is configured to execute the frequency control method described in the first aspect or any possible implementation thereof.

[0029] It should be understood that the beneficial effects of the technical solutions of the second to eighth aspects of this application and the corresponding possible implementations can be referred to the above-described technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0030] Figure 1 A schematic diagram illustrating the implementation environment of a frequency control method provided in this application embodiment;

[0031] Figure 2 A flowchart of a frequency control method provided in an embodiment of this application;

[0032] Figure 3 A schematic diagram of a frame rendering process provided in an embodiment of this application;

[0033] Figure 4 This application provides an overall flowchart of GPU frequency modulation as an embodiment of the present application;

[0034] Figure 5 This is a schematic diagram of the structure of a frequency control device provided in an embodiment of this application;

[0035] Figure 6 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0037] With the rapid development of computer technology, electronic devices such as smartphones need to flexibly control the operating frequency of the rendering device according to the load of the rendered image in order to balance the display performance of the screen and the power consumption of the rendering device, so that the operating frequency matches the load. In the embodiments of this application, the rendering device refers to any device with rendering function, such as a graphics processing unit (GPU), a digital signal processing unit (DSP), or other chips with image processing function.

[0038] Taking the GPU as the rendering device as an example, in one related technique, the GPU load for a historical period is calculated based on the ratio of GPU working time to idle time. The GPU load for the current period is then predicted based on this historical load, and the GPU's operating frequency is controlled according to the predicted load. However, since the load for the current period is predicted solely based on historical load, the predicted GPU load is inaccurate when facing scenarios with sudden load fluctuations, resulting in a low match between the GPU's operating frequency and the load.

[0039] In related technology two, the power parameters required for the current frame image (such as the number of drawing function calls and instruction count) are first predicted based on the power parameters of historical frame images. Then, the GPU load is predicted based on the predicted power parameters. Finally, the GPU's processing frequency for the current frame is controlled based on the predicted GPU load. However, due to the existence of two predictions, accuracy is difficult to guarantee. Furthermore, predicting solely based on the power parameters of historical frame images is also insufficient to handle sudden load scenarios. In other words, the accuracy of the predicted GPU load is low, resulting in a low match between the GPU's operating frequency and the load.

[0040] This application provides a frequency control method that can improve the matching degree between the operating frequency and load of the rendering device. See also Figure 1 , Figure 1 This is a schematic diagram illustrating the implementation environment of a frequency control method provided in an embodiment of this application. For example... Figure 1 As shown, the implementation environment includes an electronic device 101. The electronic device 101 includes a rendering device responsible for image rendering and other graphics data processing. This application embodiment does not limit the type of electronic device 101; it can be a terminal device or other visualization devices besides a terminal.

[0041] Optionally, the terminal device can be any electronic product that can interact with the user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device. Examples include personal computers (PCs), smartphones, cameras, personal digital assistants (PDAs), wearable devices, smartwatches, pocket PCs (PPCs), tablets, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, terminals in self-driving vehicles, smart grids, transportation safety, smart cities, smart homes, or IoT terminals, etc.

[0042] Those skilled in the art should understand that the above-described electronic device 101 is merely an example, and other existing or future electronic devices may also be applicable to this application and should be included within the scope of protection of this application, and are hereby incorporated by reference.

[0043] See Figure 2 , Figure 2 This is a flowchart illustrating a frequency control method provided in an embodiment of this application. The method is explained using an electronic device as an example; exemplarily, the electronic device can be... Figure 1 The electronic device 101 shown is an example. Figure 2 As shown, the frequency control method includes, but is not limited to, the following steps 201-203.

[0044] Step 201: Obtain the rendering parameters of the current frame image. The rendering parameters are rendering-related parameters generated before the rendering device renders the current frame image.

[0045] In this embodiment, rendering parameters refer to rendering-related parameters that are available before the rendering device renders the current frame image. Optionally, rendering parameters include at least one of the following: parameters related to IPC data transmission or parameters related to the render tree; or at least one of the following: parameters related to resources such as memory state, parameters related to pre-rendering before rendering data processing, or rendering-related parameters generated by the rendering device before starting to render the current frame image. Taking a GPU as an example, IPC data transmission can refer to data transmission from the central processing unit (CPU) process to the GPU process. The render tree is a data structure used in the rendering process to display the layout and drawing of the page; that is, the GPU renders according to the structure and content of the render tree. Specifically, the render tree contains all element nodes that need to be displayed, and each element node contains corresponding style information and geometric information.

[0046] For example, rendering parameters include, but are not limited to, at least one of the following: the number of IPC data transfers, the amount of data transferred per IPC data transfer, the number of rendered element nodes, the cache size of element nodes, the number of blurs, the blur area, the number of shadows, the shadow area, the number of brightenings, the brightening area, the size of the dirty area, the texture creation time, or the number of pixels actually rendered by the GPU. Blur is used to simulate the dynamic blur effect in photography, and shadows are used to enhance the depth of the scene, thereby enhancing the realism and artistry of the visual effect. The dirty area refers to the area that has been edited, updated, or affected by adding or deleting features. When drawing each frame, only the dirty area that needs to be redrawn can be drawn, saving rendering resources. Optionally, the rendering parameters are generated by the electronic device during the execution of system code; therefore, the rendering data can be obtained from the system code of the electronic device.

[0047] See Figure 3 , Figure 3 This is a schematic diagram illustrating a frame rendering process provided in an embodiment of this application. For example... Figure 3As shown, taking a GPU as the rendering device as an example, the electronic device achieves frame rendering through the collaboration of the CPU and GPU. For each frame image to be rendered (frame 1, frame 2, frame 3), it is first input to the CPU for processing, such as measurement, layout, animation, or drawing. Then, the CPU synchronizes the processed data to the GPU. Finally, the GPU renders each frame image based on the processed data. For each frame image, before the CPU processing stage (i.e., the GPU rendering stage), the relevant parameters for GPU rendering of that frame image can be obtained, i.e., the rendering parameters in this embodiment. In other words, in the rendering process of a frame image, the CPU is responsible for preparatory work such as layout calculation and drawing instruction generation. Before the GPU starts rendering, the rendering parameters of the current frame image can be obtained from the CPU side. For example, the rendering parameters of the current frame image are obtained from the system code executed on the CPU side.

[0048] Step 202: Predict the load of the current frame image based on the rendering parameters of the current frame image and historical data. The historical data includes the load and rendering parameters corresponding to multiple historical frame images. The load refers to the computational amount of information required by the rendering device to render the image.

[0049] Here, historical frame images refer to frame images that have been rendered before the current frame image, such as multiple consecutive frame images adjacent to the current frame image rendered before the current frame image. In this embodiment, the computational load information of the rendering device in rendering images can be measured by the number of clock cycles, running time, or number of instructions executed by the rendering device in running a program. For example, taking the number of clock cycles as the load, the load of any historical frame image can be 1 million clock cycles. Optionally, the historical data can be recorded in real time during the rendering process of the rendering device rendering historical frame images, and the method of obtaining rendering parameters can be as described in the relevant description in parameter step 201, which will not be repeated here.

[0050] After obtaining the rendering parameters corresponding to the current frame image and multiple historical frame images, as well as the load corresponding to each of the multiple historical frame images, the load of the current frame image can be predicted based on the variation patterns of the rendering parameters from the multiple historical frame images to the current frame image, and the variation patterns of the load among the multiple historical frame images. In one possible implementation, the method of predicting the load of the current frame image based on the rendering parameters of the current frame image and historical data includes, but is not limited to, the following steps 2021 and 2022.

[0051] Step 2021: Extract features from the rendering parameters and historical data of the current frame image to obtain the first feature vector.

[0052] Optionally, the methods for feature extraction of the rendering parameters and historical data of the current frame image include, but are not limited to: performing temporal feature extraction on the first time-series data and the second time-series data, and / or performing frequency domain feature extraction on the first time-series data and the second time-series data. The first time-series data includes the rendering parameters corresponding to the current frame image and multiple historical frame images, respectively, and the second time-series data includes the payloads corresponding to multiple historical frame images, respectively. For example, taking the rendering parameters including the number of IPC data transmissions C and the number of blurrings M as an example, the first time-series data includes {C1, C2, C3, ..., C...} n-1 C n} and {M1,M2,M3,…,M n-1 M n The second time series data includes {L1, L2, L3, ..., L}. n-1}, where L represents the load, n represents the current frame image, and (1,2,…,n-1) represents the n-1 historical frame images preceding the current frame image.

[0053] The time-domain features extracted through time-domain feature extraction include, but are not limited to, at least one of the following: mean, maximum, minimum, standard deviation, variance, peak period, peak-to-peak value, root mean square value, skewness coefficient, or kurtosis coefficient. The frequency features extracted through frequency feature extraction include, but are not limited to, at least one of the following: centroid frequency, mean square frequency, root mean square frequency, frequency variance, or frequency standard deviation. At least one of the extracted time-domain features or frequency features is fused to obtain a first feature vector. The fusion method may include merging, concatenating, or adding.

[0054] In one possible implementation, in addition to obtaining the first feature vector according to step 2021, the first feature vector can also be obtained by extracting features from the rendering parameters corresponding to the current frame image and multiple historical frame images respectively, while the loads corresponding to the multiple historical frame images are used to determine the following mapping relationship between the feature vector and the load.

[0055] Step 2022: Determine the first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load, and use the first load as the load of the current frame image.

[0056] In this embodiment, the mapping relationship between feature vectors and load can characterize the relationship between the changing patterns of feature vectors and load. Therefore, after determining the first feature vector, it can be mapped to the corresponding first load according to the mapping relationship. The mapping relationship can be obtained by summarizing and analyzing the changing patterns between feature vectors and load in the dataset, or it can be obtained by training a machine learning model using the dataset. The dataset includes feature vectors and loads corresponding to multiple frames of images acquired on the electronic device. The feature vector of any frame is obtained by extracting features from the rendering parameters of that frame and the historical data corresponding to that frame. The feature extraction method is the same as that in step 2021, and will not be repeated here.

[0057] In training a machine learning model, the mapping relationship between feature vectors and loads is included in the machine learning model; that is, the input of the machine learning model is the feature vector, and the output is the load. Therefore, determining the first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load can include calling the machine learning model to determine the first load corresponding to the first feature vector. For example, the model structure of the machine learning model can be a bidirectional encoder representation from transformers (BERT) model, a convolutional neural network (CNN) model, an LSTM (long short term memory) model, a bidirectional LSTM model, or a GRU (gated recurrent unit) model, etc., and this application does not limit this aspect.

[0058] Optionally, the load in this embodiment can be the numerical value of the computational workload of the rendering device in rendering the current frame image, i.e., the load value, such as 1 million clock cycles; or, the load in this embodiment can also be the range of the computational workload of the rendering device in rendering the current frame image, i.e., the load interval, such as between 1 million and 2 million clock cycles. Regarding the case where a machine learning model is called to determine the first load corresponding to the first feature vector, if the first load output by the machine learning model is a load value, then the type of the machine learning model can be a regression model; if the first load output by the machine learning model is a load interval, then the type of the machine learning model can be a classification model, in which case different load intervals are divided into different categories.

[0059] Therefore, through steps 2021 and 2022 above, the load of the current frame image can be accurately predicted based on the rendering parameters of the current frame image and historical data. Since the data used for prediction includes not only historical data, i.e., the rendering parameters and load of historical frame images, but also the rendering parameters of the current frame image, this method can learn the load change trend from historical data, and also learn the pattern of sudden load from combining historical data and the rendering data of the current frame image, thereby achieving accurate load prediction.

[0060] This application does not limit the load prediction method. Besides the prediction methods shown in steps 2021 and 2022 above, the prediction model can also be directly invoked to predict the load of the current frame image based on the rendering parameters of the current frame image and historical data. The input to the prediction model is the rendering parameters corresponding to the current frame image and multiple historical frame images, as well as the load corresponding to each of the multiple historical frame images. The output is the load of the current frame image. The prediction model is pre-trained using training data, which includes training samples and training labels corresponding to multiple frames. Taking the nth frame image as an example, the training samples are the rendering parameters corresponding to frames 1 to n and the load corresponding to frames 1 to (n-1), and the training label is the load of the nth frame image.

[0061] Step 203: Control the rendering device to render the current frame image according to the target frequency, which is matched with the load of the current frame image.

[0062] After predicting the load of the current frame image, a target frequency matching the current frame image can be determined. The rendering device is then controlled to render the current frame image at the target frequency, thus matching the rendering device's operating frequency with the load. In this embodiment, matching the target frequency with the load of the current frame image means that the rendering device's completion time for rendering the current frame image at the target frequency is less than or equal to the maximum allowable time, where the maximum allowable time is the rendering time required to meet display performance requirements.

[0063] In one possible implementation, the maximum allowable time is determined based on the display time interval between the current frame and the previous frame. Displaying the current frame and the previous frame within this time interval satisfies the display requirements. Optionally, the display time interval between the current frame and the previous frame can be determined based on the screen refresh rate of the electronic device. For example, with a current screen refresh rate of 120 frames per second, the display time interval between adjacent frames is within 1000 / 120 = 8.3 milliseconds. This means the rendering device needs to complete rendering each frame within 8.3 milliseconds, and the maximum allowable time can be 8.3 milliseconds.

[0064] Optionally, the maximum allowable time can also be determined based on the aforementioned display time interval and the rendering delay time of historical frames preceding the current frame, or based on the display time interval and the remaining rendering time of historical frames preceding the current frame. In this case, the rendering delay or early rendering of historical frames preceding the current frame is considered. The rendering delay time refers to the time during which the rendering completion time exceeds the display time interval, and the remaining rendering time refers to the time during which the rendering completion time is earlier than the display time interval. The rendered image is cached in the electronic device, which reads and displays one frame every display time interval. Therefore, it is sufficient to ensure that the rendering of the current frame is completed before it is read. If the rendering of the current frame is not completed when it is read, it will cause frame drops, resulting in display stuttering; if the current frame is rendered too early, it will lead to excessive caching time and waste of cache space.

[0065] Taking a screen refresh rate of 120 frames per second as an example, if the rendering device completes rendering of the previous frame image 1 millisecond later than the 8.3 millisecond interval (i.e., the rendering delay is 1 millisecond), then the rendering device needs to complete rendering of the current frame image within 8.3 - 1 = 7.3 milliseconds to ensure the display performance requirements of both the previous and current frames. In this case, the maximum allowable time is 7.3 milliseconds. If the rendering device completes rendering of the previous frame image 1 millisecond earlier than the 8.3 millisecond interval (i.e., the remaining rendering time is 1 millisecond), then the rendering device only needs to complete rendering of the current frame image within 8.3 + 1 = 9.3 milliseconds to avoid excessively fast rendering speed leading to too many cached images. In this case, the maximum allowable time is 9.3 milliseconds.

[0066] After determining the maximum allowed time, the target frequency can be determined by whether the completion time for the rendering device to render the current frame image at different operating frequencies is less than the maximum allowed time. Optionally, a minimum operating frequency, i.e., the lower limit of the target frequency, can be determined first based on the load and the maximum allowed time; the selectable range of the target frequency can be determined based on the lower limit of the target frequency, and a target frequency can be selected within the selectable range. In the embodiments of this application, the lower limit of the target frequency f require It can be calculated using the following formula (1), where require_time represents the maximum allowed time. When the load of the current frame image being predicted is the predicted value, load_cycles represents that predicted value. When the load of the current frame image being predicted is the prediction interval, load_cycles represents the upper limit of that prediction interval. Therefore, the selectable range of the target frequency is greater than f. require The frequency.

[0067]

[0068] In one possible implementation, the target frequency is the frequency among the selectable frequencies whose corresponding power consumption meets the power consumption requirement, and the power consumption requirement may be the lowest power consumption. The power consumption corresponding to any frequency refers to the power consumption generated by the rendering device when rendering the current frame image at any frequency. The selectable frequencies include multiple frequencies that match the load of the current frame image. For example, the selectable frequencies can be determined based on the selectable range of the above target frequency. Exemplarily, the selectable frequencies include multiple frequencies greater than f require . Thus, the frequency with the lowest power consumption can be selected within the range of selectable frequencies, so that the operating frequency of the rendering device for rendering the current frame image can not only match the load, but also reduce the energy consumed for rendering the current frame image, achieving a balance between performance and power consumption.

[0069] Among them, the power consumption of the rendering device can be determined according to the operating power and operating time of the rendering device. The operating power of the rendering device refers to the energy consumption generated by the rendering device per unit time of operation. Different operating frequencies correspond to different operating powers. Exemplarily, the operating frequency is positively correlated with the operating power, that is, the greater the operating frequency, the greater the operating power, and the smaller the operating frequency, the smaller the operating power. There is no limitation on the method for obtaining the operating power of the rendering device. For example, it can be obtained from the power meter of the rendering device hardware or manually input. Optionally, the power consumption e require generated by the rendering device when rendering the current frame image at the target frequency can be calculated by the following formula (2), where p require represents the operating power corresponding to the target frequency of the rendering device, and t require represents the completion time of the rendering device when rendering the current frame image at the target frequency.

[0070] e require = p require * t require Formula (2)

[0071] Optionally, one of the multiple frequencies can be a frequency value or a combination of at least two frequency values. Taking the multiple frequencies including multiple frequency values as an example, the selectable frequencies can include f1 and f2, that is, f1≥f require , f2≥f require . The completion time of the rendering device when rendering the current frame image at f1 is t1, that is, t1≤require_time, the operating power corresponding to f1 is p1, and the power consumption generated by the rendering device when rendering the current frame image at f1 is t1*p1 = e1; the completion time of the rendering device when rendering the current frame image at f2 is t2, that is, t2≤require_time, the operating power corresponding to f2 is p2, and the power consumption generated by the rendering device when rendering the current frame image at f2 is t2*p2 = e2. In the case of e1 < e2, f1 is selected as the target frequency.

[0072] Taking the combination of multiple frequencies including at least two frequency values as an example, the optional frequencies may include the combination of f1 and f2 and f3. For the combination of f1 and f2, the completion time for the rendering device to render the current frame image is t1 + t2, where the rendering device renders the current frame image at f1 within t1 time, and the rendering device renders the current frame image at f2 within t2 time, t1 + t2 ≤ require_time, and f1 * t1 + f2 * t2 ≥ load_cycles. The power consumption generated by the rendering device when rendering the current frame image according to the combination of f1 and f2 is t1 * p1 + t2 * p2 = e3; the power consumption generated by the rendering device when rendering the current frame image according to f3 is e4. When e3 < e4, the combination of f1 and f2 is selected as the target frequency.

[0073] Optionally, in the process of determining the target frequency based on the optional frequencies, in addition to considering the corresponding power consumption, factors such as the operating temperature of the electronic device can also be considered. For example, the target frequency is the frequency among the optional frequencies whose corresponding operating temperature is less than the temperature threshold. The operating temperature corresponding to any frequency refers to the temperature when the rendering device operates at that any frequency, and the temperature threshold can be flexibly adjusted according to the application scenario. Or, the target frequency is the frequency among the optional frequencies whose corresponding power consumption meets the power consumption requirement and whose corresponding operating temperature is less than the temperature threshold.

[0074] Thus, after determining the target frequency, the rendering device can be controlled to render the current frame image according to the target frequency. Exemplarily, the target frequency is sent to the rendering device driver to drive the rendering device to operate at the target frequency, and the operation of the rendering device is for rendering the current frame image. Thus, for different frame images, the rendering device can flexibly adjust the operating frequency of the rendering device by predicting the load of different frame images, that is, the operating frequencies of the rendering device for rendering different frame images may be different, so that the current operating frequency of the rendering device matches the current load. That is, frame-by-frame frequency modulation is achieved, and the target frequency is accurately supplied.

[0075] Exemplarily, taking the rendering device as the GPU, the above rendering parameters are obtained from the CPU side, and the rendering parameters are called CPU parameter data; the above load is obtained from the GPU side, and the load is called GPU load data. Taking the electronic device executing the GPU frequency modulation process as an example, see Figure 4 , Figure 4 which is an overall flowchart of GPU frequency modulation provided by an embodiment of the present application. The GPU frequency modulation process includes but is not limited to the following S1 - S5.

[0076] S1, record historical data, where the historical data includes CPU parameter data and GPU load data for rendering each frame image.

[0077] S2, obtain the CPU parameter data of the current frame image, and combine it with historical data to form feature data, which corresponds to the first feature vector mentioned above.

[0078] S3 calculates the GPU load prediction value for the current frame image based on feature data.

[0079] S4. Determine the target frequency based on the predicted GPU load of the current frame image and the display performance requirements of the current frame image.

[0080] S5, adjusts the GPU to render the current frame image at the target frequency.

[0081] In summary, the method provided in this application combines the rendering parameters of the current frame image with historical data to predict the load of the current frame image. It considers not only the changing trends of rendering parameters and load presented in historical data, but also the rendering parameters of the current frame image, which reflect the rendering complexity of the current frame image. Therefore, by using the rendering parameters of the current frame image, the accuracy of the predicted load can be improved, thereby improving the accuracy of frequency control based on load. This enables the method to cope with sudden load surges and improves the matching degree between the operating frequency of the rendering device and the load.

[0082] The frequency control method of this application embodiment has been described above. Corresponding to the above method, the frequency control device is also provided in this application embodiment. Figure 5 This is a schematic diagram of a frequency control device provided in an embodiment of this application. The device is applied to an electronic device, which can be the aforementioned... Figure 1 The electronic device shown. Based on Figure 5 The following modules are shown. Figure 5 The frequency control device shown can perform all or part of the operations performed by the electronic device, for example, performing... Figure 2 All or part of the operations in the illustrated method. It should be understood that the apparatus may include more additional modules than those shown, or may omit some of the modules shown; this application embodiment does not impose limitations in this regard. Figure 5 As shown, the device includes a data acquisition module 501, a load prediction module 502, a frequency modulation decision module 503, and a frequency control module 504.

[0083] The data acquisition module 501 is used to acquire the rendering parameters of the current frame image. These rendering parameters are generated before the rendering device renders the current frame image. The load prediction module 502 is used to predict the load of the current frame image based on its rendering parameters and historical data. The historical data includes the load and rendering parameters corresponding to multiple historical frame images. The load refers to the computational load information of the rendering device when rendering the image. The frequency control module 504 is used to control the rendering device to render the current frame image at a target frequency that matches the load of the current frame image. Optionally, the device also includes a frequency modulation decision module 503, used to determine the target frequency that matches the load of the current frame image.

[0084] For example, the data acquisition module 501 is also responsible for acquiring historical data such as rendering parameters on the CPU side and load on the rendering device side. The output of the data acquisition module 501 is connected to the load prediction module 502. Optionally, the data acquisition module 501 acquires rendering parameters or load from the system code of the electronic device. The load prediction module 502 receives the rendering parameters and historical data of the current frame image output by the data acquisition module 501, combines the rendering parameters and historical data of the current frame image into feature data, and predicts the load of the current frame image based on the feature data. The output of the load prediction module 502 is connected to the frequency modulation decision module 503. The load prediction module 502 can be located in the user CPU process or the kernel management process, and completes the feature processing of the data and load prediction before the rendering device starts the task.

[0085] The frequency modulation decision module 503 receives the prediction result (i.e., the load of the current frame image) output by the load prediction module 502, and determines the operating frequency of the rendering device, i.e., the target frequency, based on the load of the current frame image and the display performance requirements. The output of the frequency modulation decision module 503 is connected to the frequency control module 504. The frequency modulation decision module 503 can reside within the kernel management process of the electronic device. The frequency control module 504 receives the target frequency output by the frequency modulation decision module 503 and controls the rendering device hardware to operate at the target frequency by calling the interface of the rendering device driver, i.e., rendering the current frame image.

[0086] In one possible implementation, the load prediction module 502 is used to extract features from the rendering parameters and historical data of the current frame image to obtain a first feature vector; determine the first load corresponding to the first feature vector based on the mapping relationship between the feature vector and the load, and use the first load as the load of the current frame image.

[0087] In one possible implementation, the load prediction module 502 is used to call a machine learning model to determine the first load corresponding to the first feature vector. The machine learning model contains a mapping relationship and is trained based on a dataset, which includes feature vectors and loads corresponding to multiple historical frame images.

[0088] In one possible implementation, the rendering parameters include at least one of the parameters related to IPC data transmission or the parameters related to the rendering tree.

[0089] In one possible implementation, matching the target frequency with the load of the current frame image means that the rendering device completes rendering the current frame image at the target frequency in a time less than or equal to the maximum allowable time, where the maximum allowable time is the rendering time that meets the display performance requirements.

[0090] In one possible implementation, the maximum allowed time is determined based on the display time interval between the current frame image and the previous frame image; or, the maximum allowed time is determined based on the display time interval and the rendering delay time of historical frames preceding the current frame image; or, the maximum allowed time is determined based on the display time interval and the remaining rendering time of historical frames preceding the current frame image.

[0091] In one possible implementation, the target frequency is the frequency among the selectable frequencies that meets the power consumption requirements. The power consumption corresponding to any frequency refers to the power consumption generated by the rendering device when rendering the current frame image at any frequency. The selectable frequencies include a variety of frequencies that match the load of the current frame image.

[0092] In summary, the apparatus provided in this application combines the rendering parameters of the current frame image with historical data to predict the load of the current frame image. It considers not only the changing trends of rendering parameters and load presented in historical data, but also the rendering parameters of the current frame image, which reflect the rendering complexity of the current frame image. Therefore, by using the rendering parameters of the current frame image, the accuracy of the predicted load can be improved, thereby improving the accuracy of frequency control based on load. This enables the method to cope with sudden load surges and improves the matching degree between the operating frequency of the rendering apparatus and the load.

[0093] It should be understood that the above Figure 5 The provided device, in implementing its functions, is only illustrated by the division of the above-mentioned functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The data acquisition module 501, load prediction module 502, frequency modulation decision module 503, and frequency control module 504 can be flexibly located within the CPU process, GPU process, kernel process, etc., and this application embodiment does not impose any restrictions on this. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process and beneficial effects are detailed in the method embodiments, which will not be repeated here.

[0094] See Figure 6 , Figure 6 A schematic diagram of the structure of a terminal device 2000 provided in an exemplary embodiment of this application is shown. Figure 6 The terminal device 2000 shown is used to perform the above. Figure 2 The frequency control method shown involves operations. This terminal device 2000 can be implemented using a general bus architecture.

[0095] like Figure 6 As shown, the terminal device 2000 includes at least one processor 2001, a memory 2003, and at least one communication interface 2004.

[0096] The processor 2001 is, for example, a general-purpose central processing unit (CPU), a digital signal processor (DSP), a network processor (NP), a graphics processing unit (GPU), a neural network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the solution of this application. The GPU corresponds to the rendering device in the above embodiments.

[0097] For example, processor 2001 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A PLD may be, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof. It can implement or execute the various logic blocks, modules, and circuits described in connection with the embodiments of this invention. A processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0098] Optionally, the terminal device 2000 also includes a bus. The bus is used to transfer information between the various components of the terminal device 2000. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0099] Memory 2003 may be, for example, read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; electrically erasable programmable read-only memory (EEPROM); compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.); magnetic disk storage media or other magnetic storage devices; or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 2003 may exist independently and be connected to processor 2001 via a bus. Memory 2003 may also be integrated with processor 2001.

[0100] The communication interface 2004 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), or wireless local area networks (WLAN). The communication interface 2004 can include wired and wireless communication interfaces. Specifically, the communication interface 2004 can be an Ethernet interface, a Fast Ethernet (FE) interface, a Gigabit Ethernet (GE) interface, an Asynchronous Transfer Mode (ATM) interface, a WLAN interface, a cellular network communication interface, or a combination thereof. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. In this embodiment, the communication interface 2004 can be used by the terminal device 2000 to communicate with other devices.

[0101] In a specific implementation, as one example, the processor 2001 may include one or more CPUs, such as Figure 6 The CPUs shown are CPU0 and CPU1. Each of these processors can be a single-core CPU or a multi-core CPU. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0102] In a specific implementation, as one example, the terminal device 2000 may include multiple processors, such as... Figure 6 The processors shown are 2001 and 2005. Each of these processors can be a single-core CPU or a multi-core CPU. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0103] In a specific implementation, as one example, the terminal device 2000 may further include an output device and an input device. The output device communicates with the processor 2001 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with the processor 2001 and can receive user input in various ways. For example, the input device may be a mouse, a keyboard, a touchscreen device, or a sensor device, etc.

[0104] In some embodiments, the memory 2003 stores program code 2010 for executing the solution of this application, and the processor 2001 can execute the program code 2010 stored in the memory 2003. That is, the terminal device 2000 can implement the frequency control method provided in the method embodiment through the processor 2001 and the program code 2010 in the memory 2003. The program code 2010 may include one or more software modules. Optionally, the processor 2001 itself may also store program code or instructions for executing the solution of this application.

[0105] In a specific embodiment, the terminal device 2000 of this application embodiment can correspond to the terminal device in the above-described method embodiments. The processor 2001 in the terminal device 2000 reads the instructions in the memory 2003, causing... Figure 6 The terminal device 2000 shown can perform all or part of the operations performed by the terminal device.

[0106] Specifically, the processor 2001 is used to obtain the rendering parameters of the current frame image. The rendering parameters are rendering-related parameters generated before the GPU renders the current frame image. Based on the rendering parameters of the current frame image and historical data, the processor 2001 predicts the load of the current frame image. The historical data includes the load and rendering parameters corresponding to multiple historical frame images. The load refers to the computational amount of the GPU rendering the image. The processor 2001 controls the GPU to render the current frame image at a target frequency, which matches the load of the current frame image.

[0107] Other alternative implementation methods will not be described in detail here for the sake of brevity.

[0108] Terminal device 2000 can also correspond to the above. Figure 5The frequency control device shown in the diagram is implemented using software from the terminal device 2000 for each functional module. In other words, the functional modules included in the frequency control device are generated by the processor 2001 of the terminal device 2000 reading the program code 2010 stored in the memory 2003.

[0109] in, Figure 2 Each step of the frequency control method shown is completed through integrated logic circuits in the hardware or instructions in the software form of the processor of the terminal device 2000. The steps of the method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. Since the storage medium is located in memory, the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method; to avoid repetition, these will not be described in detail here.

[0110] It should be understood that the aforementioned processor can be a CPU, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Computing (RISC) machines (ARM) architecture.

[0111] Furthermore, in an alternative embodiment, the memory described above may include read-only memory and random access memory, and provide instructions and data to the processor. The memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0112] The memory can be volatile or non-volatile, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0113] This application embodiment also provides a visualization device, which includes a rendering device. The visualization device is used to execute any of the frequency control methods described above to control the operating frequency of the rendering device.

[0114] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to enable the computer to implement any of the frequency control methods described above.

[0115] This application also provides a computer program (product) that, when executed by a computer, causes the processor or computer to perform the corresponding steps and / or processes in the above method embodiments.

[0116] This application also provides a chip, including a processor, for calling and executing instructions stored in a memory, causing a device with the chip installed to perform any of the frequency control methods described above.

[0117] This application embodiment also provides another chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute any of the frequency control methods described above.

[0118] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to this application are generated, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk), etc.

[0119] Those skilled in the art will recognize that the method steps and modules described in conjunction with the embodiments disclosed herein can be implemented in software, hardware, firmware, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0121] When implemented using software, it can be implemented wholly or partially as a computer program product. This computer program product includes one or more computer program instructions. As an example, the methods of this application embodiment can be described in the context of machine-executable instructions, such as program modules that execute on a device on a real or virtual processor of the target. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., which perform specific tasks or implement specific abstract data structures. In various embodiments, the functionality of program modules can be combined or divided among the described program modules. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside on both local and remote storage media.

[0122] Computer program code used to implement the methods of the embodiments of this application may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the computer or other programmable data processing apparatus, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a standalone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0123] In the context of the embodiments of this application, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, etc.

[0124] Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0125] A machine-readable medium can be any tangible medium that contains or stores programs for or relating to an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More detailed examples of machine-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0126] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0127] In the embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or modules, or they may be electrical, mechanical, or other forms of connection.

[0128] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0129] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0130] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or terminal device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0131] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. Both the first image and the second image can be images, and in some cases, they can be separate and distinct images.

[0132] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0133] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple second messages refer to two or more second messages. The terms "system" and "network" are often used interchangeably in this document.

[0134] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0135] It should also be understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects are in an "or" relationship.

[0136] It should also be understood that the term “comprising” (also referred to as “includes”, “including”, “comprises” and / or “comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0137] It should also be understood that the terms “if” and “if” can be interpreted as meaning “when” or “upon”, or “in response to determination” or “in response to detection”. Similarly, depending on the context, the phrases “if determination…” or “if detection [the stated condition or event]” can be interpreted as meaning “when determination…”, or “in response to determination…”, or “when detection [the stated condition or event]” or “in response to detection [the stated condition or event]”.

[0138] It should be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0139] It should also be understood that the phrases "an embodiment," "an embodiment," and "a possible implementation" used throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment or implementation is included in at least one embodiment of this application. Therefore, the phrases "in an embodiment," "an embodiment," or "a possible implementation" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0140] The above description is only an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A frequency control method characterized by, The method comprises: obtaining rendering parameters of a current frame image, the rendering parameters being rendering-related parameters generated before a rendering device renders the current frame image; predicting a load of the current frame image based on the rendering parameters of the current frame image and historical data, the historical data comprising loads and rendering parameters corresponding to a plurality of historical frame images respectively, the load being computational load information of the rendering device rendering an image; controlling the rendering device to render the current frame image at a target frequency, the target frequency matching the load of the current frame image.

2. The method of claim 1, wherein, The rendering parameters comprise at least one of inter-process communication (IPC) data transmission-related parameters or rendering tree-related parameters.

3. The method according to claim 1 or 2, characterized in that, The prediction of the load of the current frame image based on the rendering parameters of the current frame image and the historical data comprises: performing feature extraction on the rendering parameters of the current frame image and the historical data to obtain a first feature vector; determining a first load corresponding to the first feature vector based on a mapping relationship between feature vectors and loads, and taking the first load as the load of the current frame image.

4. The method of claim 3, wherein, The determination of the first load corresponding to the first feature vector based on the mapping relationship between feature vectors and loads comprises: calling a machine learning model to determine the first load corresponding to the first feature vector, the machine learning model containing the mapping relationship, and the machine learning model being obtained based on a data set, the data set comprising feature vectors and loads corresponding to a plurality of historical frame images respectively.

5. The method according to any of claims 1 to 4, characterized in that, The target frequency matching the load of the current frame image means that the rendering device completes the rendering of the current frame image at the target frequency in a time less than or equal to a maximum allowable time, the maximum allowable time being a rendering time meeting a display performance requirement.

6. The method of claim 5, wherein, The maximum allowable time is determined based on a display time interval between the current frame image and a previous frame image, or the maximum allowable time is determined based on the display time interval and a rendering delay time of a historical frame before the current frame image, or the maximum allowable time is determined based on the display time interval and a rendering remaining time of the historical frame before the current frame image.

7. The method according to any of claims 1 to 6, characterized in that, The target frequency is a frequency corresponding to power consumption meeting a power consumption requirement in optional frequencies, the power consumption corresponding to any frequency being power consumption generated by the rendering device when rendering the current frame image at the any frequency, and the optional frequencies comprising a plurality of frequencies matching the load of the current frame image.

8. A frequency control device, characterized by The device comprises: a data acquisition module configured to obtain rendering parameters of a current frame image, the rendering parameters being rendering-related parameters generated before a rendering device renders the current frame image; a load prediction module configured to predict a load of the current frame image based on the rendering parameters of the current frame image and historical data, the historical data comprising loads and rendering parameters corresponding to a plurality of historical frame images respectively, the load being computational load information of the rendering device rendering an image; a frequency control module configured to control the rendering device to render the current frame image at a target frequency, the target frequency matching the load of the current frame image.

9. The apparatus of claim 8, wherein, The rendering parameter comprises at least one of an inter-process communication (IPC) data transmission related parameter or a rendering tree related parameter.

10. The apparatus of claim 8 or 9, wherein, The load prediction module is configured to perform feature extraction on the rendering parameter of the current frame image and the historical data to obtain a first feature vector. The first load corresponding to the first feature vector is determined based on a mapping relationship between the feature vector and the load, and the first load is taken as the load of the current frame image.

11. The apparatus of claim 10, wherein, The load prediction module is configured to call a machine learning model to determine the first load corresponding to the first feature vector, the machine learning model comprises the mapping relationship, and the machine learning model is obtained based on a data set, the data set comprises feature vectors and loads corresponding to a plurality of historical frame images respectively.

12. The apparatus of any of claims 8-11, wherein, The target frequency matches the load of the current frame image, which means that the completion time of rendering the current frame image by the rendering device according to the target frequency is less than or equal to a maximum allowed time, and the maximum allowed time is a rendering time meeting a display performance requirement.

13. The apparatus of claim 12, wherein, The maximum allowed time is determined based on a display time interval between the current frame image and a previous frame image, or the maximum allowed time is determined based on the display time interval and a rendering delay time of a historical frame before the current frame image, or the maximum allowed time is determined based on the display time interval and a rendering remaining time of the historical frame before the current frame image.

14. The apparatus of any of claims 8-13, wherein, The target frequency is a frequency corresponding to power consumption meeting a power consumption requirement in optional frequencies, the power consumption corresponding to any frequency means power consumption generated by rendering the current frame image according to the any frequency, and the optional frequencies comprise a plurality of frequencies matching the load of the current frame image.

15. A visualization device, characterized by The visualization device comprises a rendering device, and the visualization device is configured to perform the frequency control method in any one of claims 1-7 to control the operating frequency of the rendering device.

16. A terminal device, comprising: The terminal device comprises a processor coupled with a memory, and the memory stores at least one program instruction or code, the at least one program instruction or code is loaded and executed by the processor to enable the terminal device to implement the frequency control method in any one of claims 1-7.

17. A chip, characterized by The chip comprises a processor configured to call and run instructions stored in a memory to enable a device installed with the chip to perform the frequency control method in any one of claims 1-7.

18. A computer program product, characterised in that, The computer program product comprises computer program code loaded and executed by a computer to enable the computer to implement the frequency control method in any one of claims 1-7.

Images