Frequency modulation method, frequency modulation system, electronic equipment and storage medium

Abstract

The invention provides a frequency modulation method, a frequency modulation system, electronic equipment and a storage medium, and the method comprises the steps: adjusting the frequency of at least one processing module in at least two different time windows in a to-be-processed image frame according to the predicted load information of the to-be-processed image frame, therefore, the processing module does not operate at a fixed working frequency in the duration of one frame, but operates at at least two working frequencies, so that fine frequency modulation control in the duration of one frame is realized, and the power consumption of the electronic equipment is reduced.

Description

Technical Field

[0001] This application relates to the field of power consumption control technology, and in particular to a frequency modulation method, frequency modulation system, electronic device and storage medium. Background Technology

[0002] In display-related scenarios, such as gaming, video display, and image presentation, users seek optimal energy efficiency at full frame rate. Currently, the main approach is to perform frequency modulation processing on the System on Chip (SOC) side.

[0003] In related technologies, frequency modulation is usually performed using a fixed frequency modulation period with a frame duration as the unit. This frequency modulation method with a frame duration as the granularity is not precise enough, which will result in insufficient performance of electronic devices or waste of power. Summary of the Invention

[0004] This application aims to at least partially address one of the technical problems in the related art.

[0005] To address this, this application proposes a frequency modulation method, a frequency modulation system, an electronic device, and a storage medium. Based on the predicted load information of the image frame, the frequency of at least one processing module is adjusted within at least two different time windows in the image frame to be processed. This allows the processing module to operate at at least two different operating frequencies instead of a fixed operating frequency within the duration of one frame, thereby achieving fine frequency modulation control within the duration of one frame and reducing the power consumption of the electronic device.

[0006] One embodiment of this application proposes a frequency modulation method, including:

[0007] Predict the load information of the image frame to be processed;

[0008] Based on the load information, the frequency of at least one processing module is adjusted within at least two different time windows within the image frame to be processed. The processing module includes at least one of CPU, GPU, NPU, and DDR.

[0009] Another embodiment of this application proposes a frequency modulation system, including: a scheduler and at least one processing module;

[0010] The scheduler is used to predict the load information of the image frame to be processed, and adjust the frequency of the at least one processing module in at least two different time windows within the image frame to be processed according to the load information; wherein the processing module includes at least one of CPU, GPU, NPU and DDR.

[0011] Another embodiment of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method described in the foregoing aspect.

[0012] Another embodiment of this application proposes a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the foregoing aspect.

[0013] Another embodiment of this application proposes a chip including a frequency modulation system configured to perform the method as described in one aspect above.

[0014] Another embodiment of this application proposes a computer program product having a computer program stored thereon, which, when executed by a processor, implements the method described in the foregoing aspect.

[0015] The frequency modulation method, frequency modulation system, electronic device, and storage medium proposed in this application adjust the frequency of at least one processing module in at least two different time windows within the image frame to be processed, based on the predicted load information of the image frame to be processed. This allows the processing module to operate at at least two different operating frequencies within the duration of one frame, instead of using a fixed operating frequency. This achieves fine frequency modulation control within the duration of one frame and reduces the power consumption of the electronic device.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0017] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0018] Figure 1 This is a schematic flowchart illustrating a frequency modulation method provided in an embodiment of this application.

[0019] Figure 2 A flowchart illustrating another frequency modulation method provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating another frequency modulation method provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the running data within a single frame provided in an embodiment of this application;

[0022] Figure 5 A statistical diagram of runtime data within a single frame is provided as an embodiment of this application;

[0023] Figure 6 One of the schematic diagrams for dividing a time window within a single frame is provided in an embodiment of this application;

[0024] Figure 7 A second schematic diagram illustrating the division of a time window within a single frame, provided as an embodiment of this application;

[0025] Figure 8 A schematic diagram illustrating the principle of a frequency modulation method provided in an embodiment of this application;

[0026] Figure 9 This is a schematic diagram of the structure of a frequency modulation system provided in an embodiment of this application;

[0027] Figure 10 This is a schematic diagram of another frequency modulation system provided in an embodiment of this application;

[0028] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0030] In related technologies, in display scenarios such as gaming, video display, and image display, users pursue optimal energy efficiency at full frame rate. Currently, the main approach is to perform frequency modulation processing on the SOC side, that is, to perform frequency modulation processing in units of one frame duration. Typically, frequency modulation processing is performed within one frame duration according to a fixed frequency modulation cycle. This frequency modulation method with one frame duration as the granularity is not precise enough, which can lead to insufficient performance or wasted power consumption in electronic devices.

[0031] To address the aforementioned issues, embodiments of this application provide a frequency modulation method, a frequency modulation system, an electronic device, and a storage medium. Based on the predicted load information of the image frame to be processed, the frequency of at least one processing module is adjusted within at least two different time windows in the image frame to be processed. This allows the processing module to operate at at least two different operating frequencies instead of a fixed operating frequency within the duration of one frame, thereby achieving fine-grained frequency modulation control within the duration of one frame and reducing the power consumption of the electronic device.

[0032] The frequency modulation method, frequency modulation system, electronic device, and storage medium of this application are described below with reference to the accompanying drawings.

[0033] Figure 1 This is a flowchart illustrating a frequency modulation method provided in an embodiment of this application.

[0034] As one implementation, the frequency modulation method of this application embodiment can be configured in a frequency modulation device, which can be applied to any electronic device so that the electronic device can perform frequency modulation function.

[0035] Among them, electronic devices can be any device with computing capabilities, such as mobile terminals, which can be hardware devices with various operating systems, touch screens and / or displays, such as mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0036] As another implementation, the frequency modulation method in this application embodiment can also be executed by a chip with processing capabilities. The chip includes a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a system on a chip (SOC), a reduced instruction set computer (RISC), etc., which will not be listed here.

[0037] like Figure 1 As shown, the method may include the following steps:

[0038] Step 1001: Predict the load information of the image frame to be processed.

[0039] The load information includes at least one of the following: the number of instructions required by the CPU to process the image frame, the cache misses caused by the CPU processing the image frame, the number of times the drawing function is called by the GPU to process the image frame, the bandwidth required by the memory to store the image frame, or the amount of computation required by the NPU to process the image frame.

[0040] The image frame to be processed is a single image frame, and the duration of a single frame is determined by the frame rate. Taking a game scene as an example, the image frames to be processed include game image frames, and the duration of a single frame is the length of a single game frame, which refers to the length of time each game frame occupies in the video playback.

[0041] In related technologies, processing modules in electronic devices typically use a fixed frequency modulation cycle of one frame within a single frame's duration to determine the operating frequency based on the load information within that frame. This frequency modulation scheme lacks sufficient precision, leading to insufficient performance or wasted resources in the electronic device. Furthermore, since the distribution of load information within a single frame is not continuous or uniform, the frequency requirement within that frame is not constant. In other words, the device should not operate at a fixed frequency within a single frame, leaving room for improvement in energy efficiency.

[0042] Step 1002: Based on the load information, adjust the frequency of at least one processing module within at least two different time windows in the image frame to be processed.

[0043] The processing module is a processor in an electronic device, such as at least one of a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), or a double data rate (DDR) memory.

[0044] In one implementation of this application, based on load information, the positions of at least two time windows within the image frame to be processed are determined, thereby dividing the image frame to be processed into at least two time windows. The sum of the durations of the at least two time windows should be greater than or equal to the frame length of the image frame to be processed. For example, if an image frame to be processed is divided into three time windows, the sum of the durations of the two time windows is less than the frame length, while the sum of the durations of the three time windows is greater than or equal to the frame length. Ideally, the sum of the durations of the three time windows should equal the frame length, but the frame length is generally 16.667ms or 8.333ms. The total length of our frequency modulation window must be greater than or equal to the frame length of one frame, and cannot be less than the frame length of one frame. This achieves accurate division of time windows within the frame, changing the granularity of frequency modulation from one frame to multiple time windows within one frame, and realizing finer-grained frequency modulation control within the duration of one frame. Furthermore, based on the positions of at least two time windows within the image frame to be processed and the load information of the image frame, the load information within at least two time windows is determined. Based on the load information within these at least two time windows, the frequency of each processing module within these at least two time windows is determined. Therefore, in this embodiment, the processing modules are frequency-adjusted so that they no longer operate at a fixed frequency within the duration of the image frame to be processed, but instead operate at corresponding frequencies within at least two time windows. This achieves at least two operating frequencies within the duration of one frame, realizing fine-tuned frequency control and thereby reducing the power consumption of the electronic device.

[0045] In the frequency modulation method of this application embodiment, based on the predicted load information of the image frame to be processed, the frequency of at least one processing module is adjusted in at least two different time windows in the image frame to be processed, so that the processing module no longer operates at a fixed operating frequency within the duration of a frame, but operates at at least two operating frequencies, thereby realizing fine frequency modulation control within the duration of a single frame and reducing the power consumption of electronic devices.

[0046] Based on the above embodiments, Figure 2 A flowchart illustrating another frequency modulation method provided in this application embodiment is shown below. Figure 2 As shown, the method includes the following steps:

[0047] Step 2001: Determine the operating data of at least one processing module within a historical image frame, wherein the historical image frame includes multiple first statistical duration ranges.

[0048] In this embodiment, the processing module acquires operating data within a time range of historical image frames preceding the image frame to be processed. This operating data includes at least one of the following: voltage, frequency, temperature, power consumption, and key event information of the processing module. The historical image frames can be one or more. As one implementation method, historical image frames, including multiple first statistical duration ranges, are divided into more granular first statistical duration ranges at the temporal level. The sum of the durations corresponding to the multiple first statistical duration ranges is greater than or equal to the frame length of a single frame. This allows for the statistical analysis of running data within a single frame, resulting in finer-grained running data. Based on the running data of the processing modules within the multiple first statistical duration ranges, the duration range of the image frame to be processed is divided into at least two time windows. The length of each time window is not fixed but determined based on the running data of a single frame, improving the accuracy of determining at least two time windows. This allows for the determination of the operating frequencies of the processing modules within each of the at least two time windows, increasing the granularity of frequency modulation. Consequently, the operating frequencies of the processing modules within a single frame are no longer fixed but multiple, improving the accuracy of frequency modulation.

[0049] Step 2002: Based on the running data of at least one processing module within a plurality of first statistical time ranges, predict the load information of the image frame to be processed.

[0050] In this embodiment, based on the operating data of at least one processing module within multiple first statistical time ranges, the operating data of at least one processing module within multiple second statistical time ranges in the current image frame to be processed is determined, thereby enabling the prediction of the operating data of the current frame based on historical operating data. The operating data indicates the load information of the processing module's execution load. Voltage, frequency, temperature, and power consumption information are positively correlated with the number of instructions executed by the processing module, while key event information indicates the number and type of instructions required for the event. Different key events require different types and numbers of instructions. A higher number of instructions indicates a greater load. Therefore, by predicting the load information of the image frame to be processed based on the operating data of at least one processing module within multiple second statistical time ranges in the image frame, the accuracy of load information prediction is improved.

[0051] Key events include: the number of task execution instructions, the number of instructions blocked in the front-end during instruction execution, the number of three-level data caches, the number of accesses to the system bus, memory (Double Data Rate, DDR) bandwidth, etc.

[0052] Step 2003: Based on the load information, adjust the frequency of at least one processing module within at least two different time windows in the image frame to be processed.

[0053] The explanations and descriptions in the foregoing embodiments also apply to step 2003, and the principle is the same, so they will not be repeated here.

[0054] It should be noted that the sum of the working time required for each processing module to operate at its corresponding working frequency within each time window is less than or equal to the frame length of the image frame to be processed, and / or the sum of the power consumption of each processing module operating at its corresponding working frequency within each time window is minimized, so as to achieve the accuracy of the working frequency of the processing module in each time window and improve the energy efficiency and performance of the electronic device.

[0055] In the frequency modulation method of this application embodiment, based on the predicted load information of the image frame to be processed, the frequency of at least one processing module is adjusted in at least two different time windows in the image frame to be processed, so that the processing module no longer operates at a fixed operating frequency within the duration of a frame, but operates at at least two operating frequencies, thereby achieving the precision of power consumption control of the electronic device within a single frame duration, reducing the power consumption of the electronic device, and improving the performance of the electronic device.

[0056] Based on the above embodiments, Figure 3 A flowchart illustrating another frequency modulation method provided in this application embodiment is shown below. Figure 3 As shown, the method includes the following steps:

[0057] Step 3001: For any processing module, data is collected based on the sampling frequency and clock synchronization signal of the processing module to obtain the running data of the processing module at the historical collection time.

[0058] In one implementation of this application, when there is at least one processing module, for each processing module, the start time for sampling the processing module is determined according to the clock synchronization signal. Based on the start time, data is collected according to the sampling frequency of the processing module to obtain the running data of the processing module at each historical collection time.

[0059] In the case of multiple processing modules, clock synchronization signals can be sent between the multiple processing modules. As one implementation method, any one of the multiple processing modules can be designated as the master processing module, and the other processing modules as slave processing modules. The master processing module synchronizes the clock synchronization signal to each slave processing module, thereby synchronizing the clocks of each processing module and ensuring that the start time for data acquisition of each processing module is the same, which facilitates subsequent data aggregation.

[0060] Step 3002: For any first statistical duration range, determine the running data of any processing module within any first statistical duration range based on the running data of any processing module at any historical collection time within any first statistical duration range.

[0061] The duration corresponding to the first statistical duration range is determined based on the frame length of the image frame. For example, if the frame duration is 10 seconds, then the duration corresponding to the first statistical duration range is set to 1 second, that is, the statistical duration range of a frame is divided into 10 first statistical duration ranges to achieve fine-grained data statistics.

[0062] In this embodiment, when there is at least one processing module, the acquisition frequencies of each processing module may differ. To align the data acquired by each processing module, one approach is to divide the duration of a frame into finer time granularities, namely multiple first statistical duration ranges. For each statistical duration range, the running data acquired by each processing module at each moment is summarized to obtain the running data within each first statistical duration range. Therefore, by synchronizing the clock signals of each processing module, the running data of each processing module within the duration range of historical image frames can be summarized, which also facilitates the subsequent summarization and alignment of the running data of multiple processing modules.

[0063] Step 3003: Based on the running data of any processing module within any first statistical time range, determine the running data of any processing module within the corresponding second statistical time range of the image frame to be processed.

[0064] In this embodiment, the operating data of any processing module within a second statistical time range corresponding to the image frame to be processed is determined based on the operating data of any processing module within a first statistical time range. For distinction, the statistical time ranges included in historical image frames are referred to as the first statistical time ranges, and the statistical time ranges included in the current image frame to be processed are referred to as the second statistical time ranges. The number of first and second statistical time ranges is usually the same, and each first statistical time range has a corresponding second statistical time range, enabling fine-grained data statistics during the data statistics phase. Thus, for each first statistical time range, the operating data of the processing modules within the first statistical time range is determined based on the operating data of the processing modules within that range, and the operating data of the processing modules within the corresponding second statistical time range of the image frame to be processed. This achieves the determination of the operating data of the image frame to be processed based on the operating data of historical image frames. Furthermore, utilizing the operating data of historical image frames allows for the capture of dynamic changes in the operating state of the processing modules, thereby more accurately predicting the frequency requirements of the processing modules within the image frame to be processed. This helps to implement a more refined frequency modulation strategy and improve the performance and energy efficiency of the processing modules.

[0065] The following explanation addresses different scenarios based on the varying number of historical image frames:

[0066] In one scenario, a historical image frame is considered as one frame. For each first statistical duration range in the historical image frame, as an implementation method, the running data of the processing module within the first statistical duration range is used as the running data of the processing module within the second statistical duration range in the image frame to be processed.

[0067] As an example, Figure 4 This is a schematic diagram of the running data within a single frame provided in the embodiments of this application, such as... Figure 4 As shown, the historical image frames are the two image frames preceding the image frame to be processed. These two historical image frames can be either two frames that are temporally adjacent or two frames that are not temporally adjacent; this embodiment does not impose any limitations. If the historical image frames are... Figure 4 The letter A indicates a historical image frame. Taking the first statistical duration range 5 in the historical image frame as an example, the first statistical duration range 5 and the second statistical duration range 1 have a corresponding relationship. The running data obtained in the first statistical duration range 5 can be used as the running data in the second statistical duration range 1, thereby realizing the prediction of the running data in the current image frame to be processed based on the data of the historical image frame.

[0068] In another scenario, the historical image frames are multiple frames. One approach is to fuse the running data of processing modules within the same first statistical time range across multiple historical image frames, and use the fused result as the running data of the processing modules within the corresponding second statistical time range within the time range of the image frame to be processed. Another approach is to predict the running data of the processing modules within the corresponding second statistical time range within the time range of the image frame to be processed based on the running data of the processing modules within the same first statistical time range across multiple historical image frames. For example,... Figure 4 As shown, when determining the running data in the second statistical time range 1, the running data in the second statistical time range 1 can be determined based on the running data in the first statistical time range 1 and the first statistical time range 2. This realizes the change of the running data of the processing module in each first statistical time range in multiple historical image frames, accurately predicts the running data of the corresponding processing module in each second statistical time range in the image frame to be processed, and improves the accuracy of determining the running data in the image frame to be processed.

[0069] In one implementation of this application, the running data of each processing module within the duration of historical image frames is read from a set storage unit, such as system memory. For each processing module, the running data of the processing module within the duration of historical image frames is predicted and determined based on the running data of the processing module within the duration of historical image frames. As one implementation, the running data of the processing module within the duration of historical image frames can be input into a pre-trained prediction model to predict the running data of the processing module within the duration of the image frames to be processed. The foregoing explanation also applies to this embodiment and will not be repeated here.

[0070] Step 3004: Summarize the running data of at least one processing module within the second statistical time period to obtain the total running data within the second statistical time period.

[0071] In one implementation of this application, when there is only one processing module, the total running data within each second statistical time period is the running data of that processor within each second statistical time period.

[0072] In another implementation of this application, when there are multiple processing modules, for each second statistical duration range within the duration range of the image frame to be processed, it is necessary to consider the running data of multiple processing modules, that is, to summarize the running data of multiple processing modules within the second statistical duration range, that is, to place the running data of multiple processing modules within the second statistical duration range in the same data packet as the total running data of multiple processing modules within the second statistical duration range, thereby realizing the summarization of the running data of at least one processing module within each second statistical duration range.

[0073] As an example, Figure 5 This application provides a statistical diagram of runtime data within a single frame, as shown in the embodiments of this application. Figure 5 As shown, processing modules are typically the various processors in electronic devices, such as CPU, NPU, GPU, etc. Figure 5 Taking the processing module as a processor as an example, the number of processors is n, where n, indicating the number of processors, is a natural number greater than or equal to 1. In the case of multiple processors... Figure 5 The single-frame duration range in the text refers to the duration range of the image frames to be processed, that is, the duration range from time T to time T+1. This duration range from time T to time T+1 includes multiple second statistical duration ranges, for example... Figure 5 The time interval from time t to time t+1, or from time t+1 to time t+2, is not listed here. Taking the second statistical time range B1 corresponding to time t to time t+1 as an example, the total running data corresponding to the second statistical time range B1 can be stored in the form of data packets, i.e. Figure 5 Data packet 701 in the data packet includes the operating data of processors 1 to N within the second statistical time range B1. Specifically, 703-1 is the operating data of processor 1 within the second statistical time range B1, 703-2 is the operating data of processor 2 within the second statistical time range B1, and so on up to 703-n is the operating data of processor n within the second statistical time range B1. The operating data of each processor includes the processor's voltage, frequency, power consumption, temperature, and n key events. The n indicating key events is a natural number including 0, which means that key events may or may not be included.

[0074] It should be noted that when performing statistical analysis on historical image frames, each initial statistical duration range can be identified using a timestamp, for example, Figure 5 Each timestamp corresponding to 702 in the middle corresponds to, for example, the start time of a first statistical duration range.

[0075] Step 3005: Based on the total running data within multiple second statistical time ranges, predict the load information of the image frames to be processed.

[0076] In one implementation of this application, the total running data for each time window is determined for each time window's corresponding duration range. This is because the data statistics in this application are no longer based on the frame length of a single frame, but rather on multiple second statistical durations, achieving a finer data granularity. As one implementation, for each second statistical duration range, the load information within that range is predicted based on the total running data, thus obtaining the predicted load information for that range. This improves the accuracy of the load information corresponding to the second statistical duration range. By predicting the load within the second statistical duration range, compared to predicting the load within a single frame's duration range, the granularity of load information prediction is improved, and accuracy is enhanced. The prediction method can use common time series prediction algorithms, such as mean prediction, exponentially weighted moving average (EWMA) prediction, or more complex neural network prediction algorithms, etc., which are not limited in this embodiment.

[0077] In one implementation of this application, the total running data for each time window is determined for the duration range corresponding to each time window. This is because the data statistics in this application are no longer based on the frame length of a single frame, but on multiple second statistical durations, achieving a finer data granularity. Thus, the total running data for each time window is determined based on the position of each divided time window. As one implementation, the position of the time window in the frame of the image to be processed indicates at least one second duration range corresponding to the time window in the total duration range of the image frame. Therefore, for each time window, the at least one second statistical duration range included in the time window is determined according to the position of the time window in the frame of the image, and the total running data in the time window is determined according to the total running data within the at least one second statistical duration range. Furthermore, to improve the accuracy of load information within the time window, as one implementation method, for each time window, the load information within that time window is predicted based on the total running data in that time window, thereby obtaining the predicted load information for that time window. This improves the accuracy of the load information corresponding to that time window. By predicting the load within the time window, compared to predicting the load over a single frame's duration, the granularity of load information prediction is improved, thus increasing accuracy. The prediction method can use common time series prediction algorithms, such as mean prediction, exponentially weighted moving average (EWMA) prediction, or more complex neural network prediction algorithms, etc. This embodiment does not impose any limitations.

[0078] Step 3006: Based on the load information of the image frame to be processed, determine the positions of at least two time windows in the image frame to be processed.

[0079] In this embodiment, the load of the image frame to be processed in the display scene is regular and locally continuous. It is not necessary to perform frequency modulation every 1ms in a frame. Therefore, the number of times frequency modulation is needed within the duration of an image frame, and the duration of each modulation, are determined by dividing the duration of the image frame into frequency modulation time windows to improve the accuracy of frequency modulation. As one implementation method, the duration of the image frame to be processed can be divided into at least two time windows based on the load information of multiple second statistical duration ranges within the duration of the image frame to be processed. That is, the positions of at least two time windows in the image frame to be processed are determined. These positions indicate the start and end points of each time window. The load information includes the characteristics of the load distribution within the corresponding duration. The load distribution characteristics are related to the division of the frequency modulation granularity.

[0080] In one implementation of this application, the load distribution feature indicates the continuity of the load distribution. The load distribution feature can be indicated based on the load amount within each second statistical time range. Based on the load amount within multiple second statistical time ranges, the time range of the image frame to be processed is divided into at least two time windows. The load amount within multiple second statistical time ranges can show the distribution of the load amount, i.e., the statistical data of the load changing over time.

[0081] As an example, Figure 6 This is one of the schematic diagrams illustrating the division of a time window within a single frame, as provided in the embodiments of this application. Figure 6 As shown in the figure, the time occupied by each rectangle on the horizontal axis represents the duration corresponding to a range of second statistical durations, and the height of the rectangle on the vertical axis represents the load within a range of second statistical durations. Figure 6 As can be seen, different ranges of the second statistical duration correspond to different load amounts. If only the second statistical duration with a load amount greater than the set load amount is considered, then based on the continuity of the load amount, the duration range of the image frames to be processed is divided into two time windows, i.e. Figure 6 Window 1 and Window 2 in the code accurately determine the time window range of the image frame to be processed. In other words, within the time range of the image frame to be processed, the frequency of each processing module is adjusted within the time range of Window 1 and Window 2 to improve the accuracy of frequency modulation.

[0082] In another implementation of this application, the load distribution characteristics can be indicated based on the load type within each second statistical duration range. As one implementation, the load distribution characteristics can be characterized based on the Instructions Per Cycle (IPC) value. The load type is determined based on the IPC value, where different IPC values ​​correspond to different load types. Thus, the load type can be determined based on the IPC value. Furthermore, based on the load types within multiple second statistical duration ranges, the duration range of the image frame to be processed is divided into at least two time windows.

[0083] As an example, Figure 7 This is a second schematic diagram illustrating the division of a time window within a single frame, as provided in an embodiment of this application. Figure 7 As shown in the figure, the time occupied by each rectangle on the horizontal axis represents the duration corresponding to a range of second statistical durations, and the height of the rectangle on the vertical axis represents the IPC value within a range of second statistical durations. Figure 7 As can be seen, different IPC values ​​correspond to different second statistical duration ranges. The IPC values ​​of the first and second second statistical duration ranges on the time axis are relatively close, indicating they correspond to the same type of load. The IPC values ​​of the subsequent second statistical duration ranges are also relatively close. Therefore, these subsequent second statistical duration ranges are divided into the same time window. Thus, based on the difference in IPC values ​​within the duration range of the image frame to be processed, the duration range of the image frame to be processed is divided into two time windows. Figure 7 The window shown is Window 1 and Window 2.

[0084] It should be noted that time windows can also be divided based on both load volume and load type, or based on other characteristics of the load, or a combination of several characteristics. This embodiment does not impose any limitations on this.

[0085] The second statistical duration range to be counted can be the entire second statistical duration range among multiple second statistical duration ranges in the image frame to be processed, or a portion of the second statistical duration range among multiple second statistical duration ranges. This embodiment does not impose any limitation on this.

[0086] Step 3007: Determine the frequency of at least one processing module within each time window based on the position and load information of at least two time windows in the image frame.

[0087] In this embodiment, based on the positions of at least two time windows within the image frame to be processed and the image load information, where the load information of the image frame to be processed includes load information within each second statistical duration range, the load information of each time window can be determined according to the positions of the time windows within the image frame to be processed and the positions of each second statistical duration range within the image. The load information of at least two time windows is input into the energy consumption model to obtain the operating frequency corresponding to each processing module within each time window. The trained energy consumption model has learned the correspondence between the load information within at least two time windows and the operating frequencies corresponding to each processing module. To improve the accuracy of the operating frequencies corresponding to each processing module, in the case of at least two processing modules, as one implementation method, the sum of the operating times required for each processing module to run based on its corresponding operating frequency is less than or equal to the frame length of the image frame to be processed.

[0088] As a second implementation method, the sum of the power consumption of each processing module based on its corresponding operating frequency is minimized.

[0089] As a third implementation method, when the sum of the working time required for each processing module to operate based on its corresponding working frequency is less than or equal to the frame length of the image frame to be processed, and the sum of the power consumption corresponding to each processing module operating based on its corresponding working frequency is minimized, the target working frequency adopted is used as the working frequency corresponding to each processing module in each time window, thereby improving the accuracy of the working frequency corresponding to each processing module.

[0090] The operating time and power consumption are derived from the calculation of the energy efficiency model. There are many methods for energy efficiency modeling, and no restrictions are imposed here. The problem calculated using the energy efficiency model is a combinatorial optimization problem, which can be solved using exhaustive search or branch and bound methods to find the global optimum. The energy consumption model includes, for example, linear models based on key events, multinomial models, or nonlinear models based on neural networks. This embodiment does not limit the energy efficiency model.

[0091] For example, the frequency point combinations corresponding to the operating frequencies of each processing module in each time window are ((Fcpu_1, Fddr_1, Fgpu_1), (Fcpu_2, Fddr_2, Fgpu_2), ...). Taking (Fcpu_1, Fddr_1, Fgpu_1) as an example, (Fcpu_1, Fddr_1, Fgpu_1) includes the operating frequency Fcpu_1 of the CPU of the processing module, the operating frequency Fddr_1 of the DDR of the processing module, and the operating frequency Fgpu_1 of the GPU of the processing module in the first time window.

[0092] As an example, for compute-intensive workloads, it is suitable to set the CPU to a higher frequency and the DDR to a lower frequency, while for memory-intensive workloads, it is suitable to set the CPU to a lower frequency and the DDR to a higher frequency.

[0093] As another implementation, the frame length of the image frame to be processed is divided into several parts according to the number of time windows. The duration of each part is used as the performance index of the time window. Under the premise that the time required for each processing module to run does not exceed its respective performance index, the target frequency corresponding to the minimum power consumption of each processing module is found. There are also many ways to divide the frame length. One is based on the length of the time window, another is based on the proportion of the load in the time window, etc., which will not be elaborated in this embodiment.

[0094] Furthermore, by adopting a determined target frequency, the operating frequency of the processing module in each time window is adjusted so that the processing module operates at the corresponding operating frequency in each time window, thereby achieving precise frequency tuning.

[0095] It should be noted that once the operating frequency of each processing module is determined, the corresponding operating voltage of each processing module can be determined, so that each processing module can operate at the determined operating frequency and operating voltage.

[0096] In the frequency modulation method of this application embodiment, based on the operating data of the processing module, the duration range of the image frame to be processed is divided into at least two time windows. According to the load information of the processing module within the at least two time windows, the operating frequency of the processing module in each time window is determined. By dividing the duration range of a single frame into at least two time windows, the time granularity is refined. Then, the operating frequency of the processing module in each time window is determined, thereby improving the accuracy of power consumption control of the electronic device within the duration range of a single frame and improving the performance of the electronic device.

[0097] Based on the foregoing embodiments, Figure 8 This is a schematic diagram illustrating the principle of a frequency modulation method provided in an embodiment of this application, as shown below. Figure 8As shown, the processing module includes at least one, including at least one of CPU, GPU, NPU and DDR, which are not listed here. The embodiments of this application are implemented by both software algorithm and hardware. The hardware part is used to synchronize the clock signal of at least one processing module, collect the running data of at least one processing module, and summarize the collected running data of each processing module according to a set statistical duration and store it in a set storage unit. The scheduler executing the software algorithm reads the running data of each processing module stored in the set storage unit, thereby dividing the frame time window based on the running data of at least one processing module within the duration of the image frame to be processed. That is, the duration of the image frame to be processed is divided into at least two time windows, and the working frequency of each processing module in each time window is determined. Furthermore, the operating frequency corresponding to each processing module in each time window is sent to the management subsystem corresponding to each processing module in the hardware section. This allows the management subsystem to control each processing module to operate at the target frequency in each time window based on the operating frequency of each processing module in each time window, thereby achieving power consumption control. The operating frequency and operating voltage have a mapping relationship, thus enabling the management subsystem to control each processing module to operate at a determined operating frequency and at the corresponding operating voltage in each time window.

[0098] It should be noted that by adjusting the operating frequency of the processing modules in each time window, at least two operating frequencies are used within the corresponding duration of the image frame. The startup information of the next frame of the image frame to be processed is monitored, and the load execution status of any processing module is monitored. In response to receiving the startup information of the next frame of the image frame to be processed, the actual execution duration of the image frame to be processed is compared with the frame length. If the actual execution duration is longer than the frame length and the load of the processing module has not been completed, the operating frequency of the processing module within the time range exceeding the frame length is adjusted to execute the unfinished load. This situation usually occurs when the load of a certain processing module may suddenly increase during actual operation, causing the load of that processing module to not be completed when the duration of the image frame to be processed arrives. That is, the processing module still has critical events to execute. In this case, the operating frequency of the processing module within the time range exceeding the frame length is increased to achieve rapid execution of the unfinished load. Since the hardware can respond quickly, the occurrence of stuttering is reduced, and the real-world effect is improved.

[0099] In the frequency modulation method of this application embodiment, based on system-level fine-grained energy efficiency data, at least two time windows requiring frequency modulation within a frame are dynamically determined. That is, several time windows are determined from a frame according to the characteristics of load distribution, and the combination of the working frequencies of each processing module with the best energy efficiency is selected according to the load requirements of the processing modules within at least two time windows. This breaks the limitation of frame granularity, realizes finer-grained frequency modulation, and improves energy efficiency performance.

[0100] Based on the above embodiments, Figure 9 This is a schematic diagram of the structure of a frequency modulation system provided in an embodiment of this application, as shown below. Figure 9 As shown, the system includes a scheduler 500 and at least one processing module 200.

[0101] Scheduler 500 is used to predict the load information of the image frame to be processed, and adjust the frequency of at least one processing module 200 in at least two different time windows within the image frame to be processed according to the load information; wherein the processing module includes at least one of CPU, GPU, NPU and DDR.

[0102] As one implementation, the scheduler 500 is located in the processor that executes the algorithm, such as a central processing unit.

[0103] Specifically, please refer to the relevant explanations in the foregoing method embodiments, as the principles are the same and will not be repeated here.

[0104] This application proposes a frequency modulation system in which a scheduler adjusts the frequency of at least one processing module within at least two different time windows of the image frame to be processed based on the predicted load information of the image frame to be processed. This allows the processing module to operate at at least two different operating frequencies within the duration of one frame, instead of a fixed operating frequency. This achieves fine frequency modulation control within the duration of one frame and reduces the power consumption of the electronic device.

[0105] Based on the above embodiments, as one implementation method, the image frame to be processed includes a game image frame.

[0106] As one implementation, the scheduler 500 is also used for:

[0107] Based on the load information, the positions of the at least two time windows in the image frame to be processed are determined;

[0108] The frequency of the at least one processing module within each time window is determined based on the position of the at least two time windows in the image frame to be processed and the load information.

[0109] As one implementation, the scheduler 500 is also used for:

[0110] Determine the operating data of at least one processing module within a historical image frame; wherein, the historical image frame includes multiple first statistical duration ranges; predict the load information of the image frame to be processed based on the operating data of at least one processing module within the multiple first statistical duration ranges.

[0111] The explanations and descriptions in the foregoing embodiments also apply to this embodiment, and the principles are the same, so they will not be repeated here.

[0112] Based on the above embodiments, this application also provides another frequency modulation system. Figure 10 This is a schematic diagram of another frequency modulation system provided in an embodiment of this application, as shown below. Figure 10 As shown, the system also includes a management subsystem 102, which includes a data acquisition unit 301 and a data processor 302.

[0113] The data acquisition unit 301 is used to acquire data from any processing module 200 according to the sampling frequency and clock synchronization signal of the processing module 200, and obtain the operating data of the processing module 200 at the historical acquisition time.

[0114] As one implementation, the processing module 200 includes a temperature sensor 201, a power consumption detector 202, and a critical event detector 203. The data acquisition unit 301 samples temperature information of various areas of the processing module 200 at various historical acquisition times from the temperature sensor 201, samples power consumption information of various areas of the processing module 200 at various historical acquisition times from the power consumption detector 202, and samples critical event information of the operation of the processing module 200 at various historical acquisition times from the critical event detector 203.

[0115] The data processor 302 is used to determine the operating data of any processing module 200 within any first statistical time range based on the operating data of any processing module 200 at historical collection times within the first statistical time range.

[0116] As one implementation method, a frequency modulation system includes multiple management subsystems, such as Figure 10 The management subsystems 102, 103 to n are management subsystems, where n is a natural number. For example, when n is 105, it means that there are 4 management subsystems. Each management subsystem includes a processing module to be frequency regulated, a data acquisition unit and a data processor. The functions implemented by each management subsystem are similar to those of the aforementioned management subsystem 102, and are not limited in this embodiment.

[0117] As one implementation method, clock signal synchronization is also performed between the various management subsystems. As another implementation method, any management subsystem is used to synchronize the clock synchronization signal to other management subsystems among the multiple management subsystems. The synchronization method can be referred to the explanation of the clock synchronization signal in the foregoing embodiments, and the principle is the same, so it will not be repeated here.

[0118] Based on the above embodiments, as one implementation, there are multiple management subsystems, and the scheduler 500 is further used for:

[0119] For any processing module, data is collected based on the sampling frequency and clock synchronization signal of the processing module to obtain the running data of the processing module at the historical collection time.

[0120] For any given first statistical duration range, the running data of any processing module within the given first statistical duration range is determined based on the running data of any processing module at the historical collection time within the given first statistical duration range.

[0121] As one implementation, the scheduler 500 is also used for:

[0122] Based on the running data of any processing module within any first statistical time range, determine the running data of any processing module within the corresponding second statistical time range in the image frame;

[0123] The running data of at least one processing module within the second statistical time period are summarized to obtain the total running data within the second statistical time period.

[0124] Based on the total running data within the multiple second statistical time ranges, the load information of the image frame to be processed is predicted.

[0125] As one implementation, the scheduler 500 is also used for:

[0126] Based on the load distribution characteristics in the load information of the image frame to be processed, the positions of the at least two time windows in the image frame to be processed are determined.

[0127] As one implementation, the scheduler 500 is also used for:

[0128] The positions of the at least two time windows within the image frame to be processed and the load information are input into the energy consumption model to obtain the operating frequency of each processing module in each time window.

[0129] In one implementation, when there are at least two processing modules, the sum of the working time required for each processing module to operate based on its corresponding operating frequency is less than or equal to the frame length of the image frame to be processed, and / or the sum of the power consumption of each processing module operating based on its corresponding operating frequency is minimized.

[0130] As one implementation method, the management subsystem 102 also includes: a decision-maker 303;

[0131] The decision-maker 303, in response to receiving the start information of the next frame of the image frame to be processed, compares the actual execution time of the image frame to be processed with the frame length. In response to the actual execution time being longer than the frame length and the load of the processing module 200 not being completed, the decision-maker adjusts the operating frequency of the processing module 200 within the time range exceeding the frame length to execute the unfinished load. Since the hardware can respond quickly, the occurrence of lag is reduced and the real-world effect is improved.

[0132] The decision-makers included in other management subsystems have the same functions and principles as decision-maker 303, and will not be described in detail here.

[0133] It should be noted that the explanations and effects in the foregoing embodiments also apply to this embodiment, and the principle is the same, so they will not be repeated here.

[0134] In one implementation of this application, for each management subsystem, the operating frequency of each processing module corresponding to each time window sent by the scheduler 500 is obtained, and the operating frequency of each processing module in each time window is sent to the frequency-voltage V / F controller 401. The V / F controller 401 controls the clock generator 402 to adjust the frequency of each processing module. The frequency and voltage of the same processing module have a corresponding relationship, so the corresponding voltage can be determined according to the frequency of each processing module, and the power manager 403 is controlled to adjust the voltage of each processing module according to the determined voltage.

[0135] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing method embodiments.

[0136] To implement the above embodiments, this application also proposes a computer program product having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the foregoing method embodiments.

[0137] To implement the above embodiments, this application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method described in the foregoing method embodiments.

[0138] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, in-vehicle device, fitness equipment, personal digital assistant, etc.

[0139] Reference Figure 11 The electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816.

[0140] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0141] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0142] Power component 806 provides power to various components of electronic device 800. Power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0143] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0144] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0145] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0146] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0147] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0148] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0149] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0150] To achieve the above embodiments, this application also proposes a chip, including: the chip includes a frequency modulation system.

[0151] As one implementation, the chip including the frequency modulation system provided in this application is configured to perform any of the frequency modulation methods described above.

[0152] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0153] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0154] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0155] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0156] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0157] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0158] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0159] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A frequency modulation method, characterized in that, include: Predict the load information of the image frame to be processed; Based on the load information, the frequency of at least one processing module is adjusted within at least two different time windows within the image frame to be processed; wherein the processing module includes at least one of CPU, GPU, NPU and DDR.

2. The method according to claim 1, characterized in that, The image frames to be processed include game image frames.

3. The method according to claim 1, characterized in that, The step of adjusting the frequency of at least one processing module within at least two different time windows in the image frame to be processed, based on the load information, includes: Based on the load information, the positions of the at least two time windows in the image frame to be processed are determined; The frequency of the at least one processing module within each time window is determined based on the position of the at least two time windows in the image frame to be processed and the load information.

4. The method according to claim 1, characterized in that, The load information of the predicted image frame to be processed includes: Determine the operating data of at least one processing module within a historical image frame; wherein, the historical image frame includes multiple first statistical duration ranges; Based on the operating data of at least one processing module within the plurality of first statistical time ranges, the load information of the image frame to be processed is predicted.

5. The method according to claim 4, characterized in that, The determination of the operational data of at least one processing module within a historical image frame includes: For any processing module, data is collected based on the sampling frequency and clock synchronization signal of the processing module to obtain the running data of the processing module at the historical collection time. For any given first statistical duration range, the running data of any processing module within the given first statistical duration range is determined based on the running data of any processing module at the historical collection time within the given first statistical duration range.

6. The method according to claim 4, characterized in that, The step of predicting the load information of the image frame to be processed based on the operating data of at least one processing module within a plurality of first statistical time ranges includes: Based on the running data of any processing module within any first statistical time range, determine the running data of any processing module within the corresponding second statistical time range of the image frame to be processed; The running data of at least one processing module within the second statistical time period are summarized to obtain the total running data within the second statistical time period. Based on the total running data within the multiple second statistical time ranges, the load information of the image frame to be processed is predicted.

7. The method according to claim 3, characterized in that, Determining the positions of the at least two time windows within the image frame to be processed based on the load information includes: Based on the load distribution characteristics in the load information of the image frame to be processed, the positions of the at least two time windows in the image frame to be processed are determined.

8. The method according to claim 3, characterized in that, The step of determining the frequency of at least one processing module within each of the two time windows based on the position of the at least two time windows in the image frame to be processed and the load information includes: The positions of the at least two time windows within the image frame to be processed and the load information are input into the energy consumption model to obtain the operating frequency of each processing module in each time window.

9. The method according to claim 8, characterized in that, In at least two cases where the processing module includes, the sum of the working time required for each processing module to operate based on its corresponding operating frequency is less than or equal to the frame length of the image frame to be processed, and / or, the sum of the power consumption of each processing module operating based on its corresponding operating frequency is minimized.

10. A frequency modulation system, characterized in that, Includes a scheduler and at least one processing module; The scheduler is used to predict the load information of the image frame to be processed, and adjust the frequency of the at least one processing module in at least two different time windows within the image frame to be processed according to the load information; wherein the processing module includes at least one of CPU, GPU, NPU and DDR.

11. The system according to claim 10, characterized in that, The image frames to be processed include game image frames.

12. The system according to claim 10, characterized in that, The scheduler is also used for: Based on the load information, the positions of the at least two time windows in the image frame to be processed are determined; The frequency of the at least one processing module within each time window is determined based on the position of the at least two time windows in the image frame to be processed and the load information.

13. The system according to claim 12, characterized in that, The scheduler is also used for: Determine the operating data of at least one processing module within a historical image frame; wherein, the historical image frame includes multiple first statistical duration ranges; Based on the operating data of at least one processing module within the plurality of first statistical time ranges, the load information of the image frame to be processed is predicted.

14. The system according to claim 13, characterized in that, The system also includes a management subsystem, which includes a data collector and a data processor. The data acquisition device is used to acquire data for any processing module based on the sampling frequency and clock synchronization signal of the processing module, and obtain the operating data of the processing module at the historical acquisition time. The data processor is configured to determine the operating data of any processing module within any first statistical time range based on the operating data of any processing module at a historical acquisition time within the first statistical time range.

15. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method as described in any one of claims 1-9.

16. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

17. A chip, characterized in that, The chip includes a frequency modulation system configured to perform the method as described in any one of claims 1-9.

18. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, implements the method of any one of claims 1-9.

Images