Model training method and device, frequency adjustment method and device, equipment and storage medium

Abstract

The invention provides a model training method, a frequency adjusting method, a model training device, a frequency adjusting device, electronic equipment and a computer readable storage medium, and relates to the technical field of terminals.The model training method comprises the steps that a target data set is obtained from a training data playback pool, the training data playback pool comprises a plurality of data sets, the data set comprises the operation state and the instant reward of the equipment at the current moment, the working frequency required by the equipment for operation at the next moment and the operation state of the equipment at the next moment, and the reward is determined according to the frame rate and / or the power consumption of the equipment; inputting the target data set into a deep neural network model, and performing action evaluation on the model according to the training data set to obtain a current action value and a target action value of the model; and training the model based on difference information between the target action value and the current action value. According to the invention, the problems of performance reduction of the mobile equipment and power consumption waste caused by the existing frequency modulation strategy can be solved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and more specifically, to a model training method, a frequency adjustment method, a model training device, a frequency adjustment device, an electronic device and a computer-readable storage medium in the field of terminal technology. Background Art

[0002] In the Android (Chinese name: Android) system, frequency modulation mechanisms and strategies are used to manage the frequency and voltage of the processor to balance system performance and battery life. Although frequency modulation technology has been widely studied, its current implementation still has certain limitations. The frequency modulation strategy used in existing mobile devices is only based on the utilization of tasks, and does not consider the frame rate of upper-level applications. Faced with threads with low utilization but related to frame generation and threads with high utilization but unrelated to frame generation, they are often adjusted to the wrong frequency, resulting in performance degradation and power consumption waste. Therefore, how to solve the problem of performance degradation and power consumption waste of mobile devices caused by existing frequency modulation strategies has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides a model training method, a frequency adjustment method, a model training device, a frequency adjustment device, an electronic device and a computer-readable storage medium, which can solve the problems of avoiding performance degradation of mobile devices and causing power consumption waste caused by existing mobile device frequency modulation strategies, and can not only reduce the power consumption of mobile devices, but also avoid performance degradation of mobile devices.

[0004] In a first aspect, a model training method is provided, the model training method comprising: obtaining a target data group from a training data playback pool; wherein the training data playback pool comprises a plurality of data groups, the data group comprising a first sample operating state and a sample instant reward of a sample device at a current moment, a sample operating frequency of the sample device at a next moment, and a second sample operating state of the sample device at the next moment, the sample instant reward being determined according to a frame rate and / or power consumption of the sample device; inputting the target data group into a deep neural network model to perform action evaluation on the deep neural network model according to the training data group to obtain a current action value and a target action value of the deep neural network model; and training the deep neural network model based on the difference information between the target action value and the current action value.

[0005] In combination with the first aspect, in some possible implementations, the model training method also includes: updating the training data replay pool; the updating of the training data replay pool includes: obtaining a first operating state of the sample device at the current moment; determining the next moment operating frequency of the sample device at the next moment based on the first operating state; determining the current instant reward of the sample device at the current moment based on a reward function regarding the frame rate and / or power consumption of the sample device; controlling the sample device to operate at the next moment operating frequency to obtain a second operating state of the sample device at the next moment; adding the first operating state, the next moment operating frequency, the current instant reward and the second operating state to the training data replay pool to update the training data replay pool.

[0006] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, obtaining the first operating state of the sample device at the current moment includes: obtaining the current operating frequency, power consumption, current frame rate, central processing unit utilization, number of cache misses and number of I / O page errors of the sample device at the current moment to obtain the first operating state.

[0007] In combination with the first aspect and the above implementation, in some possible implementations, the formula of the reward function is as follows:

[0008] r(t)=G(t)-P(t)

[0009]

[0010] P(t)=β×p(t)

[0011] Wherein, r(t) is the reward function, p(t) is the power consumption, q(t) is the current frame rate, L is the target frame rate provided by the sample device, H is a preset frame rate that is tolerable to the user and provides the minimum user experience provided by the sample device, α is a weight for balancing the performance of the sample device, β is a weight for weighing the power consumption, and c is a constant.

[0012] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the deep neural network model includes a first sub-model for action selection and a second sub-model for action evaluation; the target data group is input into the deep neural network model to perform action evaluation on the deep neural network model according to the training data group to obtain the current action value and target action value of the deep neural network model, including: determining the current action value according to the first sample operating state, the sample operating frequency and the first model parameter of the first sub-model; inputting the second sample operating state into the second sub-model to obtain multiple to-be-selected frequencies corresponding to the second sample operating state and to-be-selected instant rewards corresponding to each of the multiple to-be-selected frequencies; obtaining the maximum to-be-selected instant reward among the multiple to-be-selected instant rewards, and determining the to-be-selected frequency among the multiple to-be-selected frequencies corresponding to the maximum to-be-selected instant reward as the maximum reward frequency; determining the target action value according to the second sample operating state, the sample instant reward, the maximum reward frequency and the second model parameter of the second sub-model.

[0013] In a second aspect, a frequency adjustment method is provided, which includes: obtaining the operating status of a target device; inputting the operating status into a deep neural network model, and having the deep neural network model output a target operating frequency; wherein the deep neural network model is trained according to the above-mentioned model training method; and controlling the target device to operate according to the target operating frequency.

[0014] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the target operating frequency includes the operating frequency required for the first cluster in the target device to operate at the next moment and the operating frequency required for the second cluster to operate at the next moment, and the computing power of the first cluster is greater than the computing power of the second cluster; the controlling the target device to operate according to the target operating frequency includes: adjusting the current operating frequency of the first cluster to the operating frequency required for the first cluster to operate at the next moment; adjusting the current operating frequency of the second cluster to the operating frequency required for the second cluster to operate at the next moment.

[0015] In a third aspect, a model training device is provided, the model training device comprising:

[0016] A data acquisition module, used to acquire a target data group from a training data playback pool; wherein the training data playback pool includes multiple data groups, the data group includes a first sample operating state and a sample instant reward of a sample device at a current moment, a sample operating frequency of the sample device at a next moment, and a second sample operating state of the sample device at a next moment, and the sample instant reward is determined according to the frame rate and / or power consumption of the sample device;

[0017] A data processing module, used for inputting the target data group into a deep neural network model, so as to perform action evaluation on the deep neural network model according to the training data group, and obtain a current action value and a target action value of the deep neural network model;

[0018] A model training module is used to train the deep neural network model based on the difference information between the target action value and the current action value.

[0019] In conjunction with the third aspect, in some possible implementations, the model training device further includes:

[0020] A data updating unit, used for updating the training data playback pool;

[0021] The data updating unit comprises:

[0022] A first acquisition subunit, used for acquiring a first operating state of the sample device at a current moment;

[0023] A first calculation subunit, configured to determine a next-moment operating frequency of the sample device at a next moment according to the first operating state;

[0024] A second calculation subunit, configured to determine a current instant reward of the sample device at a current moment according to a reward function related to a frame rate and / or power consumption of the sample device;

[0025] A second acquisition subunit, used to control the sample device to operate according to the operating frequency at the next moment, and to acquire a second operating state of the sample device at the next moment;

[0026] An updating subunit is used to add the first operating state, the next moment operating frequency, the current instant reward and the second operating state to the training data replay pool to update the training data replay pool.

[0027] In combination with the third aspect and the above-mentioned implementation methods, in some possible implementation methods, the first acquisition subunit is specifically used to: obtain the current operating frequency, power consumption, current frame rate, central processing unit utilization, number of cache misses and number of I / O page errors of the sample device at the current moment, and obtain the first operating state.

[0028] In combination with the third aspect and the above implementations, in some possible implementations, the formula of the reward function is as follows:

[0029] r(t)=G(t)-P(t)

[0030]

[0031] P(t)=β×p(t)

[0032] Wherein, r(t) is the reward function, p(t) is the power consumption, q(t) is the current frame rate, L is the target frame rate provided by the sample device, H is a preset frame rate that is tolerable to the user and provides the minimum user experience provided by the sample device, α is a weight for balancing the performance of the sample device, β is a weight for weighing the power consumption, and c is a constant.

[0033] In combination with the third aspect and the above implementation manner, in some possible implementation manners, the deep neural network model includes a first sub-model for action selection and a second sub-model for action evaluation, and the data processing module includes:

[0034] a first determining unit, configured to determine the current action value according to the first sample running state, the sample operating frequency, and a first model parameter of the first sub-model;

[0035] A second determining unit is used to input the second sample running state into the second sub-model to obtain a plurality of to-be-selected frequencies corresponding to the second sample running state and to-be-selected instant rewards corresponding to each of the plurality of to-be-selected frequencies;

[0036] A third determining unit is used to obtain a maximum instant reward to be selected from the plurality of instant rewards to be selected, and determine a frequency to be selected from the plurality of frequencies to be selected that corresponds to the maximum instant reward to be selected as a maximum reward frequency;

[0037] The fourth determination unit is used to determine the target action value according to the second sample running state, the sample instant reward, the maximum reward frequency and the second model parameter of the second sub-model.

[0038] In a fourth aspect, a frequency adjustment device is provided, the frequency adjustment device comprising:

[0039] A status acquisition module is used to obtain the operating status of the target device;

[0040] A frequency determination module, used for inputting the operating state into a deep neural network model, and the deep neural network model outputs a target operating frequency; wherein the deep neural network model is trained by the above-mentioned model training method;

[0041] The frequency adjustment module is used to control the target device to operate according to the target operating frequency.

[0042] In conjunction with the fourth aspect, in some possible implementations, the target operating frequency includes an operating frequency required for a first cluster in the target device to run at a next moment and an operating frequency required for a second cluster to run at a next moment, the computing power of the first cluster is greater than the computing power of the second cluster, and the frequency adjustment module includes:

[0043] A first adjustment unit, configured to adjust a current operating frequency of the first cluster to an operating frequency required for the first cluster to operate at a next moment;

[0044] The second adjusting unit is used to adjust the current operating frequency of the second cluster to the operating frequency required for the second cluster to operate at a next moment.

[0045] In a fifth aspect, an electronic device is provided, comprising a memory and a processor. The memory is used to store an executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the model training method in the first aspect or any possible implementation of the first aspect, or executes the frequency adjustment method in the second aspect or any possible implementation of the second aspect.

[0046] In a sixth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is run on a computer, the computer executes the model training method in the first aspect or any possible implementation of the first aspect, or executes the frequency adjustment method in the second aspect or any possible implementation of the second aspect.

[0047] In the seventh aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the model training method in the above-mentioned first aspect or any possible implementation of the first aspect, or executes the frequency adjustment method in the above-mentioned second aspect or any possible implementation of the second aspect.

[0048] The embodiments of the present application provide a model training method, a frequency adjustment method, a model training device, a frequency adjustment device, an electronic device, and a computer-readable storage medium, which have the following technical effects:

[0049] The embodiment of the present application obtains the target data group from the training data playback pool; wherein,

[0050] The training data playback pool includes multiple data groups, each data group includes a first sample operating state and a sample instant reward of the sample device at the current moment, a sample operating frequency of the sample device at the next moment, and a second sample operating state of the sample device at the next moment. The sample instant reward is determined according to the frame rate and / or power consumption of the sample device. The target data group is input into the deep neural network model to perform action evaluation on the deep neural network model according to the training data group to obtain the current action value and target action value of the deep neural network model. Based on the difference information between the target action value and the current action value, a technical solution for training the deep neural network model is developed. A deep reinforcement learning method is used to model the relationship between the system operating state of the mobile device and the frequency of heterogeneous hardware. The frequency to be adjusted can be determined through the current system operating state of the mobile device, thereby reducing the power consumption of the mobile device while satisfying the user experience, that is, maintaining the performance of the mobile device. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram showing the power consumption of three foreground applications under different frequency modulation strategies;

[0052] Figure 2 A schematic diagram of the working principle of the frequency modulation algorithm based on reinforcement learning is shown;

[0053] Figure 3 A schematic flow chart of a model training method provided in an embodiment of the present application is shown;

[0054] Figure 4 A schematic flow chart of a frequency adjustment method provided in an embodiment of the present application is shown;

[0055] Figure 5 A schematic diagram of the structure of a model training device provided in an embodiment of the present application is shown;

[0056] Figure 6 A schematic diagram of the structure of a frequency adjustment device provided in an embodiment of the present application is shown;

[0057] Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0058] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0059] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0060] In the Android system, frequency modulation mechanisms and strategies are used to manage the frequency and voltage of the processor to balance system performance and battery life. Common frequency modulation strategies include schedutil, interactive, ondemand, performance, powersave, etc. Each frequency modulation strategy has its own unique algorithm and advantages and disadvantages, and balances the performance and power requirements of the device by selecting different frequency modulation strategies. Currently, mobile devices (such as mobile phones) use schedutil as the default frequency modulator. This frequency modulator uses the load change callback mechanism in the scheduler. By registering a callback function, when the central processing unit (CPU) load changes, the frequency modulation action of schedutil is triggered. When the CPU load is low, the schedutil frequency modulator can reduce the processor frequency and voltage to reduce power consumption. When the CPU load is high, the schedutil frequency modulator can increase the processor frequency and voltage to improve performance. The specific formula for frequency modulation is as follows:

[0061] next_freq=1.25*max_freq*util / max;

[0062] Among them, next_freq represents the frequency at the next moment, max_freq represents the maximum frequency of the cluster, util represents the CPU utilization of the cluster, and max represents the computing power of the cluster.

[0063] Although frequency modulation technology has been widely studied, its current implementation still has certain limitations. First, the current frequency modulation strategy relies on load calculation or temperature control / scenario configuration, which is slow to respond and cannot achieve the optimal configuration, which easily leads to waste or insufficient system resources. In addition, the Android system is a system with high real-time requirements for interactive response. In multi-task interaction and interface drawing (User Interface Design, UI), due to the sudden nature of load changes, the current frequency modulation strategy does not respond in time and easily causes a sudden and sharp drop in system performance. Finally, the frequency modulation strategy in current mobile devices is only based on the utilization of tasks, and does not consider the frame rate of upper-level applications. Faced with threads with low utilization but related to frame generation and threads with high utilization but unrelated to frame generation, it is often adjusted to the wrong frequency, resulting in performance degradation and power consumption waste.

[0064] To verify the limitations of the existing frequency modulation strategy, an experiment was conducted on a Google pixel 3 mobile device. Tiktok, Youtube, and Netflix programs were run in the foreground, and the CPU load was run in the background to perform square root operations. The power consumption of the whole machine under different frequency modulation strategies was monitored through powermonitor. The results are shown in the figure. Figure 1 As shown, Figure 1 The schematic diagram shows the power consumption of the whole machine under different frequency modulation strategies for three foreground applications. The optimal strategy is to traverse all frequency points and meet the frame rate of the foreground application with the lowest power consumption. The native strategy is the frequency point of the native frequency modulator schedutil. By comparison, it is found that there is a huge power consumption difference between the optimal and native strategies, and the existing frequency modulator has a large power consumption waste.

[0065] Based on the problems existing in the existing frequency modulation strategies in the prior art, the embodiments of the present application provide a model training method, a frequency adjustment method, a model training device, a frequency adjustment device, an electronic device and a computer-readable storage medium. The system status and frequency of the mobile device are modeled by an enhanced method. While perceiving the user experience, the power consumption of the mobile power supply is reduced and the battery utilization efficiency is improved. This can not only reduce the power consumption of the mobile device, but also avoid the performance degradation of the mobile device.

[0066] like Figure 2 As shown, Figure 2A schematic diagram of the working principle of the frequency modulation algorithm based on reinforcement learning is shown. The agent (i.e., the neural network) reads the system environment state of the mobile device (which can also be understood as the system operation state) and takes action (i.e., the CPU frequency of each cluster in the mobile device). The goal of the agent is to learn a strategy to maximize the reward function by taking the best action. In this application, the reward function is a binary function of power consumption and frame rate. The agent selects the corresponding large and small core frequencies under different system operation states to minimize power consumption while satisfying the user experience.

[0067] Based on the working principle of the frequency modulation algorithm of reinforcement learning, it is necessary to train the frequency modulation algorithm model. The following is an embodiment of a model training method provided in an embodiment of the present application.

[0068] Figure 3 A schematic flow chart of a model training method provided in an embodiment of the present application is shown, such as Figure 3 As shown, the model training method provided in the embodiment of the present application is applied to a mobile device, such as a mobile phone. The embodiment of the present application takes a mobile phone as an example for explanation, and the sample devices and target devices involved in the following content are also mobile phones. The frequency modulation algorithm model is a deep neural network model, such as a double deep Q neural network (Double Deep Q-Network, DDQN). The embodiment of the present application takes the double deep neural network model as a DDQN network as an example for explanation.

[0069] The model training method includes the following schemes:

[0070] S110: Obtain a target data group from a training data playback pool; wherein the training data playback pool includes multiple data groups, the data groups including a first sample operating state and a sample instant reward of a sample device at a current moment, a sample operating frequency of the sample device at a next moment, and a second sample operating state of the sample device at a next moment, and the sample instant reward is determined based on the frame rate and / or power consumption of the sample device.

[0071] In an exemplary embodiment, the training data playback pool is a playback memory set in the sample device, the capacity of the training data playback pool is Nm, and its current storage capacity is expressed as M. The training data playback pool includes multiple data groups, each of which is the data required for training played back into the training data playback pool through experience playback during the operation of the sample device. For each data group, it includes the first sample operation state and sample instant reward of the sample device at the current moment, the sample operating frequency of the sample device at the next moment, and the second sample operation state of the sample device at the next moment, and the sample instant reward is calculated according to the frame rate and / or power consumption of the sample device. Among them, the current moment described in each data group is different, and the next moment described in each data group is also different. Assume that the training data playback pool includes 2 data groups, namely data group 1 and data group 2, wherein, with the current moment as the reference moment t, the moment corresponding to the first sample operation state in data group 1 is moment t, and the moment corresponding to the second sample operation state is moment t+1, then moment t is the current moment, and moment t+1 is the next moment. The time corresponding to the running state of the first sample in data group 2 is time t+2, and the time corresponding to the running state of the second sample is time t+3. Then time t+2 is the current time, and time t+3 is the next time, that is, any two data groups in the training data playback pool are different.

[0072] The target data group is obtained from the training data playback pool. The target data group includes multiple data groups randomly selected from the training data playback pool, that is, the target data group includes the first sample operation state, the sample instant reward, the sample working frequency, and the second sample operation state. Take time t as the current time, the first sample operation state at time t is represented as s, the sample instant reward at time t is represented as r, the sample working frequency of the sample device at time t+1 is obtained at time t, and the sample working frequency belongs to the action to be performed by the sample device, which is represented as a, and the second sample operation state at time t+1 is represented as s′, then the target data group is (s, a, r, s′).

[0073] This application redesigns the states, actions, and rewards used to train the DDQN network. The preparations before model training are as follows:

[0074] Regarding the problem definition: Let L be the target frame rate provided by the mobile device, G(t) represents the user experience of the application running on the mobile device at time t, and P(t) represents the power consumption of the CPU of the mobile device at time t. Let f b (t)∈F b , represents the frequency of the first cluster of mobile devices at time t, f l (t)∈F l , represents the frequency of the second cluster of mobile devices at time t, Fb represents the frequency range of the first cluster, F l represents the frequency range of the second cluster. The first cluster and the second cluster are both CPU clusters. The difference is that the computing power of the first cluster is greater than the computing power of the second cluster. Let π represent the set of available strategies for controlling the frequencies of different clusters from t = 1 to t = T (f b (1), f l (1), ..., f b (t), f l (t)), so the frequency modulation problem under user experience perception can be reduced to the following formula:

[0075]

[0076] Problem analysis: The system operation status and dynamic characteristics of mobile devices pose challenges to solving the frequency modulation problem, which requires the adoption of appropriate dynamic frequency adjustment strategies to optimize system performance and power. Since the CPU resources, memory resources, and IO resources in the system will affect the user experience of the foreground applications running on mobile devices to varying degrees and will interfere with each other, it is difficult to model the relationship between the system operation status and the frequency of each cluster.

[0077] Based on the above analysis, a method based on machine learning is sought. The dynamic frequency modulation problem is essentially a control optimization problem, and the solution is within the scope of reinforcement learning algorithms. Reinforcement learning is usually used for control problems where the control action or behavior directly affects the future state. In addition, it interacts with the environment and learns the best decision based on feedback without relying on an explicit model that matches the management problem. Considering that many factors in the system are continuously changing, such as power consumption and utilization, a DDQN network that can handle continuous values ​​is used to simulate the relationship between system state and frequency.

[0078] Regarding the design of status, action and reward: The running status of the embodiment of the present application includes eight parameters obtained by monitoring the running process of the mobile device, as shown in Table 1. In addition to monitoring the power consumption p(t) and frame rate q(t) of the foreground application at time t, the running status also monitors the frequency f of the first cluster. b (t) and the frequency f of the second cluster l (t), and the CPU utilization u of the first cluster b (t) and the CPU utilization u of the second cluster l (t). In addition, after correlation analysis of multiple performance events of mobile devices, performance events cm(t) and pf(t) are selected to reflect the characteristics of applications and the consumption of memory and I / O resources in the system, that is, cm(t) represents the number of last-level cache (Cache) misses in the memory, and pf(t) represents the number of I / O page faults.

[0079] Table 1: Detailed design of states, actions, and rewards for reinforcement learning

[0080]

[0081] The states shown in Table 1 are designed for the running states of mobile devices, including f b (t), f l (t),u b (t),u l (t), q(t), p(t), cm(t) and pf(t), that is, the number of parameters input to the model is large, which can improve the accuracy and usability of the model. b (t) and the frequency f of the second cluster l (t) is defined as the action, i.e., the action of training the DDQN network is designed for the frequency f of the first cluster. b (t) and the frequency f of the second cluster l (t), these actions are performed using the improved ε-greedy method. The DDQN network performs exploration with probability ε and exploitation with probability 1-ε, which decays over time during model training to strike a balance between exploration and exploitation. In the exploration phase, the DDQN network takes random actions across the entire frequency range to collect training data. In the exploitation phase, the action a(t) that maximizes the reward will be selected to converge the model.

[0082] For the rewards used in model training, this application redesigned the reward function, as shown in Table 1. The reward function consists of the user's experience reward G(t) and power consumption reward P(t), where the experience reward is represented by the frame rate. The design of the experience reward takes into account the following factors: 1) No additional rewards are given for exceeding the target frame rate. This application uses the min function to avoid the frequency modulation algorithm from continuously increasing the frequency to obtain higher rewards. 2) There will be severe penalties for frame rates below a certain level. This application introduces H to represent the preset frame rate that the user can tolerate and is the minimum user experience. When the user experience is less than this value, the reward becomes negative. G(t) and P(t) are defined as follows:

[0083]

[0084] P(t)=β×p(t)

[0085] Among them, α is the weight used to balance the performance of the mobile device, β is the weight used to weigh the power consumption, α and β are used to normalize the power consumption reward because the scale of p(t) is hundreds or thousands; c is a constant, for example, c=1.

[0086] Based on the above design of states, actions, and rewards required for DDQN network training, the training data preparation process required for DDQN network training is as follows:

[0087] In a possible implementation, the model training method further includes: updating the training data playback pool.

[0088] The update of the training data replay pool is to replay the experience data required for model training into the training data replay pool. The experience data replayed into the training data replay pool includes the operating status, working frequency and instant reward. The instant reward is calculated by the above reward function.

[0089] The updating of the training data playback pool comprises:

[0090] Acquire a first operating state of the sample device at a current moment;

[0091] Determine the operating frequency of the sample device at the next moment according to the first operating state;

[0092] Determine a current instantaneous reward of the sample device at a current moment according to a reward function related to a frame rate and / or power consumption of the sample device;

[0093] Controlling the sample device to operate according to the operating frequency at the next moment, and obtaining a second operating state of the sample device at the next moment;

[0094] The first operating state, the next-moment operating frequency, the current instant reward, and the second operating state are added to the training data replay pool to update the training data replay pool.

[0095] The deep neural network model includes a first sub-model for action selection and a second sub-model for action evaluation. Since the deep neural network model is a DDQN network, the first sub-model is the current Q network and the second sub-model is the target Q network. The network structures of the current Q network and the target Q network are the same.

[0096] During the operation of the sample device, the first operation state of the sample device at the current moment is obtained, that is, the first operation state includes the above eight parameters, and then the first operation state is input into the first sub-model to obtain multiple frequencies corresponding to the first operation state, and then the ε-greedy method is used to select a frequency from the multiple frequencies corresponding to the first operation state as the next moment operating frequency required for the sample device to operate at the next moment. Then the sample device is controlled to operate according to the next moment operating frequency, and the second operation state of the sample device at the next moment is obtained, and the current instant reward of the sample device at the current moment is calculated according to the above reward function and the frame rate and / or power consumption of the sample device at the current moment.

[0097] After obtaining the first operating state, the working frequency at the next moment, the current instant reward, and the second operating state, these four parameters are added to the training data replay pool. If the storage size of the training data replay pool reaches its capacity, the four parameters currently added are used to replace the four parameters stored earliest, thereby realizing the update of the data in the training data replay pool.

[0098] In one possible implementation, obtaining the first operating state of the sample device at the current moment includes: obtaining the current operating frequency, power consumption, current frame rate, central processing unit utilization, number of cache misses, and number of I / O page errors of the sample device at the current moment to obtain the first operating state.

[0099] Based on the design of the above state, during the operation of the sample device, the current operating frequency, power consumption, current frame rate, CPU utilization, number of cache misses, and number of I / O page errors of the sample device at the current moment are obtained to obtain the first operating state of the sample device at the current moment. Among them, the current operating frequency includes the frequency of the first cluster and the frequency of the second cluster of the sample device at the current moment, the CPU utilization includes the CPU utilization of the first cluster and the CPU utilization of the second cluster of the sample device at the current moment, and the power consumption and current frame rate of the sample device at the current moment are respectively the power consumption and frame rate of the foreground application running on the sample device at the current moment.

[0100] Among them, the reward function for the frame rate and / or power consumption of the sample device is the same as the reward function designed above, except that the parameter in the reward function for the frame rate and / or power consumption of the sample device is about the sample device, that is, the reward function r(t) for the frame rate and / or power consumption of the sample device is:

[0101] r(t)=G(t)-P(t)

[0102]

[0103] P(t)=β×p(t)

[0104] Among them, p(t) is the power consumption of the sample device at the current moment, q(t) is the current frame rate of the sample device at the current moment, L is the target frame rate provided by the sample device, H is used to indicate the preset frame rate that is tolerable to the user and provides the minimum user experience provided by the sample device, α is the weight used to balance the performance of the sample device, β is the weight used to weigh the power consumption, and c is a constant.

[0105] S120: Input the target data group into the deep neural network model to perform action evaluation on the deep neural network model according to the training data group to obtain a current action value and a target action value of the deep neural network model.

[0106] After obtaining the target data group, the target data group is input into the deep neural network model, and the action of the deep neural network model is evaluated through the training data group to obtain the current action value and target action value of the deep neural network model.

[0107] In a possible implementation, S120 includes the following solutions:

[0108] determining the current action value according to the first sample running state, the sample operating frequency, and the first model parameter of the first sub-model;

[0109] Inputting the second sample running state into the second sub-model, obtaining a plurality of to-be-selected frequencies corresponding to the second sample running state and to-be-selected instant rewards corresponding to each of the plurality of to-be-selected frequencies;

[0110] Obtaining a maximum instant reward to be selected from the plurality of instant rewards to be selected, and determining a frequency to be selected from the plurality of frequencies to be selected that corresponds to the maximum instant reward to be selected as a maximum reward frequency;

[0111] The target action value is determined according to the second sample running state, the sample instant reward, the maximum reward frequency and the second model parameter of the second sub-model.

[0112] The first sub-model calculates the current action value based on the first sample operating state, the sample operating frequency and the first model parameter of the first sub-model. The current action value can be understood as the current reward value of the calculated sample operating frequency. The current action value is expressed as Q(s,a;θ), where θ is the first model parameter of the first sub-model.

[0113] The second sample operation state is input into the second sub-model, and the second sub-model outputs a plurality of to-be-selected frequencies corresponding to the second sample operation state and the to-be-selected instant rewards corresponding to the plurality of to-be-selected frequencies, wherein the plurality of to-be-selected frequencies corresponding to the second sample operation state are the plurality of to-be-selected frequencies of the sample device operation at the next moment calculated at the moment corresponding to the second sample operation state. After obtaining the plurality of to-be-selected frequencies corresponding to the second sample operation state and the to-be-selected instant rewards corresponding to the plurality of to-be-selected frequencies, the maximum to-be-selected instant reward is selected from the plurality of to-be-selected instant rewards, and then the to-be-selected frequency corresponding to the maximum to-be-selected instant reward among the plurality of to-be-selected frequencies is determined as the maximum reward frequency, and the maximum reward frequency is expressed as a max ,Right now:

[0114] a max = argmax a′ Q(s′,a′;θ);

[0115] Wherein, s′ represents the running state of the second sample, and a′ represents the frequency to be selected.

[0116] Then, according to the second sample running state, the sample instant reward, the maximum reward frequency and the second model parameters of the second sub-model, the target action value is calculated. The target action value can be understood as the reward value of the maximum reward frequency, which is also the target reward value of the sample working frequency. The target action value is expressed as y, that is:

[0117] y=r+γQ(s′,a max ;θ′);

[0118] Among them, θ′ is the second model parameter of the second sub-model, r represents the sample immediate reward, and γ is the discount factor.

[0119] S130: Training the deep neural network model based on the difference information between the target action value and the current action value.

[0120] After obtaining the target action value and the current action value, the difference between the target action value and the current action value is calculated to obtain the difference information, and then the loss function Loss is constructed based on the difference information. For example, the loss function Loss is: Loss = ||yQ(s,a;θ)|| 2 After the loss function is constructed, the deep neural network model is iteratively trained according to the gradient descent method through the loss function until the model converges, so that the trained deep neural network model is used for frequency regulation of the mobile device.

[0121] The following is an algorithm flow of the deep neural network model trained based on state, action and reward in the embodiment of the present application, as shown in the following steps:

[0122] Input: training data playback pool M with capacity Nm. Initial neural network parameter θ, i.e., the first model parameter of the first sub-model, training data size Nb, and update time interval C.

[0123] Output: target neural network parameter θ′, i.e., the second model parameter of the second sub-model.

[0124]

[0125] The embodiment of the present application obtains a target data group from a training data playback pool; wherein the training data playback pool includes multiple data groups, each data group includes a first sample operating state and a sample instant reward of a sample device at a current moment, a sample operating frequency of the sample device at a next moment, and a second sample operating state of the sample device at the next moment, and the sample instant reward is determined according to the frame rate and / or power consumption of the sample device. The target data group is input into a deep neural network model to perform action evaluation on the deep neural network model according to the training data group to obtain a current action value and a target action value of the deep neural network model, and a technical solution for training the deep neural network model based on the difference information between the target action value and the current action value. A deep reinforcement learning method is used to model the relationship between the system operating state of a mobile device and the frequency of heterogeneous hardware. The frequency to be adjusted can be determined through the current system operating state of the mobile device, thereby reducing the power consumption of the mobile device while satisfying the user experience, i.e., maintaining the performance of the mobile device.

[0126] The following is an embodiment of a frequency adjustment method provided in an embodiment of the present application.

[0127] Figure 4 A schematic flow chart of a frequency adjustment method provided in an embodiment of the present application is shown. Figure 4 As shown, the frequency adjustment method provided in the embodiment of the present application is applied to a mobile device, and the frequency adjustment method includes the following schemes:

[0128] S210: Obtaining the operating status of the target device;

[0129] S220: Inputting the operating state into a deep neural network model, and having the deep neural network model output a target operating frequency; wherein the deep neural network model is trained according to the above-mentioned model training method;

[0130] S230: Control the target device to operate according to the target operating frequency.

[0131] In an exemplary embodiment, the target device is, for example, a mobile phone actually in use. The operating frequency, CPU utilization, number of cache misses, number of I / O page errors, and power consumption and frame rate of the running foreground application of the target device at the current moment during operation are obtained to obtain the operating status of the target device at the current moment.

[0132] The operating status of the target device at the current moment is input into the deep neural network model trained according to the above-mentioned model training method. The deep neural network model outputs the target operating frequency required for the target terminal to operate at the next moment, that is, the operating frequency to be adjusted, and then controls the target device to operate according to the target operating frequency.

[0133] In one possible implementation, the target operating frequency includes the operating frequency of the first cluster in the target device at the next moment and that of the second cluster, the first cluster is the first cluster in the target device, the second cluster is the second cluster in the target device, the second cluster and the first cluster are both CPU clusters, and the computing power of the first cluster is greater than the computing power of the second cluster. Since the target device includes the first cluster and the second cluster, the operating frequency at the current moment obtained in the running state includes the frequency of the first cluster and the frequency of the second cluster, and the CPU utilization includes the CPU utilization of the first cluster and the CPU utilization of the second cluster, then the output target operating frequency includes the operating frequency required for the first cluster to run at the next moment, represented as F1, and the operating frequency required for the second cluster to run at the next moment, represented as F2. S230, controlling the target device to run according to the target operating frequency includes the following schemes:

[0134] Adjusting the current operating frequency of the first cluster to F1;

[0135] The current operating frequency of the second cluster is adjusted to F2.

[0136] The current operating frequency of the first cluster is adjusted to F1, and the first cluster operates according to F1. The current operating frequency of the second cluster is adjusted to F2, and the second cluster operates according to F2, thereby adjusting the frequency of the heterogeneous hardware of the target device through the system state of the target device.

[0137] The following is a test of the trained deep neural network model:

[0138] The deep neural network model was applied to the Google Pixel 3 phone, and the results of the frequency modulation algorithm implemented based on the deep neural network model (referred to as the frequency modulation algorithm of this application) test and the schedutil benchmark test of the default frequency modulator of the Android system were compared.

[0139] Benchmark test: The benchmark test is a control for the algorithm test. It runs four scenarios to simulate the background load of the normal use of the phone:

[0140] Include only foreground applications (such as the Kuaishou app); add CPU-intensive tasks in the background; add memory-intensive tasks in the background; add IO-intensive applications in the background.

[0141] Write scripts to automatically implement the above four scenarios, and monitor the system performance and power consumption when the above tasks are performed. Finally, count and analyze the system performance and power consumption overhead, as shown in Table 2.

[0142] Table 2: Test load description

[0143] load Test environment 1 Test environment 2 CPU intensive Run 32 processes to extract the square root 4 processes compression and decompression Memory intensive The running process requests 3GB of memory Looping to load 10MB images IO intensive Run 8 processes sync Download 512MB file

[0144] Algorithm test: In the algorithm test, the test method is almost exactly the same as the benchmark test to reduce the impact of other variables on the test results. On top of the benchmark test, the background service including the frequency modulation algorithm is started to monitor the system performance and power consumption, and compared with the benchmark test to calculate the cost and benefit of the algorithm.

[0145] Test comparison results:

[0146] Table 3: Test environment 1

[0147] test Frame rate Power consumption (mW) Benchmarks 58.95 3715 Frequency modulation algorithm of this application 58.75 2606

[0148] It can be seen from Table 3 that, using the frequency modulation algorithm of the present application, the frame rate of the mobile phone is reduced by about 0.3%, and the power consumption is reduced by about 29.8%.

[0149] Table 4: Test environment 2

[0150]

[0151] It can be seen from Table 4 that, using the frequency modulation algorithm of the present application, the FPS of the mobile phone is lost by about 0.05%. In terms of power consumption, the power consumption is reduced by about 18.1%.

[0152] To sum up, the performance of the frequency modulation algorithm of the present application is almost the same as that of the native frequency modulation strategy of the Android system, and at the same time it brings about a certain decrease in power consumption, thereby reducing the power consumption of the device. Therefore, the present application will apply the deep neural network model trained according to the above-mentioned model training method to the mobile device. By modulating the frequency of the mobile device, not only the power consumption of the mobile device can be reduced, but also the performance degradation of the mobile device can be avoided.

[0153] Since the environmental changes in mobile devices are not periodic, during the use of the device, it is very likely that the device environment will not change for a long time. For example, if a short video software is always running, it is not necessary to execute the frequency modulation algorithm every second. Or, in game software with frequent performance changes, the environment changes very frequently and the frequency modulation time interval also needs to change accordingly. Therefore, the frequency modulation algorithm of the present application is not limited to being executed once per second during actual use. The two algorithm executions are separated by one second, and the execution time interval can be selected according to the specific situation.

[0154] The frequency modulation algorithm of the present application can be used for the first cluster (including 4 large cores, i.e. 4 frequencies), the second cluster (including 4 small cores, i.e. 4 frequencies), a total of 16 frequency modulation schemes, of which the first cluster can be called a large cluster and the second cluster can be called a small cluster. However, in the complex operating environment of mobile devices of the Android system, there may be other parameters that can reflect the current performance and serve as a reference for frequency modulation, and the optional frequencies of large cores and small cores are usually more than 4, and the actual number of frequency modulation schemes is much greater than 16. Therefore, more system information can be collected as the input of the neural network, increasing the input space of the model, while also increasing the output of the model, and improving the accuracy and availability of the model.

[0155] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0156] Figure 5 A schematic diagram of the structure of a model training device provided in an embodiment of the present application is shown. Figure 5 As shown, the model training device 500 includes:

[0157] A data acquisition module 510 is used to acquire a target data group from a training data playback pool; wherein the training data playback pool includes multiple data groups, the data group includes a first sample operating state and a sample instant reward of a sample device at a current moment, a sample operating frequency of the sample device at a next moment, and a second sample operating state of the sample device at a next moment, and the sample instant reward is determined according to the frame rate and / or power consumption of the sample device;

[0158] A data processing module 520 is used to input the target data group into a deep neural network model to perform action evaluation on the deep neural network model according to the training data group to obtain a current action value and a target action value of the deep neural network model;

[0159] The model training module 530 is used to train the deep neural network model based on the difference information between the target action value and the current action value.

[0160] In a possible implementation, the model training device 500 further includes:

[0161] A data updating unit, used for updating the training data playback pool;

[0162] The data updating unit comprises:

[0163] A first acquisition subunit, used for acquiring a first operating state of the sample device at a current moment;

[0164] A first calculation subunit, configured to determine a next-moment operating frequency of the sample device at a next moment according to the first operating state;

[0165] A second calculation subunit, configured to determine a current instant reward of the sample device at a current moment according to a reward function related to a frame rate and / or power consumption of the sample device;

[0166] A second acquisition subunit, used to control the sample device to operate according to the operating frequency at the next moment, and to acquire a second operating state of the sample device at the next moment;

[0167] An updating subunit is used to add the first operating state, the next moment operating frequency, the current instant reward and the second operating state to the training data replay pool to update the training data replay pool.

[0168] In one possible implementation, the first acquisition subunit is specifically used to obtain the current operating frequency, power consumption, current frame rate, central processing unit utilization, number of cache misses and number of I / O page errors of the sample device at the current moment to obtain the first operating state.

[0169] In a possible implementation, the formula of the reward function is as follows:

[0170] r(t)=G(t)-P(t)

[0171]

[0172] P(t)=β×p(t)

[0173] Wherein, r(t) is the reward function, p(t) is the power consumption, q(t) is the current frame rate, L is the target frame rate provided by the sample device, H is a preset frame rate that is tolerable to the user and provides the minimum user experience provided by the sample device, α is a weight for balancing the performance of the sample device, β is a weight for weighing the power consumption, and c is a constant.

[0174] In a possible implementation, the deep neural network model includes a first sub-model for action selection and a second sub-model for action evaluation, and the data processing module 520 includes:

[0175] a first determining unit, configured to determine the current action value according to the first sample running state, the sample operating frequency, and a first model parameter of the first sub-model;

[0176] A second determining unit is used to input the second sample running state into the second sub-model to obtain a plurality of to-be-selected frequencies corresponding to the second sample running state and to-be-selected instant rewards corresponding to each of the plurality of to-be-selected frequencies;

[0177] A third determining unit is used to obtain a maximum instant reward to be selected from the plurality of instant rewards to be selected, and determine a frequency to be selected from the plurality of frequencies to be selected that corresponds to the maximum instant reward to be selected as a maximum reward frequency;

[0178] The fourth determination unit is used to determine the target action value according to the second sample running state, the sample instant reward, the maximum reward frequency and the second model parameter of the second sub-model.

[0179] It should be noted that the model training device provided in the above embodiment only uses the division of the above functional modules as an example when executing the model training method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the model training device provided in the above embodiment belongs to the same concept as the model training method embodiment. Therefore, for details not disclosed in the device embodiment of the present application, please refer to the above-mentioned model training method embodiment of the present application, which will not be repeated here.

[0180] Figure 6 FIG. 1 shows a schematic diagram of the structure of a frequency adjustment device provided in an embodiment of the present application. For example, Figure 6 As shown, the frequency adjustment device comprises:

[0181] The status acquisition module 610 is used to acquire the operating status of the target device;

[0182] A frequency determination module 620, configured to input the operating state into a deep neural network model, and output a target operating frequency from the deep neural network model; wherein the deep neural network model is trained using the above-mentioned model training method;

[0183] The frequency adjustment module 630 is used to control the target device to operate according to the target operating frequency.

[0184] In a possible implementation, the target operating frequency includes an operating frequency required for a first cluster in the target device to run at a next moment and an operating frequency required for a second cluster to run at a next moment, the computing power of the first cluster is greater than the computing power of the second cluster, and the frequency adjustment module 630 includes:

[0185] A first adjustment unit, configured to adjust a current operating frequency of the first cluster to an operating frequency required for the first cluster to operate at a next moment;

[0186] The second adjusting unit is used to adjust the current operating frequency of the second cluster to the operating frequency required for the second cluster to operate at a next moment.

[0187] It should be noted that the frequency adjustment device provided in the above embodiment only uses the division of the above functional modules as an example when executing the frequency adjustment method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the frequency adjustment device provided in the above embodiment belongs to the same concept as the frequency adjustment method embodiment. Therefore, for details not disclosed in the device embodiment of the present application, please refer to the above-mentioned frequency adjustment method embodiment of the present application, which will not be repeated here.

[0188] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0189] Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown.

[0190] For example, Figure 7 As shown, the electronic device 700 includes: a memory 701 and a processor 702, wherein the memory 701 stores an executable program code 7011, and the processor 702 is used to call and execute the executable program code 7011 to perform a model training or a frequency adjustment method.

[0191] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be corresponded to, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0192] In the case of dividing each functional module according to each function, the electronic device may include: a data acquisition module, a data processing module, a model training module, etc., or a state acquisition module, a frequency determination module, a frequency adjustment module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0193] The electronic device provided in this embodiment is used to execute the above-mentioned model training or frequency adjustment method, and thus can achieve the same effect as the above-mentioned implementation method.

[0194] In the case of an integrated unit, the electronic device may include a processing module and a storage module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device in executing mutual program codes and data.

[0195] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits disclosed in the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0196] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a model training or a frequency adjustment method in the above-mentioned embodiment.

[0197] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a model training or a frequency adjustment method in the above-mentioned embodiment.

[0198] In addition, the electronic device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the electronic device may include a connected processor and a memory; wherein the memory is used to store instructions, and when the electronic device is running, the processor may call and execute instructions so that the chip performs a model training or a frequency adjustment method in the above-mentioned embodiments.

[0199] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding model training or frequency adjustment method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding model training or frequency adjustment method provided above, and will not be repeated here.

[0200] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0201] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0202] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model training method, characterized in that: The model training method comprises: Acquire a target data group from a training data playback pool; wherein the training data playback pool includes multiple data groups, the data group includes a first sample operating state and a sample instant reward of a sample device at a current moment, a sample operating frequency of the sample device at a next moment, and a second sample operating state of the sample device at a next moment, and the sample instant reward is determined according to a frame rate and / or power consumption of the sample device; Inputting the target data set into a deep neural network model to perform action evaluation on the deep neural network model according to the training data set to obtain a current action value and a target action value of the deep neural network model; The deep neural network model is trained based on the difference information between the target action value and the current action value.

2. The model training method according to claim 1, characterized in that: The model training method also includes: Updating the training data replay pool; The updating of the training data playback pool comprises: Acquire a first operating state of the sample device at a current moment; Determine the operating frequency of the sample device at the next moment according to the first operating state; Determine a current instantaneous reward of the sample device at a current moment according to a reward function related to a frame rate and / or power consumption of the sample device; Controlling the sample device to operate according to the operating frequency at the next moment, and obtaining a second operating state of the sample device at the next moment; The first operating state, the next-moment operating frequency, the current instant reward, and the second operating state are added to the training data replay pool to update the training data replay pool.

3. The model training method according to claim 2, characterized in that: The obtaining of the first operating state of the sample device at the current moment comprises: The current operating frequency, power consumption, current frame rate, CPU utilization, number of cache misses, and number of I / O page errors of the sample device at the current moment are obtained to obtain the first operating state.

4. The model training method according to claim 3, characterized in that: The formula of the reward function is as follows: Wherein, r(t) is the reward function, p(t) is the power consumption, q(t) is the current frame rate, L is the target frame rate provided by the sample device, H is a preset frame rate that is tolerable to the user and provides the minimum user experience provided by the sample device, α is a weight for balancing the performance of the sample device, β is a weight for weighing the power consumption, and c is a constant.

5. The model training method according to any one of claims 1 to 4, characterized in that: The deep neural network model includes a first sub-model for action selection and a second sub-model for action evaluation; The step of inputting the target data set into a deep neural network model to perform action evaluation on the deep neural network model according to the training data set to obtain a current action value and a target action value of the deep neural network model comprises: determining the current action value according to the first sample running state, the sample operating frequency, and the first model parameter of the first sub-model; Inputting the second sample running state into the second sub-model, obtaining a plurality of to-be-selected frequencies corresponding to the second sample running state and to-be-selected instant rewards corresponding to each of the plurality of to-be-selected frequencies; Obtaining a maximum instant reward to be selected from the plurality of instant rewards to be selected, and determining a frequency to be selected from the plurality of frequencies to be selected that corresponds to the maximum instant reward to be selected as a maximum reward frequency; The target action value is determined according to the second sample running state, the sample instant reward, the maximum reward frequency and the second model parameter of the second sub-model.

6. A frequency adjustment method, characterized in that: The frequency adjustment method comprises: Get the running status of the target device; Inputting the operating state into a deep neural network model, and outputting a target operating frequency from the deep neural network model; wherein the deep neural network model is trained according to the model training method according to any one of claims 1 to 5; The target device is controlled to operate according to the target operating frequency.

7. The frequency adjustment method according to claim 6, characterized in that: The target operating frequency includes an operating frequency required for a first cluster in the target device to operate at a next moment and an operating frequency required for a second cluster to operate at a next moment, and the computing power of the first cluster is greater than the computing power of the second cluster; The controlling the target device to operate according to the target operating frequency includes: Adjusting the current operating frequency of the first cluster to the operating frequency required for the first cluster to operate at the next moment; The current operating frequency of the second cluster is adjusted to the operating frequency required for the second cluster to operate at the next moment.

8. A model training device, characterized in that: The model training device comprises: A data acquisition module, used to acquire a target data group from a training data playback pool; wherein the training data playback pool includes multiple data groups, the data group includes a first sample operating state and a sample instant reward of a sample device at a current moment, a sample operating frequency of the sample device at a next moment, and a second sample operating state of the sample device at a next moment, and the sample instant reward is determined according to the frame rate and / or power consumption of the sample device; A data processing module, used for inputting the target data group into a deep neural network model, so as to perform action evaluation on the deep neural network model according to the training data group, and obtain a current action value and a target action value of the deep neural network model; A model training module is used to train the deep neural network model based on the difference information between the target action value and the current action value.

9. A frequency adjustment device, characterized in that: The frequency adjustment device comprises: A status acquisition module is used to obtain the operating status of the target device; A frequency determination module, used for inputting the operating state into a deep neural network model, and the deep neural network model outputs a target operating frequency; wherein the deep neural network model is trained according to the model training method according to any one of claims 1 to 5; The frequency adjustment module is used to control the target device to operate according to the target operating frequency.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, used to call and run the executable program code from the memory, so that the electronic device executes the model training method as described in any one of claims 1 to 5 or executes the frequency adjustment method as described in claim 6 or 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the model training method as described in any one of claims 1 to 5 or executes the frequency adjustment method as described in claim 6 or 7.

Images