EHES power management method and system based on energy neutral perception

Abstract

The invention provides an EHES power management method and system based on energy neutral perception, and belongs to the technical field of power management of an energy collection embedded system. The method comprises the steps of defining system energy models including an energy collection model, an energy storage model, a power management model and an energy consumption model; defining a task sequence gamma and an energy neutral perception model; the method comprises the following steps: acquiring an initial state of a system and initializing parameters of an Actor network and a Critic network; defining a state S (i); outputting an action according to S (i); according to the 1t corresponding to the action; fi, Uigt; adjusting the frequency and voltage of the current execution task, and giving an instant reward; estimating the state value of the current state by using a Critic network; updating Critic network parameters to minimize the prediction loss; the Actor network parameters are updated; and updating the environment state, and executing the next task until all tasks in the task sequence gamma are executed.

Description

Technical Field

[0001] The present invention relates to the technical field of power management of embedded systems, and in particular to an EHES power management method and system based on energy neutrality perception. Background Art

[0002] In recent years, small embedded devices have been used more and more widely and are becoming more and more important in our lives, covering smart cities, health detection, industrial monitoring, agricultural Internet of Things, and wearable devices. As an important component of embedded devices, batteries have always been one of their main drivers. Nowadays, some embedded devices are deployed in inaccessible areas, which are difficult to replace and maintain. In addition, the battery life is limited and the maintenance cost is high, which limits the long-term operation of the device. To solve this problem, obtaining energy from the environment (such as solar energy, mechanical energy, dynamics, etc.) is a feasible solution. Energy harvesting technology converts environmental energy into electrical energy through energy harvesters to power small embedded devices, thereby achieving permanent operation of small embedded devices. This embedded system that collects energy from the environment and stores it in an energy storage unit is defined as an energy harvesting embedded system, which consists of traditional embedded system components and energy management components. In embedded systems powered by energy harvesting, an important challenge is to balance the energy consumption and energy harvesting of embedded devices so that the system can continue to operate without relying on external power supplies. This mode is called energy neutral operation.

[0003] The dynamic characteristics of the energy harvesting process are characterized by nonlinear changes, which is particularly evident in scenarios such as solar energy harvesting. The output power of the solar energy harvesting system is affected by environmental factors such as seasonal light intensity fluctuations and cloud cover, resulting in nonlinear dynamic fluctuations in the energy supply. The nonlinear dynamic changes in the energy collected by the energy harvester may cause the system to temporarily fail due to insufficient energy supply or cause energy waste due to excess energy. Traditional power management strategies, such as dynamic voltage and frequency scaling (DVFS) and dynamic power management (DPM), show their limitations when dealing with energy harvesting embedded devices. These strategies mainly focus on reducing energy consumption by adjusting voltage and frequency, but fail to fully consider the real-time energy supply capabilities of the device. They are insufficient in adapting to the dynamics of energy harvesting and are prone to energy waste or intermittent system operation. Summary of the invention

[0004] In view of this, the present invention provides an EHES power management method and system based on energy neutrality perception, which fully considers the real-time energy supply capability of the equipment, perceives the energy neutrality state according to the energy neutrality distance, and balances the energy consumption and collection of the embedded system by adjusting the voltage and frequency of task execution, thereby shortening the energy neutrality distance and achieving energy neutral operation.

[0005] The technical solution adopted by the embodiment of the present invention to solve the technical problem is:

[0006] An EHES power management method based on energy neutrality perception, comprising:

[0007] Step S1, defining a system energy model, wherein the system energy model is composed of an energy collection model, an energy storage model, a power management model, and an energy consumption model; defining a task sequence Γ = {τ1, τ2, ..., τ i}, where τ i For the i-th executed task, each task τ i The demand vector R i ={T ai ,C i ,D i ,T i ,W i , <f i ,U i >}, where T ai Indicates the start time of the task, C i Indicates the task execution time, D i represents the task deadline, T i Represents the task cycle, <f i ,U i > represents the frequency / voltage pair that can be executed when the CPU is working. Assuming that the CPU has N discrete execution frequencies, then f i ∈{f1,f2,…,f N}, and a frequency f i Corresponding to a voltage U i ; W i is the load of the task, which indicates the number of clock cycles required for task execution;

[0008] Step S2, define the energy neutral perception model <E d (i),E s (i),EN th ,E dead ,E max >, where E d (i) is the task τ i The energy neutral distance before execution, E s (i) is the task τ iRemaining energy before execution, EN th is the energy neutral value, E dead is the minimum energy of system death, E max To store maximum energy;

[0009] Step S3, system initialization, obtaining the initial status of the scheduled execution task sequence and the current battery energy level of the system, and initializing the network parameters of the Actor network and the Critic network;

[0010] Step S4, define task τ i Energy neutral distance E before execution d (i) Task τ i-1 Energy harvesting during execution E harvest (i-1) and the task τ in the task sequence Γ to be executed i The state S(i) is formed, S(i) = [E d (i),E harvest (i-1),τ i ];

[0011] Step S5, the state S(i) is sent to the agent, and the agent outputs an action A(i) according to the state S(i), A(i) = [f i ,U i ], i∈[0,N], where A(i) is the task τ corresponding to the execution of S(i) i The required frequency f i and voltage U i ;

[0012] Step S6: The system uses DVFS technology to calculate the value of A(i) according to the corresponding <f i ,U i > Adjust the frequency and voltage of the current task to make the system energy-neutral;

[0013] Step S7, after executing action A(i), according to task τ i Whether the task is successfully executed, the current energy neutral state and the voltage frequency of the task execution, an immediate reward R(i) is given;

[0014] Step S8, using the Critic network to estimate the state value of the current state and calculate the TD error δ(i);

[0015] Step S9, using the stochastic gradient descent method, with β as the critic learning rate, updates the critic network parameters to minimize the prediction loss of the critic network, and implements the critic network update, where the prediction loss function L of the critic network is C =(δ(i)) 2 ;

[0016] Step S10, using the stochastic gradient descent method with α as the Actor learning rate to update the Actor network parameters, so that the prediction loss of the Actor network is minimized, and the Actor network update is realized, wherein the prediction loss function L of the Actor network is A =-π θ (S(i),A(i))δ(i);

[0017] Step S11, update the environment status and obtain the i+1 time slot execution task τ i+1 The previous environmental state, that is, S(i) = S(i+1), S(i+1) = [E d (i+1),E harvest (i),τ i+1 ], and further execute the next task τ based on the updated environment state i+1 ;

[0018] Step S12, repeating steps S4 to S11 until all tasks in task sequence G are completed.

[0019] Preferably, in the system energy model in step S1:

[0020] Energy harvesting model: The energy collected in the time interval [t1, t2] is defined as E harvest for:

[0021]

[0022] Among them, P h (t) is the power of energy harvested at time t;

[0023] Energy storage model: The energy E stored in the storage unit at time t storage Defined as:

[0024] E storage (t+1)=E storage (t)+E harvest (t)-E c (t)

[0025] When the stored energy exceeds the storage capacity of the energy storage unit, the energy stored in the current time slot t is defined as:

[0026] E storage (t) = min(E storage (t),E max )

[0027] Among them, E max is the maximum energy that the system can store;

[0028] Power management model: The dynamic power consumption P of the processor executing tasks at frequency f d Defined as:

[0029] P d =C×V 2 ×f

[0030] Where C is the capacitance, V is the circuit voltage, and f is the CPU frequency when executing the task;

[0031] Energy consumption model: To complete each task τ i The energy consumption required E c for:

[0032] E c =P d ×C i

[0033] In the formula, the task τ i The execution time of C i The calculation formula is:

[0034] C i =W i ×1 / f i

[0035] Where W i Represents the execution of task τ i The total number of clock cycles required, 1 / f i Indicates that at frequency f i The length of each clock cycle.

[0036] Preferably, in step S2, based on the energy neutrality perception model <E d (i),E s (i),EN th ,E dead ,E max >, the energy collected by the embedded device in a period of time T is E harvest The energy consumed is E c , energy-neutral operation is expressed as:

[0037]

[0038] Where E0 is the initial energy level of the storage unit;

[0039] Energy neutrality value EN th The formula is:

[0040]

[0041] E d (i) means Es (i) With the energy neutral value EN th The relative distance is expressed as:

[0042] E d (i) = E s (i)-EN th .

[0043] Preferably, the function expression of the instant reward R(i) in step S7 is:

[0044]

[0045] Among them, T ai +C i ≤D i Indicates that the task is completed before the deadline, then R(i) is a positive reward, T ai +C i >D i Indicates that the task missed the deadline and failed to execute, in which case R(i) is a negative reward;

[0046] In the formula, is an energy-neutral distance reward function, with a reward value range of (0, is the frequency and voltage reward function of task execution. The agent gives the task execution frequency according to the state. <f i ,U i >, DVFS achieves energy neutrality by adjusting voltage and frequency to shorten the energy neutrality distance, and the reward value range is (0,1).

[0047] Preferably, the calculation formula of the TD error δ(i) in step S8 is expressed as:

[0048] δ(i)=R(i)+γV w (S(i+1))-V w (S(i))

[0049] Where γ is the discount factor, V w (S(i)) is the state value function with w as parameter, which represents the approximate value of state S(i), V w (S(i+1)) is the state value function with w as parameter, which represents the approximate value of state S(i+1).

[0050] It can be seen from the above technical scheme that the EHES power management method and system based on energy neutrality perception provided by the embodiment of the present invention, the system perceives the energy neutrality operation during the execution of the previous task, calculates the energy neutrality distance, and uses the DVFS technology to adjust the frequency and voltage of the task being executed, thereby achieving energy neutrality operation during the execution of the task. The system can use the collected energy at an appropriate rate to ensure that the energy harvesting embedded system works permanently, improve energy utilization efficiency, and ensure that the system maintains stable operation and energy efficiency optimization in the dynamic changes of energy harvesting. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is an overall flow chart of the EHES power management method based on energy neutrality awareness. DETAILED DESCRIPTION

[0052] The technical scheme and technical effects of the present invention are further elaborated in detail below in conjunction with the accompanying drawings of the present invention.

[0053] When implementing the energy-aware power management method based on reinforcement learning and DVFS, while ensuring the success rate of task operation, the battery energy neutral operation is considered, that is, the battery energy level is maintained at E for a period of time. max and E dead In this way, the system can avoid temporary death due to insufficient energy or energy waste due to excessive energy, and the embedded system can run permanently.

[0054] The specific implementation process of the embedded system power management method based on energy neutrality perception is as follows:

[0055] Step S1, define the system energy model, which consists of energy collection model, energy storage model, power management model and energy consumption model; define the task sequence Γ={τ1,τ2,…,τ i}, where τ i For the i-th executed task, each task τ i The demand vector R i ={T ai ,C i ,D i ,T i ,W i , <f i ,U i >}, where T ai Indicates the start time of the task, C i Indicates the task execution time, D i represents the task deadline, T i Represents the task cycle, <f i ,U i> represents the frequency / voltage pair that can be executed when the CPU is working. Assuming that the CPU has N discrete execution frequencies, then f i ∈{f1,f2,…,f N}, and a frequency f i Corresponding to a voltage U i ; W i is the load of the task, which indicates the number of clock cycles required for task execution;

[0056] Step S2, define the energy neutral perception model <E d (i),E s (i),EN th ,E dead ,E max >, where E d (i) is the task τ i The energy neutral distance before execution, E s (i) is the task τ i Remaining energy before execution, EN th is the energy neutral value, E dead is the minimum energy of system death, E max To store maximum energy;

[0057] Step S3, system initialization, obtaining the initial status of the scheduled execution task sequence and the current battery energy level of the system, where E harvest (0) = 0, E storage (0) = E max / 2; Initialize the network parameters of the Actor network and the Critic network;

[0058] Step S4, define task τ i Energy neutral distance E before execution d (i) Task τ i-1 Energy harvesting during execution E harvest (i-1) and the task τ in the task sequence Γ to be executed i The state S(i) is formed, S(i) = [E d (i),E harvest (i-1),τ i ];

[0059] Step S5, the state S(i) is sent to the agent, and the agent outputs action A(i) according to the state S(i), A(i) = [f i ,U i ], i∈[0,N], where A(i) is the task τ corresponding to the execution of S(i) i The required frequency f i and voltage U i ;

[0060] Step S6: The system uses DVFS technology to calculate the corresponding <f i ,U i > Adjust the frequency and voltage of the current task to make the system energy-neutral;

[0061] Step S7, after executing action A(i), according to task τ i Whether the task is successfully executed, the current energy neutral state and the voltage frequency of the task execution, an immediate reward R(i) is given;

[0062] Step S8, using the Critic network to estimate the state value of the current state and calculate the TD error δ(i);

[0063] Step S9, using the stochastic gradient descent method, with β as the critic learning rate, updates the critic network parameters to minimize the prediction loss of the critic network, and implements the critic network update. The critic will use the new network parameters to help the actor calculate the optimal value of the state. Among them, the prediction loss function L of the critic network C =(δ(i)) 2 ;

[0064] Step S10, using the stochastic gradient descent method with α as the Actor learning rate to update the Actor network parameters, so that the prediction loss of the Actor network is minimized, and the Actor network update is realized, wherein the prediction loss function L of the Actor network is A =-π θ (S(i),A(i))δ(i);

[0065] Step S11, update the environment status and obtain the i+1 time slot execution task τ i+1 The previous environmental state, that is, S(i) = S(i+1), S(i+1) = [E d (i+1),E harvest (i),τ i+1 ], and further perform the next task τ based on the updated environment state i+1 ; E d (i+1) is task τ i+1 The energy neutral distance before execution, E harvest (i) is the task τ i Energy harvesting during execution, τ i+1 For the next task to be performed;

[0066] Step S12, repeating steps S4 to S11 until all tasks in the task sequence Γ are completed. Until all tasks in the task sequence Γ are completed.

[0067] In the system energy model in step S1:

[0068] Energy harvesting model: The function of the energy harvesting model is to efficiently collect various forms of energy in the environment and convert them into electrical energy as an energy source, but energy harvesting is random. The collected energy is stored in the energy storage module to provide energy for subsequent devices. The energy collected in the time interval [t1, t2] is defined as E harvest for:

[0069]

[0070] Among them, P h (t) is the power of energy harvested at time t;

[0071] Energy storage model: The energy storage model is a storage device that dynamically manages energy, ensuring that energy is available during peak demand and stored when energy supply is surplus. The energy E stored in the storage unit at time t storage Defined as:

[0072] E storage (t+1)=E storage (t)+E harvest (t)-E c (t) (2)

[0073] When the stored energy exceeds the storage capacity of the energy storage unit, the energy stored in the current time slot t is defined as:

[0074] E storage (t) = min(E storage (t),E max ) (3)

[0075] Among them, E max is the maximum energy that the system can store;

[0076] Power management model: The power management model is equipped with a frequency / voltage regulator to adjust and output a stable voltage to meet the voltage requirements during the execution of different tasks and achieve energy-neutral operation. The dynamic power consumption P of the processor executing a task at frequency f d Defined as:

[0077] P d =C×V 2 ×f (4)

[0078] Where C is the capacitance, V is the circuit voltage, and f is the CPU frequency when executing the task;

[0079] Energy consumption model: The energy consumption model mainly focuses on the energy consumption of tasks running in real time in the device. Task τi The demand vector is R i ={T ai ,C i ,D i ,T i ,W i , <f i ,U i >,}, complete each taskτ i The energy consumption required E c for:

[0080] E c =P d ×C i (5)

[0081] In the formula, the task τ i The execution time of C i The calculation formula is:

[0082] C i =W i ×1 / f i (6)

[0083] Where W i Represents the execution of task τ i The total number of clock cycles required, 1 / f i Indicates that at frequency f i The length of each clock cycle.

[0084] In step S2, based on the energy neutral perception model <E d (i),E s (i),EN th ,E dead ,E max >, energy-neutral operation means that the embedded device achieves a balance between energy consumption and energy harvesting, so that the system can continue to operate without relying on external power supply. Energy-neutral perception is based on the task τ i The energy neutrality distance before execution is used to perceive the energy neutrality state. That is, when the absolute value of the energy neutrality distance is smaller, the energy neutrality state is better. Conversely, when the absolute value of the energy neutrality distance is larger, the energy neutrality state is worse. The power management strategy is based on the task τ i The energy neutrality distance before execution, i.e. the difference between the remaining energy level and the energy neutrality value EN th The relative distance of the current task τ is adjusted by DVFS technology i The voltage and frequency of execution enable the embedded system to achieve a balance between energy consumption and energy collection, thereby shortening the energy neutral distance, achieving energy neutral operation, optimizing energy utilization efficiency, and enabling embedded devices to operate continuously.

[0085] The energy collected by the embedded device in a period of time T is E harvest The energy consumed is E c , energy-neutral operation is expressed as:

[0086]

[0087] Where E0 is the initial energy level of the storage unit;

[0088] Energy neutrality value EN th The formula is:

[0089]

[0090] E d (i) means E s (i) With the energy neutral value EN th The relative distance is expressed as:

[0091] E d (i) = E s (i)-EN th (9)

[0092] The function expression of the instant reward R(i) in step S7 is:

[0093]

[0094] Among them, T ai +C i ≤D i Indicates that the task is completed before the deadline, then R(i) is a positive reward, T ai +C i >D i Indicates that the task missed the deadline and failed to execute, in which case R(i) is a negative reward;

[0095] In the formula, is the energy neutral distance reward function, which is specifically expressed as follows: the smaller the energy neutral distance, the closer the current energy level of the system is to the energy neutral value. Therefore, as the energy neutral distance decreases, the reward value should be higher. The range of the reward value is is the frequency and voltage reward function of task execution. The agent gives the task execution frequency according to the state. <f i ,U i >, DVFS achieves energy neutrality by adjusting voltage and frequency to shorten the energy neutral distance. Therefore, the given <f i ,U i >The more reasonable it is, the higher the reward value should be, and the range of reward value is (0,1).

[0096] The calculation formula of TD error δ(i) in step S8 is expressed as:

[0097] δ(i)=R(i)+γV w (S(i+1))-V w (S(i))(11)

[0098] Where γ is the discount factor, V w (S(i)) is the state value function with w as parameter, which represents the approximate value of state S(i), V w (S(i)) is the state value function with w as parameter, which represents the approximate value of state S(i+1).

[0099] Through the above scheme, the system senses the energy-neutral operation during the execution of the previous task, calculates the energy-neutral distance, and uses DVFS technology to adjust the frequency and voltage of the task being executed, thereby achieving energy-neutral operation during the execution of the task. This allows the system to use the collected energy at an appropriate rate and ensure that the energy-harvesting embedded system works permanently.

[0100] Furthermore, the present invention also provides an EHES power management system based on energy neutrality perception, which is used to implement Figure 1 and the aforementioned methods.

[0101] Compared with the background technology, the present invention has the following beneficial effects:

[0102] The present invention combines the reinforcement learning method with the power management technology, and proposes an EHES power management method based on energy neutrality perception, which realizes energy neutral operation and maintains the permanent operation of the embedded system. The present invention has the following two beneficial effects:

[0103] (1) An energy-neutral sensing-based energy harvesting embedded system solution is provided, laying the foundation for a dynamic adaptive power management strategy.

[0104] (2) Combining the Actor-Critic model with the DVFS technology to adjust the frequency and voltage level of the scheduled tasks can enable the system to use the collected energy at an appropriate rate, thereby achieving energy-neutral operation and ensuring that the energy-harvesting embedded system works permanently.

[0105] The present invention provides an energy harvesting embedded system solution based on energy neutrality perception. The energy neutrality state is perceived according to the energy neutrality distance, and the energy consumption and collection of the embedded system are balanced by adjusting the voltage and frequency of task execution, thereby shortening the energy neutrality distance, achieving energy neutral operation, improving energy utilization efficiency, and ensuring stable operation of the system and optimal energy efficiency in the dynamic changes of energy harvesting.

[0106] What is disclosed above is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. A person skilled in the art can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. An EHES power management method based on energy neutrality perception, characterized in that: include: Step S1, defining a system energy model, wherein the system energy model is composed of an energy collection model, an energy storage model, a power management model, and an energy consumption model; defining a task sequence Γ = {τ1, τ2, ..., τ i }, where τ i For the i-th executed task, each task τ i The demand vector R i ={T ai ,C i ,D i ,T i ,W i , <f i ,U i >}, where T ai Indicates the start time of the task, C i Indicates the task execution time, D i represents the task deadline, T i Represents the task cycle, <f i ,U i > represents the frequency / voltage pair that can be executed when the CPU is working. Assuming that the CPU has N discrete execution frequencies, then f i ∈{f1,f2,…,f N }, and a frequency f i Corresponding to a voltage U i ; W i is the load of the task, which indicates the number of clock cycles required for task execution; Step S2, define the energy neutral perception model <E d (i),E s (i),EN th ,E dead ,E max >, where E d (i) is the task τ i The energy neutral distance before execution, E s (i) is the task τ i Remaining energy before execution, EN th is the energy neutral value, E dead is the minimum energy of system death, E max To store maximum energy; Step S3, system initialization, obtaining the initial status of the scheduled execution task sequence and the current battery energy level of the system, and initializing the network parameters of the Actor network and the Critic network; Step S4, define task τ i Energy neutral distance E before execution d (i) Task τ i-1 Energy harvesting during execution E harvest (i-1) and the task τ in the task sequence Γ to be executed i The state S(i) is formed, S(i) = [E d (i),E harvest (i-1),τ i ]; Step S5, the state S(i) is sent to the agent, and the agent outputs an action A(i) according to the state S(i), A(i) = [f i ,U i ], i∈[0,N], where A(i) is the task τ corresponding to the execution of S(i) i The required frequency f i and voltage U i ; Step S6: The system uses DVFS technology to calculate the value of A(i) according to the corresponding <f i ,U i > Adjust the frequency and voltage of the current task to make the system energy-neutral; Step S7, after executing action A(i), according to task τ i Whether the task is successfully executed, the current energy neutral state and the voltage frequency of the task execution, an immediate reward R(i) is given; Step S8, using the Critic network to estimate the state value of the current state, and calculating the TD error δ(i) according to the instant reward R(i); Step S9, using the stochastic gradient descent method, with β as the critic learning rate, updates the critic network parameters to minimize the prediction loss of the critic network, and implements the critic network update, where the prediction loss function L of the critic network is C =(δ(i)) 2 ; Step S10, using the stochastic gradient descent method with α as the Actor learning rate to update the Actor network parameters, so that the prediction loss of the Actor network is minimized, and the Actor network update is realized, wherein the prediction loss function L of the Actor network is A =-π θ (S(i),A(i))δ(i); Step S11, update the environment status and obtain the i+1 time slot execution task τ i+1 The previous environmental state, that is, S(i) = S(i+1), S(i+1) = [E d (i+1),E harvest (i),τ i+1 ], and further execute the next task τ based on the updated environment state i+1 ; Step S12, repeating steps S4 to S11 until all tasks in task sequence G are completed.

2. The EHES power management method based on energy neutrality perception as claimed in claim 1, characterized in that: In the system energy model in step S1: Energy harvesting model: The energy collected in the time interval [t1, t2] is defined as E harvest for: Among them, P h (t) is the power of energy harvested at time t; Energy storage model: The energy E stored in the storage unit at time t storage Defined as: E storage (t+1)=E storage (t)+E harvest (t)-E c (t) When the stored energy exceeds the storage capacity of the energy storage unit, the energy stored in the current time slot t is defined as: E storage (t)=min(E storage (t),E max ) Among them, E max is the maximum energy that the system can store; Power management model: The dynamic power consumption P of the processor executing tasks at frequency f d Defined as: P d =C×V 2 ×f Where C is the capacitance, V is the circuit voltage, and f is the CPU frequency when executing the task; Energy consumption model: To complete each task τ i The energy consumption required E c for: AND c =P d ×C i In the formula, the task τ i The execution time of C i The calculation formula is: C i =W i ×1 / f i Where W i Represents the execution of task τ i The total number of clock cycles required, 1 / f i Indicates that at frequency f i The length of each clock cycle.

3. The EHES power management method based on energy neutrality perception as claimed in claim 2, characterized in that: In step S2, based on the energy neutral perception model <E d (i),E s (i),EN th ,E dead ,E max >, the energy collected by the embedded device in a period of time T is E harvest The energy consumed is E c , energy-neutral operation is expressed as: Where E0 is the initial energy level of the storage unit; Energy neutrality value EN th The formula is: E d (i) means E s (i) With the energy neutral value EN th The relative distance is expressed as: E d (i)=E s (i)-EN th 。 4. The EHES power management method based on energy neutrality perception as claimed in claim 3, characterized in that: The functional expression of the instant reward R(i) in step S7 is: Among them, T ai +C i ≤D i Indicates that the task is completed before the deadline, then R(i) is a positive reward, T ai +C i >D i Indicates that the task missed the deadline and failed to execute, in which case R(i) is a negative reward; In the formula, is an energy-neutral distance reward function, with a reward value range of (0, is the frequency and voltage reward function of task execution. The agent gives the task execution frequency according to the state. <f i ,U i >, DVFS achieves energy neutrality by adjusting voltage and frequency to shorten the energy neutrality distance, and the reward value range is (0,1).

5. The EHES power management method based on energy neutrality perception as claimed in claim 4, characterized in that: The calculation formula of the TD error δ(i) in step S8 is expressed as: δ(i)=R(i)+γV w (S(i+1))-V w (S(i)) Where γ is the discount factor, V w (S(i)) is the state value function with w as parameter, which represents the approximate value of state S(i), V w (S(i)) is the state value function with w as parameter, which represents the approximate value of state S(i+1).

6. The EHES power management method based on energy neutrality perception as claimed in claim 5, characterized in that: In step S3, in the initial state, E harvest (0) = 0, E storage (0) = E max / 2.

7. An EHES power management system based on energy neutrality perception, characterized in that: Used to implement the method according to any one of claims 1 to 6.

Images