Household energy optimization management method and equipment based on electricity price and photovoltaic output prediction, and medium

Abstract

The invention discloses a household energy optimization management method and device based on electricity price and photovoltaic output prediction, and a medium, and the method comprises the steps: obtaining historical electricity price data and historical photovoltaic data, and predicting the electricity price and photovoltaic output according to the obtained data; modeling is carried out on the HEMS electric appliances with different electrical characteristics; according to the established model and the predicted electricity price and photovoltaic output, calculating to obtain an optimal scheduling scheme of various electric appliances; and calculating an energy consumption condition according to the obtained scheduling scheme, and sending the energy consumption condition to an electricity selling company, so that the electricity selling company calculates and returns an electricity bill according to the electricity use condition. According to the invention, by predicting the electricity price and the photovoltaic output, various types of household appliances can be individually scheduled through the DRL algorithm. According to the method, the uncertainty of HEMS electricity price and photovoltaic output can be managed, and the high-dimensional comprehensive decision problem can be solved. The method can help residents reasonably plan power consumption plans, and can be widely applied to the field of household energy management.

Description

Technical Field

[0001] The present invention relates to the field of household energy management, and in particular to a household energy optimization management method, device and medium based on electricity price and photovoltaic output prediction. Background Art

[0002] Smart grid is a modern power system that achieves efficient transmission and management of electricity by integrating advanced communication technologies, sensors and control systems. The realization of smart grid is conducive to improving the efficiency of electric energy use, improving the stability of power system, promoting users to participate in the power market and reducing the impact on the environment. As an important component of smart grid energy consumption, smart home encourages residents to participate in the power market through demand response (DR) management and use their smart devices, which promotes the development of modern power and energy systems. In order to maximize the peak load shaving and reduce the cost of electricity, a photovoltaic output and electricity price prediction model based on Bidirectional Gated Recurrent Unit (BiGRU) algorithm is developed, and the optimal home energy management is achieved through deep reinforcement learning (DRL) algorithm. The home energy management system (HEMS) consists of photovoltaic (PV) power generation system, battery energy storage system (ESS), electric vehicle (EV) and a variety of household appliances.

[0003] At present, HEMS face challenges in how to effectively handle uncertainty and solve high-dimensional decision-making problems. To deal with the uncertainty of HEMS, such as electricity price and photovoltaic output, many researchers have proposed corresponding methods, such as autoregressive models, exponential smoothing models, etc. However, these methods are usually unable to effectively predict the nonlinear time series data required in HEMS. In recent years, deep learning (DL) algorithms have developed rapidly in the prediction of time series data in the power field and have become a potential solution to the challenge of uncertain information processing in HEMS. Common DL algorithms, such as gated recurrent unit (GRU) and long short-term memory (LSTM), have been used to predict time series data. However, GRU has insufficient long-term dependency modeling, which may make it difficult to capture complex patterns and dependencies in long-term time series; LSTM models have the risk of overfitting when processing complex data, especially when the amount of training data is insufficient. Neural networks are used to predict users' comfort temperature and other information to improve the performance of HEMS. Multi-head convolutional neural networks (CNNs) are used to predict solar irradiance and wind speed. However, these models can only consider one-way information flow. This limitation of one-way information flow makes it difficult for traditional recurrent neural network models such as GRU to fully capture the bidirectional dependencies and contextual information in time series data. Therefore, they may perform poorly in processing complex time series data prediction tasks, and thus cannot fully mine bidirectional information to make more accurate predictions.

[0004] In terms of decision-making methods, traditional optimization algorithms such as Mixed Integer Linear Programming (MILP) and stochastic dynamic optimization have been applied to HEMS. However, traditional algorithms face challenges in solving high-dimensional comprehensive decision-making problems with uncertainty in HEMS. In recent years, DRL has gradually been applied to HEMS due to its strong nonlinear fitting ability. DRL algorithms can be divided into three types according to their optimization strategies: value-based algorithms, policy-based algorithms, and actor-critic-based algorithms. Value-based algorithms use iterative update strategies and need to compare the value function of each action, which makes this method difficult to handle high-dimensional continuous space problems and random policy problems. Policy-based algorithms can solve problems in high dimensions and continuous spaces, but they require a complete state sequence when updating, making them inefficient and time-consuming, and have poor convergence performance. Actor-critic-based algorithms combine the advantages of value-based algorithms and policy-based algorithms. It uses a value function to estimate the return value, uses the estimated return value to estimate the policy gradient, and then updates the parameters of the policy. The DRL algorithm combines the perception capabilities of the DL algorithm and the decision-making capabilities of the reinforcement learning algorithm, making it a more powerful approach to solving complex problems. Some studies have proposed DRL-based load models to adaptively respond to changing environments. However, few studies have combined DRL with factors such as uncertain electricity prices and uncertain photovoltaic output, which limits the full utilization of DRL's environmental uncertainty processing capabilities. In order to solve these problems, a method is needed that can manage the uncertainty of HEMS electricity prices and photovoltaic output and solve high-dimensional comprehensive decision-making problems. This method can effectively solve high-dimensional comprehensive decision-making problems with uncertainty and is suitable for complex home energy system management. Summary of the invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a method, device and medium for optimizing household energy management based on electricity price and photovoltaic output prediction.

[0006] The first technical solution adopted by the present invention is:

[0007] A method for optimizing household energy management based on electricity price and photovoltaic output prediction includes the following steps:

[0008] Obtain historical electricity price data and historical photovoltaic data, and predict electricity prices and photovoltaic output based on the obtained data;

[0009] Modeling of HEMS appliances with different electrical characteristics;

[0010] Calculate and obtain the best dispatching scheme for various electrical appliances based on the established model and predicted electricity prices and photovoltaic output;

[0011] The energy consumption is calculated according to the obtained dispatching plan, and the energy consumption is sent to the power sales company, so that the power sales company calculates and returns the electricity bill according to the electricity usage.

[0012] Furthermore, a prediction model based on the BiGRU algorithm is used to predict electricity prices and PV output;

[0013] The prediction model includes an input layer, a BiGRU layer and a neural network layer;

[0014] The input layer is used to preprocess the original data and input the processed data into the BiGRU layer;

[0015] The BiGRU layer is used to extract potential bidirectional features of time series data and input the extracted features into the neural network layer;

[0016] The neural network layer is used to make predictions based on the extracted features and output the final prediction results.

[0017] Furthermore, the BiGRU layer includes a GRU module and a final output unit;

[0018] The GRU module consists of a reset gate r t , update gate z t , candidate unit and hidden units h t The calculation formula is as follows:

[0019] r t =σ(W r x t +U r h t-1 )

[0020] z t =σ(W z x t +U z h t-1 )

[0021]

[0022] Where W r , U r , W z , U z , W and U are weight matrices; σ is the Sigmoid function, x t is the input at the current moment, h t-1 is the hidden state of the previous moment, ⊙ is the element-by-element multiplication;

[0023] The final output unit GRU hidden unit by forward propagation and back-propagated GRU hidden units Determined jointly, the calculation formula is as follows:

[0024]

[0025] In the formula, γ t and is the output weight, b t is bias; is the GRU hidden unit for forward propagation, is the GRU hidden unit for back propagation.

[0026] Furthermore, the input data X for electricity price prediction is 1 It consists of 6 dimensions, input data X 1 Including the historical electricity prices in the past 48 hours ρ = [ρ t-48 ,ρ t-47 ,...ρ t-1 ], and the corresponding month M, day D, hour K, week number W and holiday symbol H;

[0027] Input data for photovoltaic output prediction X 2 It consists of 9 dimensions, and the input data X 2 Includes PV output for the last 48 hours Month M, day D, hour K, solar zenith angle [sza t-48 ,sza t-47 ,...sza t-1 ], extraterrestrial irradiance [etr t-48 ,etr t-47 ,...etr t-1 ], global horizontal irradiance [ghi t-48 ,ghi t-47 ,... t-1 ], wind speed [ws t-48 ,ws t-47 ,...ws t-1 ], and temperature [tem t-48 ,tem t-47 ,...tem t-1 ].

[0028] Furthermore, the HEMS appliances with different electrical characteristics include: uncontrollable appliances, time-controllable appliances, power-controllable appliances, battery energy storage systems, electric vehicles, and photovoltaic power generation systems;

[0029] The HEMS electrical appliance modeling is as follows:

[0030] 1) Uncontrollable electrical appliances

[0031] For the i-th uncontrollable appliance, its rated power is The running time is The power is:

[0032]

[0033] 2) Time-controlled electrical appliances

[0034] The start time of the jth time-controllable appliance is exist But its power Fixed to rated power Duration

[0035]

[0036] 3) Power controllable electrical appliances

[0037] The power of power-controllable electrical appliances can be adjusted as electricity prices and demand change. The expression is:

[0038]

[0039] Where P k represents the energy consumption of the kth PSL device at time step t, Indicates its power ratio;

[0040] 4) Battery Energy Storage System ESS

[0041] ESS needs to consider both the charging and discharging processes, so the SOC change formula of ESS is given by the following formula:

[0042]

[0043] In the formula, is the charging state of ESS at time t, η ESS,ch , η ESS,dch are the charging and discharging efficiency of ESS, P t ESS,ch , P t ESS,dch are the charging and discharging power of ESS respectively, and the constraints are as follows:

[0044]

[0045] P t ESS,ch ·P t ESS,dch =0

[0046]

[0047] In the formula, is the minimum charging state of the energy storage system, is the maximum charging state of the energy storage system;

[0048] 5) Electric Vehicles (EV)

[0049] The relationship between the SOC of EV at time t and the SOC of the previous time step:

[0050]

[0051] In the formula, is the charging state at time t, η EV,ch , η EV,dch Charge and discharge efficiency, P t EV,ch , P t EV,dch is the charge and discharge power, represents the time when the EV arrives and leaves the home; similarly, the constraints of the EV are as follows:

[0052]

[0053] P t EV,ch ·P t EV,dch =0

[0054]

[0055] In the formula, is the lowest state of charge of an electric vehicle. The maximum charging state of the electric vehicle;

[0056] 6) Photovoltaic power generation system

[0057] Photovoltaic power generation is within the predicted value Nearby fluctuations:

[0058]

[0059] Furthermore, the optimal dispatching schemes for various electrical appliances are calculated based on the established model and the predicted electricity price and photovoltaic output, including:

[0060] Based on the established appliance model, the home energy management problem is transformed into a Markov decision process;

[0061] A decision-making method using a double-delayed deep deterministic policy gradient algorithm is used to solve the problem of optimal household energy scheduling.

[0062] Furthermore, the home energy management problem is converted into a Markov decision process, including:

[0063] The Markov decision process consists of a tuple (S, A, P, R, γ), where S is the state space, A is the action space, P is the transition probability, R is the reward function, and γ is the discount factor between 0 and 1; the agent is set as a HEMS agent, and the environment is the entire house including various electrical appliances;

[0064] 1) Status: in is the power of the uncontrollable electrical appliance, is the power of the time-controllable electrical appliance, is the power of the kth power-controllable electrical appliance at time t, is the battery status of the electric vehicle at time t-1, is the battery status of the energy storage system at time t-1, is the power prediction value of the photovoltaic system at time t, ρ t is the electricity price at time t;

[0065] 2) Action: [P t N ,P t T ,P t PSL ,P t EV,ch ,P t EV,dch ,P t ESS,ch ,P t ESS,dch ,P t PV ];in is the power of the uncontrollable electrical appliance at time t, is the power of the time-controllable electrical appliance at time t, is the power of the power-controllable electrical appliance at time t, is the charging power of the electric vehicle at time t, is the discharge power of the electric vehicle at time t, is the charging power of energy storage at time t, is the discharge power of energy storage at time t, is the photovoltaic power at time t;

[0066] 3) Reward: -r t Price ;

[0067] Since users can both buy electricity from the grid and sell electricity to the grid, t PriceIt is expressed as follows:

[0068] r t Price =ρ t P t Grid Δt

[0069] In the formula, ρ t It's the electricity price. It is the power purchased or sold by the user to the grid at time t, and Δt is the scheduling interval.

[0070] Furthermore, the dual-delayed deep deterministic policy gradient algorithm works as follows:

[0071] The double-delayed deep deterministic policy gradient algorithm uses an online policy network π during training. θ , two online critic networks and and three corresponding target networks, namely π θ′ , and To reduce the overestimation bias, the TD target is calculated using the minimum of the two target critic networks:

[0072]

[0073] In the double-delayed deep deterministic policy gradient algorithm, two techniques for reducing variance are proposed: target policy smoothing regularization and delayed update to reduce the cumulative error; then the TD target is:

[0074]

[0075] ε~clip(Ν(0,σ 2 ),-c,c)

[0076] Where ε is the clipped Gaussian noise and c is its boundary; the two online critic networks are updated according to the following formula:

[0077]

[0078] Update the actor network using policy gradients to maximize long-term reward:

[0079]

[0080] Considering the instability of the training process, a delayed strategy update rule is used to periodically update the online actor network and the target network with a fixed interval length; finally, a soft update mechanism with a learning rate of τ is used to update the target network, as shown below:

[0081] θ′←τθ+(1-τ)θ′

[0082] φ′←τφ+(1-τ)φ′.

[0084] The second technical solution adopted by the present invention is:

[0085] An electronic device comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, the code set or the instruction set is loaded and executed by the processor to implement a household energy optimization management method based on electricity price and photovoltaic output prediction as described above.

[0086] The third technical solution adopted by the present invention is:

[0087] A computer-readable storage medium, wherein at least one instruction, at least one program, code set or instruction set is stored in the storage medium, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement a household energy optimization management method based on electricity price and photovoltaic output prediction as described above.

[0088] The fourth technical solution adopted by the present invention is:

[0089] A computer program product or a computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

[0090] The beneficial effects of the present invention are as follows: the present invention predicts electricity prices and photovoltaic output, thereby enabling personalized scheduling of various types of household appliances through the DRL algorithm. The present invention can manage the uncertainty of HEMS electricity prices and photovoltaic output, and solve high-dimensional comprehensive decision-making problems. The present invention can help residents rationally plan their electricity consumption plans and optimize the allocation of demand-side resources, playing an important role in reducing users' electricity costs and improving system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0092] Figure 1 is a schematic diagram of a home energy management system (HEMS) according to an embodiment of the present invention;

[0093] Figure 2 is a schematic diagram of a HEMS architecture combining prediction and scheduling in an embodiment of the present invention;

[0094] Figure 3 Schematic diagram of the structure of the prediction model based on BiGRU in an embodiment of the present invention;

[0095] Figure 4 is a flow chart of a HEMS combining prediction and scheduling in an embodiment of the present invention;

[0096] Figure 5 It is a flowchart of the steps of a household energy optimization management method based on electricity price and photovoltaic output prediction in an embodiment of the present invention. DETAILED DESCRIPTION

[0097] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0098] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0099] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0100] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0101] In response to the existing technical problems, the present invention combines DRL with factors such as uncertain electricity prices and uncertain photovoltaic output, and proposes a prediction-based HEMS optimization solution. This solution can more accurately predict future conditions through the prediction model, adapt to changes in electricity prices and photovoltaic output in real time, improve the flexibility of energy management and enhance the decision-making ability of the system. Secondly, the present invention adopts a prediction model based on BiGRU, which can utilize forward and reverse information, so that it can more fully limit the time dependence in time series data, thereby effectively mining two-way information to make more accurate predictions. Finally, the solution of the present invention adopts the TD3 algorithm, which can effectively solve high-dimensional comprehensive decision-making problems with uncertainty and can achieve more efficient energy consumption optimization. In summary, the solution of the present invention combines the prediction model with the scheduling model, which can handle the uncertainty problems of electricity prices and photovoltaic output in HEMS and high-dimensional comprehensive decision-making problems, minimize household energy costs, and maintain safe and stable operation of the system.

[0102] Example 1

[0103] like Figure 5 As shown, this embodiment provides a household energy optimization management method based on electricity price and photovoltaic output prediction, including the following steps:

[0104] S1. Obtain historical electricity price data and historical photovoltaic data, and predict electricity price and photovoltaic output based on the obtained data;

[0105] S2. Modeling HEMS appliances with different electrical characteristics;

[0106] S3. Calculate and obtain the best dispatching scheme for various electrical appliances based on the established model and the predicted electricity price and photovoltaic output;

[0107] S4. Calculate the energy consumption according to the obtained dispatching plan, and send the energy consumption to the power selling company, so that the power selling company can calculate and return the electricity bill according to the electricity usage.

[0108] The HEMS (home energy management system) in the embodiment of the present invention takes into account various household appliances such as Figure 1As shown, it includes non-schedulable load (NSL; i.e., non-controllable electrical appliances, such as refrigerators, televisions, computers, etc.), time-schedulable load (TSL; i.e., time-controllable electrical appliances, such as washing machines, dishwashers, dryers, etc.), power-schedulable load (PSL; i.e., power-controllable electrical appliances, such as air conditioners, etc.), electric vehicles EV, battery energy storage systems ESS and PV systems. In this embodiment, a prediction model is first used to predict electricity prices and photovoltaic output; secondly, HEMS electrical appliances with different electrical characteristics are modeled and converted into Markov decision processes (MDP); finally, based on the MDP model of HEMS, a decision method using a twin delayed deep deterministic policy gradient (TD3) algorithm is proposed to solve the complex household energy optimal scheduling problem.

[0109] As an optional implementation mode, the overall architecture of the embodiment of the present invention is as follows: Figure 2 As shown in the figure, firstly, the historical data of electricity prices are obtained from the power sales company, and the electricity price and photovoltaic output are predicted using a prediction model based on the BiGRU algorithm; secondly, the HEMS agent obtains the predicted price and photovoltaic output from the BiGRU model; then the agent uses the TD3 algorithm to calculate the best scheduling plan for various electrical appliances according to the optimization goal; finally, the agent calculates the energy consumption and sends it to the power sales company, which calculates and returns the electricity bill based on the electricity usage.

[0110] The above method is explained in detail below with reference to the accompanying drawings and specific embodiments.

[0111] (1) Prediction model based on BiGRU algorithm

[0112] Neural networks can effectively handle nonlinear features and are widely used in photovoltaic forecasting and electricity price forecasting. However, most existing forecasting methods only use past information and current inputs. BiGRU can utilize both forward and backward information, enabling it to more fully constrain the temporal dependencies in time series data.

[0113] The prediction model based on the BiGRU algorithm is as follows Figure 3 As shown in the figure. First, the raw data is preprocessed in the input layer; second, the BiGRU layer is used to extract the potential bidirectional features of the time series data; finally, the deep neural network layer is used to extract features and output the final prediction results.

[0114] The GRU module consists of a reset gate r t , update gate zt , candidate unit and hidden units h t Composition, calculated by formula (1)-(4), where W r , U r , W z , U z , W and U are weight matrices. The reset gate focuses more on capturing short-term information features, while the update gate focuses more on capturing long-term information features. As the output of the GRU block, the hidden unit h t Contains these two information features.

[0115] r t =σ(W r x t +U r h t-1 ) (1)

[0116] z t =σ(W z x t +U z h t-1 ) (2)

[0117]

[0118] The final output unit of the BiGRU algorithm Calculated by formula (5), the GRU hidden unit of the forward propagation and back-propagated GRU hidden units Determine together, where γ t and is the output weight, b t For bias.

[0119]

[0120] For electricity price forecast, input data X 1 Including the historical electricity prices in the past 48 hours ρ = [ρ t-48 ,ρ t-47 ,...ρ t-1 ], the corresponding month M (such as integer value 1, 2, ..., 12), day D (such as integer value 1, 2, ..., 31), hour K (such as integer value 0, 2, ..., 23), week number W (such as integer value 1, 2, ..., 52) and holiday symbol H (Boolean value 0 or 1). Input data X for electricity price prediction 1 It consists of 6 dimensions.

[0121] For photovoltaic power generation prediction, input data X 2It consists of weather data from the weather station and historical data of household photovoltaic power generation. It includes the photovoltaic output in the past 48 hours Month M, day D, hour K, solar zenith angle [sza t-48 ,sza t-47 ,...sza t-1 ], extraterrestrial irradiance [etr t-48 ,etr t-47 ,...etr t-1 ], global horizontal irradiance [ghi t-48 ,ghi t-47 ,... t-1 ], wind speed [ws t-48 ,ws t-47 ,...ws t-1 ], and temperature [tem t-48 ,tem t-47 ,...tem t-1 ]. Input data X for photovoltaic output prediction 2 It consists of 9 dimensions.

[0122] (2) HEMS model

[0123] 2.1) Uncontrollable electrical appliances

[0124] For the i-th uncontrollable appliance, its rated power is The running time is The power is:

[0125]

[0126] 2.2) Time-controlled electrical appliances

[0127] The start time of the jth dispatchable load is exist But its power Fixed to rated power Duration

[0128]

[0129] 2.3) Power controllable electrical appliances

[0130] PSL power can be adjusted according to changes in electricity prices and demand, among which P k represents the energy consumption of the kth PSL device at time step t, Indicates its power ratio.

[0131]

[0132] 2.4) Battery Energy Storage System ESS

[0133] ESS needs to consider both the charging and discharging processes, so the SOC change formula of ESS is given by the following formula:

[0134]

[0135] in is the charging state of ESS at time t, η ESS,ch , η ESS,dch are the charging and discharging efficiency of ESS, P t ESS,ch , P t ESS,dch are the charging and discharging powers of ESS respectively. The constraints are as follows:

[0136]

[0137] P t ESS,ch ·P t ESS,dch =0 (11)

[0138]

[0139] 2.5) Electric Vehicles (EV)

[0140] The relationship between the SOC of EV at time t and the SOC of the previous time step:

[0141]

[0142] in is the charging state at time t, η EV,ch , η EV,dch Charge and discharge efficiency, P t EV,ch , P t EV,dch is the charge and discharge power, Indicates the time when the EV arrives and leaves home. Similarly, the constraints of the EV are as follows:

[0143]

[0144] P t EV,ch ·P t EV,dch =0 (15)

[0145]

[0146] 2.6) Photovoltaic power generation system

[0147] Photovoltaic power generation is within the predicted value Fluctuation nearby.

[0148]

[0149] 2.7) Energy balance constraints

[0150] In order to ensure the reliability of household energy systems, it is necessary to ensure a balance between supply and demand.

[0151]

[0152] Where P t Grid is the transaction power between HEMS and the grid. is the demand of the i-th undispatched load, is the demand for dispatchable load at the jth time, is the demand for the kth power dispatchable load.

[0153] (3) Modeling the home energy management problem as an MDP

[0154] The DRL algorithm is based on the Markov decision process and consists of a tuple (S, A, P, R, γ). S is the state space, A is the action space, P is the transition probability, R is the reward function, and γ is the discount factor between 0 and 1. The agent is set as a HEMS agent and the environment is the entire house including various electrical appliances.

[0155] a) Status:

[0156] b) Action: [P t N ,P t T ,P t PSL ,P t EV,ch ,P t EV,dch ,P t ESS,ch ,P t ESS,dch ,P t PV ]

[0157] c) Reward: -r t Price

[0158] Since users can both buy electricity from the grid and sell electricity to the grid, t Price as follows:

[0159] r t Price=ρ t P t Grid Δt (19)

[0160] Among them, ρ t is the electricity price, and Δt is the dispatch interval.

[0161] (4) TD3 algorithm

[0162] The TD3 algorithm uses an online policy network π during training θ , two online critic networks and and three corresponding target networks, namely π θ′ , and To reduce the overestimation bias, the TD target is calculated using the minimum of the two target critic networks:

[0163]

[0164] In addition, two skill target strategies for reducing variance are proposed in TD3: smooth regularization and delayed update to reduce the cumulative error. Then the TD target is:

[0165]

[0166] ε~clip(Ν(0,σ 2 ),-c,c)(22)

[0167] Where ε is the clipped Gaussian noise and c is its boundary. The two online critic networks can be updated according to the following formula:

[0168]

[0169] Update the actor network using policy gradients to maximize long-term reward:

[0170]

[0171] Considering the instability of the training process, a delayed strategy update rule is used to periodically update the online actor network and the target network with a fixed interval length. Finally, a soft update mechanism with a learning rate of τ is used to update the target network as shown below:

[0172] θ′←τθ+(1-τ)θ′ (25)

[0173] φ′←τφ+(1-τ)φ′ (26)

[0174] The flowchart of HEMS combining prediction and scheduling considering uncertainty is as follows: Figure 4 As shown in Figure 1. The scheme first uses historical data to train electricity price and photovoltaic power generation prediction models. Combined with these models, the DRL algorithm is trained to obtain the optimal energy usage strategy for HEMS through step-by-step iteration. In order to improve the efficiency of algorithm training, experience replay is used to store the data of each training step into the replay buffer. During training, a set of data is randomly sampled from the replay buffer for training.

[0175] In summary, the existing DRL-based home energy management schemes usually only optimize the energy consumption of various electrical appliances, without fully considering the impact of uncertain factors such as electricity prices and photovoltaic output in HEMS on the scheduling scheme. This not only limits the ability of DRL to handle environmental uncertainty, but may also lead to reduced efficiency of HEMS in practical applications and failure to achieve the best energy management effect. In addition, traditional prediction models usually only consider unidirectional information flow, which makes it difficult for traditional recurrent neural network models such as GRU to fully capture bidirectional dependencies and contextual information in time series data, and thus it is impossible to fully mine bidirectional information to make more accurate predictions. In view of the randomness and uncertainty of information such as real-time electricity prices and photovoltaic output, the present invention combines DRL with uncertain electricity prices, uncertain photovoltaic output and other factors, and proposes a prediction-based HEMS optimization method. The invention predicts electricity prices and photovoltaic output based on the prediction model, so that various types of household appliances can be personalized and scheduled through the DRL algorithm. The present invention can not only manage the uncertainty of HEMS electricity prices and photovoltaic output, but also solve high-dimensional comprehensive decision-making problems. This invention can help residents rationally plan their electricity consumption plans and optimize the allocation of demand-side resources, playing an important role in reducing users' electricity costs and improving system stability.

[0176] Example 2

[0177] An embodiment of the present invention further provides an electronic device, the electronic device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the following Figure 5 A household energy optimization management method based on electricity price and photovoltaic output prediction is shown.

[0178] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.

[0179] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0180] Since the electronic device is an electronic device corresponding to a household energy optimization management method based on electricity price and photovoltaic output prediction in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0181] Example 3

[0182] The embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 5 A household energy optimization management method based on electricity price and photovoltaic output prediction is shown.

[0183] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0184] Since the storage medium is a storage medium corresponding to a household energy optimization management method based on electricity price and photovoltaic output prediction in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0185] Example 4

[0186] In some possible implementations, various aspects of the method of the embodiment of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of a household energy optimization management method based on electricity price and photovoltaic output prediction according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" for executing each embodiment can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0187] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0188] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0189] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A household energy optimization management method based on electricity price and photovoltaic output prediction, characterized in that: The following steps are involved: Obtain historical electricity price data and historical photovoltaic data, and predict electricity prices and photovoltaic output based on the obtained data; Modeling of HEMS appliances with different electrical characteristics; Calculate and obtain the best dispatching scheme for various electrical appliances based on the established model and predicted electricity prices and photovoltaic output; The energy consumption is calculated based on the obtained dispatch plan and sent to the power sales company.

2. A household energy optimization management method based on electricity price and photovoltaic output prediction according to claim 1, characterized in that: A forecasting model based on the BiGRU algorithm is used to predict electricity prices and PV output; The prediction model includes an input layer, a BiGRU layer and a neural network layer; The input layer is used to preprocess the original data and input the processed data into the BiGRU layer; The BiGRU layer is used to extract potential bidirectional features of time series data and input the extracted features into the neural network layer; The neural network layer is used to make predictions based on the extracted features and output the final prediction results.

3. A household energy optimization management method based on electricity price and photovoltaic output prediction according to claim 2, characterized in that: The BiGRU layer includes a GRU module and a final output unit; The GRU module consists of a reset gate r t , update gate z t , candidate unit and hidden units h t The calculation formula is as follows: r t =σ(W r x t +U r h t-1 ) z t =σ(W z x t +U z h t-1 ) Where W r , U r , W z , U z , W and U are weight matrices; σ is the Sigmoid function, x t is the input at the current moment, h t-1 is the hidden state of the previous moment, ⊙ is the element-by-element multiplication; The final output unit GRU hidden unit by forward propagation and back-propagated GRU hidden units Determined jointly, the calculation formula is as follows: In the formula, γ t and is the output weight, b t is bias; is the GRU hidden unit for forward propagation, is the GRU hidden unit for back propagation.

4. The method for optimizing household energy management based on electricity price and photovoltaic output prediction according to claim 1, characterized in that: The input data X1 for electricity price prediction consists of 6 dimensions. The input data X1 includes the historical electricity prices for the past 48 hours ρ = [ρ t-48 ,ρ t-47 ,...ρ t-1 ], and the corresponding month M, day D, hour K, week number W and holiday symbol H; The input data X2 for photovoltaic output prediction consists of 9 dimensions. The input data X2 includes the photovoltaic output in the past 48 hours. Month M, day D, hour K, solar zenith angle [sza t-48 ,sza t-47 ,...sza t-1 ], extraterrestrial irradiance [etr t-48 ,etr t-47 ,...etr t-1 ], global horizontal irradiance [ghi t-48 ,ghi t-47 ,... t-1 ], wind speed [ws t-48 ,ws t-47 ,...ws t-1 ], and temperature [tem t-48 ,tem t-47 ,...tem t-1 ].

5. The method for optimizing household energy management based on electricity price and photovoltaic output prediction according to claim 1, characterized in that: The HEMS appliances with different electrical characteristics include: uncontrollable appliances, time-controllable appliances, power-controllable appliances, battery energy storage systems, electric vehicles, and photovoltaic power generation systems; The HEMS electrical appliance modeling is as follows: 1) Uncontrollable electrical appliances For the i-th uncontrollable appliance, its rated power is The running time is The power is: 2) Time-controlled electrical appliances The start time of the jth time-controllable appliance is exist But its power Fixed to rated power Duration 3) Power controllable electrical appliances The power of power-controllable electrical appliances can be adjusted as electricity prices and demand change. The expression is: Where P k represents the energy consumption of the kth PSL device at time step t, Indicates its power ratio; 4) Battery Energy Storage System ESS ESS needs to consider both the charging and discharging processes, so the SOC change formula of ESS is given by the following formula: In the formula, is the charging state of ESS at time t, η ESS,ch , η ESS,dch are the charging and discharging efficiency of ESS, P t ESS ,ch , P t ESS,dch are the charging and discharging power of ESS respectively, and the constraints are as follows: P t ESS,ch ·P t ESS,dch =0 In the formula, is the minimum charging state of the energy storage system ESS, The highest charging state of the energy storage system ESS; 5) Electric Vehicles (EV) The relationship between the SOC of the EV at time t and the SOC of the previous time step: In the formula, is the charging state at time t, η EV,ch , η EV,dch Charge and discharge efficiency, P t EV,ch , P t EV,dch is the charge and discharge power, represents the time when the EV arrives and leaves the home; similarly, the constraints of the EV are as follows: P t EV,ch ·P t EV,dch =0 In the formula, is the lowest charging state of the electric vehicle EV, The highest charging state of electric vehicles EV; 6) Photovoltaic power generation system Photovoltaic power generation is within the predicted value Nearby fluctuations:

6. A household energy optimization management method based on electricity price and photovoltaic output prediction according to claim 1, characterized in that: The optimal dispatching scheme for various electrical appliances is calculated based on the established model and the predicted electricity price and photovoltaic output, including: Based on the established appliance model, the home energy management problem is transformed into a Markov decision process; A decision-making method using a double-delayed deep deterministic policy gradient algorithm is used to solve the problem of optimal household energy scheduling.

7. A household energy optimization management method based on electricity price and photovoltaic output prediction according to claim 6, characterized in that: The home energy management problem is transformed into a Markov decision process, including: The Markov decision process consists of a tuple (S, A, P, R, γ), where S is the state space, A is the action space, P is the transition probability, R is the reward function, and γ is the discount factor between 0 and 1; the agent is set as a HEMS agent, and the environment is the entire house including various electrical appliances; 1) Status: in is the power of the uncontrollable electrical appliance, is the power of the time-controllable electrical appliance, is the power of the kth power-controllable electrical appliance at time t, is the battery status of the electric vehicle at time t-1, is the battery status of the energy storage system at time t-1, is the power prediction value of the photovoltaic system at time t, ρ t is the electricity price at time t; 2) Action: [P t N ,P t T ,P t PSL ,P t EV,ch ,P t EV,dch ,P t ESS,ch ,P t ESS,dch ,P t PV ];in is the power of the uncontrollable electrical appliance at time t, is the power of the time-controllable electrical appliance at time t, is the power of the power-controllable electrical appliance at time t, is the charging power of the electric vehicle at time t, is the discharge power of the electric vehicle at time t, is the charging power of energy storage at time t, is the discharge power of energy storage at time t, is the photovoltaic power at time t; 3) Reward: -r t Price ; Since users can both buy electricity from the grid and sell electricity to the grid, t Price It is expressed as follows: r t Price =ρ t P t Grid Δt In the formula, ρ t It's the electricity price. is the power purchased or sold by the user to the grid at time t, and Δt is the scheduling interval.

8. A household energy optimization management method based on electricity price and photovoltaic output prediction according to claim 6, characterized in that: The dual-delayed deep deterministic policy gradient algorithm works as follows: The double-delayed deep deterministic policy gradient algorithm uses an online policy network π during training. θ , two online critic networks and and three corresponding target networks, namely π θ′ , and The TD target is calculated using the minimum of the two target critic networks: In the double-delayed deep deterministic policy gradient algorithm, two techniques for reducing variance are proposed: target policy smoothing regularization and delayed update to reduce the cumulative error; then the TD target is: ε~clip(Ν(0,σ 2 ),-c,c) Where ε is the clipped Gaussian noise and c is its boundary; the two online critic networks are updated according to the following formula: Update the actor network using policy gradients to maximize long-term reward: The online actor network and the target network are periodically updated with a delay strategy update rule of fixed interval length; finally, the target network is updated with a soft update mechanism with a learning rate of τ, as shown below: θ′←τθ+(1-τ)θ′ φ′←τφ+(1-τ)φ′.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

Images