Energy scheduling method, device and equipment and storage medium

Abstract

The invention relates to an energy scheduling method and device, equipment and a storage medium. According to the invention, based on the power state data of the energy storage system and the power state data of the electric vehicle in the current time period, the electricity price and the net load of the user, the optimal charging and discharging strategy corresponding to the energy storage system and the electric vehicle in the next target time period of the current time period is predicted through the hybrid prediction network; the energy scheduling strategy of the energy storage system and the electric vehicle can be timely adjusted according to current needs, so that the energy cost consumed by the management object of the user energy management system is minimum, the energy overhead of the user can be reduced, and the optimal scheduling of the energy consumption of the user is realized; and the timeliness, adaptability and flexibility of scheduling the energy storage system and the electric vehicle by the user energy management system are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of energy management, and particularly relates to an energy scheduling method and device, equipment and a storage medium. BACKGROUND

[0002] In recent years, with the rising energy costs and the increasing environmental sustainability problems, user energy management systems (UEMS) have gradually become an important means to improve user energy use efficiency and reduce energy costs. The user energy management system monitors and schedules various energy devices in real time to optimize energy consumption and reduce power expenditure.

[0003] In the user energy management system, the charge and discharge management of the energy storage system (ESS) and the electric vehicle (EV) is a key component of energy scheduling. The energy storage system can store excess power generated from the solar photovoltaic system (PV), or purchase power from the grid at a low price, and release power for user use when the electricity price is higher. The electric vehicle, as a schedulable load, can be charged when the electricity price is lower, discharged when the electricity price is higher, or participate in the demand response (DR) plan to balance the grid load. The scheduling strategy of the energy storage system and the electric vehicle directly affects the user's energy cost and energy utilization efficiency.

[0004] Currently, the energy scheduling of the energy storage system and the electric vehicle in the user energy management system usually relies on simple rules or optimization algorithms, which cannot adjust the energy scheduling strategy of the energy storage system and the electric vehicle in time according to the changing environmental conditions and user demands, and has certain limitations. SUMMARY

[0005] To solve the above technical problems, the present disclosure provides an energy scheduling method, device, equipment and storage medium.

[0006] The first aspect of the present disclosure provides an energy scheduling method, comprising:

[0007] obtaining the net load of the user managed by the user energy management system in the current period, the power state data of the energy storage system and the electric vehicle managed by the user energy management system in the current period, and the electricity price of the current period;

[0008] based on the electricity price, the net load and the power state data of the energy storage system, constructing a first state vector and a first charge and discharge action vector of the energy storage system in the current period;

[0009] construct a second state vector and a second charging and discharging action vector of the electric vehicle in the current period based on the electricity price, the net load and the power state data of the electric vehicle;

[0010] The preset hybrid prediction network predicts a target charging and discharging strategy corresponding to the energy storage system and the electric vehicle in a next target period of the current period based on the first state vector, the first charging and discharging action vector, the second state vector and the second charging and discharging action vector, and the energy cost of the management object of the user energy management system under the target charging and discharging strategy is the minimum;

[0011] The charging and discharging actions of the energy storage system and the electric vehicle are controlled based on the target charging and discharging strategy in the target period.

[0012] The second aspect of the present disclosure provides an energy scheduling device, comprising:

[0013] The acquisition module is configured to acquire the net load of a user managed by a user energy management system, power state data of an energy storage system and power state data of an electric vehicle managed by the user energy management system in a current period, and an electricity price of the current period;

[0014] The first construction module is configured to construct a first state vector and a first charging and discharging action vector of the energy storage system in the current period based on the electricity price, the net load and the power state data of the energy storage system;

[0015] The second construction module is configured to construct a second state vector and a second charging and discharging action vector of the electric vehicle in the current period based on the electricity price, the net load and the power state data of the electric vehicle;

[0016] The prediction module is configured to predict a target charging and discharging strategy corresponding to the energy storage system and the electric vehicle in a next target period of the current period based on the first state vector, the first charging and discharging action vector, the second state vector and the second charging and discharging action vector by the preset hybrid prediction network, and the energy cost of the management object of the user energy management system under the target charging and discharging strategy is the minimum;

[0017] The control module is configured to control the charging and discharging actions of the energy storage system and the electric vehicle based on the target charging and discharging strategy in the target period.

[0018] The third aspect of the present disclosure provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the energy scheduling method of the first aspect can be realized.

[0019] The fourth aspect of the present disclosure provides a computer readable storage medium, and the storage medium stores a computer program, and when the computer program is executed by the processor, the energy scheduling method of the first aspect can be realized.

[0020] The technical scheme provided by the present disclosure has the following advantages compared with the prior art.

[0021] The present disclosure obtains the net load of the user managed by the user energy management system in the current period, the power state data of the energy storage system managed by the user energy management system and the power state data of the electric vehicle in the current period, and the electricity price of the current period; based on the electricity price, the net load and the power state data of the energy storage system, a first state vector and a first charging and discharging action vector of the energy storage system in the current period are constructed; based on the electricity price, the net load and the power state data of the electric vehicle, a second state vector and a second charging and discharging action vector of the electric vehicle in the current period are constructed; the first state vector, the first charging and discharging action vector, the second state vector and the second charging and discharging action vector are input into the preset hybrid prediction network to predict the target charging and discharging strategy of the energy storage system and the electric vehicle in the next target period of the current period, and the energy cost of the management object of the user energy management system under the target charging and discharging strategy is the minimum; the charging and discharging action of the energy storage system and the electric vehicle is controlled based on the target charging and discharging strategy in the target period. The present disclosure can predict the optimal charging and discharging strategy of the energy storage system and the electric vehicle in the next target period of the current period based on the power state data of the energy storage system and the electric vehicle in the current period, the electricity price and the net load of the user, through the hybrid prediction network, can timely adjust the energy scheduling strategy of the energy storage system and the electric vehicle according to the current needs, so that the energy cost of the management object of the user energy management system is the minimum, the energy consumption of the user can be reduced, the optimal scheduling of the user energy consumption can be realized, and the timeliness, adaptability and flexibility of the user energy management system in scheduling the energy storage system and the electric vehicle can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings required to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0024] Figure 1 is a flowchart of an energy scheduling method provided by an embodiment of the present disclosure;

[0025] Figure 2 is a flowchart of a hybrid prediction network training method provided by an embodiment of the present disclosure;

[0026] Figure 3 is a structural schematic diagram of an energy scheduling device provided by an embodiment of the present disclosure;

[0027] Figure 4 is a structural schematic diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other manners different from those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present disclosure, and not all the embodiments.

[0030] It should be understood that each of the steps recorded in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] It should be noted that, in this document, relational terms such as “first” and “second”, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between or among the entities or actions. Moreover, the terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase “comprising a...” does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0032] It should be noted that the modification of “one” and “multiple” mentioned in the present disclosure is illustrative rather than limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as “one or more”.

[0033] In order to better understand the inventive concept of the embodiments of the present disclosure, the technical schemes of the embodiments of the present disclosure will be described below in conjunction with exemplary embodiments.

[0034] Figure 1 is a flowchart of an energy scheduling method provided by an embodiment of the present disclosure, which can be executed by a user energy management system in a computer device, which can be understood as any device with computing function and processing capability, such asFigure 1 As shown, the energy scheduling method provided by the embodiment includes the following steps:

[0035] In step 110, the user's net load managed by the user energy management system in the current period, the power state data of the energy storage system and the electric vehicle managed by the user energy management system in the current period, and the electricity price in the current period are obtained.

[0036] In the embodiment of the present disclosure, the user's net load can be understood as the actual electric power taken by the user from the power system.

[0037] The electricity price can include a purchase price and a sales price, the purchase price being the unit price of the power purchased from the power grid, and the sales price being the unit price of the power sold to the power grid.

[0038] The management object of the user energy management system can include an energy storage system, an electric vehicle, a solar photovoltaic system, and a user's home appliance.

[0039] The energy storage system can be used to store electric energy, for example, to store excess electric energy generated from the solar photovoltaic system, or to purchase electric energy from the power grid at a low price and release the electric energy for the user to use when the electricity price is high.

[0040] The electric vehicle, as a dispatchable load, can be charged when the electricity price is low, discharged when the electricity price is high, or participate in a demand response plan to balance the load of the power grid.

[0041] The solar photovoltaic system can convert solar energy into electric energy.

[0042] The user energy management system in the computer device can obtain the user's net load managed by the user energy management system in the current period, the power state data of the energy storage system and the electric vehicle managed by the user energy management system in the current period, and the electricity price in the current period.

[0043] In some embodiments, the user's net load managed by the user energy management system in the current period can be obtained through S11-S14:

[0044] S11, obtaining the historical energy consumption power of the home appliance in a preset period before the current period.

[0045] The preset period can be set as needed, for example, 168 hours before the current period, which is not limited here.

[0046] S12, predicting the energy consumption power of the home appliance in the current period based on the historical energy consumption power, to obtain the target energy consumption power of the home appliance in the current period.

[0047] For example, the historical energy consumption power of the home appliance device in a preset time period before the current time period can be input into a preset recurrent neural network (RNN), and the recurrent neural network is used to predict the energy consumption power of the home appliance device in the current time period based on the historical energy consumption power, to obtain the target energy consumption power of the home appliance device in the current time period.

[0048] For example, the target energy consumption power of the home appliance device in the current time period t The target energy consumption power of the home appliance device in the current time period t can be represented by formula (1):

[0049]

[0050] wherein, represents the historical energy consumption power of the home appliance device in 168 hours before the current time period t.

[0051] S13, based on the conversion efficiency and the maximum power of the solar photovoltaic system, and the solar irradiance in the current time period, calculating the target power generation power of the solar photovoltaic system in the current time period.

[0052] For example, the target power generation power of the solar photovoltaic system in the current time period The target power generation power of the solar photovoltaic system in the current time period t can be represented by formula (2):

[0053]

[0054] wherein, v t represents the solar irradiance in the current time period t; η PV represents the conversion efficiency of the solar photovoltaic system; represents the maximum power of the solar photovoltaic system.

[0055] S14, calculating the difference between the target energy consumption power and the target power generation power, to obtain the net load of the user managed by the user energy management system in the current time period.

[0056] For example, the net load of the user managed by the user energy management system in the current time period t The net load of the user managed by the user energy management system in the current time period t can be represented by formula (3):

[0057]

[0058] Step 120, based on the electricity price, the net load and the power state data of the energy storage system, constructing a first state vector and a first charging and discharging action vector of the energy storage system in the current time period.

[0059] In the disclosed embodiments, the energy storage system's state vector can be used as the first state vector, and the energy storage system's charge and discharge action vector can be used as the first charge and discharge action vector. The user energy management system in the computer device can construct the energy storage system's first state vector and first charge and discharge action vector for the current time period based on the current time period's electricity price, the user's net load, and the energy storage system's power state data.

[0060] In some embodiments, the charging power of the energy storage system can be used as the first charging power, the discharging power of the energy storage system can be used as the first discharging power, and the remaining power of the energy storage system can be used as the first remaining power; the power status data of the energy storage system in the current time period can include the first charging power and the first discharging power of the energy storage system in the current time period and the first remaining power of the energy storage system in the previous time period of the current time period.

[0061] The user energy management system in the computer device can construct the state vector of the energy storage system in the current period as the first state vector based on the electricity price of the current period, the net load of the user, and the first remaining power of the energy storage system in the previous period. For example, the first state vector of the energy storage system in any period t can be expressed by formula (4):

[0062]

[0063] in, represents the first state vector of the energy storage system in time period t; t represents the electricity price in time period t; represents the net load of the user in time period t; Indicates the first remaining capacity of the energy storage system in the previous period of time period t.

[0064] The user energy management system in the computer device can construct the charge and discharge action vector of the energy storage system in the current period as the first charge and discharge action vector based on the first charge power and the first discharge power of the energy storage system in the current period. For example, the first charge and discharge action vector of the energy storage system in any period t can be expressed by formula (5):

[0065]

[0066] in, represents the first charge and discharge action vector of the energy storage system in time period t; represents the first charge and discharge power of the energy storage system in time period t; P t ESS,ch represents the first charging power of the energy storage system in time period t; P t ESS,dch Indicates the first discharge power of the energy storage system.

[0067] In step 130, a second state vector of the electric vehicle and a second charging and discharging action vector of the electric vehicle in the current time period are constructed based on the electricity price, the net load and the power state data of the electric vehicle.

[0068] In the embodiments of the present disclosure, the second state vector of the electric vehicle can be taken as the second state vector, and the charging and discharging action vector of the electric vehicle can be taken as the second charging and discharging action vector. The user energy management system in the computer device can construct the second state vector of the electric vehicle and the second charging and discharging action vector of the electric vehicle in the current time period based on the electricity price of the current time period, the net load of the user and the power state data of the electric vehicle.

[0069] In some embodiments, the charging power of the electric vehicle can be taken as the second charging power, the discharging power of the electric vehicle can be taken as the second discharging power, and the remaining power of the electric vehicle can be taken as the second remaining power; the power state data of the electric vehicle in the current time period can include the second charging power and the second discharging power of the electric vehicle in the current time period and the second remaining power of the electric vehicle in the previous time period of the current time period.

[0070] The user energy management system in the computer device can construct the second state vector of the electric vehicle in the current time period based on the electricity price of the current time period, the net load of the user and the second remaining power of the electric vehicle in the previous time period; for example, the second state vector of the electric vehicle in any time period t can be represented by formula (6):

[0071]

[0072] wherein, represents the second state vector of the electric vehicle in the time period t; λ t represents the electricity price in the time period t; represents the net load of the user in the time period t; represents the second remaining power of the electric vehicle in the previous time period of the time period t.

[0073] The user energy management system in the computer device can construct the charging and discharging action vector of the electric vehicle in the current time period as the second charging and discharging action vector based on the second charging power and the second discharging power of the electric vehicle in the current time period; for example, the second charging and discharging action vector of the electric vehicle in any time period t can be represented by formula (7):

[0074]

[0075] wherein, represents the second charging and discharging action vector of the electric vehicle in the time period t; represents the second charging and discharging power of the electric vehicle in the time period t; P t EV,ch represents the second charging power of the electric vehicle in the time period t; P t ESS,dchThe second discharge power of the electric vehicle is represented.

[0076] Step 140, the preset hybrid prediction network predicts a target charging and discharging strategy corresponding to the energy storage system and the electric vehicle in the next target period of the current period based on the first state vector, the first charging and discharging action vector, the second state vector and the second charging and discharging action vector, and the energy cost of the management object of the user energy management system under the target charging and discharging strategy is the minimum.

[0077] In the embodiments of the present disclosure, the user energy management system in the computer device can input the first state vector, the first charging and discharging action vector, the second state vector and the second charging and discharging action vector into the preset hybrid prediction network, and the preset hybrid prediction network predicts a target charging and discharging strategy corresponding to the energy storage system and the electric vehicle in the next target period of the current period based on the first state vector, the first charging and discharging action vector, the second state vector and the second charging and discharging action vector. The energy cost of the management object of the user energy management system under the target charging and discharging strategy is the minimum.

[0078] Among them, the hybrid prediction network can be understood as a kind of deep neural network (Deep Neural Network, DNN). For example, the dimension of the input layer can be consistent with the state vector (for example, 4 dimensions); the hidden layer can be three full connection layers, the number of neurons can be 200, 100 and 50 respectively, and the activation function is linear rectification function (Linear rectification function, ReLU); the output layer can be a linear activation function, and the output dimension is consistent with the charging and discharging action vector (for example, 1 dimension).

[0079] The hybrid prediction network is trained based on a supervised learning method and a deep reinforcement learning method.

[0080] Supervised learning (supervised learning, SL) is a process of adjusting the parameters of a classifier using a set of samples with known categories to achieve the required performance, also known as supervised training.

[0081] Deep reinforcement learning (Deep Reinforcement Learning, DRL) is a machine learning method that combines deep learning and reinforcement learning. It uses deep neural networks to approximate the policy or value function in reinforcement learning, thereby solving complex decision-making problems.

[0082] Step 150, control the charging and discharging action of the energy storage system and the electric vehicle based on the target charging and discharging strategy in the target period.

[0083] In the embodiments of the present disclosure, the user energy management system in the computer device can control the charging and discharging actions of the energy storage system and the electric vehicle in the target period based on the target charging and discharging strategy, so as to minimize the energy cost consumed by the management object of the user energy management system under the target charging and discharging strategy.

[0084] Therefore, the optimal charging and discharging strategy of the energy storage system and the electric vehicle in the next target period of the current period can be predicted by the hybrid prediction network based on the power state data of the energy storage system and the electric vehicle in the current period, the electricity price and the net load of the user, the energy scheduling strategy of the energy storage system and the electric vehicle can be adjusted in time according to the current needs, the energy cost consumed by the management object of the user energy management system can be minimized, the energy expenditure of the user can be reduced, the optimal scheduling of the user energy consumption can be realized, and the timeliness, adaptability and flexibility of the user energy management system in scheduling the energy storage system and the electric vehicle can be improved.

[0085] In some embodiments of the present disclosure, before the preset hybrid prediction network predicts the target charging and discharging strategy of the energy storage system and the electric vehicle in the next target period of the current period based on the first state vector, the first charging and discharging action vector, the second state vector and the second charging and discharging action vector, the computer device can train the hybrid prediction network based on a supervised learning method and a deep reinforcement learning method, specifically, the following steps can be performed. Figure 2 The flowchart of the hybrid prediction network training method provided in the present embodiment is shown in Figure 2 The hybrid prediction network training method provided in the present embodiment includes the following steps:

[0086] Step 210, constructing a target data set, the target data set including the first historical state vector and the first historical charging and discharging action vector of the energy storage system and the second historical state vector and the second historical charging and discharging action vector of the electric vehicle in each of the preset number of historical periods.

[0087] In the embodiments of the present disclosure, the preset number can be set as needed, which is not limited here.

[0088] In some embodiments, constructing the target data set can include S21-S24:

[0089] S21, obtaining the historical net load of the user managed by the user energy management system, the historical power state data of the energy storage system and the historical power state data of the electric vehicle managed by the user energy management system in each of the preset number of historical periods, and the historical electricity price of each historical period;

[0090] S22, for each historical period, based on the historical electricity price, the historical net load and the historical power state data of the energy storage system of the historical period, a first historical state vector and a first historical charging and discharging action vector of the energy storage system in the historical period are constructed;

[0091] S23, for each historical period, based on the historical electricity price, the historical net load and the historical power state data of the electric vehicle of the historical period, a second historical state vector and a second historical charging and discharging action vector of the electric vehicle in the historical period are constructed;

[0092] S24, based on the first historical state vector and the first historical charging and discharging action vector of the energy storage system and the second historical state vector and the second historical charging and discharging action vector of the electric vehicle in each historical period in the preset number of historical periods, a target data set is constructed.

[0093] Step 220, input the target data set into a preset supervised learning network, solve the optimal charging and discharging strategy of the energy storage system and the electric vehicle in each historical period in the target data set based on the supervised learning network, obtain the optimal charging and discharging strategy corresponding to each historical period in the target data set, and construct an optimal charging and discharging strategy set corresponding to the target data set based on the optimal charging and discharging strategy corresponding to each historical period in the target data set. The energy cost of the management object of the user energy management system under the optimal charging and discharging strategy is the smallest.

[0094] For example, in the supervised learning phase, a mixed integer linear programming (MILP) solver can be used to solve the optimal charging and discharging strategy of the energy storage system and the electric vehicle in each historical period in the target data set, obtain the optimal charging and discharging strategy corresponding to each historical period in the target data set, and construct an optimal charging and discharging strategy set corresponding to the target data set based on the optimal charging and discharging strategy corresponding to each historical period in the target data set.

[0095] The training process of the supervised learning method is realized by minimizing the mean-square error (MSE) or the mean absolute deviation (MAE). For details, please refer to the related technology, which will not be described here.

[0096] Step 230, input the target data set into a preset deep reinforcement learning network, predict the charging and discharging strategy of the energy storage system and the electric vehicle in each historical period in the target data set based on the deep reinforcement learning network, obtain the predicted charging and discharging strategy corresponding to each historical period in the target data set, and construct a predicted charging and discharging strategy set corresponding to the target data set based on the predicted charging and discharging strategy corresponding to each historical period in the target data set.

[0097] Step 240, input the optimal charging and discharging strategy set corresponding to the target data set and the predicted charging and discharging strategy set into the preset deep reinforcement learning network, minimize the error between the predicted charging and discharging strategy and the optimal charging and discharging strategy corresponding to each historical period based on the deep reinforcement learning network, and obtain a trained hybrid prediction network.

[0098] For example, the error L between the predicted charging and discharging strategy and the optimal charging and discharging strategy can be represented by formula (8):

[0099]

[0100] Wherein, a i represents the optimal charging and discharging strategy corresponding to the i-th historical period; represents the predicted charging and discharging strategy corresponding to the i-th historical period; N represents the preset number of historical periods.

[0101] In the deep reinforcement learning stage, a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm can be used, and each agent (energy storage system ESS and electric vehicle EV) learns the strategy independently; the agent selects an action It can be represented by formula (9):

[0102]

[0103] Wherein, μ p represents the output of the action (Actor) network; N t represents the exploration noise; the agent selects an action It can be updated according to the reward function.

[0104] The MADDPG algorithm can include:

[0105] Centralized Critic and Distributed Actor: ESS agents and EV agents each have independent Actor networks and Critic networks; the Critic network input is the global state st and all agent actions at, and the output is the action state value;

[0106] Experience replay mechanism: store historical interaction data and randomly sample to update the network;

[0107] Target network soft update: update the target network weights by a preset learning rate parameter:

[0108] The reward function is designed as the sum of negative energy cost and penalty term.

[0109] The core idea of MADDPG is to improve learning effectiveness and stability by adopting an independent Actor-Critic architecture for each agent and considering the strategy information of other agents during the training process. The Actor network is responsible for outputting actions, while the Critic network is responsible for evaluating the goodness of the current strategy. Centralized learning is adopted during training, but each agent only makes decisions based on its own local information when executing.

[0110] Thus, the hybrid prediction network can be trained based on the supervised learning method and the deep reinforcement learning method, thereby effectively improving the optimization effect of the user energy management system. First, the deep reinforcement learning strategy can enable the system to automatically learn and adjust energy scheduling decisions when facing complex and dynamic energy demands. In particular, in terms of electric vehicle (EV) and energy storage system (ESS) scheduling, through the feedback of real-time environmental data (such as electricity prices, solar power generation, user loads, etc.), the system can continuously optimize the charging and discharging strategy to minimize user energy costs and improve energy utilization efficiency. Second, the supervised learning strategy uses a deep neural network (DNN) to predict the near-optimal scheduling decisions of electric vehicles and energy storage systems, and uses the best behavior set generated from historical data to make real-time decisions, thereby avoiding the high computational complexity and poor real-time performance of traditional optimization algorithms. By combining the two, the present scheme can not only ensure the accuracy and efficiency of the decision, but also improve the scalability and flexibility of the system. In addition, the multi-agent system (MADDPG) is introduced in the scheme to simulate the collaborative work between ESS and EV, so that each agent can learn independently while considering the interaction with other agents, thereby realizing a more accurate and globally optimized scheduling strategy. Overall, the present scheme not only reduces the user's energy expenditure, but also significantly improves the adaptive ability and stability of the UEMS system, and has strong practical application value.

[0111] In the above embodiment, the first power P t G2U and the second power P t U2G satisfies the constraint, as shown in equation (10):

[0112]

[0113] wherein G2U represents the mode activation state of the power grid to the user managed by the user energy management system in any time period t; U2G represents the mode activation state of the user managed by the user energy management system to the power grid in any time period t; and are binary variables, Indicates the upper limit of the interaction power from the grid to the user's energy management system; Indicates the upper limit of the interaction power between the user's energy management system and the power grid;

[0114] The first charging power P of the energy storage system in any time period t t ESS,ch and the first discharge power P of the energy storage system t ESS,dch Satisfy the constraints, as shown in formula (11):

[0115]

[0116] in, Indicates the charging mode of the energy storage system; Indicates the discharge mode of the energy storage system; and are binary variables (0 or 1); Indicates the maximum charging power of the energy storage system; Indicates the maximum discharge power of the energy storage system;

[0117] The energy state of the energy storage system in any period t satisfies the constraints, as shown in formula (12):

[0118]

[0119] in, Represents the remaining capacity of the energy storage system within time period t; η ESS represents the charging and discharging efficiency of the energy storage system; Δτ represents the time step; Represents the remaining capacity of the energy storage system in the previous period of time t; ε ESS Indicates the minimum capacity of the energy storage system; Indicates the maximum capacity of the energy storage system; DOC ESS Indicates the discharge depth of the energy storage system;

[0120] The charging power P of the electric vehicle in any period t t EV,ch and the discharge power P of the electric vehicle t EV,dch The constraints are satisfied as shown in formula (13):

[0121]

[0122] in, Indicates the charging mode of the electric vehicle; Indicates the discharge mode of the electric vehicle; and are binary variables (0 or 1); Indicates the maximum charging power of the electric vehicle; denotes the maximum discharge power of the electric vehicle;

[0123] The energy state of the electric vehicle in any time period t satisfies the constraint as shown in formula (14):

[0124]

[0125] wherein, denotes the remaining power of the electric vehicle in the time period t; η EV denotes the charge-discharge efficiency of the electric vehicle; Δτ denotes a time step; denotes the remaining power of the electric vehicle in the last time period of the time period t; ε EV denotes the minimum power of the electric vehicle; denotes the maximum power of the electric vehicle; DOC EV denotes the discharge depth of the electric vehicle.

[0126] Figure 3 is a structural schematic diagram of an energy scheduling device provided by an embodiment of the present disclosure. The device can be understood as the above computer device or part of the functional modules in the above computer device. As shown in Figure 3 the energy scheduling device 300 comprises:

[0127] an acquisition module 310, configured to acquire a net load of a user managed by a user energy management system, power state data of an energy storage system managed by the user energy management system and power state data of an electric vehicle in a current time period, and an electricity price of the current time period;

[0128] a first construction module 320, configured to construct a first state vector and a first charge-discharge action vector of the energy storage system in the current time period based on the electricity price, the net load and the power state data of the energy storage system;

[0129] a second construction module 330, configured to construct a second state vector and a second charge-discharge action vector of the electric vehicle in the current time period based on the electricity price, the net load and the power state data of the electric vehicle;

[0130] a prediction module 340, configured to predict, by a preset hybrid prediction network, a target charge-discharge strategy corresponding to the energy storage system and the electric vehicle in a next target time period of the current time period based on the first state vector, the first charge-discharge action vector, the second state vector and the second charge-discharge action vector, the target charge-discharge strategy being a charge-discharge strategy under which an energy cost consumed by a management object of the user energy management system is minimum;

[0131] a control module 350, configured to control charge-discharge actions of the energy storage system and the electric vehicle in the target time period based on the target charge-discharge strategy.

[0132] Optionally, the management object of the user energy management system comprises an energy storage system, an electric vehicle, a solar photovoltaic system, and home appliances of the user.

[0133] The acquisition module comprises:

[0134] An acquisition sub-module is configured to acquire a historical energy consumption power of the home appliance in a preset time period before a current time period.

[0135] A prediction sub-module is configured to predict the energy consumption power of the home appliance in the current time period based on the historical energy consumption power, and obtain a target energy consumption power of the home appliance in the current time period.

[0136] A first calculation sub-module is configured to calculate a target power generation of the solar photovoltaic system in the current time period based on a conversion efficiency and a maximum power of the solar photovoltaic system and a solar irradiance in the current time period.

[0137] A second calculation sub-module is configured to calculate a difference between the target energy consumption power and the target power generation, and obtain a net load of the user energy management system in the current time period.

[0138] Optionally, the power state data of the energy storage system comprises a first charging power and a first discharging power of the energy storage system in the current time period and a first residual power of the energy storage system in a previous time period of the current time period.

[0139] The first construction module comprises:

[0140] A first construction sub-module is configured to construct a state vector of the energy storage system in the current time period as a first state vector based on the electricity price, the net load, and the first residual power.

[0141] A second construction sub-module is configured to construct a charging and discharging action vector of the energy storage system in the current time period as a first charging and discharging action vector based on the first charging power and the first discharging power.

[0142] Optionally, the power state data of the electric vehicle comprises a second charging power and a second discharging power of the electric vehicle in the current time period and a second residual power of the electric vehicle in a previous time period of the current time period.

[0143] The second construction module comprises:

[0144] A third construction sub-module is configured to construct a second state vector of the electric vehicle in the current time period based on the electricity price, the net load, and the second residual power.

[0145] a fourth constructing sub-module, configured to construct, based on the second charging power and the second discharging power, a charging and discharging action vector of the electric vehicle in the current time period as a second charging and discharging action vector.

[0146] Optionally, the energy scheduling device comprises:

[0147] a data set constructing module, configured to construct a target data set, the target data set comprising a first historical state vector and a first historical charging and discharging action vector of the energy storage system and a second historical state vector and a second historical charging and discharging action vector of the electric vehicle in each of a preset number of historical time periods;

[0148] a supervised training module, configured to input the target data set into a preset supervised learning network, solve, based on the supervised learning network, an optimal charging and discharging strategy of the energy storage system and the electric vehicle in each of the historical time periods in the target data set, obtain the optimal charging and discharging strategy corresponding to each of the historical time periods in the target data set, and construct a set of optimal charging and discharging strategies corresponding to the target data set based on the optimal charging and discharging strategy corresponding to each of the historical time periods in the target data set, the optimal charging and discharging strategy being the one under which the energy cost consumed by the management object of the user energy management system is the minimum;

[0149] a deep reinforcement training module, configured to input the target data set into a preset deep reinforcement learning network, predict, based on the deep reinforcement learning network, a charging and discharging strategy of the energy storage system and the electric vehicle in each of the historical time periods in the target data set, obtain a predicted charging and discharging strategy corresponding to each of the historical time periods in the target data set, and construct a set of predicted charging and discharging strategies corresponding to the target data set based on the predicted charging and discharging strategy corresponding to each of the historical time periods in the target data set;

[0150] a minimization module, configured to input the set of optimal charging and discharging strategies and the set of predicted charging and discharging strategies corresponding to the target data set into the preset deep reinforcement learning network, minimize, based on the deep reinforcement learning network, an error between the predicted charging and discharging strategy and the optimal charging and discharging strategy corresponding to each of the historical time periods, and obtain a trained hybrid prediction network.

[0151] Optionally, the data set constructing module comprises:

[0152] an acquisition sub-module, configured to acquire a historical net load of a user managed by the user energy management system, historical power state data of the energy storage system and historical power state data of the electric vehicle managed by the user energy management system in each of the historical time periods, and a historical electricity price of each of the historical time periods;

[0153] a first data construction submodule configured to, for each historical period, construct a first historical state vector and a first historical charging and discharging action vector of the energy storage system in the historical period based on historical electricity price, historical net load and historical power state data of the energy storage system of the historical period;

[0154] a second data construction submodule configured to, for each historical period, construct a second historical state vector and a second historical charging and discharging action vector of the electric vehicle in the historical period based on historical electricity price, historical net load and historical power state data of the electric vehicle of the historical period;

[0155] a third data construction submodule configured to construct a target data set based on the first historical state vector and the first historical charging and discharging action vector of the energy storage system and the second historical state vector and the second historical charging and discharging action vector of the electric vehicle in each of the preset number of historical periods.

[0156] Optionally, the first power P t G2U and the second power P t U2G satisfy the constraint:

[0157]

[0158] wherein G2U represents the mode activation state of the grid to the user of the user energy management system in any period t; U2G represents the mode activation state of the user of the user energy management system to the grid in any period t; and are binary variables, represents the upper limit of the interactive power of the grid to the user energy management system; represents the upper limit of the interactive power of the user energy management system to the grid;

[0159] the first charging power P t ESS,ch and the first discharging power P t ESS,dch satisfy the constraint:

[0160]

[0161] wherein, represents the charging mode of the energy storage system; represents the discharging mode of the energy storage system; and are binary variables (0 or 1), respectively; denotes the maximum charging power of the energy storage system; denotes the maximum discharging power of the energy storage system;

[0162] the energy state of the energy storage system in any time period t satisfies the constraint:

[0163]

[0164] wherein, denotes the remaining electric quantity of the energy storage system in time period t; η ESS denotes the charging and discharging efficiency of the energy storage system; Δτ denotes the time step; denotes the remaining electric quantity of the energy storage system in the last time period of time period t; ε ESS denotes the minimum electric quantity of the energy storage system; denotes the maximum electric quantity of the energy storage system; DOC ESS denotes the depth of discharge of the energy storage system;

[0165] the charging power P t EV,ch and the discharging power P t EV,dch satisfy the constraint:

[0166]

[0167] wherein, denotes the charging mode of the electric vehicle; denotes the discharging mode of the electric vehicle; and are binary variables (0 or 1), respectively; denotes the maximum charging power of the electric vehicle; denotes the maximum discharging power of the electric vehicle;

[0168] the energy state of the electric vehicle in any time period t satisfies the constraint:

[0169]

[0170] wherein, denotes the remaining electric quantity of the electric vehicle in time period t; η EV denotes the charging and discharging efficiency of the electric vehicle; Δτ denotes the time step; denotes the remaining electric quantity of the electric vehicle in the last time period of time period t; ε EV denotes the minimum electric quantity of the electric vehicle; denotes the maximum electric quantity of the electric vehicle; DOC EV denotes the depth of discharge of the electric vehicle.

[0171] The energy scheduling apparatus provided by the embodiments of the present disclosure can implement the method of any of the above embodiments, and has similar implementation manners and beneficial effects, which will not be repeated here.

[0172] The embodiments of the present disclosure also provide a computer device, which comprises a processor and a memory, wherein the memory stores a computer program which, when executed by the processor, can implement the method of any of the above embodiments, and has similar implementation manners and beneficial effects, which will not be repeated here.

[0173] The computer device in the embodiments of the present disclosure can be understood as any device with processing and computing capabilities, which can include but is not limited to mobile terminals such as smart phones, notebook computers, personal digital assistants (PDAs), tablet computers (PADs), and the like, and fixed electronic devices such as digital TVs, desktop computers, and the like.

[0174] Figure 4 is a structural schematic diagram of a computer device provided by the embodiments of the present disclosure, as Figure 4 shown, the computer device 400 can include a processor 410 and a memory 420, wherein the memory 420 stores a computer program 421, which, when executed by the processor 410, can implement the method provided by any of the above embodiments, and has similar implementation manners and beneficial effects, which will not be repeated here.

[0175] Of course, in order to simplify, Figure 4 only some of the components related to the present disclosure in the computer device 400 are shown, and components such as buses, input / output interfaces, input devices and output devices are omitted. In addition, according to specific application conditions, the computer device 400 can also include any other appropriate components.

[0176] The embodiments of the present disclosure provide a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the method of any of the above embodiments can be implemented, and has similar implementation manners and beneficial effects, which will not be repeated here.

[0177] The computer readable storage medium can be a combination of one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0178] The computer program can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer device, partly on the user's device, as a stand-alone software package, partly on the user's computer device and partly on a remote computer device or entirely on the remote computer device or server. The computer program can be downloaded to the user's computer device from an external computer device or server via a communication interface.

[0179] The above description merely provides an overview of the exemplary embodiments and inventive concepts of the present disclosure and is not intended to limit the scope of the disclosure. Accordingly, the disclosure is not limited to the specific embodiments described above, but only by the claims. It should be understood that the present disclosure includes any alterations and modifications to the specific embodiments set forth in the above description within the scope of the present disclosure and within the scope of equivalents of the claims. For example, the features of the above-described embodiments can be combined with other features disclosed in the present disclosure (but not limited to) to form a technical solution.

[0180] Furthermore, while operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order depicted. On the contrary, many of the operations can be performed in a different order or concurrently with one another. In addition, while an exemplary method implemented by a computer is described with respect to the exemplary embodiments, steps of the method can be performed at least in part by one or more computers. Also, it will be appreciated that specific internal hardware like Application Specific Integrated Circuits (ASICs) can be used to carry out the operations of the method. Similarly, it will be appreciated that the functionality of the method, as described above, can be split into multiple parts or combined. In addition, the functionality can be performed using virtual or physical components and data can be stored in virtual or physical storage.

[0181] The foregoing is merely illustrative of the various implementations of the present disclosure and the general principles thereof. Numerous modifications can be made to these illustrations, and equivalents can be substituted therefor, without departing from the scope of the present disclosure. The specific embodiments commensurate with the specific application are intended to be illustrative only and not limiting of the scope of the application as set forth in the following claims.

Claims

1. An energy scheduling method, characterized in that: include: Obtaining the net load of users managed by the user energy management system in the current period, power status data of the energy storage system and electric vehicle managed by the user energy management system in the current period, and the electricity price in the current period; constructing a first state vector and a first charge-discharge action vector of the energy storage system in the current time period based on the electricity price, the net load, and the power state data of the energy storage system; constructing a second state vector and a second charge-discharge action vector of the electric vehicle in the current time period based on the electricity price, the net load, and the power state data of the electric vehicle; A preset hybrid prediction network predicts, based on the first state vector, the first charge-discharge action vector, the second state vector, and the second charge-discharge action vector, a target charge-discharge strategy corresponding to the energy storage system and the electric vehicle in a target period next to the current period, wherein the energy cost consumed by the managed object of the user energy management system is minimized under the target charge-discharge strategy; The charging and discharging actions of the energy storage system and the electric vehicle are controlled based on a target charging and discharging strategy within a target period.

2. The method according to claim 1, characterized in that The management objects of the user energy management system include energy storage systems, electric vehicles, solar photovoltaic systems, and users' home appliances; The obtaining of the net load of the user managed by the user energy management system in the current period includes: Obtaining historical energy consumption power of the household appliance during a preset period before the current period; Predicting the energy consumption power of the household appliance in a current period based on the historical energy consumption power to obtain a target energy consumption power of the household appliance in the current period; Calculating a target power generation of the solar photovoltaic system in the current period based on the conversion efficiency and maximum power of the solar photovoltaic system and the solar irradiance in the current period; The difference between the target energy consumption power and the target power generation power is calculated to obtain the net load of the user managed by the user energy management system in the current time period.

3. The method according to claim 1, characterized in that The power status data of the energy storage system includes a first charging power and a first discharging power of the energy storage system in the current time period and a first remaining power of the energy storage system in the previous time period of the current time period; The constructing, based on the electricity price, the net load, and the power state data of the energy storage system, a first state vector and a first charge-discharge action vector of the energy storage system in the current time period includes: constructing a state vector of the energy storage system in the current time period as a first state vector based on the electricity price, the net load, and the first remaining power; Based on the first charging power and the first discharging power, a charging and discharging action vector of the energy storage system in the current time period is constructed as a first charging and discharging action vector.

4. The method according to claim 1, wherein The power status data of the electric vehicle includes the second charging power and the second discharging power of the electric vehicle in the current time period and the second remaining power of the electric vehicle in the previous time period of the current time period; The constructing, based on the electricity price, the net load and the power state data of the electric vehicle, a second state vector and a second charge-discharge action vector of the electric vehicle in the current time period includes: constructing a second state vector of the electric vehicle in the current time period based on the electricity price, the net load, and the second remaining power; Based on the second charging power and the second discharging power, a charging and discharging action vector of the electric vehicle in the current time period is constructed as a second charging and discharging action vector.

5. The method according to claim 1, wherein Before the preset hybrid prediction network predicts target charge and discharge strategies corresponding to the energy storage system and the electric vehicle within a target period next to the current period based on the first state vector, the first charge and discharge action vector, the second state vector, and the second charge and discharge action vector, the method further includes: Constructing a target data set, the target data set including a first historical state vector and a first historical charge-discharge action vector of the energy storage system and a second historical state vector and a second historical charge-discharge action vector of the electric vehicle in each historical period of a preset number of historical periods; Inputting the target data set into a preset supervised learning network, solving the optimal charging and discharging strategies corresponding to the energy storage system and the electric vehicle in each historical period in the target data set based on the supervised learning network, obtaining the optimal charging and discharging strategies corresponding to each historical period in the target data set, and constructing an optimal charging and discharging strategy set corresponding to the target data set based on the optimal charging and discharging strategies corresponding to each historical period in the target data set, wherein the energy cost consumed by the management objects of the user energy management system is minimized under the optimal charging and discharging strategies; Inputting the target data set into a preset deep reinforcement learning network, predicting the charging and discharging strategies corresponding to the energy storage system and the electric vehicle in each historical period in the target data set based on the deep reinforcement learning network, obtaining the predicted charging and discharging strategies corresponding to each historical period in the target data set, and constructing a predicted charging and discharging strategy set corresponding to the target data set based on the predicted charging and discharging strategies corresponding to each historical period in the target data set; The optimal charge and discharge strategy set and the predicted charge and discharge strategy set corresponding to the target data set are input into a preset deep reinforcement learning network. Based on the deep reinforcement learning network, the error between the predicted charge and discharge strategy corresponding to each historical period and the optimal charge and discharge strategy is minimized to obtain a trained hybrid prediction network.

6. The method according to claim 5, characterized in that The constructing of the target data set includes: Obtaining historical net loads of users managed by the user energy management system in each of a preset number of historical time periods, historical power status data of energy storage systems and electric vehicles managed by the user energy management system in each of the historical time periods, and historical electricity prices in each of the historical time periods; For each historical period, constructing a first historical state vector and a first historical charge-discharge action vector of the energy storage system in the historical period based on the historical electricity price, the historical net load, and the historical power state data of the energy storage system; For each historical period, constructing a second historical state vector and a second historical charge and discharge action vector of the electric vehicle in the historical period based on the historical electricity price, the historical net load and the historical power state data of the electric vehicle in the historical period; A target data set is constructed based on the first historical state vector and the first historical charge and discharge action vector of the energy storage system and the second historical state vector and the second historical charge and discharge action vector of the electric vehicle in each historical period of a preset number of historical periods.

7. The method according to any one of claims 1 to 6, characterized in that Also includes: The first power P from the power grid to the user managed by the user energy management system in any period t t G2U and the second power P from the user's energy management system to the grid t U2G Satisfy the constraints: Among them, G2U represents the activation state of the mode from the grid to the user managed by the user energy management system in any period t; U2G represents the activation state of the mode from the user managed by the user energy management system to the grid in any period t; and It is a binary variable. Indicates the upper limit of the interaction power from the grid to the user's energy management system; Indicates the upper limit of the interaction power between the user's energy management system and the power grid; The first charging power P of the energy storage system in any time period t t ESS,ch and the first discharge power P of the energy storage system t ESS,dch Satisfy the constraints: in, Indicates the charging mode of the energy storage system; Indicates the discharge mode of the energy storage system; and are binary variables (0 or 1); Indicates the maximum charging power of the energy storage system; Indicates the maximum discharge power of the energy storage system; The energy state of the energy storage system in any period t satisfies the constraints: in, Represents the remaining capacity of the energy storage system within time period t; η ESS represents the charging and discharging efficiency of the energy storage system; Δτ represents the time step; Represents the remaining capacity of the energy storage system in the previous period of time t; ε ESS Indicates the minimum capacity of the energy storage system; Indicates the maximum capacity of the energy storage system; DOC ESS Indicates the discharge depth of the energy storage system; The charging power P of the electric vehicle in any period t t EV,ch and the discharge power P of the electric vehicle t EV,dch Satisfy the constraints: in, Indicates the charging mode of the electric vehicle; Indicates the discharge mode of the electric vehicle; and are binary variables (0 or 1); Indicates the maximum charging power of the electric vehicle; Indicates the maximum discharge power of the electric vehicle; The energy state of the electric vehicle in any period t satisfies the constraints: in, Represents the remaining power of the electric vehicle in time period t; η EV represents the charging and discharging efficiency of the electric vehicle; Δτ represents the time step; Indicates the remaining power of the electric vehicle in the previous period of time t; ε EV Indicates the minimum power of the electric vehicle; Indicates the maximum power of the electric vehicle; DOC EV Indicates the discharge depth of the electric vehicle.

8. An energy scheduling device, characterized in that: include: An acquisition module, configured to acquire the net load of users managed by the user energy management system in the current period, the power status data of the energy storage system and the electric vehicle managed by the user energy management system in the current period, and the electricity price in the current period; A first construction module is configured to construct a first state vector and a first charge-discharge action vector of the energy storage system in the current time period based on the electricity price, the net load, and the power state data of the energy storage system; a second constructing module, configured to construct a second state vector and a second charge-discharge action vector of the electric vehicle in the current time period based on the electricity price, the net load, and the power state data of the electric vehicle; a prediction module, configured to predict, via a preset hybrid prediction network, a target charge and discharge strategy corresponding to the energy storage system and the electric vehicle within a target period next to the current period based on the first state vector, the first charge and discharge action vector, the second state vector, and the second charge and discharge action vector, wherein the target charge and discharge strategy minimizes the energy cost consumed by the managed objects of the user energy management system; A control module is used to control the charging and discharging actions of the energy storage system and the electric vehicle based on a target charging and discharging strategy within a target time period.

9. A computer device, characterized in that: include: A memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the energy scheduling method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the energy scheduling method according to any one of claims 1 to 7 is implemented.

Images