Charging and discharging behavior decision-making method, system, equipment and medium

Abstract

The invention provides a charging and discharging behavior decision-making method and system, electronic equipment and a readable storage medium, and relates to the field of energy storage system optimization configuration, and the method comprises the steps: obtaining basic information of a household energy storage body; the basic information comprises current charging and discharging power and current state information; inputting the basic information into an energy storage behavior prediction model, and predicting charging and discharging behaviors of the energy storage body; wherein single-step income information is determined according to the current charging and discharging power and the current state information; generating expected revenue information based on the single-step revenue information and the revenue strategy; and in combination with the current charging and discharging power, the current state information, the expected income information and the charging and discharging adjustment strategy, a new energy storage execution result is predicted and used for executing the next charging and discharging behavior, effective utilization and storage of energy are realized, and the intelligent level of household energy storage is improved.

Description

Technical Field

[0001] The present invention relates to the field of energy storage system optimization configuration, and in particular to a charging and discharging behavior decision-making method, system, electronic device and readable storage medium. Background Art

[0002] With the rapid development of the global economy, the demand for electricity continues to rise. However, this trend has exacerbated the global energy crisis and global warming. In order to achieve sustainable energy development, the development and application of renewable energy, such as solar energy and wind power, are becoming more and more extensive. At the same time, energy storage systems have gradually become an effective means to solve energy storage problems. However, the existing home energy storage systems are mainly battery energy storage and lack intelligent management strategies. Therefore, there is still a lot of room for improvement in the process of using energy storage systems to optimize household electricity consumption.

[0003] Therefore, it is necessary to propose a decision-making method, system, electronic device and readable storage medium for charging and discharging behavior. Summary of the invention

[0004] This specification provides a decision-making method, system, electronic device and readable storage medium for charging and discharging behavior. By obtaining the current charging and discharging power and state information of a household energy storage body and inputting it into the energy storage behavior prediction model, this method can accurately predict the charging and discharging behavior of the energy storage body. This process not only takes into account the current charging and discharging efficiency and state, but also generates expected profit information by combining single-step profit information with profit strategy, providing a more comprehensive decision-making basis for the prediction model. Combined with the current state, power, expected profit and charging and discharging adjustment strategy, it is possible to predict and optimize the execution results of the energy storage body, thereby guiding the next charging and discharging behavior, achieving effective utilization and storage of energy, and improving the intelligence level of household energy storage.

[0005] A charging and discharging behavior decision method provided in the present application adopts the following technical solution, including:

[0006] Obtaining basic information of the household energy storage body; the basic information includes: current charging and discharging power and current status information;

[0007] Inputting the basic information into an energy storage behavior prediction model to predict the charge and discharge behavior of the energy storage body;

[0008] Optionally, the basic information is input into an energy storage behavior prediction model to predict the charging and discharging behavior of the energy storage body, including: determining single-step profit information based on the current charging and discharging power and the current state information; generating expected profit information based on the single-step profit information and the profit strategy; and predicting a new energy storage execution result in combination with the current charging and discharging power, the current state information, the expected profit information, and the charging and discharging adjustment strategy for executing the next charging and discharging behavior.

[0009] Optionally, the obtaining of basic information of the household energy storage body includes:

[0010] Execute the previous energy storage execution strategy to obtain the current charging and discharging behavior;

[0011] Determine the current charge and discharge power by combining the current charge and discharge behavior and the rate coefficient;

[0012] The current status information is collected, and the current status information includes: current photovoltaic power generation power, current household load power, and current electricity price.

[0013] Optionally, the determining the current charge and discharge power by combining the current charge and discharge behavior and the rate coefficient includes:

[0014] Determining action information according to the current charging and discharging behavior;

[0015] The current charging and discharging power is obtained by combining the action information and the rate coefficient.

[0016] Optionally, generating single-step benefit information according to the current charging and discharging power and the current state information includes:

[0017] Combining the current photovoltaic power generation, the current household load power and the current charging and discharging power, generating a current docking power;

[0018] Determining a current actual price according to the current connection power, the current electricity price and the preset judgment condition;

[0019] The single-step revenue information is obtained by combining the current connection power, the current actual price and the unit power.

[0020] Optionally, it also includes: verifying the energy storage execution result based on the constraint condition;

[0021] The verifying the energy storage execution result based on the constraint condition includes:

[0022] Extracting a target charge and discharge rate from the energy storage execution result;

[0023] Determining whether the target charge and discharge rate meets the rigid constraint condition to obtain a first determination result;

[0024] Determining whether the target charge and discharge rate meets the auxiliary constraint condition to obtain a second determination result;

[0025] The first judgment result and the second judgment result are combined to generate a verification result of the energy storage execution result.

[0026] Optionally, the determining whether the target charge and discharge rate meets the rigid constraint condition to obtain a first determination result includes:

[0027] Determine whether the target charge and discharge power is less than or equal to the maximum discharge rate, and if so, determine that the target charge and discharge power meets the first constraint sub-condition;

[0028] Determine whether the target charge and discharge power is less than or equal to the maximum charge rate, and if so, determine that the target charge and discharge power meets the second constraint sub-condition;

[0029] Determine whether the target charge and discharge power is greater than or equal to the minimum charge power, and if so, determine that the target charge and discharge power meets the third constraint sub-condition;

[0030] It is determined whether the target charge and discharge power is greater than or equal to the maximum discharge power. If so, it is determined that the target charge and discharge power meets the fourth constraint sub-condition.

[0031] Optionally, the determining whether the target charge and discharge rate meets the auxiliary constraint condition to obtain a second determination result includes:

[0032] Determine whether the target charge and discharge power is within a preset load range.

[0033] The present application provides a charging and discharging behavior decision system that adopts the following technical solutions, including:

[0034] An information acquisition module is used to acquire basic information of the household energy storage body; the basic information includes: current charging and discharging power and current status information;

[0035] A prediction module, used for inputting the basic information into an energy storage behavior prediction model to predict the charging and discharging behavior of the energy storage body;

[0036] The prediction module comprises:

[0037] A single-step benefit determination submodule, used to determine single-step benefit information according to the current charge and discharge power and the current state information;

[0038] An expected return determination submodule, used to generate expected return information based on the single-step return information and return strategy;

[0039] The prediction submodule is used to combine the current charging and discharging power, the current state information, the expected benefit information and the charging and discharging adjustment strategy to predict a new energy storage execution result for executing the next charging and discharging behavior.

[0040] The prediction module comprises:

[0041] A single-step benefit determination submodule, used to determine single-step benefit information according to the current charge and discharge power and the current state information;

[0042] An expected return determination submodule, used to generate expected return information based on the single-step return information and return strategy;

[0043] The prediction submodule is used to combine the current charging and discharging power, the current state information, the expected benefit information and the charging and discharging adjustment strategy to predict a new energy storage execution result for executing the next charging and discharging behavior.

[0044] Optionally, the information acquisition module includes:

[0045] The first acquisition submodule is used to execute the previous energy storage execution strategy and obtain the current charging and discharging behavior;

[0046] A charge and discharge power determination submodule, used to determine the current charge and discharge power in combination with the current charge and discharge behavior and the rate coefficient;

[0047] The collection submodule is used to collect the current status information, and the current status information includes: current photovoltaic power generation power, current household load power, and current electricity price.

[0048] Optionally, the charging and discharging power determination submodule includes:

[0049] an action information determining unit, configured to determine action information according to the current charging and discharging behavior;

[0050] The charge and discharge power determination unit is used to obtain the current charge and discharge power by combining the action information and the rate coefficient.

[0051] Optional, single-step profit determination submodule, including:

[0052] A first determining unit, configured to generate a current docking power by combining the current photovoltaic power generation power, the current household load power and the current charging and discharging power;

[0053] A second determining unit, configured to determine a current actual price according to the current connection power, the current electricity price and the preset judgment condition;

[0054] The third determining unit is used to obtain the single-step revenue information by combining the current docking power, the current actual price and the unit power.

[0055] Optionally, it also includes: a verification module, used to verify the energy storage execution result based on the constraint condition;

[0056] Optionally, the verification module includes:

[0057] An extraction submodule, used to extract a target charge and discharge rate from the energy storage execution result;

[0058] A first judgment submodule, used to judge whether the target charge and discharge rate meets the rigid constraint condition, and obtain a first judgment result;

[0059] A second judgment submodule is used to judge whether the target charge and discharge rate meets the auxiliary constraint condition and obtain a second judgment result;

[0060] A verification submodule is used to generate a verification result of the energy storage execution result by combining the first judgment result and the second judgment result.

[0061] Optionally, the first judgment submodule includes:

[0062] A first judgment unit, used to judge whether the target charge and discharge power is less than or equal to the maximum discharge rate, and if so, determine that the target charge and discharge power meets the first constraint sub-condition;

[0063] A second judgment unit is used to judge whether the target charge and discharge power is less than or equal to the maximum charging rate, and if so, it is determined that the target charge and discharge power meets the second constraint sub-condition;

[0064] A third judgment unit, used to judge whether the target charge and discharge power is greater than or equal to the minimum charge power, and if so, determine that the target charge and discharge power meets the third constraint sub-condition;

[0065] The fourth judgment unit is used to judge whether the target charge and discharge power is greater than or equal to the maximum discharge power. If so, it is determined that the target charge and discharge power meets the fourth constraint sub-condition.

[0066] Optionally, the second judgment submodule includes:

[0067] The fifth judgment unit is used to judge whether the target charge and discharge power is within a preset load range.

[0068] This specification also provides an electronic device, wherein the electronic device includes:

[0069] processor; and,

[0070] A memory storing computer executable instructions, which when executed cause the processor to perform any of the above methods.

[0071] The present specification also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, any of the above methods is implemented.

[0072] In the present application, basic information of a household energy storage body is obtained; the basic information includes: current charging and discharging power and current state information; the basic information is input into an energy storage behavior prediction model to predict the charging and discharging behavior of the energy storage body; wherein, single-step profit information is determined according to the current charging and discharging power and the current state information; based on the single-step profit information and the profit strategy, expected profit information is generated; in combination with the current charging and discharging power, the current state information, the expected profit information and the charging and discharging adjustment strategy, a new energy storage execution result is predicted for executing the next charging and discharging behavior, thereby realizing effective utilization and storage of energy and improving the intelligence level of household energy storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of the principle of a charging and discharging behavior decision method provided in an embodiment of this specification;

[0074] Figure 2 A schematic diagram of a model training method for a decision-making method of charging and discharging behavior provided in an embodiment of this specification;

[0075] Figure 3 A schematic diagram of a model update of a charging and discharging behavior decision method provided in an embodiment of this specification;

[0076] Figure 4 A schematic diagram of the structure of a charging and discharging behavior decision system provided in an embodiment of this specification;

[0077] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0078] Figure 6 A schematic diagram of a computer-readable medium provided for an embodiment of this specification. DETAILED DESCRIPTION

[0079] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0080] Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in a variety of forms, and should not be construed as limiting the present invention to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, and is more convenient for fully conveying the inventive concept to those skilled in the art. The same reference numerals in the figures represent the same or similar elements, components or parts, and thus their repeated description will be omitted.

[0081] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0082] In the description of specific embodiments, the features, structures, characteristics or other details described in the present invention are intended to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics or other details.

[0083] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0084] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0085] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0086] Figure 1 A schematic diagram of a charging and discharging behavior decision method provided in an embodiment of this specification, the method comprising:

[0087] S1 obtains basic information of the household energy storage body; the basic information includes: current charging and discharging power and current status information;

[0088] S2 inputs the basic information into the energy storage behavior prediction model to predict the charging and discharging behavior of the energy storage body; wherein, according to the current charging and discharging power and the current state information, the single-step profit information is determined; based on the single-step profit information and the profit strategy, the expected profit information is generated; in combination with the current charging and discharging power, the current state information, the expected profit information and the charging and discharging adjustment strategy, a new energy storage execution result is predicted for executing the next charging and discharging behavior.

[0089] In the smart home energy management system, home energy storage planning is a key task. The main goal of this task is to use energy storage devices to participate in regulating home electricity consumption behavior, while meeting home electricity demand and reducing home electricity costs as much as possible, and maximizing benefits when conditions permit.

[0090] Existing energy storage system planning mainly involves linear programming, evolutionary algorithms and reinforcement learning technology, which mainly have the following problems:

[0091] (1) Linear programming methods are suitable for dealing with simple resource allocation and scheduling problems, but their applicability to nonlinear or complex problems is limited. Complex mathematical derivations are required to transform nonlinear conditions, and the reasoning is complex and the process is prone to errors.

[0092] (2) Evolutionary algorithms are suitable for dealing with nonlinear and complex multidimensional problems, but they usually require a long running time to solve the calculations and are insufficient in summarizing existing data and experience. As the scale of the problem continues to increase, the effect and solution time will obviously deteriorate.

[0093] (3) Reinforcement learning methods require a large amount of training data and computing resources, have high requirements on the accuracy of the environmental model, and have the disadvantage of unstable convergence effect.

[0094] Based on this, in order to improve the optimization efficiency and optimization effect, the present invention provides a decision method for charging and discharging behavior, which specifically includes:

[0095] S1 obtains basic information of household energy storage;

[0096] The basic information includes: current charging and discharging power and current status information;

[0097] S11 executes the previous energy storage execution strategy to obtain the current charging and discharging behavior;

[0098] S12 determines the current charge and discharge power in combination with the current charge and discharge behavior and the rate coefficient;

[0099] S121 pre-constructs the action space;

[0100] The action space (Action) is a continuous space that stores the action information (i.e., action value) at each moment. The range of the action information a in the action space is [-1, 1]. t Represents the direction and intensity of the charge and discharge behavior at time t.

[0101] Specifically, when the action information is a positive number, that is, 0<a≤1, it indicates charging behavior, and the larger the value, the higher the charging intensity; when the action information is a negative number, that is, -1≤a<0, it indicates discharging behavior, and the closer the value is to -1, the higher the discharge intensity; when the action information is 0, that is, a=0, it indicates a static state, without charging and discharging. S122 determines the action information according to the current charging and discharging behavior;

[0102] In one embodiment of the present specification, the charging and discharging behavior of the household energy storage body at the current time t is obtained in real time; based on the current charging and discharging behavior at the current time t, the action information corresponding to the current time t is extracted from the action space. The charging and discharging behavior refers to the charging and discharging state of the household energy storage body, such as the charging state, the discharging state, and the static state. .

[0103] In another embodiment of the present specification, the charging and discharging behavior is acquired through a monitoring system or interface to obtain the action information at the current moment. It is necessary to standardize and convert the output data provided by the monitoring system or interface in advance so as to appropriately adjust and extract the action information according to the output data of the monitoring system or interface.

[0104] In yet another embodiment of the present specification, based on the last energy storage execution strategy, the action information corresponding to the current time t is extracted from the action space.

[0105] S123 combines the action information and the rate coefficient to obtain the current charge and discharge power.

[0106] Combined with the current action information a t Determine the charge and discharge state, and select the rate coefficient rate at the current moment based on the charge and discharge state;

[0107] In one embodiment of the present specification, a corresponding maximum charge and discharge rate is selected as the rate coefficient rate according to the charge and discharge state.

[0108] Specifically, when the action information is a positive number (indicates the charging state), the maximum charging rate ChargePower is used as the rate coefficient rate.

[0109] When the action information is negative (indicated as a discharge state), the maximum discharge rate DischargePower is used as the rate coefficient rat e .

[0110] Combined with the current action information a t and rate coefficient rate, determine the current charging and discharging power battery t , as the current charging and discharging power.

[0111] Charge and discharge power battery t Refers to the charging power and / or discharging power of the battery of the household energy storage body. t =a t *rate; where, where, t=1…N.

[0112] For example, when the action information is 0.68, the corresponding rate coefficient rate is ChargePower.

[0113] The charging and discharging power battery = 0.68 × ChargePower, which is 0.68 times the maximum charging power.

[0114] S13 collects current status information;

[0115] S131 pre-constructs the state space;

[0116] The state space (State) includes: remaining battery power (SoC), load power (load) in the next 24 hours, photovoltaic power generation power pv in the next 24 hours, electricity purchase price buy in the next 24 hours, and electricity selling price sell in the next 24 hours.

[0117] That is, State: [SoC, load 1 , ..., load N , pv 1 , ..., pv N , buy 1 , ..., buy N , sell 1 ,...,sell N ];

[0118] Among them, photovoltaic power is affected by historical power generation power, historical weather and future weather.

[0119] S132 collects the current state information;

[0120] The status information includes: photovoltaic power generation, household load power, and electricity price. The electricity price includes: electricity purchase price and electricity selling price.

[0121] Therefore, the current status information includes: current photovoltaic power generation, current household load power, and current electricity price.

[0122] Among them, the current photovoltaic power generation power refers to the photovoltaic power generation power pv at the current time t t ; The current household load power refers to the household load power load at the current time t t ; The current electricity price includes: the electricity purchase price at the current time t Buy t 、The electricity selling price at the current time t is Sell t .

[0123] S2 inputs the basic information into an energy storage behavior prediction model to predict the charging and discharging behavior of the energy storage body;

[0124] S21 builds a prediction model for energy storage behavior;

[0125] Specifically, a storage energy behavior prediction model is constructed through reinforcement learning algorithm, deep reinforcement learning algorithm, deep Q learning algorithm, etc.

[0126] In one embodiment of the present specification, the present invention adopts the SAC method to calculate the energy storage planning strategy, which performs well in dealing with continuous and high-dimensional action spaces in energy storage systems. Compared with traditional reinforcement learning algorithms, SAC has faster convergence speed and better stability, can obtain optimization results in a shorter time, and can maintain good performance. In addition, SAC also has a soft policy update mechanism that can strike a balance between exploring unknown areas and using known information, thereby overcoming the contradiction between exploration and decision-making.

[0127] S22 constructs a training sample set to train the energy storage behavior prediction model;

[0128] S221 constructs a training sample set.

[0129] The process of energy storage scheduling needs to consider the influence of various factors, including but not limited to electricity trading prices, weather characteristics, user behavior, power information of other controllable / uncontrollable loads, etc. In one embodiment of this specification, it supports setting a flag to indicate whether the photovoltaic panel is reversely connected to the grid, and the flag is affected by the selling price. In the negative value state, the photovoltaic panel is prohibited from reversely connecting to the grid. The storage cost of this section of electricity within the coverage of the power generation capacity is 0, and the selling cost of electricity is the selling price at that time.

[0130] Therefore, in order to improve the optimization effect of the energy storage behavior prediction model, in the process of generating the energy storage scheduling strategy, it is necessary to take the above information into consideration and formulate the best energy storage scheduling strategy to meet the needs of the family and thus improve the user experience.

[0131] Obtain some historical data of household energy storage as training samples.

[0132] The historical data includes: multiple historical states and multiple historical behaviors of the household energy storage body in the historical period. The historical states include: the load power, photovoltaic power generation power, and electricity price of the household energy storage body in the historical period. The electricity price includes: the electricity purchase price and / or the grid price.

[0133] The historical actions include: the charging behavior and / or discharging behavior of the household energy storage body in the historical period.

[0134] Preferably, the historical data includes influencing factors, including timestamps, weather conditions, and holidays.

[0135] Clean the historical data to remove outliers and missing values. Standardize or normalize the historical data to ensure the stability and accuracy of subsequent model training.

[0136] Each historical data is used as a training sample, and all training samples are summarized to obtain a training sample set for model training.

[0137] S222 trains the energy storage behavior prediction model in combination with the training sample set.

[0138] In one embodiment of the present specification, a quaternary group (S, A, r, S') is constructed according to the training sample, and each quaternary group contains a value at a time, that is:

[0139] "S" indicates the status information at the current moment;

[0140] “A” represents the action taken in the current state, i.e., the charging or discharging behavior of the energy storage body;

[0141] “R” represents the immediate reward calculated according to the reward function;

[0142] "S"' indicates the state information at the next moment.

[0143] The following is a brief description of the training process:

[0144] like Figure 2 As shown, the present invention performs environmental modeling and intelligent agent design, that is, the environment (Environment) is determined to include load power, photovoltaic power generation power, and electricity price. The intelligent agent (agent) is a household energy storage body, and the action is to charge / discharge the household energy storage body. After the historical data is collected, the strategy network is updated to perform soft strategy updates.

[0145] In one embodiment of the present specification, an initial state is randomly selected before the algorithm starts, and then the execution action is selected based on this state. In the early stage, the action is basically randomly selected. After the action selection is completed, the agent will perform this action (A) in the environment (Environment), and then the environment (Environment) will return the next state (S') and the current single-step profit (R). At this time, the four-tuple (S, A, r, S') is stored in the experience playback buffer D; next, the next state (S') is regarded as the current state (S), and the above steps are repeated until the maximum number of iterations is reached or the model converges. After the training is completed, the model is evaluated and tuned.

[0146] S23, determining single-step benefit information according to the current charging and discharging power and the current state information;

[0147] S231 generates a current connection power by combining the current photovoltaic power generation power, the current household load power and the current charging and discharging power;

[0148] Combined with the photovoltaic power generation power pv at the current time t t , the household load power load at the current time t t 、The charge and discharge power battery at the current time t t , get the docking power GridPower at the current time t t , as the current docking power.

[0149] That is, GridPower t =pv t -load t +battery t .

[0150] S232: determining a current actual price according to the current connection power, the current electricity price and the preset judgment condition;

[0151] According to the docking power GridPower at the current time t t And the preset judgment conditions, predict the actual price Price at the current time t t .

[0152] The preset judgment conditions include:

[0153] Among them, when the docking power GridPower at the current time t t When <0, it means that electricity needs to be purchased from the power grid. At this time, the actual price at the current time t is Price t The electricity purchase price at the current time t is Buy t .

[0154] When the docking power GridPower at the current time t t When ≥0, it means that the excess electricity needs to be sold to the grid. At this time, the actual price at the current time t is t The electricity selling price Sell at the current time t t

[0155] S233 combines the current docking power, the current actual price and the unit power to obtain the single-step profit information.

[0156] Specifically, the docking power GridPower at the current time t t 、Actual Price t The product of the unit power unit is used to generate the single-step profit r(s) at the current time t. t , a t ), that is, in state s t Next, perform action a t Instant rewards.

[0157] That is, r(s t , a t )=GridPower t ×Price t ×unit;

[0158] At each time step, the present invention calculates the actual charging and discharging power according to the current state and action value, and applies it to the building energy system. At the same time, it calculates the connection power with the mains by comparing with the household load power, so as to determine the predicted value of the electricity purchase / sale price at this time as the actual price, which is used to calculate the single-step profit, and finally the final profit / cost is obtained by summarizing the N-step profits.

[0159] The present invention has a clear definition of the state space and action space. When calculating the electricity cost and benefit, the changes in the mains electricity caused by multiple factors including photovoltaics and loads are taken into account, and the final calculation is based on the mains electricity information.

[0160] S24 generates expected profit information based on the single-step profit information and profit strategy;

[0161] Expected return information Q π is the state action value function, which is used to determine the state of the state s at the current time t. t Next, perform action a t expected return information.

[0162] In one embodiment of the present specification, the revenue strategy includes:

[0163]

[0164] Among them, p is the transition probability, which describes the transition from the current state s t To the next state s t+1 γ is the discount factor, which is used to control the current value of future rewards. α is the temperature parameter, which is used to balance the maximization of benefits and the maximization of strategy entropy. t , a t ) means in state s t Next, perform action a t The single-step profit.

[0165] S25 combines the current charging and discharging power, the current state information, the expected benefit information and the charging and discharging adjustment strategy to predict a new energy storage execution result for executing the next charging and discharging behavior.

[0166] The charge and discharge adjustment strategies include:

[0167]

[0168] Among them, J(π) represents the objective function of the charge and discharge adjustment strategy, that is, adjusting the next energy storage execution plan by maximizing the profit;

[0169] D stands for experience replay buffer; a t represents the action at time t, s t represents the state at time t; Q π represents expected return information; Q π (s t , a t ) means that in state s t Next, perform action a t The expected return after that; H stands for entropy, which indicates the uncertainty of the strategy.

[0170] In the process of generating the execution plan, the present invention adopts the SAC method to better adapt the continuous action space, and improves its performance in higher dimensions by effectively utilizing historical samples. The charge and discharge adjustment strategy can effectively deal with the uncertainty of the environment by utilizing the characteristics of maximum entropy. The charge and discharge adjustment strategy based on the SAC method performs prediction planning through a priori models, which is more efficient.

[0171] S26 verifies the energy storage execution plan based on the constraint conditions;

[0172] Constraints include: rigid constraints and auxiliary constraints.

[0173] S261 extracts a target charge and discharge rate from the energy storage execution result;

[0174] The energy storage execution plan includes: the next target charging and discharging behavior.

[0175] Based on the target charging and discharging behavior, the corresponding target action information is matched to determine the corresponding target rate coefficient;

[0176] The target charge and discharge rate is obtained by combining the product of the target action information and the target rate coefficient.

[0177] S262 determines whether the target charge and discharge rate meets the rigid constraint condition, and obtains a first determination result;

[0178] Rigid constraints include: several constraint sub-conditions.

[0179] In one embodiment of the present specification, the constraint condition includes: a first constraint sub-condition, a second constraint sub-condition, a third constraint sub-condition and a fourth constraint sub-condition.

[0180] The first constraint sub-condition: target charge and discharge power (discharge rate) ≤ maximum discharge rate.

[0181] That is, Action*rate≥-DischargePower.

[0182] Among them, Action*rate represents the charging and discharging power corresponding to the target action information.

[0183] Negative values ​​indicate discharge. Therefore, the first constraint indicates that when the household energy storage is discharging, the absolute value of the product of the rate coefficient rate and the current action Action does not exceed the maximum discharge power -DischargePower allowed by its design.

[0184] Second constraint sub-condition: target charge and discharge power (charging rate) ≤ maximum charging rate.

[0185] That is, Action*rate≤ChargePower.

[0186] A positive value indicates charging. Therefore, the second constraint condition is that when the household energy storage body is charging, the absolute value of the product of the rate coefficient rate and the current action Action does not exceed the maximum discharge power ChargePower allowed by its design.

[0187] The third constraint sub-condition: target charging and discharging power (charging rate) ≥ minimum charging power.

[0188] Right now,

[0189] Among them, the minimum charging rate is SoC indicates the remaining battery power. max Indicates the maximum battery power, Capacity indicates the battery status, and unit indicates the unit time.

[0190] Through the above steps, the minimum charging power required to achieve the maximum charge of the battery is obtained.

[0191] When the remaining battery capacity SoC of the household energy storage body is lower than the maximum battery capacity SoC max Therefore, when increasing the power, in order to increase the power to the maximum, the household energy storage body needs to be charged at a rate not less than the minimum charging rate.

[0192] Fourth constraint sub-condition: target charge and discharge power (discharge rate) ≥ maximum discharge power.

[0193] Right now,

[0194] The maximum discharge rate is SoC indicates the remaining battery power. min Indicates the minimum battery power, Capacity indicates the battery status, and unit indicates the unit time.

[0195] Through the above steps, the maximum discharge power required to achieve the minimum battery power is obtained.

[0196] When the remaining battery power SoC of the household energy storage body is higher than the minimum battery power SoC min Therefore, when reducing the power, in order to avoid the power being lower than the negative value of the maximum discharge power required for the minimum power (discharge is negative power), the household energy storage body needs to be charged at a power not higher than the maximum discharge power.

[0197] Through the above constraints, the boundaries that the energy storage agent must abide by when performing charging or discharging operations are determined, so that the future charging and discharging power is within the preset upper and lower power limits, and the effective rechargeable power is within the upper and lower power limits of the battery, preventing the action from crossing the boundary, while ensuring that the operation will neither exceed its physical limitations nor cause the power to be too low or too high.

[0198] In one embodiment of the present invention, it is determined whether the target charge and discharge power is less than or equal to the maximum discharge rate. If so, it is determined that the target charge and discharge power meets the first constraint sub-condition; it is determined whether the target charge and discharge power is less than or equal to the maximum charging rate. If so, it is determined that the target charge and discharge power meets the second constraint sub-condition; it is determined whether the target charge and discharge power is greater than or equal to the minimum charging power. If so, it is determined that the target charge and discharge power meets the third constraint sub-condition; it is determined whether the target charge and discharge power is greater than or equal to the maximum discharge power. If so, it is determined that the target charge and discharge power meets the fourth constraint sub-condition. When the target charge and discharge power meets all constraint sub-conditions, it is determined that the target charge and discharge rate meets the rigid constraint condition.

[0199] S263 determines whether the target charge and discharge rate meets the auxiliary constraint condition, and obtains a second determination result;

[0200] In order to further reduce the user's cost and meet the user's cost requirements, auxiliary constraints are also constructed.

[0201] Cost requirements include: Demand charges. Demand charges are based on the user's maximum electricity demand (or demand) in a specific period of time (such as a month).

[0202] The auxiliary constraint conditions include: the target charging and discharging power is within a preset load range.

[0203] That is, Action*rate≤DemandPower+pv-load;

[0204] Among them, the preset load range is DemandPower+pv-load, where DemandPower represents the maximum power demand threshold, pv represents the power output of photovoltaic power generation (Photovoltaic), and load represents the user's power load, that is, the power consumed by the user at the current moment.

[0205] In one embodiment of the present specification, it is determined whether the target charge and discharge power is within a preset load range; if so, it is determined that the target charge and discharge power meets the auxiliary constraint condition.

[0206] Through the above steps, the user's charging and discharging power (Action*rate) is limited to a preset load range. The upper limit of the preset load range is the user's maximum power demand threshold (DemandPower) plus the power output of photovoltaic power generation (pv), minus the user's power load (load).

[0207] Through this auxiliary constraint, it is ensured that the user's power demand will not exceed the maximum power demand threshold by too much when taking into account the photovoltaic power generation and the current power load.

[0208] In practical applications, the power output of photovoltaic power generation (pv) and the power load of users (load) both change over time, so the preset load range in the auxiliary constraint condition is changing and dynamic.

[0209] Through a variety of constraints, it is easier to meet market and user needs.

[0210] S264 combines the first judgment result and the second judgment result to generate a verification result of the energy storage execution result.

[0211] When the target charge and discharge rate meets the rigid constraint condition, and the target charge and discharge rate meets the auxiliary constraint condition, the verification result of the energy storage execution plan is deemed to be passed, and the charge and discharge behavior of the home energy storage body is updated based on the energy storage execution plan. After receiving the new action value, the home energy storage body will perform the corresponding charge and discharge operation and update the battery status.

[0212] When the target charge and discharge rate does not meet the rigid constraint condition, and / or the target charge and discharge rate meets the auxiliary constraint condition, the verification result of the energy storage execution plan is deemed to be failed.

[0213] In order to optimize the energy storage execution plan of the home energy storage body, the energy storage can also be reset. Specifically, a reset function is provided, and when an energy storage reset instruction is received, the battery state of the home energy storage body is reset to the initial state, so as to restart the optimization scheduling in a new training process.

[0214] like Figure 3 As shown, in order to further improve the accuracy of the energy storage behavior prediction model, at fixed intervals, the newly collected data is fused with the past historical data for offline data aggregation;

[0215] Offline training is performed through offline data, that is, the newly collected data is integrated with the past historical data to recalculate and obtain a new energy storage behavior prediction model;

[0216] Compare the average returns of the new energy storage behavior prediction model and the old energy storage behavior prediction model in the past 7 days, and update the model based on the comparison results. Among them, by comparing the average returns of the new and old models in the past 7 days to see whether the energy storage behavior prediction model is updated, if it needs to be updated, replace the file of the old energy storage behavior prediction model with the file of the new energy storage behavior prediction model for online reasoning. After determining the fixed model, after receiving the calculation request, obtain the latest prediction data from the interface and perform reasoning, complete the decision on future energy storage behavior and send it to the hardware for execution.

[0217] The present invention uses an energy storage behavior prediction model to dynamically adjust the energy storage operation plan of the energy storage device to optimize the electricity cost and maximize the benefit. It has stronger adaptability and intelligence, and can flexibly adjust the strategy according to environmental changes, so as to better meet the actual needs of household electricity consumption.

[0218] Figure 4 A schematic diagram of a charging and discharging behavior decision system provided in an embodiment of this specification, the system comprising:

[0219] The information acquisition module 410 is used to acquire basic information of the household energy storage body; the basic information includes: current charging and discharging power and current status information;

[0220] A prediction module 420, configured to input the basic information into an energy storage behavior prediction model to predict the charge and discharge behavior of the energy storage body;

[0221] The prediction module 420 includes:

[0222] A single-step benefit determination submodule 423 is used to determine single-step benefit information according to the current charge and discharge power and the current state information;

[0223] An expected profit determination submodule 424 is used to generate expected profit information based on the single-step profit information and profit strategy;

[0224] The prediction submodule 425 is used to predict a new energy storage execution result in combination with the current charging and discharging power, the current state information, the expected benefit information and the charging and discharging adjustment strategy, so as to execute the next charging and discharging behavior.

[0225] The prediction module 420 includes:

[0226] A single-step benefit determination submodule 423 is used to determine single-step benefit information according to the current charge and discharge power and the current state information;

[0227] An expected profit determination submodule 424 is used to generate expected profit information based on the single-step profit information and profit strategy;

[0228] The prediction submodule 425 is used to predict a new energy storage execution result in combination with the current charging and discharging power, the current state information, the expected benefit information and the charging and discharging adjustment strategy, so as to execute the next charging and discharging behavior.

[0229] Optionally, the information acquisition module 410 includes:

[0230] The first acquisition submodule is used to execute the previous energy storage execution strategy and obtain the current charging and discharging behavior;

[0231] A charge and discharge power determination submodule, used to determine the current charge and discharge power in combination with the current charge and discharge behavior and the rate coefficient;

[0232] The collection submodule is used to collect the current status information, and the current status information includes: current photovoltaic power generation power, current household load power, and current electricity price.

[0233] Optionally, the charging and discharging power determination submodule includes:

[0234] an action information determining unit, configured to determine action information according to the current charging and discharging behavior;

[0235] The charge and discharge power determination unit is used to obtain the current charge and discharge power by combining the action information and the rate coefficient.

[0236] Optionally, the single-step benefit determination submodule 423 includes:

[0237] A first determining unit, configured to generate a current docking power by combining the current photovoltaic power generation power, the current household load power and the current charging and discharging power;

[0238] A second determining unit, configured to determine a current actual price according to the current connection power, the current electricity price and the preset judgment condition;

[0239] The third determining unit is used to obtain the single-step revenue information by combining the current docking power, the current actual price and the unit power.

[0240] Optionally, it also includes: a verification module, used to verify the energy storage execution result based on the constraint condition;

[0241] Optionally, the verification module includes:

[0242] An extraction submodule, used to extract a target charge and discharge rate from the energy storage execution result;

[0243] A first judgment submodule, used to judge whether the target charge and discharge rate meets the rigid constraint condition, and obtain a first judgment result;

[0244] A second judgment submodule is used to judge whether the target charge and discharge rate meets the auxiliary constraint condition and obtain a second judgment result;

[0245] A verification submodule is used to generate a verification result of the energy storage execution result by combining the first judgment result and the second judgment result.

[0246] Optionally, the first judgment submodule includes:

[0247] A first judgment unit, used to judge whether the target charge and discharge power is less than or equal to the maximum discharge rate, and if so, determine that the target charge and discharge power meets the first constraint sub-condition;

[0248] A second judgment unit is used to judge whether the target charge and discharge power is less than or equal to the maximum charging rate, and if so, it is determined that the target charge and discharge power meets the second constraint sub-condition;

[0249] A third judgment unit, used to judge whether the target charge and discharge power is greater than or equal to the minimum charge power, and if so, determine that the target charge and discharge power meets the third constraint sub-condition;

[0250] The fourth judgment unit is used to judge whether the target charge and discharge power is greater than or equal to the maximum discharge power. If so, it is determined that the target charge and discharge power meets the fourth constraint sub-condition.

[0251] Optionally, the second judgment submodule includes:

[0252] The fifth judgment unit is used to judge whether the target charge and discharge power is within a preset load range.

[0253] The functions of the system of the embodiment of the present invention have been described in the above method embodiment, so for details not fully described in this embodiment, please refer to the relevant description in the above embodiment, and no further description will be given here.

[0254] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0255] The electronic device embodiment of the present invention is described below, and the electronic device can be regarded as an implementation of the method and device embodiments of the present invention in a physical form. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.

[0256] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Figure 5 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0257] like Figure 5 As shown, the computer device 500 of this exemplary embodiment is in the form of a general data processing device. The components of the computer device 500 may include but are not limited to: at least one processor 510, at least one memory 520, a network interface 530, a display unit 540, an input component 550, etc.

[0258] The memory 520 stores a computer-readable program, which may be a source program or a code of a read-only program. The program may be executed by the processor 510, so that the processor 510 performs the steps of various embodiments of the present invention. For example, the processor 510 may perform the following steps: Figure 1 Steps shown.

[0259] The memory 520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only memory unit (ROM). The memory 520 may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment.

[0260] Also included is a bus (not shown) which may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0261] The computer device 500 may also communicate with one or more external devices (e.g., keyboard, display, network device, Bluetooth device, etc.) so that a user can interact with the computer device 500 via these external devices, and / or so that the computer device 500 can communicate with one or more other data processing devices (e.g., routers, modems, etc.). Such communication may be performed through the network interface 530, and may also be performed through a network adapter with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet). The network adapter may communicate with other modules of the computer device 500 via a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in the computer device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0262] Figure 6 Schematic diagram of a computer readable medium embodiment of the present invention. Figure 6As shown, the computer program can be stored on one or more computer-readable media. The computer-readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. When the computer program is executed by one or more data processing devices, the computer-readable medium is enabled to implement the above method of the present invention.

[0263] Through the description of the above implementation modes, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation mode of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a data processing device (which can be a personal computer, a server, or a network device, etc.) to perform the above method according to the present invention.

[0264] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0265] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0266] In summary, the present invention can be implemented by a method, apparatus, electronic device or computer-readable medium that executes a computer program. In practice, a general data processing device such as a microprocessor or a digital signal processor (DSP) can be used to implement some or all functions of the present invention.

[0267] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A decision-making method for charging and discharging behavior, characterized in that: include: Obtain basic information about household energy storage; The basic information includes: current charging and discharging power and current status information; The basic information is input into an energy storage behavior prediction model to predict the charging and discharging behavior of the energy storage body; wherein, single-step profit information is determined according to the current charging and discharging power and the current state information; expected profit information is generated based on the single-step profit information and the profit strategy; and a new energy storage execution result is predicted in combination with the current charging and discharging power, the current state information, the expected profit information and the charging and discharging adjustment strategy for executing the next charging and discharging behavior.

2. A charging and discharging behavior decision method as claimed in claim 1, characterized in that: The basic information of the household energy storage device is obtained, including: Execute the previous energy storage execution strategy to obtain the current charging and discharging behavior; Determine the current charge and discharge power by combining the current charge and discharge behavior and the rate coefficient; The current status information is collected, and the current status information includes: current photovoltaic power generation power, current household load power, and current electricity price.

3. A charging and discharging behavior decision method as claimed in claim 2, characterized in that: The determining the current charge and discharge power by combining the current charge and discharge behavior and the rate coefficient includes: Determining action information according to the current charging and discharging behavior; The current charging and discharging power is obtained by combining the action information and the rate coefficient.

4. A charging and discharging behavior decision method as claimed in claim 2, characterized in that: The generating single-step benefit information by using the current charge and discharge power and the current state information includes: Combining the current photovoltaic power generation, the current household load power and the current charging and discharging power, generating a current docking power; Determining a current actual price according to the current connection power, the current electricity price and the preset judgment condition; The single-step revenue information is obtained by combining the current connection power, the current actual price and the unit power.

5. A charging and discharging behavior decision method as claimed in claim 2, characterized in that: Also includes: Verifying the energy storage execution result based on the constraint condition; The verifying the energy storage execution result based on the constraint condition includes: Extracting a target charge and discharge rate from the energy storage execution result; Determining whether the target charge and discharge rate meets the rigid constraint condition to obtain a first determination result; Determining whether the target charge and discharge rate meets the auxiliary constraint condition to obtain a second determination result; The first judgment result and the second judgment result are combined to generate a verification result of the energy storage execution result.

6. A charging and discharging behavior decision method as claimed in claim 5, characterized in that: The step of determining whether the target charge and discharge rate meets the rigid constraint condition to obtain a first determination result includes: Determine whether the target charge and discharge power is less than or equal to the maximum discharge rate, and if so, determine that the target charge and discharge power meets the first constraint sub-condition; Determine whether the target charge and discharge power is less than or equal to the maximum charge rate, and if so, determine that the target charge and discharge power meets the second constraint sub-condition; Determine whether the target charge and discharge power is greater than or equal to the minimum charge power, and if so, determine that the target charge and discharge power meets the third constraint sub-condition; It is determined whether the target charge and discharge power is greater than or equal to the maximum discharge power. If so, it is determined that the target charge and discharge power meets the fourth constraint sub-condition.

7. A charging and discharging behavior decision method as claimed in claim 5, characterized in that: The determining whether the target charge and discharge rate meets the auxiliary constraint condition to obtain a second determination result includes: Determine whether the target charge and discharge power is within a preset load range.

8. A decision-making system for charging and discharging behavior, characterized in that: include: Information acquisition module, used to obtain basic information of household energy storage; The basic information includes: current charging and discharging power and current status information; A prediction module, used for inputting the basic information into an energy storage behavior prediction model to predict the charging and discharging behavior of the energy storage body; The prediction module comprises: A single-step benefit determination submodule, used to determine single-step benefit information according to the current charge and discharge power and the current state information; An expected return determination submodule, used to generate expected return information based on the single-step return information and return strategy; The prediction submodule is used to combine the current charging and discharging power, the current state information, the expected benefit information and the charging and discharging adjustment strategy to predict a new energy storage execution result for executing the next charging and discharging behavior.

9. An electronic device, wherein: The electronic device includes: processor; and, A memory storing computer executable instructions which, when executed, cause the processor to perform a method according to any one of claims 1-7.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method of any one of claims 1 to 7 is implemented.

Images