Energy storage equipment scheduling method and device, equipment, storage medium and program product

Abstract

The invention provides an energy storage equipment scheduling method and device, equipment, a storage medium and a program product, and relates to the technical field of power systems. According to the method, electric power data of a target station area are obtained, the electric power data comprise a power grid load demand of the target station area, the storage electric quantity of target energy storage equipment in the target station area, the charging power of the target energy storage equipment and the discharging power of the target energy storage equipment, the electric power data are preprocessed, and first target electric power data are obtained; the first target power data comprises a plurality of time steps and second target power data corresponding to each time step, inputting the second target power data corresponding to the time step into the MILP model for optimal scheduling for each time step in the plurality of time steps, obtaining a first scheduling strategy corresponding to the time step output by the MILP model, and obtaining a second scheduling strategy corresponding to the time step output by the MILP model according to the first scheduling strategy; the second scheduling strategy corresponding to the target energy storage equipment is obtained, and the scheduling precision and the calculation efficiency of the scheduling strategy are improved.

Description

Technical Field

[0001] This application relates to the field of power system technology, and in particular to a method, apparatus, equipment, storage medium and program product for dispatching energy storage devices. Background Technology

[0002] In power systems, distribution area energy storage devices serve as a crucial energy storage and regulation tool, effectively balancing grid load fluctuations and enhancing grid responsiveness. These devices can store excess electrical energy and release it during peak grid load periods, thus achieving grid load balancing. Therefore, optimal configuration of energy storage devices is essential for efficient scheduling.

[0003] Currently, traditional methods for dispatching energy storage devices in distribution substations typically rely on simple grid load forecasting. Specifically, by performing regression analysis on historical data of energy storage device dispatching, a corresponding dispatching strategy for the energy storage devices is obtained, and this strategy is then used to dispatch the energy storage devices in the distribution substations.

[0004] However, the inventors discovered that the above-mentioned energy storage equipment scheduling method has the problem of poor scheduling accuracy. Summary of the Invention

[0005] This application provides a method, apparatus, equipment, storage medium, and program product for scheduling energy storage devices, in order to solve the problem of poor scheduling accuracy in related technologies.

[0006] In a first aspect, this application provides a method for scheduling energy storage devices, comprising: acquiring power data of a target distribution area, the power data including the grid load demand of the target distribution area, the stored power of a target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device; preprocessing the power data to obtain first target power data, the first target power data including multiple time steps and second target power data corresponding to each time step; for each of the multiple time steps, inputting the second target power data corresponding to the time step into a mixed-integer linear programming (MILP) model for optimized scheduling to obtain a first scheduling strategy corresponding to the time step output by the MILP model, and obtaining a second scheduling strategy corresponding to the target energy storage device based on the first scheduling strategy.

[0007] In one possible implementation, the MILP model takes minimizing the cost of energy storage devices as its objective function; the second objective power data corresponding to the time step is input into the mixed integer linear programming MILP model for optimization scheduling, and the first scheduling strategy corresponding to the time step output by the MILP model is obtained, including: inputting the second objective power data corresponding to the time step into the objective function, solving the objective function based on the constraints satisfied by the objective function, and obtaining the first scheduling strategy.

[0008] In one possible implementation, the constraints include physical constraints on the charging power of the energy storage device, physical constraints on the discharging power of the energy storage device, physical constraints on the stored capacity of the energy storage device, and matching constraints between grid load demand and energy storage device scheduling strategies.

[0009] In one possible implementation, after obtaining the second scheduling strategy corresponding to the target energy storage device according to the first scheduling strategy, the method further includes: optimizing the second scheduling strategy according to the grid load demand and the stored energy to obtain the third scheduling strategy corresponding to the target energy storage device.

[0010] In one possible implementation, the second scheduling strategy includes the optimal charging power and optimal discharging power corresponding to each time step. Based on the grid load demand and the stored energy, the second scheduling strategy is optimized to obtain the third scheduling strategy corresponding to the target energy storage device. This includes: for each time step in the second scheduling strategy, the reinforcement learning model is initialized based on the optimal charging power, the optimal discharging power, the grid load demand, and the stored energy to obtain the initial parameters of the reinforcement learning model. The initial parameters include the state space, the action space, the reward function, and the expected benefit of taking any action in any state.

[0011] Based on the state space, action space, and reward function, update the expected benefit until the preset iteration conditions are met to obtain the target expected benefit;

[0012] Based on the expected benefits, the fourth scheduling strategy corresponding to the time step is determined, and the third scheduling strategy is obtained based on the fourth scheduling strategy.

[0013] In one possible implementation, the fourth scheduling strategy corresponding to the time step is determined based on the expected benefits of the target, including: mapping the expected benefits of the target to the fourth scheduling strategy based on a preset mapping function, wherein the mapping includes linear mapping and nonlinear mapping.

[0014] Secondly, this application provides an energy storage device dispatching apparatus, comprising:

[0015] The acquisition module is used to acquire the power data of the target distribution area. The power data includes the grid load demand of the target distribution area, the stored power of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device.

[0016] The preprocessing module is used to preprocess the power data to obtain the first target power data, which includes multiple time steps and the second target power data corresponding to each time step.

[0017] The first optimization scheduling module is used to input the second target power data corresponding to each time step in multiple time steps into the MILP model for optimization scheduling, to obtain the first scheduling strategy corresponding to the time step output by the MILP model, and to obtain the second scheduling strategy corresponding to the target energy storage device based on the first scheduling strategy.

[0018] In one possible implementation, the MILP model takes minimizing the cost of energy storage devices as its objective function; the first optimization scheduling module is specifically used to: input the second target power data corresponding to the time step into the objective function, solve the objective function based on the constraints satisfied by the objective function, and obtain the first scheduling strategy.

[0019] In one possible implementation, the constraints include physical constraints on the charging power of the energy storage device, physical constraints on the discharging power of the energy storage device, physical constraints on the stored capacity of the energy storage device, and matching constraints between grid load demand and energy storage device scheduling strategies.

[0020] In one possible implementation, the energy storage device scheduling device further includes a second optimization scheduling module (not shown). The second optimization scheduling module is used to optimize the second scheduling strategy according to the grid load demand and the stored energy after obtaining the second scheduling strategy corresponding to the target energy storage device according to the first scheduling strategy, so as to obtain the third scheduling strategy corresponding to the target energy storage device.

[0021] In one possible implementation, the second scheduling strategy includes the optimal charging power and optimal discharging power corresponding to each time step. The second optimization scheduling module is specifically used for: for each time step in the second scheduling strategy, initializing the reinforcement learning model based on the optimal charging power, the optimal discharging power, the grid load demand, and the stored energy, to obtain the initial parameters of the reinforcement learning model, including the state space, action space, reward function, and the expected benefit of taking any action in any state; updating the expected benefit to meet the preset iteration conditions based on the state space, action space, and reward function to obtain the target expected benefit; determining the fourth scheduling strategy corresponding to the time step based on the target expected benefit, and obtaining the third scheduling strategy based on the fourth scheduling strategy.

[0022] In one possible implementation, the second optimization scheduling module is also used to: map the expected benefits of the target to a fourth scheduling strategy based on a preset mapping function, the mapping including linear mapping and nonlinear mapping.

[0023] Thirdly, this application provides an electronic device, including a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided in the first aspect above.

[0024] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method provided in the first aspect above.

[0025] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in the first aspect above.

[0026] The energy storage device scheduling method, apparatus, equipment, storage medium, and program product provided in this application acquire power data of a target distribution area. This power data includes the grid load demand of the target distribution area, the stored capacity of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device. The power data is preprocessed to obtain first target power data, which includes multiple time steps and second target power data corresponding to each time step. For each time step, the second target power data corresponding to that time step is input into a MILP model for optimized scheduling, resulting in a first scheduling strategy corresponding to the time step output by the MILP model. Based on the first scheduling strategy, a second scheduling strategy corresponding to the target energy storage device is obtained. This application, by optimizing the scheduling of the grid load demand of the target distribution area, the stored capacity of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device for each time step based on the MILP model, improves the scheduling accuracy and the computational efficiency of the scheduling strategy. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0028] Figure 1 This is a schematic diagram of the structure of the energy storage device scheduling system provided in the embodiments of this application;

[0029] Figure 2 A flowchart illustrating the energy storage device scheduling method provided in this application embodiment. Figure 1 ;

[0030] Figure 3 A flowchart illustrating the energy storage device scheduling method provided in this application embodiment. Figure 2 ;

[0031] Figure 4 This is a schematic diagram of the structure of the energy storage equipment scheduling device provided in the embodiments of this application;

[0032] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0033] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0035] In related technologies, traditional methods for scheduling energy storage devices in distribution substations rely on two main approaches. First, they depend on simple grid load forecasting, which involves performing regression analysis on historical data to derive scheduling strategies for the energy storage devices, and then using these strategies to schedule the devices. Second, they rely on setting fixed charging and discharging time periods to schedule the devices. However, these traditional methods suffer from low computational efficiency and poor scheduling accuracy. Furthermore, they have limited applicability in complex and variable grid demand response scenarios.

[0036] Based on the problems existing in related technologies, the embodiments of this application are based on the MILP model to optimize the scheduling of the grid load demand of the distribution area, the stored power of the energy storage devices in the distribution area, the charging power of the energy storage devices, and the discharging power of the energy storage devices, thereby improving the scheduling accuracy and the computational efficiency of the scheduling strategy.

[0037] Figure 1 This is a schematic diagram of the structure of an energy storage device scheduling system provided in an embodiment of this application. Figure 1 As shown, the energy storage equipment scheduling system includes a data acquisition and preprocessing module, an energy storage equipment optimization scheduling module, a reinforcement learning dynamic optimization module, and a data visualization and strategy support module.

[0038] The data acquisition and preprocessing module is used to monitor the energy storage equipment in the distribution area in real time, and to collect grid load data such as grid load demand, as well as the operating data of the energy storage equipment in the distribution area such as the stored power, charging power and discharging power of the energy storage equipment. The module also performs data cleaning and normalization on the collected data to ensure the accuracy and reliability of the data used to obtain the scheduling strategy corresponding to the energy storage equipment.

[0039] The energy storage equipment optimization scheduling module is used to optimize and schedule the data processed by the data acquisition and preprocessing module based on the constructed MILP model, so as to achieve the initial optimization scheduling of energy storage equipment.

[0040] The reinforcement learning dynamic optimization module is used to further optimize the initial scheduling strategy of the energy storage devices obtained by the energy storage device optimization scheduling module based on the reinforcement learning model. By continuously feeding back grid load fluctuations and changes in the charging and discharging power of the energy storage devices, the module adjusts the initial scheduling strategy of the energy storage devices to make the scheduling strategy of the energy storage devices more compatible with the actual grid load demand.

[0041] The data visualization and strategy support module provides operators of the energy storage equipment dispatching system with a real-time data visualization interface, and displays the status of energy storage equipment, grid load demand in the distribution area, and optimized dispatching results of energy storage equipment based on the visualization interface.

[0042] The energy storage device scheduling method provided in this application will be described in detail below with reference to specific embodiments.

[0043] Figure 2 A flowchart illustrating the energy storage device scheduling method provided in this application embodiment. Figure 1 .like Figure 2 As shown, the implementation of this energy storage device scheduling method may include the following steps:

[0044] S201, Obtain the power data of the target distribution area. The power data includes the grid load demand of the target distribution area, the stored power of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device.

[0045] For example, the power data of the target transformer area can be obtained through sensors.

[0046] For example, the power data of the target distribution area can be real-time power data collected within a preset period, that is, the grid load demand of the target distribution area can be real-time grid load demand data collected within a preset period, the stored power of the target energy storage device in the target distribution area can be real-time stored power collected within a preset period, the charging power of the target energy storage device can be real-time charging power collected within a preset period, and the discharging power of the target energy storage device can be real-time discharging power collected within a preset period.

[0047] For example, the preset period can be one day or one week, etc. This application embodiment does not limit the size of the preset period, and it can be determined according to the actual application requirements.

[0048] S202, preprocess the power data to obtain first target power data, which includes multiple time steps and second target power data corresponding to each time step.

[0049] For example, preprocessing may include data cleaning, data normalization, and data segmentation.

[0050] Understandably, by cleaning and normalizing power data, noise can be eliminated, and data without practical physical meaning and abnormal data can be removed, thereby improving the accuracy and reliability of power data.

[0051] In this step, one possible implementation is as follows: First, perform data cleaning and normalization on the power data to obtain the first target power data. Then, divide the time period corresponding to the first target power data into multiple time steps based on preset rules, and mark the first target power data in each time step as the second target power data corresponding to that time step.

[0052] For example, a time step can be a time period.

[0053] For example, the preset rule can be equal-interval division.

[0054] For example, when the time period corresponding to the first target power data is one week, each time step can be one day, and correspondingly, the second target power data corresponding to each time step can be the first target power data corresponding to each day.

[0055] It is understandable that by dividing the time period corresponding to the first target power data into multiple time steps and optimizing the scheduling of the second target power data corresponding to each time step, the scheduling strategy corresponding to the target energy storage device can be obtained, thereby improving the scheduling accuracy.

[0056] S203, for each of the multiple time steps, input the second target power data corresponding to the time step into the mixed integer linear programming (MILP) model for optimized scheduling, obtain the first scheduling strategy corresponding to the time step output by the MILP model, and obtain the second scheduling strategy corresponding to the target energy storage device based on the first scheduling strategy.

[0057] For example, the first scheduling strategy can be the charging and discharging strategy of the target energy storage device within that time step.

[0058] For example, the first scheduling strategy includes the optimal charging power and optimal discharging power of the target energy storage device within that time step.

[0059] For example, the second scheduling strategy can be the charging and discharging strategy of the target energy storage device within the time period corresponding to the first target power data.

[0060] For example, the second scheduling strategy includes the optimal charging power and optimal discharging power of the target energy storage device in each time step.

[0061] In this step, one possible implementation is as follows: For multiple time steps, optimize the scheduling of the second target power data corresponding to each time step using the MIPL model to obtain the first scheduling strategy corresponding to each time step. Then, arrange the first scheduling strategies corresponding to all time steps in the time order corresponding to the time steps to obtain the second scheduling strategy of the target energy storage device.

[0062] In this embodiment, power data of a target distribution area is acquired. This power data includes the grid load demand of the target distribution area, the stored capacity of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device. The power data is preprocessed to obtain first target power data. The first target power data includes multiple time steps and second target power data corresponding to each time step. For each time step, the second target power data corresponding to the time step is input into the MILP model for optimized scheduling, resulting in a first scheduling strategy corresponding to the time step output by the MILP model. Based on the first scheduling strategy, a second scheduling strategy corresponding to the target energy storage device is obtained. In this embodiment, by optimizing the scheduling of the grid load demand of the target distribution area, the stored capacity of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device corresponding to each time step based on the MILP model, the scheduling accuracy and the computational efficiency of the scheduling strategy are improved.

[0063] Optionally, the MILP model provided in this application embodiment takes minimizing the cost of energy storage devices as its objective function.

[0064] For example, minimizing the cost of an energy storage device includes minimizing the charging power cost of the energy storage device, minimizing the discharging power cost of the energy storage device, and minimizing the deviation penalty of the target storage capacity of the target energy storage device.

[0065] For example, the objective function can be expressed by the following formula:

[0066]

[0067] Where t represents the time step, and T represents the number of time steps. This represents the charging power of the target energy storage device at time step t. This represents the discharge power of the target energy storage device at time step t. This represents the stored energy capacity of the target energy storage device at time step t. This represents the target energy storage capacity of the target energy storage device corresponding to time step t. The cost factor representing charging power, The cost factor representing discharge power, Indicates the trade-off coefficient. This represents the storage power error penalty coefficient corresponding to time step t.

[0068] It is understandable that in a power system, the charging power, discharging power, and stored capacity of energy storage devices may change in real time. That is, within a certain period of time, the charging power, discharging power, and stored capacity of energy storage devices may fluctuate. In one possible implementation of this application embodiment, when constructing the objective function of the MIPL model, the charging power, discharging power, and stored capacity of the target energy storage device within the corresponding time step can be the average value of the real-time data within that time step.

[0069] For example, the charging power of the target energy storage device corresponding to time step t. This can be the average real-time charging power of the target energy storage device within time step t; or the discharge power of the target energy storage device corresponding to time step t. This can be the average real-time discharge power of the target energy storage device within the time step t; or the stored energy of the target energy storage device corresponding to time step t. It can be the average value of the real-time stored power of the target energy storage device within the time step t.

[0070] For example, the cost factor of charging power This can be the average of the real-time electricity price data of the target energy storage device within the time step t, and the cost coefficient of the discharge power. It can be the average value of the real-time electricity price data of the target energy storage device within the time step t.

[0071] It should be noted that the cost coefficients of charging power, discharging power, and storage capacity error penalty coefficients for different time steps can be the same or different, depending on the actual application requirements. This application does not impose any restrictions on this.

[0072] For example, when the number of time steps is 5, the cost coefficients of the charging power corresponding to each time step can be [1, 1.2, 1.1, 1, 1.5] in sequence according to the time order of the time steps; the cost coefficients of the discharging power corresponding to each time step can be [1.3, 1.4, 1.2, 1.1, 1.3] in sequence; and the storage capacity error penalty coefficients corresponding to each time step can be [0.5, 0.4, 0.6, 0.3, 0.5], etc.

[0073] It should be noted that the embodiments of this application do not limit the cost coefficient of charging power, the cost coefficient of discharging power, and the error penalty coefficient of stored capacity corresponding to each time step. They can be determined according to the actual application requirements.

[0074] For example, the target stored capacity of the target energy storage device corresponding to time step t. This can be defined as the minimum amount of electricity that the target energy storage device needs to store within time step t. This application specifies the target stored capacity of the target energy storage device corresponding to time step t. The size and method of determination are not limited; they can be determined based on the actual application requirements.

[0075] Optionally, in step S203, the second target power data corresponding to the time step is input into the mixed integer linear programming (MILP) model for optimization scheduling to obtain the first scheduling strategy corresponding to the time step output by the MILP model can be implemented as follows: the second target power data corresponding to the time step is input into the objective function, and the objective function is solved based on the constraints satisfied by the objective function to obtain the first scheduling strategy.

[0076] For example, the objective function is as described above, and will not be repeated here.

[0077] In one possible implementation, the second target power data corresponding to the time step is input into the objective function. Based on the constraints satisfied by the objective function, the objective function is solved by a solver to obtain the optimal charging power and optimal discharging power corresponding to the time step output by the MIPL model, which is the first scheduling strategy.

[0078] For example, the solver can be a mathematical programming optimizer or a numerical optimization technique, etc.

[0079] Understandably, by using the solver, the results will provide a preliminary optimized scheduling scheme for the charging and discharging strategy of the target energy storage device, thereby minimizing charging and discharging costs and power errors, while also meeting the grid load demand.

[0080] Optionally, the constraints satisfied by the objective function may include: physical constraints on the charging power of the energy storage device, physical constraints on the discharging power of the energy storage device, physical constraints on the stored power of the energy storage device, and matching constraints between grid load demand and energy storage device scheduling strategy.

[0081] Each constraint is explained in detail below.

[0082] 1) Physical constraints on the charging power of energy storage devices

[0083] For example, the physical constraint on the charging power of the energy storage device can be that the charging power of the target energy storage device at time step t is less than or equal to a predefined charging power threshold.

[0084] For example, the physical constraints on the charging power of energy storage devices can be expressed by the following formula:

[0085]

[0086] in, This indicates the charging power threshold.

[0087] For example, the charging power threshold can be 100kWh.

[0088] This application does not limit the size of the charging power threshold; it can be determined according to the actual application requirements.

[0089] 2) Physical constraints on the discharge power of energy storage devices

[0090] For example, the physical constraint on the discharge power of the energy storage device can be that the discharge power of the target energy storage device at time step t is less than or equal to a predefined discharge power threshold.

[0091] For example, the physical constraint on the discharge power of an energy storage device can be expressed by the following formula:

[0092]

[0093] in, This indicates the discharge power threshold.

[0094] For example, the discharge power threshold can be 100 kWh.

[0095] The embodiments of this application do not limit the size of the discharge power threshold; it can be determined according to the actual application requirements.

[0096] 3) Physical constraints on the amount of electricity stored in energy storage devices

[0097] For example, the physical constraint on the energy storage capacity of an energy storage device can be that the energy storage capacity of the target energy storage device at time step t is between the lower limit threshold and the upper limit threshold of the energy storage capacity.

[0098] For example, the physical constraints on the amount of electricity that an energy storage device can store can be expressed by the following formula:

[0099]

[0100] in, This indicates the lower limit threshold for storage capacity. This indicates the upper limit threshold for the amount of electricity stored.

[0101] For example, the lower limit threshold for storage capacity can be 20kWh, and the upper limit threshold for storage capacity can be 100kWh.

[0102] This application does not limit the size of the lower and upper limits of the storage capacity threshold; the specific values ​​can be determined according to the actual application requirements.

[0103] 4) Matching constraints between grid load demand and energy storage device dispatching strategies

[0104] For example, the matching constraint between grid load demand and energy storage device scheduling strategy can be the grid load demand of the target distribution area within time step t, and the matching grid demand with the charging power and discharging power of the target energy storage device within time step t.

[0105] For example, the matching constraint between grid load demand and energy storage device scheduling strategy can be expressed as: and It should match the needs of the power grid.

[0106] in, This indicates the power grid load demand of the target distribution area corresponding to time step t.

[0107] For example, the grid load demand of the target distribution area corresponding to time step t can be the average value of the real-time grid load demand of the target distribution area within that time step t.

[0108] Optionally, in the energy storage device scheduling method provided in this application embodiment, after obtaining the second scheduling strategy corresponding to the target energy storage device according to the first scheduling strategy in step S203, the method further includes: optimizing the scheduling of the second scheduling strategy according to the grid load demand and the stored power to obtain the third scheduling strategy corresponding to the target energy storage device.

[0109] Optionally, the second scheduling strategy includes the optimal charging power and the optimal discharging power for each time step.

[0110] In one possible implementation, a reinforcement learning model is used to optimize the second scheduling strategy based on the grid load demand and the amount of stored energy, thereby obtaining a third scheduling strategy corresponding to the target energy storage device.

[0111] Understandably, by using reinforcement learning algorithms to dynamically optimize the second scheduling strategy based on grid load demand and the actual stored capacity of the target energy storage device, scheduling accuracy and flexibility can be further improved.

[0112] The following is combined with Figure 3 This paper provides a detailed explanation of the specific implementation method of optimizing the second scheduling strategy based on the grid load demand and the stored energy to obtain the third scheduling strategy corresponding to the target energy storage device.

[0113] Figure 3 A flowchart illustrating the energy storage device scheduling method provided in this application embodiment. Figure 2 .like Figure 3 As shown, the specific implementation of the method, which optimizes the second scheduling strategy based on the grid load demand and the stored energy, to obtain the third scheduling strategy corresponding to the target energy storage device, may include the following steps:

[0114] S301, for each time step in the second scheduling strategy, the reinforcement learning model is initialized based on the optimal charging power, the optimal discharging power, the grid load demand, and the stored energy corresponding to the time step, to obtain the initial parameters of the reinforcement learning model. The initial parameters include the state space, action space, reward function, and the expected benefit of taking any action in any state.

[0115] In one possible implementation, the reinforcement learning model is initialized based on the optimal charging power at the corresponding time step, the optimal discharging power at the corresponding time step, the grid load demand, and the stored energy, including:

[0116] 1) Define the state space

[0117] For example, the state space of a reinforcement learning model includes the stored energy of the energy storage device, the charging power of the energy storage device, the discharging power of the energy storage device, and the grid load demand of the distribution area.

[0118] For example, the dimension of the state space can be represented as:

[0119]

[0120] in, Represents the state space corresponding to time step t. This represents the stored energy capacity of the target energy storage device at time step t. This represents the optimal charging power of the target energy storage device at time step t. This represents the optimal discharge power of the target energy storage device at time step t. This indicates the power grid load demand of the target distribution area corresponding to time step t.

[0121] 2) Define the action space

[0122] For example, the action space includes a charge / discharge strategy for each time step, wherein each action is used to adjust the optimal charging power of the target energy storage device for each time step, and / or the optimal discharging power of the target energy storage device.

[0123] In some embodiments, each action is used to adjust the optimal charging power of the target energy storage device corresponding to each time step; in some embodiments, each action is used to adjust the optimal discharging power of the target energy storage device corresponding to each time step; in some embodiments, each action is used to adjust both the optimal charging power and the optimal discharging power of the target energy storage device corresponding to each time step.

[0124] 3) Define the reward function

[0125] For example, the reward function is used to measure the scheduling effect of the second scheduling strategy by the storage capacity error of the energy storage device, the power fluctuation of the energy storage device, and the grid load demand of the distribution area.

[0126] For example, the reward function can be the sum of the storage capacity error penalty, the charging and discharging power fluctuation penalty, and the grid load demand matching penalty.

[0127] For example, the reward function can be expressed by the following formula:

[0128]

[0129] in, Represents the reward function, This represents the penalty coefficient for fluctuations in charging and discharging power. Penalty coefficient for matching power grid load demand.

[0130] 4) Initialize the expected benefits of taking any action in any state.

[0131] For example, the expected benefits can be expressed as , used to indicate the state Take action below The expected benefits, where n is a non-negative integer. Represents the action space.

[0132] For example, initial expected benefits value This value can be any value. This application does not impose any restrictions on this value; it can be determined based on the specific application requirements.

[0133] S302, based on the state space, action space and reward function, update the expected benefit until the preset iteration conditions are met, and obtain the target expected benefit.

[0134] For example, the realization of the expected benefits can be represented by the following formula:

[0135]

[0136] in, This represents the expected benefit obtained after updating n+1 times. This represents the expected benefit obtained after n updates. Indicates the learning rate. Indicates the discount factor. state All possible actions of The maximum value among the values.

[0137] Understandably, the updated This will affect the scheduling strategy at each time step, i.e., through the selection of the optimal action. To adjust the optimal charging power and optimal discharging power of the target energy storage device.

[0138] For example, the preset iteration condition can be a preset iteration number threshold, or it can be that the difference in expected benefits after zero consecutive updates is less than or equal to a preset threshold. This application does not limit the preset iteration number threshold or the preset threshold itself; it can be determined according to the actual application requirements.

[0139] S303. Based on the expected benefits of the target, determine the fourth scheduling strategy corresponding to the time step, and based on the fourth scheduling strategy, obtain the third scheduling strategy.

[0140] For example, the fourth scheduling strategy includes the optimal charging power and optimal discharging power of the target energy storage device at the time step after optimization scheduling by the reinforcement learning model.

[0141] For example, the charging power and discharging power of the target energy storage device corresponding to the expected benefits are determined as the optimal charging power and optimal discharging power of the target energy storage device for that time step.

[0142] For example, the third scheduling strategy for the target energy storage device is obtained by arranging the fourth scheduling strategies corresponding to all time steps in the time order of the time steps.

[0143] Optionally, one possible implementation of determining the fourth scheduling strategy corresponding to the time step based on the expected benefits can be: mapping the expected benefits to the fourth scheduling strategy based on a preset mapping function, where the mapping includes linear mapping and nonlinear mapping.

[0144] For example, the mapping function can be represented as f(∙).

[0145] It should be noted that the form of the mapping function f(∙) can be obtained by fitting empirical data or a model. The mapping function is a function that transforms the expected benefit Q value into the charging power and discharging power of the specific energy storage device. It can be designed according to different application requirements. This application does not limit this, and it can be determined according to the actual application requirements.

[0146] For example, mapping the expected benefits of the objective to the fourth scheduling strategy can be expressed by the following formula:

[0147]

[0148] in, This represents the optimal charging power of the target energy storage device at time step t after optimization scheduling via the reinforcement learning model. The optimal discharge power of the target energy storage device at time step t is shown after optimization scheduling by the reinforcement learning model.

[0149] For example, the mapping function can be implemented as a linear mapping or a non-linear mapping.

[0150] The following explains linear and nonlinear mappings.

[0151] 1) Linear mapping: If the range of expected benefit Q value has been reasonably normalized, the charging power and discharging power of energy storage equipment can be directly mapped by linear equations.

[0152] 2) Nonlinear mapping: When the energy storage device is in the form of a battery, if the battery's charging and discharging characteristics are nonlinear (e.g., battery charging efficiency changes with the amount of charge), the mapping function may be nonlinear. This mapping function can be fitted using machine learning methods based on historical data. By obtaining the charging and discharging power of the energy storage device through the updated target expected benefit mapping, the power system can predict future charging and discharging strategies, such as whether to perform charging, discharging, or maintain the current charging state. Simultaneously, based on the predicted charging and discharging strategies, the power system can adjust the power output of the energy storage device in real time, thereby optimizing grid load management.

[0153] In this embodiment, for each time step in the second scheduling strategy, a reinforcement learning model is initialized based on the optimal charging power, optimal discharging power, grid load demand, and stored energy at that time step. This yields the state space, action space, reward function, and expected benefit of taking any action in any state. The expected benefit is then updated to meet preset iteration conditions based on the state space, action space, and reward function to obtain the target expected benefit. Furthermore, based on the target expected benefit, a fourth scheduling strategy corresponding to the time step is determined, and a third scheduling strategy is obtained based on the fourth scheduling strategy. This embodiment employs a reinforcement learning model to dynamically optimize the scheduling results of the MILP model, enabling the target energy storage device scheduling to respond in real-time to grid load fluctuations, improving scheduling accuracy, flexibility, and adaptability, and avoiding the shortcomings of static scheduling.

[0154] In summary, the energy storage device scheduling method provided in this application has two main advantages. First, by using a MIPL model and reinforcement learning model, it can calculate the optimal charging and discharging strategy (i.e., scheduling strategy) of the energy storage device based on the real-time power data of the energy storage device and the real-time grid load demand of the distribution area, thereby improving scheduling accuracy. Furthermore, by continuously adjusting the scheduling strategy through reinforcement learning, it enhances the scheduling efficiency of the power system in long-term operation. Second, the energy storage device scheduling method provided in this application is a demand response (DR)-oriented scheduling method. By using a MIPL model and reinforcement learning model, it optimizes the scheduling of the stored energy, charging power, and discharging power of the energy storage devices in the distribution area according to the grid load demand of the distribution area. This effectively responds to grid load fluctuations, improves response flexibility, and enhances the reliability and efficiency of the power system. Demand response, as a technology that achieves grid load balance by adjusting the electricity consumption behavior of users, can improve the stability of the grid load through flexible scheduling during peak user electricity consumption periods or when there is a supply-demand imbalance in the grid load.

[0155] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0156] Figure 4 This is a schematic diagram of the energy storage device scheduling apparatus provided in an embodiment of this application. Figure 4 As shown, the energy storage device scheduling device 40 includes: an acquisition module 410, a preprocessing module 420, and a first optimization scheduling module 430.

[0157] The acquisition module 410 is used to acquire the power data of the target distribution area. The power data includes the grid load demand of the target distribution area, the stored power of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device.

[0158] Preprocessing module 420 is used to preprocess the power data to obtain first target power data, which includes multiple time steps and second target power data corresponding to each time step;

[0159] The first optimization scheduling module 430 is used to input the second target power data corresponding to each time step in multiple time steps into the MILP model for optimization scheduling, to obtain the first scheduling strategy corresponding to the time step output by the MILP model, and to obtain the second scheduling strategy corresponding to the target energy storage device based on the first scheduling strategy.

[0160] In one possible implementation, the MILP model takes minimizing the cost of energy storage devices as the objective function; the first optimization scheduling module 430 is specifically used to: input the second target power data corresponding to the time step into the objective function, solve the objective function based on the constraints satisfied by the objective function, and obtain the first scheduling strategy.

[0161] In one possible implementation, the constraints include physical constraints on the charging power of the energy storage device, physical constraints on the discharging power of the energy storage device, physical constraints on the stored capacity of the energy storage device, and matching constraints between grid load demand and energy storage device scheduling strategies.

[0162] In one possible implementation, the energy storage device scheduling device further includes a second optimization scheduling module (not shown). The second optimization scheduling module is used to optimize the second scheduling strategy according to the grid load demand and the stored energy after obtaining the second scheduling strategy corresponding to the target energy storage device according to the first scheduling strategy, so as to obtain the third scheduling strategy corresponding to the target energy storage device.

[0163] In one possible implementation, the second scheduling strategy includes the optimal charging power and optimal discharging power corresponding to each time step. The second optimization scheduling module is specifically used for: for each time step in the second scheduling strategy, initializing the reinforcement learning model based on the optimal charging power, the optimal discharging power, the grid load demand, and the stored energy, to obtain the initial parameters of the reinforcement learning model, including the state space, action space, reward function, and the expected benefit of taking any action in any state; updating the expected benefit to meet the preset iteration conditions based on the state space, action space, and reward function to obtain the target expected benefit; determining the fourth scheduling strategy corresponding to the time step based on the target expected benefit, and obtaining the third scheduling strategy based on the fourth scheduling strategy.

[0164] In one possible implementation, the second optimization scheduling module is also used to: map the expected benefits of the target to a fourth scheduling strategy based on a preset mapping function, the mapping including linear mapping and nonlinear mapping.

[0165] The energy storage device scheduling device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0166] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0167] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0168] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0169] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0170] The memory may include random access memory (RAM) and non-volatile memory (NVM), such as at least one disk storage device.

[0171] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0172] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0173] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0174] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0175] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0176] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0177] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0179] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0180] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0181] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for scheduling energy storage devices, characterized in that, include: Acquire power data for the target distribution area, the power data including the grid load demand of the target distribution area, the stored power of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device; The power data is preprocessed to obtain first target power data, which includes multiple time steps and second target power data corresponding to each time step. For each of the plurality of time steps, the second target power data corresponding to the time step is input into a mixed integer linear programming (MILP) model for optimized scheduling to obtain a first scheduling strategy corresponding to the time step output by the MILP model, and a second scheduling strategy corresponding to the target energy storage device is obtained based on the first scheduling strategy.

2. The energy storage device scheduling method according to claim 1, characterized in that, The MILP model takes minimizing the cost of energy storage devices as its objective function; the step of inputting the second objective power data corresponding to the time step into the mixed integer linear programming (MILP) model for optimization scheduling, and obtaining the first scheduling strategy corresponding to the time step output by the MILP model, includes: The second target power data corresponding to the time step is input into the objective function. Based on the constraints satisfied by the objective function, the objective function is solved to obtain the first scheduling strategy.

3. The energy storage device scheduling method according to claim 2, characterized in that, The constraints include physical constraints on the charging power of the energy storage device, physical constraints on the discharging power of the energy storage device, physical constraints on the stored power of the energy storage device, and matching constraints between grid load demand and energy storage device scheduling strategies.

4. The energy storage device scheduling method according to any one of claims 1 to 3, characterized in that, After obtaining the second scheduling strategy corresponding to the target energy storage device according to the first scheduling strategy, the method further includes: Based on the grid load demand and the stored energy, the second scheduling strategy is optimized to obtain the third scheduling strategy corresponding to the target energy storage device.

5. The energy storage device scheduling method according to claim 4, characterized in that, The second scheduling strategy includes the optimal charging power and optimal discharging power for each time step. The step of optimizing the second scheduling strategy based on the grid load demand and the stored energy to obtain the third scheduling strategy corresponding to the target energy storage device includes: For each time step in the second scheduling strategy, the reinforcement learning model is initialized based on the optimal charging power, the optimal discharging power, the grid load demand, and the stored energy corresponding to the time step, so as to obtain the initial parameters of the reinforcement learning model. The initial parameters include the state space, action space, reward function, and the expected benefit of taking any action in any state. Based on the state space, the action space, and the reward function, the expected benefit is updated to meet the preset iteration conditions to obtain the target expected benefit; Based on the expected benefits of the target, a fourth scheduling strategy corresponding to the time step is determined, and based on the fourth scheduling strategy, the third scheduling strategy is obtained.

6. The energy storage device scheduling method according to claim 5, characterized in that, The step of determining the fourth scheduling strategy corresponding to the time step based on the expected benefits includes: Based on a preset mapping function, the target expected benefits are mapped to the fourth scheduling strategy, and the mapping includes linear mapping and nonlinear mapping.

7. A scheduling device for energy storage equipment, characterized in that, include: The acquisition module is used to acquire power data of the target distribution area. The power data includes the grid load demand of the target distribution area, the stored power of the target energy storage device in the target distribution area, the charging power of the target energy storage device, and the discharging power of the target energy storage device. The preprocessing module is used to preprocess the power data to obtain first target power data, which includes multiple time steps and second target power data corresponding to each time step. The first optimization scheduling module is used to input the second target power data corresponding to each of the plurality of time steps into a mixed integer linear programming (MILP) model for optimization scheduling, to obtain the first scheduling strategy corresponding to the time step output by the MILP model, and to obtain the second scheduling strategy corresponding to the target energy storage device based on the first scheduling strategy.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

Images