Control optimization method and device on user energy storage system based on multiple time scales

Abstract

The invention discloses a control optimization method and device on a user energy storage system based on multiple time scales, and relates to the technical field of electric power. According to the main technical scheme, on the short-term time scale and the long-term time scale, different state information is processed through a double-layer processing frame, and first decision information for energy storage control on the short-term time scale and second decision information for energy storage control on the long-term time scale are output; and then the second decision information is used as a control constraint condition to constrain the first decision information, so that the purpose of influencing short-term scheduling by using long-term planning is realized, and effective control on the user energy storage system is realized by integrating a short-term time scale and a long-term time scale. Therefore, an effective and reasonable energy storage control strategy is provided for the user energy storage system by integrating short-term and long-term time scales, and the purpose is to maximize the overall benefit of the user energy storage system in a dynamically changing environment as much as possible.

Description

Technical Field

[0001] The present application relates to the field of electric power technology, and in particular to a control optimization method and device for a user energy storage system based on multiple time scales. Background Art

[0002] In the field of power technology, user energy storage systems usually refer to energy storage devices installed at the power user end. This system can store electrical energy and release it for use when needed. For example, its main functions include peak shaving, improving power supply reliability, promoting renewable energy consumption, and participating in grid services.

[0003] At present, there are multiple decision-making factors in the operation of user energy storage systems, including changes in electricity prices, fluctuations in power loads, fluctuations in renewable energy, etc. These factors are not only time-varying, but also have different time scales. For example, electricity prices may show seasonal changes, and load demand may fluctuate within hours or daily ranges. Therefore, how to design reasonable control strategies on different time scales to maximize the overall benefits of user energy storage systems in a dynamically changing environment is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a control optimization method and device for a user energy storage system based on multiple time scales. The main purpose is to achieve effective control of the user energy storage system by integrating short-term time scales and long-term time scales, and to provide an effective and reasonable energy storage control strategy, with the aim of maximizing the overall benefit of the user energy storage system in a dynamically changing environment.

[0005] In order to achieve the above objectives, this application mainly provides the following technical solutions:

[0006] In a first aspect, the present application provides a control optimization method for a user energy storage system based on multiple time scales, the method comprising:

[0007] Constructing a state space corresponding to the operating environment of the user energy storage system, wherein a first state variable and a second state variable are pre-set in the state space, wherein the first state variable is used to assist in making a decision on a control operation of the user energy storage system at a first time scale, and the second state variable is used to assist in making a decision on a control operation of the user energy storage system at a second time scale, wherein the first time scale is equal to the time length of a time step in a time series represented by the second time scale;

[0008] Obtaining current environment information corresponding to the operating environment of the user energy storage system;

[0009] Based on the state space, parse out the first state information corresponding to the first state variable and the second state information corresponding to the second state variable from the current environment information;

[0010] Process the first state information using a first-layer preset processing framework to output first decision information, where the first preset processing framework makes a control decision on the user energy storage system at the first time scale;

[0011] Process the second state information using a second-layer preset processing framework to output second decision information, where the second preset processing framework makes a control decision on the user energy storage system at the second time scale; wherein, in the first preset processing framework and the second preset processing framework, the first preset processing framework is the underlying optimization framework, and the second preset processing framework is the top-level optimization framework;

[0012] Execute a control operation on the user energy storage system by using the second decision information as a control constraint condition to constrain the first decision information.

[0013] A second aspect of the present application provides an upper control optimization device for a user energy storage system based on multiple time scales, and the device includes:

[0014] A construction unit for constructing a state space corresponding to the operating environment where the user energy storage system is located. The state space is pre-set with a first state variable and a second state variable. The first state variable is used to assist in making a decision on the control operation of the user energy storage system at the first time scale, and the second state variable is used to assist in making a decision on the control operation of the user energy storage system at the second time scale. The first time scale is equal to the time length of one time step in the time series represented by the second time scale;

[0015] An acquisition unit for acquiring the current environment information corresponding to the operating environment where the user energy storage system is located;

[0016] An analysis unit for parsing out the first state information corresponding to the first state variable and the second state information corresponding to the second state variable from the current environment information based on the state space;

[0017] A first processing unit for processing the first state information using a first-layer preset processing framework to output first decision information, where the first preset processing framework makes a control decision on the user energy storage system at the first time scale;

[0018] A second processing unit, configured to process the second status information by using a second preset processing framework and output second decision information, where the second preset processing framework is to make a control decision on the user energy storage system at the second time scale; wherein, in the first preset processing framework and the second preset processing framework, the first preset processing framework is a bottom-layer optimization framework, and the second preset processing framework is a top-layer optimization framework;

[0019] A first execution unit, configured to perform a control operation on the user energy storage system by using the second decision information as a control constraint condition to constrain the first decision information.

[0020] A third aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the control optimization method for a user energy storage system based on multiple time scales as described above is implemented.

[0021] A fourth aspect of the present application provides an electronic device, where the device includes at least one processor, at least one memory connected to the processor, and a bus;

[0022] wherein, the processor and the memory communicate with each other through the bus;

[0023] The processor is configured to call program instructions in the memory to execute the control optimization method for a user energy storage system based on multiple time scales as described above.

[0024] By means of the above technical solutions, the technical solutions provided by the present application have at least the following advantages:

[0025] The present application provides a control optimization method and device for a user energy storage system based on multiple time scales. First, the present application constructs a corresponding state space for the operating environment where the user energy storage system is located. In this state space, a first state variable and a second state variable are preset. Both of these state variables actually come from the variables included in the environmental information. Based on such a preset, when processing the current environmental information subsequently, useful state variables for assisting in making decisions about the operation of the user energy storage system can be parsed from it. And when setting, the state variables are distinguished by the words "first" and "second" to be used to distinguish the first state variable for assisting in making decisions about controlling the user energy storage system on a short-term time scale (i.e., the first time scale), and the second state variable corresponding to the long-term time scale (the second time scale). Then, the present application parses the current environmental information based on the pre-constructed state space to obtain the first state information corresponding to the first state variable and the second state information corresponding to the second state variable. Then, a two-layer processing framework is used to process different state information respectively, and the energy storage decision control information on the short-term time scale (i.e., the first decision information) and the energy storage decision control information on the long-term time scale (i.e., the second decision information) are output. Then, the first decision information is constrained by using the second decision information as a control constraint condition, thereby achieving the purpose of using long-term planning to influence short-term scheduling, and effectively controlling the user energy storage system by integrating the short-term time scale and the long-term time scale.

[0026] Compared with the need to deal with different time scales involved in the decision-making factors on the user energy storage system, the present application effectively controls the user energy storage system by integrating the short-term time scale and the long-term time scale, and provides an effective and reasonable energy storage control strategy, aiming to maximize the overall benefit of the user energy storage system in a dynamically changing environment as much as possible.

[0027] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specifically illustrates the specific implementation manners of the present application. Brief Description of the Drawings

[0028] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0029] Figure 1 It is a flowchart of a control optimization method for a user energy storage system based on multiple time scales provided by an embodiment of the present application;

[0030] Figure 2 Another flowchart of the control optimization method for the user energy storage system based on multiple time scales provided by the embodiment of the present application;

[0031] Figure 3 A block diagram of the composition of a control optimization device for the user energy storage system based on multiple time scales provided by the embodiment of the present application;

[0032] Figure 4 Another block diagram of the composition of a control optimization device for the user energy storage system based on multiple time scales provided by the embodiment of the present application. Detailed implementation manners

[0033] Hereinafter, the exemplary embodiments of the present application will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be fully conveyed to those skilled in the art.

[0034] A user energy storage system generally refers to an energy storage device installed at the power user side. Such a system can store electrical energy and release it for use when needed. Taking the energy storage system installed on the residential user side as an example, a solar + energy storage solution can be implemented. For example, many families have installed a solution combining solar photovoltaic panels (PV) and a battery energy storage system. For example, when there is sufficient sunlight during the day, the solar power generation may exceed the immediate consumption demand of the family, and the excess energy can be stored in the battery for use at night or on cloudy days. This not only reduces the dependence on grid power but also further reduces the electricity bill by participating in the net metering program.

[0035] Now, during the operation of the user energy storage system, there are multiple decision-making factors, including electricity price changes, power load fluctuations, renewable energy fluctuations, etc. These factors not only have time-varying characteristics but also have different time scales.

[0036] Therefore, through research, the inventor found that if, due to the different impacts of different decision-making factors on different time scales, a distinction is made between the short-term time scale and the long-term time scale, and different decision-making factor combinations are used to make decision control on the user energy storage system respectively, and then integrated to give the final decision control, such an integration of multiple time scales realizes a more reasonable control strategy for the user energy storage system.

[0037] Based on the above considerations, the embodiment of the present application provides a control optimization method for the user energy storage system based on multiple time scales, as Figure 1 shown. The following specific steps are provided for the embodiment of the present invention:

[0038] 101. Construct a state space corresponding to the operating environment of the user energy storage system. In the state space, a first state variable and a second state variable are preset in advance. The first state variable is used to assist in making decisions on the control operation of the user energy storage system on the first time scale, and the second state variable is used to assist in making decisions on the control operation of the user energy storage system on the second time scale. The first time scale is equal to the time length of one time step in the time series represented by the second time scale.

[0039] It should be noted that, in order to distinguish different state variables, state information, preset processing frameworks, and decision-making information, the embodiments of the present application use the words "first" and "second" for identification, and they only serve as identification and there is no ambiguity in the order of precedence.

[0040] Among them, the state space contains multiple preset state variables. The state variables are used to represent different influencing factors on the user energy storage system during operation. The embodiments of the present application list several preset state variables as follows:

[0041] (1) State of Charge (SOC) of the battery: SOC is a key parameter reflecting the remaining battery power, usually ranging from 0% to 100%. It directly affects the discharge capacity and strategy selection of the energy storage system.

[0042]

[0043] (2) Current electricity price: The change of the electricity price determines the economic benefits of charging and discharging. The energy storage system usually charges at a low electricity price and discharges at a high electricity price. Therefore, the electricity price is a key influencing factor for the optimization strategy. Denote the current electricity price as e(t).

[0044] (3) Load demand (power load status): The load demand directly affects the working state and load balance of the energy storage system. It is necessary to give priority to providing power support during high load periods and reduce discharge during low load periods. Denote the current load demand as P(t).

[0045] (4) Ambient temperature: Temperature affects the battery life and performance. In high or low temperature environments, the charging and discharging efficiency and safety of the battery may both decrease.

[0046] (5) Time characteristics: Time characteristics such as intra-day time periods and days of the week within a week have a significant impact on the changes in electricity price and load, and can be used to help the model predict demand peaks and electricity price fluctuations.

[0047] It should be noted that the above are only exemplary examples in the embodiments of the present application. In order to make more accurate decisions on the user energy storage system, a more optimal way is to construct more multi-dimensional preset state variables in the state space, so as to obtain more dimensional decision-making factors.

[0048] Further, among these preset state variables, considering the different decision-making impacts on the user energy storage system at different time scales, for the purpose of dividing into short-term and long-term time scales in the embodiments of the present application, the preset state variables at different time scales are correspondingly divided and associated with different time scales. It should be noted that, for example, for the preset state variable "electricity price", its impacts on the short-term and long-term time scales are equally important (i.e., both are very important). Therefore, preset state variables like this can be repeatedly classified and associated with both time scales. Accordingly, if the short-term time scale is defined as the "first time scale" and the long-term time scale is defined as the "second time scale", then the preset state variables are correspondingly divided into "first state variables" and "second state variables", and based on actual decision-making requirements, there can be the same preset state variables in the two types of state variables obtained by the division.

[0049] Further, to characterize the difference between the short-term and long-term time scales, in the embodiments of the present application, it is set that the first time scale is equal to the time length of one time step in the time series represented by the second time scale. For example, taking the time units "day" and "hour" as an example: the short-term time scale makes a decision every hour, while the long-term time scale makes a decision every day.

[0050] 102. Obtain the current environment information corresponding to the operating environment of the user energy storage system.

[0051] The operating environment of the user energy storage system is very complex and changeable, such as it will involve the comprehensive impacts of factors such as electricity price fluctuations, load demand changes, environmental temperature fluctuations, and battery health status. In the embodiments of the present application, the current environment information is obtained for decision-making control of the user energy storage system in the future.

[0052] 103. Based on the state space, parse out the first state information corresponding to the first state variable and the second state information corresponding to the second state variable from the current environment information.

[0053] In the embodiments of the present application, the obtained current environment information can be, but is not limited to, including structured data and unstructured data. Therefore, through parsing and processing, it is compared with the preset state variables in the state space, and considering the division of the short-term and long-term time scales, it is equivalent to performing transformation processing on the current state information to obtain the first state information corresponding to the first state variable and the second state information corresponding to the second state variable.

[0054] 104. Process the first state information using the first-layer preset processing framework and output the first decision information. The first preset processing framework makes a control decision on the user energy storage system at the first time scale.

[0055] The embodiments of the present application provide a two - layer optimization framework for processing. For example, in the top - layer optimization framework (the second - layer preset processing framework), decision - making information is output to the user energy storage system on a long - term time scale, and in the bottom - layer optimization framework (the first - layer preset processing framework), decision - making information is output to the user energy storage system on a short - term time scale.

[0056] It should be noted that on a short - term time scale, more emphasis is placed on the decision - making of the daily power dispatching of the user energy storage system. For example, on a short - term time scale, the first state variables pre - assigned can include, but are not limited to, electricity price, load demand, and state of charge of the battery (i.e., the current state SOC of the energy storage device). Based on the first - state information of such first state variables, the embodiments of the present application can make decisions within a short time in the first - layer preset processing framework. The purpose is to adjust the charge - discharge operation of the energy storage system to cope with peak electricity prices, changes in load demand, or emergencies in grid operation.

[0057] Exemplarily, in the first - layer preset processing framework, algorithms that can be used include, but are not limited to, dynamic programming, heuristic algorithms, and model - predictive - control - based methods.

[0058] Dynamic programming decomposes the optimization problem into multiple stages, gradually calculates the optimal decision for each stage, and thus obtains the global optimal solution. Heuristic algorithms such as genetic algorithms and particle swarm optimization (PSO) search by simulating natural processes to find approximate optimal solutions. The model - predictive - control - based method predicts future electricity prices and load demands and adjusts the control strategy in real - time, enabling the energy storage system to maximize economic benefits while ensuring system stability.

[0059] 105. The second - layer preset processing framework is used to process the second - state information and output the second decision - making information. The second preset processing framework makes control decisions for the user energy storage system on the second time scale; among them, in the first preset processing framework and the second preset processing framework, the first preset processing framework is the bottom - layer optimization framework, and the second preset processing framework is the top - layer optimization framework.

[0060] In the top - layer optimization framework (the second - layer preset processing framework), decision - making information is output to the user energy storage system on a long - term time scale. It should be noted that different from the emphasis on the short - term time scale, on a long - term time scale, more emphasis is placed on making decision control for the user energy storage system considering long - term economic benefits, energy storage device maintenance, battery life, etc. For example, on a long - term time scale, the second state variables pre - assigned can include, but are not limited to, parameter variables of battery health, charge - discharge cycle number variables, meteorological data variables, etc. Based on the second - state information of such second state variables, the embodiments of the present application can make decisions on a long - term time scale. The purpose is to comprehensively consider, for example, but not limited to, the following decision - making factors:

[0061] (1) Battery health management: The lifespan of a battery is usually affected by factors such as depth of discharge (DoD), charging rate, and temperature. Long-term optimization requires designing charge and discharge strategies to extend the battery's lifespan. For example, by controlling the depth of each charge and reducing over-discharge or over-charging, the battery can be prevented from aging too quickly.

[0062] (2) Investment and replacement of energy storage devices: Long-term optimization also involves investment decisions regarding energy storage devices. For example, the initial investment cost of the energy storage system, the capacity selection of the equipment, and the maintenance and replacement strategies for the equipment in the next few years. These decisions not only affect the economic return of the system but also determine the lifespan of the equipment.

[0063] (3) Charge and discharge cycle control: The number of charge and discharge cycles of a battery is an important factor affecting the battery's lifespan. The long-term optimization problem needs to consider how to control the number of charge and discharge cycles while maintaining the economy of the system to avoid premature degradation of the battery.

[0064] Exemplarily, in the second-layer preset processing framework, it is possible but not limited to adopt multi-objective programming methods, genetic algorithms, particle swarm optimization algorithms (PSO), etc. The multi-objective programming method can help balance multiple objectives. For example, while maximizing economic benefits, it can extend the battery lifespan. The genetic algorithm searches for the optimal solution in a vast decision space by simulating the process of natural selection; while the particle swarm optimization algorithm finds a balance among multiple objectives by simulating the behavior of particles in a group.

[0065] 106. By using the second decision information as a control constraint condition to constrain the first decision information, a control operation is performed on the user energy storage system.

[0066] In the embodiments of the present application, the decision-making controls made on short-term and long-term time scales do not exist independently but need to be coordinated with each other and mutually constrained. Short-term decisions usually respond quickly to the dynamic changes in the power market and grid load fluctuations, while long-term decisions focus on the health management of energy storage devices, battery lifespan, and investment decisions. Therefore, fusing the two to coordinate possible conflicts and achieve a certain balance will bring the greatest possible benefits to the decision-making control of the user energy storage system. For example, as provided in the embodiments of the present application, using the decision on the long-term time scale as a control constraint condition to constrain the decision information on the short-term time scale, and finally outputting comprehensive decision information for performing a control operation on the user energy storage system.

[0067] As described above, the embodiments of the present application provide a control optimization method for a user energy storage system based on multiple time scales. Compared with the need to deal with different time scales involved in the decision-making factors of the user energy storage system, the embodiments of the present application comprehensively consider the short-term time scale and the long-term time scale to achieve effective control of the user energy storage system, providing an effective and reasonable energy storage control strategy, aiming to maximize the overall benefits of the user energy storage system in a dynamically changing environment as much as possible.

[0068] In some alternative embodiments, for more detailed explanation, the embodiments of the present application also provide another control optimization method for the user energy storage system based on multiple time scales, as Figure 2 shown. The embodiments of the present invention provide the following specific steps for this:

[0069] 201. Construct a state space corresponding to the operating environment where the user energy storage system is located. In the state space, a first state variable and a second state variable are preset in advance. The first state variable is used to assist in making decisions on the control operations of the user energy storage system on the first time scale, and the second state variable is used to assist in making decisions on the control operations of the user energy storage system on the second time scale. The first time scale is equal to the time length of one time step in the time series represented by the second time scale.

[0070] In the embodiments of the present application, for the explanation of this step, refer to 101, which will not be elaborated here.

[0071] 202. Obtain the current environment information corresponding to the operating environment where the user energy storage system is located.

[0072] 203. Based on the state space, parse out the first state information corresponding to the first state variable and the second state information corresponding to the second state variable from the current environment information.

[0073] In the embodiments of the present application, for the explanation of steps 202-203, refer to 102-103, which will not be elaborated here.

[0074] 204. Process the first state information using the first preset processing framework to output the first decision information. The first preset processing framework is to make control decisions on the user energy storage system on the first time scale.

[0075] Among them, the first state variables exemplified in the embodiments of the present application may include, but are not limited to: electricity price, load demand, and state of charge of the battery. Based on this exemplification, the detailed steps provided in this step are as follows:

[0076] A1. Use a pre-trained first reinforcement learning model to perform perceptual interaction processing with the state information corresponding to the current electricity price, load demand, and state of charge of the battery respectively, and output the current decision information corresponding to the current first time scale.

[0077] The first reinforcement learning model can be, but is not limited to, a model trained based on a reinforcement learning framework for implementing control on a user energy storage system. This model includes a policy network and a value network. Subsequently, in the control scenario of the user energy storage system, during the process of the model processing data, the specific implementation methods provided by these two networks are as follows:

[0078] For example, in a user-side energy storage system, the policy network determines the charge and discharge rate for the next moment based on the current state information (such as the current electricity price, load demand, and state of charge of the battery). For example, it increases the charging rate when the electricity price is low and increases the discharging rate when the electricity price is high; in the user-side energy storage system, the value network evaluates the effect of the current charge and discharge strategy, such as predicting the electricity cost savings brought by adopting a specific charge and discharge rate in the next period of time. This evaluation result is used to adjust the action selection of the policy network to ensure that the system is optimized in the direction of maximizing revenue.

[0079] A2. Predict the new state information corresponding to the electricity price, load demand, and state of charge of the battery respectively at the next first time scale after executing the current decision information.

[0080] A3. Use the pre-trained first reinforcement learning model to perform perceptual interaction processing with the new state information and output the new decision information corresponding to the next first time scale.

[0081] A4. At consecutive time steps, by iteratively performing perceptual interaction processing on the first state information corresponding to the first state variable at each time scale, obtain the first decision information corresponding to each first time scale.

[0082] As can be seen from A2 - A4, it is applicable to each short-term time scale. According to the current state information at that time scale, the first reinforcement learning model is used for perceptual processing to output a decision information, and based on this decision information, the corresponding new current state information is obtained at the next adjacent time scale. Subsequently, based on the consecutive time steps formed by multiple short-term time scales, iterative execution operations are realized, such that the decision information made at the previous time scale will affect the current state information at the next time scale, thereby realizing how to iteratively predict the control operation of the user energy storage system on multiple short-term time scales.

[0083] Therefore, on a short-term time scale, more emphasis is placed on the decision-making for the daily power dispatching of the user energy storage system. For example: collect real-time electricity price, load demand, and SOC data of the energy storage device every certain time interval (such as every 1 hour), and calculate the optimal charge and discharge strategy based on the current input variables.

[0084] 205. Use the second preset processing framework to process the second state information and output the second decision information. The second preset processing framework makes control decisions for the user energy storage system on the second time scale. Among them, in the first preset processing framework and the second preset processing framework, the first preset processing framework is the underlying optimization framework, and the second preset processing framework is the top-level optimization framework.

[0085] Among them, the second state variables exemplified in the embodiments of the present application may include, but are not limited to: parameter variables of battery health, charge and discharge cycle count variables, and meteorological data variables. The specific steps include: using a pre-trained second reinforcement learning model to perform perceptual interaction processing with the second state information of the second state variable and output the second decision information corresponding to the second time scale.

[0086] The second reinforcement learning model may include, but is not limited to, having the same processing framework as the first reinforcement learning model, so as to obtain decision information about the user energy storage system on a long-term time scale.

[0087] 206. Use the second decision information as a control constraint condition to constrain the first decision information and perform a control operation on the user energy storage system.

[0088] The embodiments of the present application provide a two-layer optimization framework for processing (such as the first preset processing framework and the second preset processing framework). Since the first time scale is equal to the time length of one time step in the time series represented by the second time scale. For example, taking the time units "day" and "hour" as an example: the short-term time scale makes a decision every hour, while the long-term time scale makes a decision every day. Therefore, relative to the first time scale, these two-layer optimization frameworks form a relationship between the top-level optimization framework and the underlying optimization framework. Subsequently, the embodiments of the present application use the top-level optimization framework (such as decision information on a long-term time scale) as a control constraint condition to constrain the decision information on a short-term time scale, so as to finally perform a control operation on the user energy storage system.

[0089] It should be noted that the embodiments of the present application explain in detail the characteristics and concepts of the provided two-layer optimization framework as follows:

[0090] The purpose of the top-level optimization framework (planned on a long-term time scale) provided by the embodiments of the present application is to be responsible for formulating long-term strategies and goals, such as annual or quarterly energy management plans, equipment maintenance arrangements, etc. These decisions are usually based on historical data, prediction models, and external factors (such as electricity price trends, weather forecasts, etc.).

[0091] The purpose of the embodiment of this application to provide an underlying optimization framework (real-time scheduling on a short-term time scale) is: based on the current state information (such as real-time electricity price, load demand, etc.), make short-term operation decisions, such as adjusting the charge and discharge rate to maximize the immediate revenue.

[0092] Thus, a feedback mechanism is implemented based on the two-layer optimization framework, including: through this hierarchical optimization framework, decisions on the long-term scale can affect the decisions on the short-term scale, and the scheduling on the short-term scale can also inversely affect the long-term scale. For example, the long-term scale determines the investment and replacement cycle of energy storage devices, while the short-term scale scheduling can adjust the charge and discharge strategies of the energy storage system according to electricity price and load fluctuations, which can ensure a positive interaction between the two.

[0093] In short, the result of the top-level optimization provides a guiding framework for the bottom-level optimization, while the actual operation result of the bottom-level optimization is fed back to the top-level optimization for correcting the long-term plan. According to the concept of this feedback mechanism, the embodiment of this application provides 206 above and 207 below.

[0094] 207. By iteratively executing to adjust the decision information output by the first-layer preset processing framework and the second-layer preset processing framework respectively, to balance the decision control on the user energy storage system at the first time scale and the second time scale respectively to increase the decision-making revenue.

[0095] In the embodiment of this application, it is actually to couple the decision information on the short-term time scale and the decision information on the long-term time scale to coordinate the conflicts between the two, so that the user energy storage system can maximize the overall benefit in a dynamically changing environment. The specific implementation steps are as follows:

[0096] B1. Select a target first time scale from multiple first time scales as the base point.

[0097] B2. Starting from the target first time scale, by accumulating the first decision information corresponding to multiple first time scales at consecutive time steps, obtain the decision data set corresponding to the underlying optimization framework.

[0098] Among them, the decision data set refers to the data set obtained by summarizing the decision information on multiple short-term time scales at consecutive time steps.

[0099] B3. By using a preset reward function to evaluate the decision data set, obtain the predicted return corresponding to performing control charging or discharging operations on the user energy storage system.

[0100] Among them, the preset reward function includes, for example, the positive reward, neutral reward, and negative reward given when the decision and control energy storage system executes control charging and discharging; the role of this reward function is to provide a feedback signal for each decision, which is used to evaluate the control effect of the energy storage system achieved based on this decision. Several factors considered in the design of the preset reward function in the embodiments of this application are enumerated as follows:

[0101] (1) Economic reward: Design the reward by calculating the revenue obtained from the electricity price difference. For example, when the energy storage system discharges at a high electricity price, a large positive reward can be obtained, while discharging at a low electricity price will generate a negative reward.

[0102] (2) Stability reward: Encourage the behavior of peak shaving and valley filling to reduce the impact of load fluctuations on the power grid. When the energy storage system discharges during peak load and charges during valley load, the system will receive a positive reward.

[0103] (3) Safety penalty: To protect the battery life, when the charging and discharging behavior causes the state of charge (SOC) to exceed the reasonable range or the temperature to exceed the safe range, the system will be penalized, so as to encourage the agent to learn reasonable charging and discharging behaviors.

[0104] B4. Compare the predicted return with the expected revenue on the second time scale to obtain a comparison result.

[0105] Among them, the expected revenue referred to in the embodiments of this application is the revenue return on a long-term time scale. Since the first time scale provided in the embodiments of this application is equal to the time length of one time step in the time series represented by the second time scale, then the more the first time scales are accumulated, the closer it should be to reaching the second time scale. Therefore, when evaluating the decision data set obtained as in B2 in the embodiments of this application, it is actually to evaluate whether, when getting closer to reaching the second time scale, the decision information obtained based on multiple short-term time scales is also getting closer to the expected revenue on the long-term time scale.

[0106] B5. Use the analysis information feedback by the comparison result to adjust the processing parameters in the top-level optimization framework, and apply it to adjust the second decision information output by the second-layer preset processing framework to obtain the third decision information.

[0107] B6. Use the third decision information as the new control constraint condition for the first decision information.

[0108] If there is a difference in trends (such as running counter or conflicting) between the decision-making information obtained based on multiple short-term time scales and the expected benefits on the long-term time scale, then it should be considered whether the expected benefits on the long-term time scale meet the current user's electricity demand. Therefore, the second decision information output by the second-layer preset processing framework can be adjusted to obtain the third decision information, which is used as the new decision information to apply new control constraints in 206 and feedback to the short-term time scale. Based on this, reciprocal iterative adjustment / constraint operations can be realized in the two-layer optimization framework, so as to balance the decision-making controls of the first time scale and the second time scale on the user energy storage system respectively to increase the decision-making benefits, that is, to maximize the overall benefits of the user energy storage system in a dynamically changing environment.

[0109] Combined with the above B1 - B6, for example, the optimization scheme for decision-making on the long-term time scale (also known as the slow time scale, and the corresponding short time scale is also known as the fast time scale) in the embodiments of the present application includes the following:

[0110] On the slow time scale, the optimization objective is to formulate operation strategies on the long-term time scale, and these strategies usually involve a relatively long time range, such as daily, monthly or annual planning. The optimization of the slow time scale aims to determine the best decisions within these longer time periods to maximize the system benefits in the overall operation, which is expressed by formula (1):

[0111]

[0112] where, x s represents the decision variable of the slow time scale, C t represents the cost at time t, and x f represents the decision variable on the fast time scale. By optimizing this objective function, the system can make optimal decisions on the long-term time scale while considering the impact of decisions on the short-term time scale, thereby achieving the balance and optimization of the overall objective. For example, the optimization of the slow time scale can help formulate reasonable daily, monthly or annual plans, and at the same time, by coordinating the real-time decisions on the fast time scale, the objectives on the short-term time scale can be matched with the long-term plan.

[0113] In some modified embodiments, during the process of data processing using the first-layer preset processing framework and the second-layer preset processing framework, the embodiments of the present application can also add multiple target conditions as multiple constraint conditions, adopt the preset optimal solution algorithm, seek a balance among multiple target conditions on the user energy storage system, and allocate weights to each of the multiple target conditions; according to the weights of each of the multiple target conditions, participate in the respective processing processes on the first-layer preset processing framework and the second-layer preset processing framework.

[0114] In practical applications, the optimization of energy storage systems usually involves multiple objectives, and there are often complex interactions and potential contradictions among these objectives. Taking the goals of economic efficiency, battery life, and system stability as an example, to maximize economic efficiency, the system may need to perform deep charge and discharge operations frequently, but this will accelerate the aging of the battery and affect its life; to optimize the battery life, the charge and discharge operations may need to be restricted, but this will reduce the economic efficiency of the system. In addition, there may also be a trade-off between the system stability and economic efficiency. For example, frequent charge and discharge in response to load fluctuations may lead to a decrease in economic returns.

[0115] The preset optimal solution algorithm provided by the embodiments of this application, such as multi-objective optimization (Multi-Objective Optimization, MOO), is usually represented by the following mathematical model for multi-objective optimization problems, as shown in formula (2) below;

[0116] min F(x)={f1(x),f2(x),…,f m (x)}

[0117] s.t.g i (x)≤0,i=1,2,3…

[0118] h j (x)=0,j=1,2,3…

[0119] In multi-objective optimization, since multiple competing objectives are involved, the optimal solution usually cannot be directly defined by a single solution. This is because during the optimization process, some objectives may conflict with other objectives. For example, only when a solution cannot be replaced by other solutions in some objectives for all objectives, this solution is a Pareto optimal solution.

[0120] The preset optimal solution algorithm provided by the embodiments of this application is implemented by using weights. A classic method is the weighting method. The weighting method linearly combines multiple objectives into a single-objective optimization problem through weight coefficients: as shown in formula (3) below;

[0121]

[0122] In addition, there is also the goal priority method. By setting priorities for multiple objectives, each objective function is optimized in turn. The written formula (4) is:

[0123] minf1(x)

[0124] s.t.f2(x)≤α,f3(x)≤β…

[0125] Optimization methods for directly finding the Pareto optimal solution set are usually based on evolutionary computing methods, such as the genetic algorithm mentioned above. These methods use a population search mechanism to obtain an approximate solution of the Pareto front in one run.

[0126] In the actual application process, the objective function and constraint conditions in a dynamic environment may change over time, such as the electricity price fluctuation in the energy market. Therefore, it is necessary to adjust the optimization strategy in real time. A feasible solution is to achieve dynamic response based on a rolling optimization window or an online learning mechanism.

[0127] Above, in some alternative embodiments, the embodiments of the present application propose control decisions for the user energy storage system on multiple time scales and under multiple objective conditions. For example, deep reinforcement learning (DRL) is adopted. Deep learning can automatically discover the potential laws and uncertainties in the system by learning a large amount of historical data, while reinforcement learning can help optimize the dynamic decision-making process. Then, the trained deep reinforcement learning model can make more intelligent decisions through self-learning and interaction with the environment, further improving the optimization performance.

[0128] In an example scenario, such a deep learning reinforcement model is used to make control decisions for the user energy storage system, achieving comprehensive consideration of time scales and multiple objective conditions. For example, in the user energy storage system, it is not only necessary to optimize the load scheduling within multiple time periods (i.e., on multiple time scales), but also necessary to make trade-offs among multiple objectives (such as economy, stability, and environmental protection) to ultimately maximize the overall benefit of the user energy storage system in a dynamically changing environment.

[0129] Further, as an implementation of the method shown above Figure 1 、 Figure 2 The embodiments of the present application provide a control optimization device for the user energy storage system based on multiple time scales. The device embodiments correspond to the foregoing method embodiments. For the convenience of reading, the device embodiments will not repeat the detailed content in the foregoing method embodiments one by one. However, it should be clear that the device in this embodiment can correspondingly implement all the content in the foregoing method embodiments. This device is applied to make energy storage decisions for the user energy storage system. Specifically, as shown in Figure 3 The device includes:

[0130] A building unit 31 is configured to build a state space corresponding to the operating environment of the user energy storage system. In the state space, a first state variable and a second state variable are preset in advance. The first state variable is used to assist in making decisions on the control operation of the user energy storage system on a first time scale, and the second state variable is used to assist in making decisions on the control operation of the user energy storage system on a second time scale. The first time scale is equal to the time length of one time step in the time series represented by the second time scale;

[0131] An acquisition unit 32 is configured to acquire the current environment information corresponding to the operating environment of the user energy storage system;

[0132] An analysis unit 33 is configured to analyze, based on the state space, the first state information corresponding to the first state variable and the second state information corresponding to the second state variable from the current environment information;

[0133] A first processing unit 34 is configured to process the first state information by using a first preset processing framework and output first decision information. The first preset processing framework is to make a control decision on the user energy storage system on the first time scale;

[0134] A second processing unit 35 is configured to process the second state information by using a second preset processing framework and output second decision information. The second preset processing framework is to make a control decision on the user energy storage system on the second time scale. Among them, in the first preset processing framework and the second preset processing framework, the first preset processing framework is a bottom-layer optimization framework, and the second preset processing framework is a top-layer optimization framework;

[0135] A first execution unit 36 is configured to perform a control operation on the user energy storage system by using the second decision information as a control constraint condition to constrain the first decision information.

[0136] Further, as Figure 4 shown, the device further includes: a second execution unit 37; specifically, the second execution unit 37 is configured to:

[0137] Select a target first time scale from multiple first time scales as a base point;

[0138] Starting from the target first time scale, by accumulating the first decision information corresponding to multiple first time scales at consecutive time steps, obtain a decision data set corresponding to the bottom-layer optimization framework;

[0139] Evaluate the decision data set by using a preset reward function to obtain the predicted return corresponding to performing a control charging or discharging operation on the user energy storage system;

[0140] By comparing the predicted return with the expected benefit on the second time scale, a comparison result is obtained;

[0141] Using the analysis information feedback from the comparison result, adjust the processing parameters in the top-level optimization framework, and apply it to adjust the second decision information output by the second-layer preset processing framework to obtain the third decision information;

[0142] Using the third decision information as a new control constraint condition for the first decision information;

[0143] By iteratively executing the adjustment of the decision information output by the first-layer preset processing framework and the second-layer preset processing framework respectively, to balance the first time scale and the second time scale in making decision control for the user energy storage system to increase decision benefits.

[0144] Further, as Figure 4 shown, the first state variable at least includes electricity price, load demand, and state of charge of the battery, and the first processing unit 34 includes:

[0145] A first processing module 341, configured to use a pre-trained first reinforcement learning model to perform perception interaction processing through the state information corresponding to the current electricity price, the load demand, and the state of charge of the battery respectively, and output the current decision information corresponding to the current first time scale;

[0146] A prediction module 342, configured to predict the new state information corresponding to the electricity price, the load demand, and the state of charge of the battery respectively on the next first time scale after executing the current decision information;

[0147] A second processing module 343, configured to use the pre-trained first reinforcement learning model to perform perception interaction processing through the new state information, and output the new decision information corresponding to the next first time scale;

[0148] An execution module 344, configured to perform perception interaction processing on the first state information corresponding to the first state variable on each time scale by iterative execution at continuous time steps, to obtain the first decision information corresponding to each first time scale.

[0149] Further, as Figure 4 shown, the second state variable at least includes parameter variables of battery health condition, charge and discharge cycle number variables, and meteorological data variables, and the second processing unit 35 is specifically configured to:

[0150] Using a pre-trained second reinforcement learning model, through perceptual interaction processing with the second state information of the second state variable, output the second decision information corresponding to the second time scale.

[0151] Further, as Figure 4 shown, the device further includes:

[0152] An adding unit 38, configured to add multiple target conditions as multiple constraint conditions during the process of data processing using the first-layer preset processing framework and the second-layer preset processing framework;

[0153] A solving unit 39, configured to use a preset optimal solution algorithm to seek a balance among multiple target conditions on the user energy storage system, so as to allocate weights to each of the multiple target conditions;

[0154] A third execution unit 310, configured to participate in the respective processing processes on the first-layer preset processing framework and the second-layer preset processing framework according to the weights of each of the multiple target conditions.

[0155] As above, the control optimization device on the user energy storage system based on multiple time scales includes a processor and a memory. The above-mentioned construction unit, acquisition unit, parsing unit, first processing unit, second processing unit, first execution unit, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions.

[0156] The processor includes a kernel, and the kernel retrieves the corresponding program units from the memory. One or more kernels can be set, and by adjusting the kernel parameters, the effective control of the user energy storage system is achieved by integrating the short-term time scale and the long-term time scale, providing an effective and reasonable energy storage control strategy, with the aim of maximizing the overall benefit of the user energy storage system in a dynamically changing environment as much as possible.

[0157] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the control optimization method on the user energy storage system based on multiple time scales as described above is implemented.

[0158] An embodiment of the present application provides an electronic device, the device includes at least one processor, and at least one memory and a bus connected to the processor; wherein, the processor and the memory complete mutual communication through the bus; the processor is used to call program instructions in the memory to execute the control optimization method on the user energy storage system based on multiple time scales as described above.

[0159] The present application also provides a computer program product which, when executed on a data processing device, is adapted to execute a program initialized with the steps of the control optimization method for a user energy storage system based on multiple time scales.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general purpose computers, special purpose computers, embedded processors, or other programmable data processing devices to produce a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0161] In a typical configuration, the device includes one or more processors (CPUs), a memory, and a bus. The device may also include an input / output interface, a network interface, etc.

[0162] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip. The memory is an example of a computer-readable medium.

[0163] Computer-readable media includes permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0164] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

[0165] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A control optimization method for a user energy storage system based on multiple time scales, characterized in that, The method includes: Constructing a state space corresponding to the operating environment of the user energy storage system, where a first state variable and a second state variable are preset in the state space. The first state variable is used to assist in making decisions on the control operation of the user energy storage system on a first time scale, and the second state variable is used to assist in making decisions on the control operation of the user energy storage system on a second time scale. The first time scale is equal to the time length of one time step in the time series represented by the second time scale; Obtaining the current environment information corresponding to the operating environment of the user energy storage system; Based on the state space, parsing out the first state information corresponding to the first state variable and the second state information corresponding to the second state variable from the current environment information; Processing the first state information using a first preset processing framework to output first decision information. The first preset processing framework makes a control decision on the user energy storage system on the first time scale; Processing the second state information using a second preset processing framework to output second decision information. The second preset processing framework makes a control decision on the user energy storage system on the second time scale; wherein, in the first preset processing framework and the second preset processing framework, the first preset processing framework is the underlying optimization framework, and the second preset processing framework is the top-level optimization framework; Performing a control operation on the user energy storage system by using the second decision information as a control constraint condition to constrain the first decision information.

2. The method according to claim 1, wherein The method further includes: Selecting a target first time scale from multiple first time scales as a base point; Starting from the target first time scale, by accumulating the first decision information corresponding to multiple first time scales at consecutive time steps, obtaining a decision data set corresponding to the underlying optimization framework; Evaluating the decision data set by using a preset reward function to obtain the predicted return corresponding to performing a control charging or discharging operation on the user energy storage system; Comparing the predicted return with the expected revenue on the second time scale to obtain a comparison result; Using the analysis information feedback by the comparison result to adjust the processing parameters in the top-level optimization framework, and applying it to adjust the second decision information output by the second preset processing framework to obtain third decision information; Using the third decision information as a new control constraint condition for the first decision information; By iteratively executing and adjusting the decision information output by the first preset processing framework and the second preset processing framework respectively, to balance the decision control made by the first time scale and the second time scale on the user energy storage system to increase the decision revenue.

3. The method according to claim 1 or 2, characterized in that, The first state variable at least includes electricity price, load demand, and state of charge of the battery. Processing the first state information using the first preset processing framework to output first decision information includes: Using a pre-trained first reinforcement learning model, through perceptual interaction processing with the state information corresponding to the current electricity price, the load demand, and the battery state of charge respectively, output the current decision information corresponding to the current first time scale; Predict the new state information corresponding to the electricity price, the load demand, and the battery state of charge respectively at the next first time scale after executing the current decision information; Using the pre-trained first reinforcement learning model, through perceptual interaction processing with the new state information, output the new decision information corresponding to the next first time scale; At consecutive time steps, through iterative execution of perceptual interaction processing on the first state information corresponding to the first state variable at each time scale, obtain the first decision information corresponding to each first time scale.

4. The method according to claim 1 or 2, characterized in that, The second state variable at least includes parameter variables of the battery health condition, charge and discharge cycle number variables, and meteorological data variables. The second state information is processed using a second preset processing framework to output second decision information, including: Using a pre-trained second reinforcement learning model, through perceptual interaction processing with the second state information of the second state variable, output the second decision information corresponding to the second time scale.

5. The method according to claim 1, wherein During the process of data processing using the first preset processing framework and the second preset processing framework, add multiple target conditions as multiple constraint conditions; Using a preset optimal solution algorithm, seek a balance among multiple target conditions on the user energy storage system to allocate weights to each of the multiple target conditions; According to the weights of each of the multiple target conditions, participate in the respective processing processes on the first preset processing framework and the second preset processing framework.

6. A control optimization device for a user energy storage system based on multiple time scales, characterized in that, The device includes: A construction unit for constructing a state space corresponding to the operating environment of the user energy storage system. The first state variable and the second state variable are preset in the state space. The first state variable is used to assist in making decisions on the control operation of the user energy storage system at the first time scale, and the second state variable is used to assist in making decisions on the control operation of the user energy storage system at the second time scale. The first time scale is the time length equal to one time step in the time series represented by the second time scale; An acquisition unit for acquiring the current environment information corresponding to the operating environment of the user energy storage system; An analysis unit for parsing, based on the state space, the first state information corresponding to the first state variable and the second state information corresponding to the second state variable from the current environment information; A first processing unit for processing the first state information using a first preset processing framework to output first decision information. The first preset processing framework is to make a control decision on the user energy storage system at the first time scale; A second processing unit, configured to process the second state information by using a second preset processing framework and output second decision information, where the second preset processing framework is to make a control decision on the user energy storage system at the second time scale; wherein, in the first preset processing framework and the second preset processing framework, the first preset processing framework is a bottom-layer optimization framework and the second preset processing framework is a top-layer optimization framework; A first execution unit, configured to perform a control operation on the user energy storage system by using the second decision information as a control constraint condition to constrain the first decision information.

7. The device according to claim 6, characterized in that The apparatus further includes: a second execution unit; specifically, the second execution unit is configured to: Select a target first time scale from multiple first time scales as a base point; Starting from the target first time scale, obtain a decision data set corresponding to the bottom-layer optimization framework by accumulating the first decision information corresponding to multiple first time scales at consecutive time steps; Evaluate the decision data set by using a preset reward function to obtain a predicted return corresponding to performing a control charge or discharge operation on the user energy storage system; Compare the predicted return with the expected revenue at the second time scale to obtain a comparison result; Use the analysis information fed back by the comparison result to adjust the processing parameters in the top-layer optimization framework, and apply the adjustment to the second decision information output by the second preset processing framework to obtain a third decision information; Use the third decision information as a new control constraint condition for the first decision information; By iteratively executing and adjusting the decision information output by the first preset processing framework and the second preset processing framework respectively, balance the decision control made by the first time scale and the second time scale on the user energy storage system to increase the decision revenue.

8. The device according to claim 6 or 7, characterized in that The first state variable at least includes electricity price, load demand, and state of charge of the battery, and the first processing unit includes: A first processing module, configured to use a pre-trained first reinforcement learning model to perform a perception interaction process with the state information corresponding to the current electricity price, the load demand, and the state of charge of the battery respectively, and output current decision information corresponding to the current first time scale; A prediction module, configured to predict, after executing the current decision information, new state information corresponding to the electricity price, the load demand, and the state of charge of the battery respectively at the next first time scale; A second processing module, configured to use the pre-trained first reinforcement learning model to perform a perception interaction process with the new state information and output new decision information corresponding to the next first time scale; An execution module, configured to, at consecutive time steps, obtain the first decision information corresponding to each first time scale by iteratively performing a perception interaction process on the first state information corresponding to the first state variable at each time scale.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the control optimization method for a user energy storage system based on multiple time scales as described in any one of claims 1-5.

10. An electronic device, characterized in that, The device includes at least one processor, at least one memory connected to the processor, and a bus; wherein the processor and the memory communicate with each other through the bus; The processor is configured to call program instructions in the memory to execute the control optimization method for a user energy storage system based on multiple time scales as described in any one of claims 1-5.

Images