Operation optimization method and system for energy storage participation auxiliary service market

Abstract

The invention provides an operation optimization method and system for an energy storage participation auxiliary service market, and the method comprises the steps: determining an energy storage performance aging factor, and defining an energy storage aging time function according to the energy storage performance aging factor; the energy storage performance aging factors comprise calendar aging and cycle aging; based on the energy storage aging time function, constructing an intra-day operation model taking energy storage expected life maximization and intra-day benefit maximization as objective functions; and solving the intra-day operation model by adopting a multi-agent algorithm of a non-adversarial game to obtain an optimal strategy of energy storage intra-day operation. According to the invention, the relation between the energy storage aging characteristic and the working characteristic is introduced to carry out the energy storage intra-day operation decision, the economic benefit can be truly evaluated, the energy efficiency of energy storage and the stability of scheduling are considered, and the operation maintenance level is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of energy storage scheduling, and in particular to an operation optimization method and system for energy storage participating in ancillary service markets. Background Art

[0002] Energy storage, particularly electrochemical energy storage, is widely used in the power sector due to its superior electrical properties and rapid response capabilities. As the proportion of renewable energy in the power grid continues to increase, the demand for energy storage to smooth out peak-load fluctuations and mitigate frequency fluctuations across the entire grid is rapidly increasing. For the power market, peak-shaving and frequency regulation services within the ancillary services market are the primary applications for energy storage.

[0003] As a fixed asset, energy storage operations must consider its return on investment. Generally, energy storage capacity allocation, as an investment planning issue, fully considers the operation and maintenance costs and return on investment over the operating cycle. However, as specialized electrical equipment, energy storage's operating characteristics and aging characteristics are significantly time-dependent. Previous operational decision-making methods have only considered operation and maintenance costs and specific depreciation methods, without considering the performance losses caused by aging.

[0004] Currently, energy storage aging causes losses, including calendar aging and cyclic aging. Calendar aging is generally calculated using linear depreciation, amortized over the available lifecycle. Cyclic aging, on the other hand, refers to the significant changes in core parameters such as output characteristics and state of charge over time as the number of charging cycles increases. Previous energy storage benefit assessment and operational scheduling decision-making methods have not considered the additional benefit losses or changes in the actual operating cycle caused by the aging characteristics of energy storage, nor have they incorporated aging characteristics into operational decisions. This has prevented a complete and objective assessment of the actual operational evaluation cycle or true return on investment of energy storage. Furthermore, they have not considered how to improve energy storage maintenance efficiency and overall grid stability while maximizing benefits from the perspective of the energy storage lifecycle. Summary of the Invention

[0005] To this end, the present invention provides an operational optimization method and system for energy storage participating in the ancillary service market, aiming to solve the technical problems in the existing technology of inaccurate operational evaluation and low energy storage maintenance efficiency caused by the failure to consider the performance loss caused by aging characteristics in energy storage operations.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions:

[0007] According to a first aspect of the present invention, the present invention provides an operation optimization method for energy storage participating in ancillary service markets, the method comprising:

[0008] Determining an energy storage performance aging factor, and defining an energy storage aging time function according to the energy storage performance aging factor; the energy storage performance aging factor includes calendar aging and cycle aging;

[0009] Based on the energy storage aging time function, a daily operation model is constructed with maximizing the expected life of energy storage and maximizing the daily benefit as the objective function;

[0010] A non-adversarial multi-agent algorithm is used to solve the intraday operation model and obtain the optimal strategy for energy storage intraday operation.

[0011] According to a second aspect of the present invention, the present invention provides an operation optimization system for energy storage participating in ancillary service markets, the system comprising:

[0012] An aging function determination module is used to determine an energy storage performance aging factor and define an energy storage aging time function according to the energy storage performance aging factor; the energy storage performance aging factor includes calendar aging and cycle aging;

[0013] An operation model construction module is used to construct a daily operation model with the objective functions of maximizing the expected life of energy storage and maximizing the daily benefit based on the energy storage aging time function;

[0014] The operation strategy solving module is used to solve the intraday operation model using a non-adversarial multi-agent algorithm to obtain the optimal strategy for the energy storage intraday operation.

[0015] The present invention adopts the above technical solution and has at least the following beneficial effects:

[0016] Through the solution of the present invention, energy storage performance aging factors are determined, and an energy storage aging time function is defined based on these energy storage performance aging factors; these energy storage performance aging factors include calendar aging and cyclic aging. Based on the energy storage aging time function, an intraday operation model is constructed with maximizing the expected lifespan of energy storage and maximizing intraday benefits as objective functions. A non-adversarial multi-agent algorithm is used to solve the intraday operation model, resulting in the optimal strategy for intraday energy storage operations. Thus, by incorporating the relationship between energy storage aging characteristics and operating characteristics into intraday energy storage operation decisions, economic benefits can be truly assessed while taking into account both the energy efficiency of energy storage and the stability of scheduling, significantly improving the level of operation and maintenance.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram illustrating a flow chart of an operation optimization method for energy storage participating in ancillary service markets provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram showing the structure of an operation optimization system for energy storage participating in ancillary service markets provided by an embodiment of the present invention is shown;

[0021] Figure 3 A schematic diagram of the physical structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0023] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0024] The embodiment of the present invention provides an operation optimization method for energy storage participating in ancillary service market, such as Figure 1 As shown, it may at least include the following steps S101 to S103:

[0025] Step S101 : determining an energy storage performance aging factor, and defining an energy storage aging time function according to the energy storage performance aging factor.

[0026] Energy storage performance aging factors mainly include calendar aging and cycle aging. Calendar aging refers to linear depreciation, so the performance loss caused by daily calendar aging can be considered fixed. Cycle aging, on the other hand, is mainly caused by the increase in charge and discharge times. Numerous experiments have shown that the relationship between cycle aging and charge and discharge times is not a simple linear relationship. Specifically, for a given charge and discharge depth, the mathematical expression for the relationship between the maximum charge and discharge times and the charge and discharge depth can be:

[0027]

[0028] in, Indicates the number of charge and discharge times during the service life of the energy storage; φ represents the charge and discharge depth, that is, the ratio of the power released / absorbed in a complete charge and discharge cycle to the rated capacity; It indicates the number of times the energy storage can be charged and discharged within its service life at 100% charge and discharge depth; δ indicates the fitting parameter. and δ are parameters set when the energy storage system leaves the factory.

[0029] Referring to formula (1), given the charge and discharge depth, it can be converted into the number of charge and discharge times at 100% charge and discharge depth. The formula is expressed as:

[0030]

[0031] in, Indicates that Ω φ Converted to the equivalent cumulative number of charging times at 100% charging depth.

[0032] Furthermore, as the energy storage ages, its energy conversion efficiency will change. According to the empirical formula for the internal resistance of the energy storage battery, there is the following relationship between its energy conversion efficiency and battery resistance, which can be expressed as:

[0033]

[0034] Among them, U rate Indicates the rated operating voltage of the energy storage battery; R0 indicates the internal resistance of the energy storage battery when it leaves the factory; ζ0 indicates the energy conversion efficiency of the energy storage battery when it leaves the factory; R n and ζ n They represent the internal resistance and energy conversion efficiency of energy storage in the nth year respectively.

[0035] The above formula (3) can be further deduced as follows:

[0036]

[0037] Similarly, the relationship between the maximum output power and rated power of energy storage in year n can be derived, and the formula is expressed as:

[0038]

[0039] in, Indicates the maximum output power of energy storage in year n; Indicates the rated power of the energy storage.

[0040] At the same time, according to the empirical formula, the law of change of the internal resistance of energy storage in the nth year with time can be expressed as follows:

[0041] R n =R0[1+λ(T float -n)] (6)

[0042] Among them, λ is the fitting parameter, which can be obtained by fitting the sample data; T float The floating charge life of the energy storage can be replaced by the expected life. According to the above formula (1), the expected life of the energy storage can be expressed as follows:

[0043]

[0044] Among them, T E represents the expected life of energy storage; D represents the number of working days of energy storage per year; Indicates the cumulative number of daily charges at 100% equivalent depth of charge and discharge.

[0045] Then, the equivalent cumulative number of charging times at 100% depth of charge in the nth year can be expressed as:

[0046]

[0047] Furthermore, according to the energy conversion efficiency ζ of the resistance of the energy storage in the nth year n , Maximum output power Equivalent cumulative charging times at 100% charging depth Combining the above equations (1) to (8), we can obtain the following derivation:

[0048]

[0049] in, Indicates the maximum output power of energy storage in the nth year; Indicates the rated power of energy storage; Indicates the equivalent cumulative number of charging times at 100% depth of charge in the nth year; Indicates the number of times the energy storage can be charged and discharged within its service life at 100% charge and discharge depth; ζ0 indicates the energy conversion efficiency of the energy storage when it leaves the factory; ζ n represents the energy conversion efficiency of energy storage in the nth year;

[0050] The derivation of formula (9) Substituting into the Arrhenius empirical formula, we can get the remaining capacity of energy storage in the nth year, which is expressed as:

[0051]

[0052] Among them, Q n Indicates the actual remaining capacity of energy storage in year n; Q rate It represents the rated capacity of the energy storage; η is the forward factor, which is related to the discharge rate of the energy storage; K is the environmental constant, which is the product of a constant and the thermodynamic temperature of the energy storage; RAE represents the activation energy of the electrochemical reaction generated when the energy storage is working; is a parameter that can be obtained through experiments.

[0053] Step S102 : Based on the energy storage aging time function, a daily operation model is constructed with the objective functions of maximizing the expected life of the energy storage and maximizing the daily benefit.

[0054] In the embodiment of the present invention, in order to take into account both the benefits of energy storage operation and the improvement of maintenance level, the objective function of the daily operation model should include maximizing the daily benefits and maximizing the expected life of energy storage.

[0055] Regarding the maintenance level target of energy storage, it can be seen from the above formula (7) that maximizing the expected life of energy storage is equivalent to minimizing the equivalent charge and discharge times of the cumulative 100% charge and discharge depth within a day. Therefore, the first objective function can be obtained, and the mathematical expression is:

[0056]

[0057] It is understandable that to achieve this goal, the corresponding measures to be taken are to optimize the charge and discharge cycle of energy storage, that is, to reasonably arrange the number of charge and discharge times, duration, and corresponding charge and discharge depth of energy storage. For a charge and discharge cycle of energy storage, it can be identified by its net power output. The charge and discharge depth of energy storage can be expressed as follows according to the definition:

[0058]

[0059] in, Indicates the charge and discharge depth of the mth charge or discharge cycle; It represents the net output of the mth dispatch instruction or other task energy storage; represents the discharge power of energy storage during period t; represents the charging power of the energy storage during period t; Indicates the moment when discharge ends; Indicates the moment when discharge begins; Indicates the moment when charging is completed; Indicates the time when charging starts; Indicates the energy loss coefficient of energy storage charging and discharging; ζ n It represents the energy conversion efficiency of energy storage in the nth year; the unit of t is one hour.

[0060] Based on the above formula (12), the formula for expressing the number of charge and discharge times at 100% charge and discharge depth accumulated in a day is defined as follows:

[0061]

[0062] Based on the above formula (13), the expression formula of the first objective function is improved, and the mathematical expression is:

[0063]

[0064] in, represents the depth of charge and discharge of the mth charge or discharge cycle; t represents the charge and discharge period; m represents the number of charge and discharge times; δ represents the fitting parameter.

[0065] Furthermore, with respect to the operational benefit goal of energy storage, when energy storage participates in both peak-shaving and frequency-regulating ancillary service markets, the mathematical expression of the second objective function for maximizing intraday benefits can be expressed as:

[0066] F2=max(R frs +R pss -C op -C mc -C alc ) (15)

[0067] Among them, R frs represents the benefits of energy storage participating in frequency regulation services; R pss represents the benefits of energy storage participating in peak load regulation services; C op Represents the operating cost of energy storage; C mc represents the daily maintenance cost of energy storage; C alc Represents the loss cost caused by aging of energy storage performance.

[0068] In formula (15), the revenue of energy storage participating in frequency regulation service is R frs Including frequency regulation capacity income and frequency regulation mileage income, it can be expressed as follows:

[0069]

[0070] Among them, R cr Indicates the total frequency regulation capacity benefit for the day; represents the compensation unit price of frequency regulation capacity during period t; μ represents the frequency regulation performance assessment coefficient set by the regulator; R represents the frequency regulation power of the energy storage system in the day-ahead period during period t; mi Indicates the total frequency modulation mileage income for the day; represents the unit price of mileage income subsidy for frequency modulation in period t; k represents the kth effective response in period t; K represents the total number of effective responses in period t; It represents the actual FM response power of the kth response in the t period, which is specifically expressed as:

[0071]

[0072] Among them, β t,k It represents the effective coefficient of the response recognized by ACG for the kth time in period t.

[0073] Furthermore, the revenue of energy storage participating in peak load regulation service is R pss , the mathematical expression can be:

[0074]

[0075] Among them, ρ t represents the time-of-use electricity price during period t; represents the discharge power of energy storage during period t; Represents the charging power of the energy storage during period t.

[0076] Furthermore, in the embodiment of the present invention, the operating cost of energy storage C op Considering the loss cost due to calendar aging, the mathematical expression can be:

[0077]

[0078] Among them, ρ op represents the operating cost coefficient of energy storage; ξ t represents the calendar aging loss coefficient of energy storage in period t; represents the frequency regulation power won by the energy storage system in the day-ahead period t; represents the discharge power of energy storage during period t; Represents the charging power of the energy storage during period t.

[0079] Furthermore, the daily maintenance cost C of energy storage is mc , the mathematical expression is:

[0080] C mc =ρ mc ·C tot (20)

[0081] Among them, C tot represents the book value of energy storage; ρ mc Indicates the conversion factor of energy storage maintenance cost allocated to each day.

[0082] Furthermore, the loss cost C caused by aging of energy storage performance alcConsidering the loss cost caused by cycle aging, the formula (10) can be Taking the partial derivative, we get:

[0083]

[0084] Approximately deducing formula (12), we get:

[0085]

[0086] The embodiment of the present invention takes one day as the inspection period, and the following results can be obtained:

[0087]

[0088] Combining equations (22) and (23), we can get the total cost of aging loss within a storage day, which is expressed as:

[0089]

[0090] Where Γ represents the loss cost coefficient; Q rate Indicates the rated capacity of energy storage; represents the depth of charge or discharge of the mth charge or discharge cycle; η is the forward factor, which is related to the discharge rate of the energy storage; K is the environmental constant; RAE represents the activation energy of the electrochemical reaction generated when the energy storage is working; z is a parameter that can be obtained through experiments; Indicates the cumulative equivalent number of charges in the nth year.

[0091] Combining equations (15) to (24) above, we can derive the intraday operation model for energy storage participating in the ancillary services market. It should be noted that the constraints of the intraday operation model include various parameter constraints and power constraints for energy storage, which are not detailed in this invention.

[0092] Step S103 , using a non-adversarial game multi-agent algorithm to solve the intraday operation model and obtain the optimal strategy for the intraday operation of energy storage.

[0093] The intraday operation model in this embodiment of the present invention includes two objective functions, sharing variables and parameters. Because some of these functions are nonlinear, conventional solutions are difficult to implement. Considering the conflicting objectives of minimizing the number of equivalent charge and discharge times per day and maximizing daily operating revenue, this embodiment of the present invention employs a multi-agent non-adversarial game (MAPPO) algorithm to solve the intraday operation model.

[0094] The following is an introduction to the multi-agent (MAPPO) algorithm, which includes at least the following steps S1 to S5:

[0095] Step S1: Construct a maintenance agent for seeking a working strategy that maximizes the expected life of energy storage under the condition of maximizing the expected benefit of energy storage, and a benefit agent for seeking a working strategy that maximizes the expected benefit of energy storage under the condition of maximizing the expected life of energy storage; and initialize the agent's strategy network, critic network, and target network.

[0096] First, two intelligent agents and corresponding networks (including a policy network, a critic network, and a target network) are constructed. Under the condition of shared parameters and state space, one intelligent agent focuses on seeking a working strategy that maximizes the expected lifespan of energy storage while maximizing the expected return of energy storage, and is defined as the maintenance intelligent agent; the other intelligent agent solves the problem of the previous intelligent agent in reverse, that is, it focuses on seeking a working strategy that maximizes the expected return of energy storage while maximizing the expected lifespan of energy storage, and is defined as the benefit intelligent agent.

[0097] Step S2: Construct an action space with decision variables, and define a state space with other parameters in the intraday operation model except the decision variables.

[0098] In the embodiment of the present invention, the action space represents the set of actions that the agent can take. The specific formula can be:

[0099]

[0100] Among them, M represents the decision variables, such as the number of charge and discharge times, the number of responses participating in frequency modulation, and other variables; represents the output decision for each time period, including charging and discharging power, bid power, etc.; τ represents the scheduling decision, including variables such as charging and discharging time and charging and discharging duration. It is understood that all parameters other than the decision variables in the intraday operation model are considered elements of the state space. Due to the large number of parameters, this invention does not elaborate on them here.

[0101] During the specific implementation process, the parameters in the intraday operation model can be input into the network to generate the state space and action space, and initialize the policy network, critic network and target network of the maintenance agent and benefit agent.

[0102] Step S3: The maintenance agent and the benefit agent share the critic network and state space for iterative training; and, for the preset objective function, the time difference method is used to calculate the reward function and advantage function of each agent, so that the agent can select the current operation strategy from the feasible set of action space.

[0103] During the training process for the intraday operation model, the agent selects a sequence of certain variables from the feasible set of action space, that is, an operation strategy. For two agents, the following preset objective function can be constructed simultaneously:

[0104]

[0105] In formula (26), θ represents the value of the joint strategy to be sought in the critic network; the subscript t represents the iteration step; E t Indicates the expectation of the function at the t-th iteration step; Represents the estimated value of the advantage function, which measures the action a t In state s t advantage under the above conditions; clip(·) indicates that the sequence in the brackets is clipped and assigned a value; ∈ indicates the clipping threshold, which is used to control the amplitude of the strategy update; γ t (θ) represents the update ratio of the current strategy to the old strategy, which can be expressed as:

[0106]

[0107] Among them, π θ (a t |s t ) represents the specific strategy of the t-th iteration step corresponding to θ in the policy network.

[0108] The advantage function is used to measure the t The mathematical expression for the quality of the average action relative to the current strategy is:

[0109]

[0110] in, is the discount factor; is the weight parameter in GAE (generalized advantage estimation); T represents the maximum iteration step; δ t It represents the time difference error of the t-th iteration step, which can be expressed as:

[0111]

[0112] In formula (29), V(s t ) indicates that based on state s t The value function of each agent is estimated by the Critic network. Multiple agents share the Critic network and consider the actions of other agents when taking actions, thereby learning the most effective strategy.

[0113] During the specific implementation process, the iteration step and other network parameters can be set to allow each agent to store and interact with the actions, states and parameters of each iteration step in the network; and the time difference method can be used to calculate the reward function and advantage function of each agent.

[0114] Step S4: During the iterative training process, the minimum mean square error of the loss function is calculated according to the current operation strategy, and the parameters of the critic network, policy network and target network are updated.

[0115] In the embodiment of the present invention, the critic network updates parameters by finding the minimum mean square error of the loss function. The specific form of the loss function can be:

[0116]

[0117] In formula (30), χ i represents the network parameters of agent i; Indicates based on s t ,γ t The expectation of ; σ represents the standard deviation; V i (s t ,χ i ) represents agent i based on (s t ,χ i )’s value function estimate; R t Represents the reward function, which means the cumulative reward from the current iteration step t to the future. It can be expressed as:

[0118]

[0119] in, represents the target network parameters of agent i.

[0120] In the specific implementation process, the MSE (minimum mean square error) iterative loss function is used to update the parameters of the Critic network; the preset objective function is clipped according to the Clip function The policy network of each agent is then propagated and updated through policy gradients; at the same time, a soft update mechanism is used to gradually update the parameters in the target network.

[0121] Step S5: Repeat steps S3 to S4 until the model converges and outputs the optimal strategy for daily energy storage operation.

[0122] The embodiment of the present invention provides an evaluation and decision-making method that can truly describe the benefits and costs of energy storage participating in the ancillary service market. That is, by modeling the relationship between the actual working parameters of energy storage and time, and incorporating it into the benefit objective function of energy storage and the decision-making of intraday charging and discharging arrangements, and finally using a multi-agent algorithm to provide the optimal strategy for the intraday operation of energy storage. Specifically, for the decision-making of energy storage participating in the peak-shaving and frequency regulation markets, in order to truly reflect the intraday benefits of energy storage, the present invention describes the time characteristics of energy storage performance and uses it as the goal of operation or scheduling. Specifically, maximizing intraday benefits and maximizing the service life of energy storage are simultaneously used as operational goals. The intraday operation of energy storage can be disassembled into two parts: one is the day-ahead declaration decision for participating in the ancillary service market, and the other is to consider the aging characteristics of energy storage and arrange the charging and discharging process of energy storage. By describing the model of aging characteristics and energy storage working characteristics, these two decisions are mutually coupled, that is, sharing parameters and boundary conditions. Through the present invention, the performance aging of energy storage is incorporated into the influencing factors of energy storage operation decision-making, which can accurately evaluate the actual benefits of energy storage daily operation; the equivalent number of energy storage charging times and the benefits of participating in the ancillary service market are taken together as operational decision-making goals, taking into account both economy and energy storage stability, and improving the level of operation and maintenance; the introduction of the MAPPO algorithm to solve the relatively conflicting dual problems can quickly and in real time propose the best solution for energy storage daily operation.

[0123] Further, as Figure 1 The embodiment of the present invention provides an operation optimization system for energy storage participating in the auxiliary service market, such as Figure 2 As shown, the system may include: an aging function determination module 210 , an operation model construction module 220 , and an operation strategy solution module 230 .

[0124] The aging function determination module 210 can be used to determine the energy storage performance aging factor and define the energy storage aging time function according to the energy storage performance aging factor; the energy storage performance aging factor includes calendar aging and cycle aging;

[0125] The operation model construction module 220 can be used to construct a daily operation model with the objective functions of maximizing the expected life of energy storage and maximizing the daily benefit based on the energy storage aging time function;

[0126] The operation strategy solving module 230 can be used to solve the intraday operation model using a non-adversarial multi-agent algorithm to obtain the optimal strategy for the energy storage intraday operation.

[0127] It should be noted that for other corresponding descriptions of the functional modules involved in the operation optimization system for energy storage participating in the ancillary service market provided by the embodiment of the present invention, please refer to Figure 1 The corresponding description of the method shown will not be repeated here.

[0128] Based on the above Figure 1 The method shown, accordingly, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the operation optimization method of energy storage participating in the auxiliary service market in any of the above embodiments are implemented.

[0129] Based on the above Figure 1 The method shown and Figure 2 The embodiment of the system shown in FIG. 1 further provides a physical structure diagram of a computer device, such as Figure 3 As shown, the computer device may include a communication bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device. The various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor is configured to execute the program stored in the memory and perform the steps of the above-described method for optimizing the operation of energy storage participating in ancillary service markets.

[0130] Those skilled in the art will clearly understand that the specific working processes of the systems, devices, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they will not be further described here.

[0131] In addition, the functional units in various embodiments of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into a single processing unit. The above-mentioned integrated functional units may be implemented in the form of hardware, software, or firmware.

[0132] Those skilled in the art will understand that if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can essentially or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes a number of instructions for enabling a computing device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention when running the instructions. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0133] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as a computing device such as a personal computer, a server, or a network device), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of a computing device, the computing device executes all or part of the steps of the method described in each embodiment of the present invention.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate from the scope of protection of the present invention.

Claims

1. An operational optimization method for energy storage participating in ancillary service markets, characterized in that: The method comprises: Determining an energy storage performance aging factor, and defining an energy storage aging time function according to the energy storage performance aging factor; the energy storage performance aging factor includes calendar aging and cycle aging; Based on the energy storage aging time function, a daily operation model is constructed with maximizing the expected life of energy storage and maximizing the daily benefit as the objective function; A non-adversarial multi-agent algorithm is used to solve the intraday operation model and obtain the optimal strategy for energy storage intraday operation.

2. The method according to claim 1, characterized in that Defining the energy storage aging time function according to the energy storage performance aging factor includes: According to the relationship between the maximum charge and discharge times and the charge and discharge depth, the charge and discharge times at 100% charge and discharge depth are determined; Based on the relationship between the energy conversion efficiency of energy storage and battery resistance, the energy conversion efficiency and maximum output power of the resistance of energy storage in the nth year are determined. At the same time, based on the time-varying pattern of the internal resistance of energy storage in the nth year, the equivalent cumulative number of charging times at 100% depth of charge in the nth year is obtained. According to the energy conversion efficiency of the resistor in the nth year, the maximum output power and the equivalent cumulative number of charging times at 100% depth of charge, the following derivation is obtained: in, Indicates the maximum output power of energy storage in the nth year; Indicates the rated power of energy storage; Indicates the equivalent cumulative number of charging times at 100% depth of charge in the nth year; Indicates the number of times the energy storage can be charged and discharged within its service life at 100% charge and discharge depth; ζ0 indicates the energy conversion efficiency of the energy storage when it leaves the factory; ζ n represents the energy conversion efficiency of energy storage in the nth year; The derivation in Substituting into the Arrhenius empirical formula, we can get the remaining capacity of energy storage in the nth year, which is expressed as: Among them, Q n Indicates the actual remaining capacity of energy storage in year n; Q rate It represents the rated capacity of energy storage; η is the forward factor; K is the environmental constant; RAE represents the electrochemical reaction activation energy generated when the energy storage is working; z is a parameter.

3. The method according to claim 2, characterized in that Determining the charge and discharge times at 100% charge and discharge depth based on the relationship between the maximum charge and discharge times and the charge and discharge depth includes: The mathematical expression for the relationship between the maximum charge and discharge times and the charge and discharge depth is: in, Indicates the number of charge and discharge times during the service life of the energy storage; φ represents the charge and discharge depth, that is, the ratio of the power released / absorbed in a complete charge and discharge cycle to the rated capacity; It represents the number of times the energy storage can be charged and discharged within its service life at 100% charge and discharge depth; δ represents the fitting parameter; Determine the charge and discharge depth and obtain the charge and discharge times at 100% charge and discharge depth. The formula is: in, Indicates that Ω φ Converted to the equivalent cumulative number of charging times at 100% charging depth.

4. The method according to claim 3, characterized in that The determining of the energy conversion efficiency and the maximum output power of the resistance of the energy storage in the nth year based on the relationship between the energy conversion efficiency of the energy storage and the battery resistance includes: The relationship between the energy conversion efficiency of the energy storage and the battery resistance is expressed as follows: Among them, U rate Indicates the rated operating voltage of the energy storage battery; R0 indicates the internal resistance of the energy storage battery when it leaves the factory; ζ0 indicates the energy conversion efficiency of the energy storage battery when it leaves the factory; R n and ζ n They represent the internal resistance and energy conversion efficiency of energy storage in the nth year respectively; The relationship between the energy conversion efficiency of the energy storage and the battery resistance is deduced to obtain: The maximum output power of the energy storage in year n is expressed as follows: in, Indicates the maximum output power of energy storage in year n; Indicates the rated power of energy storage; and / or, According to the time-varying law of the internal resistance of the energy storage in the nth year, the equivalent cumulative number of charging times at 100% depth of charge in the nth year is obtained, including: The internal resistance of energy storage in the nth year changes with time, and the formula is expressed as: R n =R0[1+λ(T float -n)] Among them, λ is the fitting parameter; T float Indicates the floating charge life of energy storage, with the expected life T E Instead, the formula is expressed as: Among them, T E represents the expected life of energy storage; D represents the number of working days of energy storage per year; Indicates the cumulative number of daily charges at 100% charge and discharge equivalent depth; The equivalent cumulative number of charging times at 100% depth of charge in the nth year is expressed as follows:

5. The method according to claim 1, wherein The construction of a daily operation model with maximizing the expected life of energy storage and maximizing daily benefits as objective functions includes: Maximizing the expected life of energy storage is equivalent to minimizing the number of charge and discharge times accumulated at 100% depth of charge and discharge within a day, and the first objective function is obtained. The mathematical expression is: in, Indicates the cumulative number of charge and discharge times at 100% depth of charge and discharge within a day; Define the formula for expressing the charge and discharge depth of energy storage. The mathematical expression is: in, Indicates the charge and discharge depth of the mth charge or discharge cycle; It represents the net output of the mth dispatch instruction or other task energy storage; represents the discharge power of energy storage during period t; represents the charging power of the energy storage during period t; Indicates the moment when discharge ends; Indicates the moment when discharge begins; Indicates the moment when charging is completed; Indicates the time when charging starts; Indicates the energy loss coefficient of energy storage charging and discharging; ζ n represents the energy conversion efficiency of energy storage in the nth year; Based on the energy storage charge and discharge depth expression formula, the expression formula for the charge and discharge times accumulated at 100% charge and discharge depth within the day is defined, and the mathematical expression is: Based on the expression formula of the charge and discharge times accumulated at 100% charge and discharge depth within the day, the expression formula of the first objective function is improved, and the mathematical expression is: in, represents the depth of charge and discharge of the mth charge or discharge cycle; t represents the charge and discharge period; m represents the number of charge and discharge times; δ represents the fitting parameter.

6. The method according to claim 1, characterized in that The construction of a daily operation model with maximizing the expected life of energy storage and maximizing daily benefits as objective functions includes: When energy storage participates in both peak load regulation and frequency regulation ancillary service markets, the mathematical expression of the second objective function for maximizing intraday benefits is: F2=max(R frs +R pss -C op -C mc -C alc ) Among them, R frs represents the benefits of energy storage participating in frequency regulation services; R pss represents the benefits of energy storage participating in peak load regulation services; C op Represents the operating cost of energy storage; C mc represents the daily maintenance cost of energy storage; C alc Represents the loss cost caused by aging of energy storage performance.

7. The method according to claim 6, characterized in that The revenue R of energy storage participating in frequency regulation service frs Including frequency regulation capacity income and frequency regulation mileage income, the formula is expressed as: Among them, R cr Indicates the total frequency regulation capacity benefit for the day; represents the compensation unit price of frequency regulation capacity during period t; μ represents the frequency regulation performance assessment coefficient set by the regulator; R represents the frequency regulation power of the energy storage system in the day-ahead period during period t; mi Indicates the total frequency modulation mileage income for the day; represents the unit price of mileage income subsidy for frequency modulation in period t; k represents the kth effective response in period t; K represents the total number of effective responses in period t; It represents the actual FM response power of the kth response in the t period, which is specifically expressed as: Among them, β t,k It represents the effective coefficient of the response of ACG for the kth time in period t; and / or, The revenue R of energy storage participating in peak load regulation service pss , the mathematical expression is: Among them, ρ t represents the time-of-use electricity price during period t; represents the discharge power of energy storage during period t; represents the charging power of the energy storage during period t; and / or, The operating cost of the energy storage is C op Considering the loss cost due to calendar aging, the mathematical expression is: Among them, ρ op represents the operating cost coefficient of energy storage; ξ t represents the calendar aging loss coefficient of energy storage in period t; represents the frequency regulation power won by the energy storage system in the day-ahead period at time t; represents the discharge power of energy storage during period t; represents the charging power of the energy storage during period t; and / or, The daily maintenance cost C of the energy storage mc , the mathematical expression is: C mc =ρ mc ·C tot Among them, C tot represents the book value of energy storage; ρ mc Indicates the daily conversion factor of energy storage maintenance costs; and / or, The loss cost C caused by the aging of the energy storage performance alc Considering the loss cost due to cycle aging, the mathematical expression is: Where Γ represents the loss cost coefficient; Q rate Indicates the rated capacity of energy storage; represents the depth of charge or discharge of the mth charge or discharge cycle; η is the forward factor, which is related to the discharge rate of the energy storage; K is the environmental constant; RAE represents the activation energy of the electrochemical reaction generated when the energy storage is working; z is a parameter; Indicates the cumulative equivalent number of charges in the nth year.

8. The method according to any one of claims 1 to 7, characterized in that The multi-agent algorithm using non-adversarial game to solve the intraday operation model includes: Step S1: constructing a maintenance agent for seeking a working strategy that maximizes the expected lifespan of energy storage under the condition of maximizing the expected return of energy storage, and a benefit agent for seeking a working strategy that maximizes the expected return of energy storage under the condition of maximizing the expected lifespan of energy storage; and initializing the strategy network, critic network, and target network of the agents; Step S2: Construct an action space with decision variables, and define a state space with other parameters in the intraday operation model except the decision variables; wherein the action space represents the set of actions that the agent can take, and the specific formula is: Where M represents the decision variable; represents the output decision in each period; τ represents the time scheduling decision; Step S3: The maintenance agent and the benefit agent share the critic network and the state space for iterative training; and, for a preset objective function, the temporal difference method is used to calculate the reward function and advantage function of each agent, so that the agent selects the current operation strategy from the feasible set of the action space; Step S4: During the iterative training process, the minimum mean square error of the loss function is calculated according to the current operation strategy, and the parameters of the critic network, policy network and target network are updated; the mathematical expression of the loss function is: Among them, χ i represents the network parameters of agent i; Indicates based on s t ,γ t The expectation of ; σ represents the standard deviation; V i (s t ,χ i ) represents agent i based on (s t ,χ i )’s value function estimate; R t Represents the reward function, the specific formula is: in, represents the target network parameters of agent i; Step S5: Repeat steps S3 to S4 until the model converges and outputs the optimal strategy for daily energy storage operation.

9. The method according to claim 8, characterized in that The mathematical expression of the preset objective function is: Where θ represents the value of the joint strategy to be sought in the Critic network; the subscript t represents the iteration step; E t Indicates the expectation of the function at the t-th iteration step; Represents the estimated value of the advantage function; clip(·) represents the clipping assignment of the sequence; ∈ represents the clipping threshold, which is used to control the amplitude of the policy update; γ t (θ) represents the update ratio of the current strategy to the old strategy. The specific formula is: Among them, π θ (a t |s t ) represents the specific strategy of the t-th iteration step corresponding to θ in the strategy network; The advantage function is used to measure an action a t The mathematical expression for the quality of the average action relative to the current strategy is: in, is the discount factor; is the weight parameter in the GAE generalized advantage estimation; T represents the maximum iteration step; δ t It represents the time difference error of the t-th iteration step. The specific formula is: Among them, V(s t ) indicates that based on state s t The value function of .

10. An operation optimization system for energy storage participating in ancillary service market, characterized in that: The system comprises: An aging function determination module is used to determine an energy storage performance aging factor and define an energy storage aging time function according to the energy storage performance aging factor; the energy storage performance aging factor includes calendar aging and cycle aging; An operation model construction module is used to construct a daily operation model with the objective functions of maximizing the expected life of energy storage and maximizing the daily benefit based on the energy storage aging time function; The operation strategy solving module is used to solve the intraday operation model using a non-adversarial multi-agent algorithm to obtain the optimal strategy for the energy storage intraday operation.

Images