A method and device for an auxiliary primary frequency regulation of a thermal power unit by a chemical energy storage device

Abstract

The present application relates to a method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation, including the following steps: collecting historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device, respectively training a simulation model of the thermal power unit and a simulation model of the chemical energy storage device according to the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device, establishing a reinforcement learning model by using the simulation model of the thermal power unit and the simulation model of the chemical energy storage device, and determining an optimization strategy for the chemical energy storage device to assist the thermal power unit in primary frequency regulation. The present application can determine the frequency regulation strategy of the chemical energy storage device, relieve the frequency regulation pressure of the thermal power unit, and improve the safe operation level of the power grid.

Description

Technical Field

[0001] The present invention relates to the technical field of frequency regulation of thermal power generation units, and in particular to a method and a device for assisting primary frequency regulation of thermal power generation units with a chemical energy storage device. Background Art

[0002] Due to the strong randomness and intermittency of new energy, when wind power, solar power and other large-scale grid-connected power are connected in a high proportion, the inertia level of the power grid system continues to decline, and the amplitude and frequency of grid disturbances are showing an aggravated trend. The task of the primary frequency regulation of thermal power units is to quickly change the valve opening according to the changes in the grid frequency through the primary frequency regulation function of the turbine speed regulation system, and use the unit heat storage to quickly increase or decrease the unit output according to the grid load requirements, balance the power supply and demand relationship, and maintain the frequency stability of the grid.

[0003] In the context of the rapid development of new energy, if we rely solely on traditional thermal power units to participate in primary frequency regulation, on the one hand, it will increase the fatigue and wear of the equipment, and on the other hand, the environmental protection requirements of thermal power will also limit the regulation ability of the units, bringing risks to the safety of the power grid. Electrochemical energy storage has the advantages of sensitive response, two-way regulation, and high energy conversion efficiency. By combining thermal power and energy storage equipment for primary frequency regulation of the power grid, the frequency characteristics of the power grid can be effectively improved. Using energy storage equipment as a means of auxiliary frequency regulation of the power system can reduce the frequency regulation pressure of thermal power units, reduce frequency regulation costs, and improve the safe operation level of the power grid. However, there is currently no ideal method to achieve chemical energy storage devices to assist thermal power units in primary frequency regulation and achieve good results.

[0004] Therefore, how to provide a method for using a chemical energy storage device to assist in primary frequency regulation of thermal power units is a problem that needs to be solved urgently. Summary of the invention

[0005] The embodiment of the present invention provides a method for assisting the primary frequency regulation of a thermal power unit with a chemical energy storage device, so as to solve the problem that there is still a lack of a method for assisting the primary frequency regulation of a thermal power unit with a chemical energy storage device in the prior art. In order to have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a general review, nor is it intended to identify key / important components or to describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a preface to the detailed description that follows.

[0006] In a first aspect, the present application provides a method for using a chemical energy storage device to assist in primary frequency regulation of a thermal power unit, comprising the following steps:

[0007] Collect historical data of state variables of thermal power units and state observation variables of chemical energy storage devices;

[0008] Train the simulation model of the thermal power unit and the simulation model of the chemical energy storage device respectively according to the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device;

[0009] Use the simulation model of the thermal power unit and the simulation model of the chemical energy storage device to establish a reinforcement learning model, and determine the primary frequency regulation optimization strategy for the chemical energy storage device to assist the thermal power unit.

[0010] Optionally, the state variables of the thermal power unit include one or more of active power, main steam temperature, main steam flow rate, and boiler master control output, and the state observation variables of the chemical energy storage device include state of charge and discharge power.

[0011] Optionally, the simulation model of the thermal power unit is a gated recurrent control network model, and the calculation formula for the input quantity of the gated recurrent control network model is:

[0012]

[0013] The calculation formula for the output quantity of the gated recurrent control network model is:

[0014]

[0015] Where: is the active power of the thermal power unit at time t, is the main steam temperature of the thermal power unit at time t, is the main steam flow rate of the thermal power unit at time t, is the boiler master control output of the thermal power unit at time t, is the active power output by the thermal power unit at time t + 1, is the main steam temperature of the thermal power unit at time t + 1, is the main steam flow rate of the thermal power unit at time t + 1, is the boiler master control output of the thermal power unit at time t + 1.

[0016] Optionally, the gated recurrent control network model is a three-layer network, where is the input quantity of the first layer network of the gated recurrent control network model, the output quantity of the first layer network is used as the input quantity of the second layer network, and the output quantity of the second layer network is used as the input quantity of the third layer network, is the output quantity of the third layer network of the gated recurrent control network model.

[0017] Optionally, the gated recurrent control network model uses the stochastic gradient descent method for training as the method for updating the parameters of the three-layer network, and uses the loss function to optimize the three-layer network.

[0018] Optionally, the calculation formula for the loss function is:

[0019]

[0020] Where: N G represents the number of samples, and 4 represents the state variables of the thermal power unit for the 4 dimensions of represents the estimated value of the state variables of the thermal power unit, represents the actual value of the state variables of the thermal power unit.

[0021] Optionally, the calculation formula for the input quantity of the simulation model of the chemical energy storage device is:

[0022]

[0023] The calculation formula for the output quantity of the simulation model of the chemical energy storage device is:

[0024]

[0025] Where: is the state of charge of the chemical energy storage device at time j, is the discharge power of the chemical energy storage device at time j, is the state of charge of the chemical energy storage device at time j + 1, is the discharge power of the chemical energy storage device at time j + 1.

[0026] Optionally, the simulation model of the chemical energy storage device is a support vector regression model with slack variables, and the support vector regression model with slack variables converts the non-linear prediction problem of data into a linear prediction problem in a high-dimensional space through mapping.

[0027] Optionally, the calculation formula for the support vector regression model with slack variables is:

[0028]

[0029] And the support vector regression model with slack variables satisfies:

[0030]

[0031] Where: x i represents the state of the chemical energy storage device of the i-th training sample x i is equivalent to the foregoing ; N B represents the total number of samples; is the weight coefficient of Equation (6), and the arithmetic formula is the function margin, representing the distance between the sample point and the hyperplane, and || || represents the calculation formula of the two-norm; δ(xi ) represents the kernel function transformation of the sample point x i , that is, K(x, x i ) = δ(x) T δ(x i ), T represents vector transpose; C is the penalty coefficient, used to eliminate the noise points in the data structure, ξ i and are slack variables, ε is the error threshold between the support vector regression model f(x) with slack variables and the corresponding actual value z i , and b is a constant.

[0032] Optionally, the kernel function of the support vector regression model with slack variables is selected as the Gaussian kernel function, and the calculation formula is:

[0033]

[0034] where: σ is the parameter of the kernel function, and exp represents the exponential function with the natural constant e as the base.

[0035] Optionally, the simulation model of the chemical energy storage device is trained using a loss function, and the calculation formula is:

[0036]

[0037] where: N B represents the total number of samples; j represents the number of dimensions of the state of the chemical energy storage device.

[0038] Optionally, the steps of establishing a reinforcement learning model using the simulation model of the thermal power unit and the simulation model of the chemical energy storage device to determine the primary frequency regulation optimization strategy of the chemical energy storage device assisting the thermal power unit include: determining the state evaluation index of the thermal power unit and determining the state evaluation index of the chemical energy storage device, and the two satisfy:

[0039] R = α1R1 + α2R2 (10)

[0040] where: R1 is the state evaluation index of the thermal power unit, R2 is the state evaluation index of the chemical energy storage device, α1 and α2 are the weight coefficients of the state evaluation index of the thermal power unit and the weight coefficient of the evaluation index of the chemical energy storage device set respectively, α1 and α2 are both greater than 0, and the sum of α1 and α2 is 1.

[0041] Optionally, the weight coefficient α1 of the state evaluation index of the thermal power unit = 0.7, and the weight coefficient α2 of the evaluation index of the chemical energy storage device = 0.3.

[0042] Optionally, the calculation formula of the state evaluation index of the thermal power unit is:

[0043]

[0044] The calculation formula for the evaluation index of the chemical energy storage device is as follows:

[0045]

[0046] Where: SOC t is the state of charge of the battery of the chemical energy storage device at time t, S * is the state of charge of the battery of the chemical energy storage device that meets the requirements, take S * = 0.5 * S Max S Max is the state of charge of the battery of the chemical energy storage device in the fully charged state, represents the actual power of the thermal power unit at time T1, P t G and P t B are the actual power of the thermal power unit and the discharge power of the chemical energy storage device respectively starting from time T1 + 1, T3 represents the position at 15 seconds since the start of primary frequency modulation, P2 represents the theoretical power change of the thermal power unit within 15 seconds, and its calculation formula is:

[0047]

[0048] Where: Δn represents the change in the speed of the thermal power unit after ignoring the speed dead zone part during primary frequency modulation, n0 represents the rated speed of the steam turbine of the thermal power unit, and after grid connection, it is taken as 3000 revolutions per minute, δ represents the speed regulation rate of the thermal power unit, and the selected value is 5%, P e is the rated power of the thermal power unit.

[0049] Optionally, the reinforcement learning model includes a policy network and an evaluation network. The policy network is used to output a power recommendation value for the chemical energy storage device to discharge at the current moment according to the states of the thermal power unit and the chemical energy storage device at the previous moment. The evaluation network is used to calculate the training error of the evaluation network according to the power recommendation value for the chemical energy storage device to discharge, and update the training parameters of the policy network and the evaluation network.

[0050] Optionally, the evaluation network includes a first evaluation network and a second evaluation network. After the power recommendation value for the chemical energy storage device to discharge is respectively input into the first evaluation network and the second evaluation network, scores of the first evaluation network and the second evaluation network are respectively obtained. Among them, the calculation formulas for the scores of the first evaluation network and the second evaluation network at time t are:

[0051]

[0052] Where: V1t and V 2t are the scores of the first evaluation network and the second evaluation network at time t, respectively, and X t S is the simulation state value of the thermal power unit and the chemical energy storage device at time t, and its expression is:

[0053]

[0054] Using V t+1 -V t represents the training error of the evaluation network, and:

[0055]

[0056] where: γ represents the discount factor, which is set to 0.9.

[0057] Optionally, the target parameters of the policy network are updated through an objective function, and the calculation formula of the objective function is:

[0058]

[0059] where: θ Actor is the target parameter of the policy network, N represents the number of samples, and T represents the total training duration.

[0060] In a second aspect, the present application provides a chemical energy storage device-assisted primary frequency regulation device for a thermal power unit, including an acquisition unit and a calculation unit, where:

[0061] The acquisition unit is configured to acquire historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device,

[0062] The calculation unit is configured to train a simulation model of the thermal power unit and a simulation model of the chemical energy storage device respectively according to the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device, and establish a reinforcement learning model by using the simulation model of the thermal power unit and the simulation model of the chemical energy storage device, and determine an optimal primary frequency regulation strategy for the chemical energy storage device to assist the thermal power unit.

[0063] In a third aspect, the present application provides a computer storage medium, on which a program is stored, and when the program is executed by a processor, it implements the steps in the method for chemical energy storage device-assisted primary frequency regulation of a thermal power unit as described in any one of the above.

[0064] In a fourth aspect, the present application provides a computer device, including a memory, a processor, and a program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in the method for chemical energy storage device-assisted primary frequency regulation of a thermal power unit as described in any one of the above.

[0065] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0066] Based on the historical operation data of the chemical energy storage device and the thermal power unit, this application establishes a corresponding simulation model, which can train and fit the operation states of both; and through reinforcement learning, determines the final frequency modulation optimization strategy, realizes the optimal discharge power strategy of the chemical energy storage device in the primary frequency modulation response interval of the comprehensive thermal power operation state, and realizes frequency modulation optimization.

[0067] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. Brief Description of the Drawings

[0068] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments in accordance with the present invention, and are used together with the specification to explain the principles of the present invention.

[0069] Figure 1 It is a schematic flow chart of a method for a chemical energy storage device to assist a thermal power unit in primary frequency modulation shown according to an exemplary embodiment;

[0070] Figure 2 It is a schematic structural diagram of a gated recurrent control unit shown according to an exemplary embodiment;

[0071] Figure 3 It is a schematic diagram of a training method of a reinforcement learning model shown according to an exemplary embodiment;

[0072] Figure 4 It is a schematic diagram of the primary frequency modulation action process of a thermal power unit shown according to an exemplary embodiment;

[0073] Figure 5 It is a schematic structural diagram of a device for a chemical energy storage device to assist a thermal power unit in primary frequency modulation shown according to an exemplary embodiment;

[0074] Figure 6 It is a schematic structural diagram of a computer device shown according to an exemplary embodiment. Detailed Embodiments

[0075] The following description and the accompanying drawings fully disclose specific embodiments herein, enabling those skilled in the art to practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims and all available equivalents of the claims. In this document, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a structure, device or equipment comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such structure, device or equipment. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the structure, device or equipment comprising the element. In the description of the embodiments herein, unless otherwise specified and limited, the terms "mounted", "connected", "coupled" shall be construed broadly, for example, it may be a mechanical connection or an electrical connection, or may also be the internal connection of two elements, it may be directly connected, or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0076] In this document, the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing this document and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In the description of this document, unless otherwise specified and limited, the terms "mounted", "connected", "coupled" shall be construed broadly. For example, it may be a mechanical connection or an electrical connection, or may also be the internal connection of two elements, it may be directly connected, or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0077] In this document, unless otherwise stated, the term "plurality" means two or more.

[0078] In this document, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0079] In this document, the term "and / or" is an associative relationship describing an object, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B these three relationships.

[0080] Without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0081] Please refer to Figure 1 , this embodiment provides a method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation, including the following steps:

[0082] Collect historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device;

[0083] Train the simulation model of the thermal power unit and the simulation model of the chemical energy storage device respectively according to the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device;

[0084] Establish a reinforcement learning model by using the simulation model of the thermal power unit and the simulation model of the chemical energy storage device, and determine the primary frequency regulation optimization strategy for the chemical energy storage device to assist the thermal power unit.

[0085] Optionally, the state variables of the thermal power unit include one or several of active power, main steam temperature, main steam flow rate, and boiler master control output, and the state observation variables of the chemical energy storage device include state of charge and discharge power.

[0086] Figure 2 The schematic diagram of the working principle of the gated recurrent unit (GRU Cell) in an embodiment is given. Among them, the solid arrow indicates that a weight needs to be multiplied on this line, and the arrow indicates that the initial end of the solid line is the independent variable of the end. Among them represents the information input state caused by the newly input variable, r t represents the reset gate, i t represents the update gate, which is expressed by the formula as:

[0087] r t =σ sig (W r s t-1 +U r x t +b r ) (21)

[0088] i t =σ sig (W i s t-1 +U i x t +b i ) (22)

[0089]

[0090] As can be seen from the above formula, when the reset gate r t is close to 0, it means that the information of the previous state s t-1 is ignored, and only the new information x t at the current moment is used as the input to calculate the current state s t , and the update gate i t controls the degree to which the information of the previous moment is brought into the current state. The smaller i t is, the more information the previous state s t-1 provides. The output value of the gated recurrent control network at time t needs s t to be obtained through a dense layer. Among them, the expressions of the Sigmoid function and the Tanh function are as follows:

[0091]

[0092] tanh(x) = 2*sig(x) - 1(26)

[0093] where ⊙ represents the element-wise multiplication of the elements in the corresponding element representation matrix; s t represents the state value at time t,; σ sig represents the activation function of the reset gate and the update gate, and usually the Sigmoid function is used as the activation function; φ tanh represents the activation function of the new information state , and usually the Tanh function is used as the activation function; W r , U r , b r represent the reset gate parameters respectively, W i , U i , b i represent the update gate parameters, X n and rnn out represent the input and output of the gated recurrent control network respectively.

[0094] In one embodiment, the simulation model of the thermal power unit is a gated recurrent control network model (GRU model, Gated Recurrent Unit model), and the calculation formula for the input of the gated recurrent control network model is:

[0095]

[0096] The calculation formula for the output of the gated recurrent control network model is:

[0097]

[0098] where: is the active power of the thermal power unit at time t, is the main steam temperature of the thermal power unit at time t, is the main steam flow rate of the thermal power unit at time t, is the boiler master control output of the thermal power unit at time t, is the active power output of the thermal power unit at time t + 1, is the main steam temperature of the thermal power unit at time t + 1, is the main steam flow rate of the thermal power unit at time t + 1, is the boiler master control output of the thermal power unit at time t + 1.

[0099] In one embodiment, the gated recurrent control network model is a three-layer network, where is the input of the first layer network of the gated recurrent control network model, the output of the first layer network is used as the input of the second layer network, and the output of the second layer network is used as the input of the third layer network, is the output of the third layer network of the gated recurrent control network model.

[0100] The gated recurrent control network model is trained using the stochastic gradient descent method as the method for updating the parameters of the three-layer network, and a loss function is used to optimize the three-layer network. The calculation formula of the loss function is:

[0101]

[0102] where: N G represents the number of samples, and 4 represents the 4 dimensions of the state variables of the thermal power unit of the 4 dimensions, represents the estimated value of the state variables of the thermal power unit, represents the actual value of the state variables of the thermal power unit.

[0103] The calculation formula for the input of the simulation model of the chemical energy storage device is:

[0104]

[0105] The calculation formula for the output of the simulation model of the chemical energy storage device is:

[0106]

[0107] where: is the state of charge of the chemical energy storage device at time j, is the discharge power of the chemical energy storage device at time j, is the state of charge of the chemical energy storage device at time j + 1, is the discharge power of the chemical energy storage device at time j + 1.

[0108] The simulation model of the chemical energy storage device is a support vector regression model with slack variables (SVM model, Support Vector Machine model). The support vector regression model with slack variables converts the non-linear prediction problem of data into a linear prediction problem in a high-dimensional space through mapping. The calculation formula of the support vector regression model with slack variables is:

[0109]

[0110] And the support vector regression model with slack variables satisfies:

[0111]

[0112] Where: x i represents the state of the chemical energy storage device of the i-th training sample, x i is equivalent to the aforementioned ; N B represents the total number of samples; is the weight coefficient of Equation (6), and the formula is the function margin, which represents the distance between the sample point and the hyperplane. |||| represents the two-norm calculation formula; δ(x i ) represents the kernel function transformation of the sample point x i , that is, K(x, x i ) = δ(x) T δ(x i ), T represents the vector transpose; C is the penalty coefficient, used to eliminate the noise points in the data structure, ξ i and are slack variables, ε is the error threshold between the support vector regression model f(x) with slack variables and the corresponding actual value z i , and b is a constant.

[0113] The kernel function of the support vector regression model with slack variables selects the Gaussian kernel function, and the calculation formula is:

[0114]

[0115] Where: σ is the parameter of the kernel function, and exp represents the exponential function with the natural constant e as the base.

[0116] The simulation model of the chemical energy storage device is trained using a loss function, and the calculation formula is:

[0117]

[0118] Where: N B represents the total number of samples; j represents the number of dimensions of the chemical energy storage device state.

[0119] An enhanced learning model is established by using the simulation models of thermal power units and chemical energy storage devices, and the steps to determine the primary frequency regulation optimization strategy of chemical energy storage devices assisting thermal power units include: determining the state evaluation index of thermal power units and determining the state evaluation index of chemical energy storage devices, and the two satisfy:

[0120] R = α1R1 + α2R2 (10)

[0121] Where: R1 is the state evaluation index of the thermal power unit, R2 is the state evaluation index of the chemical energy storage device, α1 and α2 are the weight coefficients of the state evaluation index of the set thermal power unit and the evaluation index of the chemical energy storage device respectively, both α1 and α2 are greater than 0, and the sum of α1 and α2 is 1.

[0122] The weight coefficient α1 of the state evaluation index of the thermal power unit = 0.7, and the weight coefficient α2 of the evaluation index of the chemical energy storage device = 0.3.

[0123] The calculation formula for the state evaluation index of the thermal power unit is:

[0124]

[0125] The calculation formula for the evaluation index of the chemical energy storage device is:

[0126]

[0127] Since the state of charge (SOC) of the energy storage device is closely related to the discharge, and the charge and discharge at different SOC ratios also determine the service life of the device, it is necessary to evaluate the self-condition of the energy storage device while evaluating the primary frequency regulation performance, where: SOC t is the state of charge of the battery of the chemical energy storage device at time t, S * is the state of charge of the battery of the chemical energy storage device that meets the requirements, take S * = 0.5*S Max , S Max is the state of charge of the battery of the chemical energy storage device in the fully charged state, represents the actual power of the thermal power unit at time T1, P t G and P t B are the actual power of the thermal power unit and the discharge power of the chemical energy storage device respectively starting from time T1 + 1, T3 represents the position of 15 seconds from the start of the primary frequency regulation, P2 represents the theoretical power change of the thermal power unit within 15 seconds, and its calculation formula is:

[0128]

[0129] Where: Δn represents the change in the speed of the thermal power unit after ignoring the speed dead zone part during primary frequency regulation. n0 represents the rated speed of the steam turbine of the thermal power unit, which is taken as 3000 r / min after grid connection. δ represents the speed regulation rate of the thermal power unit, and the selected value is 5%. P e is the rated power of the thermal power unit, and the rated power of the thermal power unit can be modified according to different unit capacities. In addition, during the process of primary frequency regulation, the starting response time is also used as a performance evaluation index. However, since energy storage is used in this application and it has the characteristic of fast discharge speed, the response time is no longer used as the primary frequency regulation performance evaluation index.

[0130] Combined with the above embodiments, please refer to Figure 3 and Figure 4 , in one embodiment, based on the historical massive operation data of the energy storage device and the thermal power unit, the SVM (Support Vector Machine) and GRU (Gated Recurrent Unit) models are respectively used to train, fit and realize single-step prediction of their operation states, so as to obtain corresponding simulation models. The reinforcement learning model includes a policy network (Actor network) and an evaluation network (Critic network). The policy network is used to output the recommended power value for the chemical energy storage device to discharge at the current moment according to the states of the thermal power unit and the chemical energy storage device at the previous moment. The evaluation network is used to calculate the training error (TD) of the evaluation network according to the recommended power value for the chemical energy storage device to discharge, and update the training parameters of the policy network and the evaluation network. The evaluation network completes the calculation of the overall reward according to the operation states of the thermal power unit and the energy storage device and the current discharge action of the energy storage device. The policy network is used to receive the states of the thermal power unit and the energy storage device at the previous moment and output the recommended power value for the energy storage device to discharge at the current moment.

[0131] Optionally, the evaluation network includes a first evaluation network and a second evaluation network. After the recommended power value for the chemical energy storage device to discharge is respectively input into the first evaluation network and the second evaluation network, the scores of the first evaluation network and the second evaluation network are respectively obtained. The calculation formulas for the scores of the first evaluation network and the second evaluation network at time t are as follows:

[0132]

[0133] Where: V 1t and V 2t are respectively the scores of the first evaluation network and the second evaluation network at time t. is the state value of the thermal power unit and the chemical energy storage device at time t, and its expression is:

[0134]

[0135] Using V t+1 -V t represents the training error of the evaluation network, and:

[0136]

[0137] where: γ represents the discount factor, which is set to 0.9.

[0138] The target parameters of the policy network are updated through the objective function, and the calculation formula of the objective function is:

[0139]

[0140] where: θ Actor is the target parameter of the policy network, N represents the number of samples, and T represents the total training duration.

[0141] When the trained reinforcement learning model is actually applied, it receives the state variables at time t given by the thermal power unit and the chemical energy storage device, and outputs the discharge charge of the corresponding chemical energy storage device thus realizing the optimized policy output and completing the optimized policy output for the primary frequency regulation of the thermal power unit.

[0142] Please refer to Figure 5 , in one embodiment, there is provided a chemical energy storage device-assisted primary frequency regulation device for a thermal power unit, including an acquisition unit and a calculation unit, where:

[0143] The acquisition unit is used to acquire the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device,

[0144] The calculation unit is used to train the simulation model of the thermal power unit and the simulation model of the chemical energy storage device respectively according to the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device, and to establish a reinforcement learning model using the simulation models of the thermal power unit and the chemical energy storage device, and to determine the optimized primary frequency regulation strategy for the chemical energy storage device to assist the thermal power unit.

[0145] In one embodiment, there is provided a chemical energy storage device-assisted primary frequency regulation device for a thermal power unit, which realizes the optimized control of the chemical energy storage device to assist the primary frequency regulation of the thermal power unit through the steps in the method of chemical energy storage device-assisted primary frequency regulation of a thermal power unit disclosed in any of the above embodiments.

[0146] In one embodiment, there is provided a computer storage medium, on which a program is stored, and when the program is executed by a processor, it realizes the steps in the method of chemical energy storage device-assisted primary frequency regulation of a thermal power unit disclosed in any of the above embodiments.

[0147] In one embodiment, a computer device is provided, including a memory, a processor, and a program stored on the memory and executable on the processor. When the processor executes the program, the steps in the method for assisting a thermal power unit in primary frequency regulation by a chemical energy storage device disclosed in any of the foregoing embodiments are implemented.

[0148] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as Figure 6 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them: The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the foregoing method embodiments are implemented.

[0149] Those skilled in the art can understand that Figure 6 the structure shown in

[0150] is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the device to which the solution of the present invention is applied. The specific device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0151] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

[0152] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them: Any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0153] It should be noted that the above description is only some embodiments of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in the present application.

[0154] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present application. Certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0155] Although the subject matter has been described in language specific to structural features and / or method logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

Claims

1. A method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation, characterized in that, It includes the following steps: Collect historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device; Train the simulation model of the thermal power unit and the simulation model of the chemical energy storage device respectively according to the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device; Establish a reinforcement learning model by using the simulation model of the thermal power unit and the simulation model of the chemical energy storage device, and determine the primary frequency regulation optimization strategy for the chemical energy storage device to assist the thermal power unit, including: determining the state evaluation index of the thermal power unit and determining the state evaluation index of the chemical energy storage device, and the two satisfy: R = α1R1 + α2R2 (10) Where: R1 is the state evaluation index of the thermal power unit, R2 is the state evaluation index of the chemical energy storage device, α1 and α2 are the weight coefficients of the state evaluation index of the set thermal power unit and the evaluation index of the chemical energy storage device respectively, both α1 and α2 are greater than 0, and the sum of α1 and α2 is 1; The calculation formula for the state evaluation index of the thermal power unit is: The calculation formula for the evaluation index of the chemical energy storage device is: SOC t is the state of charge of the battery of the chemical energy storage device at time t, S * is the state of charge of the battery of the chemical energy storage device whose state meets the requirements, and take S * = 0.5 * S Max , S Max is the state of charge of the battery of the chemical energy storage device in the fully charged state, represents the actual power of the thermal power unit at time T1, P t G and P t B are respectively the actual power of the thermal power unit and the discharge power of the chemical energy storage device starting from time T1 + 1. T3 represents the position at 15 seconds since the start of primary frequency regulation, and P2 represents the theoretical power change of the thermal power unit within 15 seconds; The reinforcement learning model includes a policy network and an evaluation network. The policy network is used to output the power recommendation value that the chemical energy storage device should discharge at the current moment according to the state values of the thermal power unit and the chemical energy storage device at the previous moment. The evaluation network is used to calculate the training error of the evaluation network according to the power recommendation value that the chemical energy storage device should discharge, and update the training parameters of the policy network and the evaluation network.

2. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 1, characterized in that, The state variables of the thermal power unit include one or several of active power, main steam temperature, main steam flow rate, and boiler master control output. The state observation variables of the chemical energy storage device include state of charge and discharge power.

3. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 2, characterized in that, The simulation model of the thermal power unit is a gated recurrent control network model, and the calculation formula for the input quantity of the gated recurrent control network model is: The calculation formula for the output quantity of the gated recurrent control network model is: Wherein: is the active power of the thermal power unit at time t, is the main steam temperature of the thermal power unit at time t, is the main steam flow rate of the thermal power unit at time t, is the boiler master control output of the thermal power unit at time t, is the active power output by the thermal power unit at time t+1, is the main steam temperature of the thermal power unit at time t+1, is the main steam flow rate of the thermal power unit at time t+1, is the boiler master control output of the thermal power unit at time t+1.

4. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 3, characterized in that, The gate recurrent control network model is a three-layer network, where: is the input of the first layer network of the gate recurrent control network model. The output of the first layer network serves as the input of the second layer network, and the output of the second layer network serves as the input of the third layer network. is the output of the third layer network of the gate recurrent control network model.

5. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 4, characterized in that, The gated recurrent control network model uses the stochastic gradient descent method for training as the method for updating the parameters of the three-layer network, and uses a loss function to optimize the three-layer network. The calculation formula for the loss function is: Where: N G represents the number of samples, and 4 represents the state variables of the thermal power unit for the 4 dimensions of represents the estimated value of the state variables of the thermal power unit, and represents the actual value of the state variables of the thermal power unit.

6. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 5, characterized in that, The calculation formula for the input quantity of the simulation model of the chemical energy storage device is: The calculation formula for the output quantity of the simulation model of the chemical energy storage device is: Wherein: is the charge state of the chemical energy storage device at time j, is the discharge power of the chemical energy storage device at time j, is the charge state of the chemical energy storage device at time j+1, is the discharge power of the chemical energy storage device at time j+1.

7. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 6, characterized in that, The simulation model of the chemical energy storage device is a support vector regression model with slack variables, and the calculation formula for the support vector regression model with slack variables is: And the support vector regression model with slack variables satisfies: Where: x i represents the state of the chemical energy storage device of the i-th training sample, and x i is equivalent to the aforementioned N B represents the total number of samples, is the weight coefficient of Equation (6), and the formula is the functional margin, which represents the distance between the sample point and the hyperplane. |||| represents the two-norm calculation formula, and δ(x i ) represents the kernel function transformation of the sample point x i , that is, K(x, x i ) = δ(x) T δ(x i ). T represents the vector transpose, C is the penalty coefficient, which is used to eliminate the noise points in the data structure, ξ i and are slack variables, ε is the error threshold between the support vector regression model f(x) with slack variables and the corresponding actual value z i , and b is a constant.

8. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 7, characterized in that, The calculation formula for the kernel function of the support vector regression model with slack variables is: Where: σ is the parameter of the kernel function, and exp represents the exponential function with the natural constant e as the base.

9. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 8, characterized in that, The simulation model of the chemical energy storage device is trained using a loss function, and the calculation formula is: Where: N B represents the total number of samples; j represents the number of dimensions of the state of the chemical energy storage device.

10. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 1, characterized in that, The evaluation network includes a first evaluation network and a second evaluation network. After the power recommended values at which the chemical energy storage device should discharge are respectively input into the first evaluation network and the second evaluation network, scores of the first evaluation network and the second evaluation network are respectively obtained. The calculation formulas for the scores of the first evaluation network and the second evaluation network at time t are respectively: Where: V 1t and V 2t are the scores of the first evaluation network and the second evaluation network at time t, respectively, is the simulation state value of the thermal power unit and the chemical energy storage device at time t, and its expression is: Using V t+1 -V t represents the training error of the evaluation network, and: where: γ represents the discount factor, which is set to 0.

9.

11. The method for a chemical energy storage device to assist a thermal power unit in primary frequency regulation according to claim 10, characterized in that, The target parameters of the policy network are updated through an objective function, and the calculation formula of the objective function is: where: θ Actor is the target parameter of the policy network, N represents the number of samples, and T represents the total training duration.

12. A device for a chemical energy storage device to assist a thermal power unit in primary frequency regulation, characterized in that, including a collection unit and a calculation unit, where: The collection unit is used to collect historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device; The calculation unit is used to respectively train a simulation model of the thermal power unit and a simulation model of the chemical energy storage device according to the historical data of the state variables of the thermal power unit and the state observation variables of the chemical energy storage device, and use the simulation model of the thermal power unit and the simulation model of the chemical energy storage device to establish a reinforcement learning model, and determine an optimized primary frequency regulation strategy for the chemical energy storage device to assist the thermal power unit, including: determining the state evaluation index of the thermal power unit and determining the state evaluation index of the chemical energy storage device, and the two satisfy: R = α1R1 + α2R2 (10) where: R1 is the state evaluation index of the thermal power unit, R2 is the state evaluation index of the chemical energy storage device, α1 and α2 are respectively the weight coefficients of the state evaluation index of the thermal power unit and the evaluation index of the chemical energy storage device set, α1 and α2 are both greater than 0, and the sum of α1 and α2 is 1; The calculation formula for the state evaluation index of the thermal power unit is: The calculation formula for the evaluation index of the chemical energy storage device is: SOC t is the state of charge of the battery of the chemical energy storage device at time t, S * is the state of charge of the battery of the chemical energy storage device whose state meets the requirements, taking S * = 0.5 * S Max , S Max is the state of charge of the battery of the chemical energy storage device in the fully charged state, represents the actual power of the thermal power unit at time T1, P t G and P t B are respectively the actual power of the thermal power unit and the discharge power of the chemical energy storage device starting from time T1 + 1. T3 represents the position 15 seconds after the start of primary frequency regulation, and P2 represents the theoretical power change of the thermal power unit within 15 seconds; The reinforcement learning model includes a policy network and an evaluation network. The policy network is used to output the power recommended value at which the chemical energy storage device should discharge at the current moment according to the state values of the thermal power unit and the chemical energy storage device at the previous moment. The evaluation network is used to calculate the training error of the evaluation network according to the power recommended value at which the chemical energy storage device should discharge, and update the training parameters of the policy network and the evaluation network.

13. A chemical energy storage device-assisted primary frequency regulation device for a thermal power unit, characterized in that, The state variables of the thermal power unit include one or several of active power, main steam temperature, main steam flow rate, and boiler master control output. The state observation variables of the chemical energy storage device include state of charge and discharge power.

14. A chemical energy storage device-assisted primary frequency regulation device for a thermal power unit according to claim 13, characterized in that, The simulation model of the thermal power unit is a gated recurrent control network model, and the calculation formula for the input quantity of the gated recurrent control network model is: The calculation formula for the output quantity of the gated recurrent control network model is: Wherein: is the active power of the thermal power unit at time t, is the main steam temperature of the thermal power unit at time t, is the main steam flow rate of the thermal power unit at time t, is the boiler master control output of the thermal power unit at time t, is the active power output by the thermal power unit at time t + 1, is the main steam temperature of the thermal power unit at time t + 1, is the main steam flow rate of the thermal power unit at time t + 1, is the boiler master control output of the thermal power unit at time t + 1.

15. A computer storage medium, on which a program is stored, characterized in that, When the program is executed by a processor, it implements the steps in the method for the chemical energy storage device to assist the thermal power unit in primary frequency regulation according to any one of claims 1-11.

16. A computer device, comprising a memory, a processor, and a program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for the chemical energy storage device to assist the thermal power unit in primary frequency regulation according to any one of claims 1-11.

Images