A multi-time scale hierarchical frequency regulation control method and system for a wind storage system

Abstract

The application relates to the technical field of power system frequency regulation and energy storage collaborative control, and discloses a multi-time scale hierarchical frequency regulation control method and system of a wind storage system. The method comprises the following steps: constructing a multi-rate hierarchical control structure, including a fast execution layer and a slow operation coordination layer; modeling a wind power-flywheel energy storage combined frequency regulation system under a fast time scale and a slow time scale respectively, constructing an equivalent discrete state space model, and obtaining a coupling relationship between fast dynamic states and slow dynamic states; constructing an operation layer optimization control framework, and generating optimal frequency regulation reference power; constructing a time sequence difference error, and adopting a recursive least square method to perform online update on a Q function parameter; and under a fast sampling period of a basic loop layer, designing a controller to realize fast and stable tracking of the operation layer reference power by the basic loop layer, and adopting model predictive control to improve the response speed of a slow dynamic object. The application improves the stability of the regulation and control of the energy storage system.

Description

A multi-time-scale hierarchical frequency regulation control method and system for wind storage systems Technical Field

[0001] This application relates to the field of frequency regulation and energy storage coordinated control technology in power systems, and in particular to a multi-time-scale hierarchical frequency regulation control method and system for wind energy storage systems. Background Technology

[0002] With the large-scale grid connection of new energy sources such as wind power, the equivalent inertia of the power system continues to decline, significantly increasing the pressure on system frequency regulation. The dynamic evolution process after a disturbance exhibits obvious multi-stage characteristics: the frequency change rate is large in the initial stage of the disturbance, requiring rapid power support; subsequently, in the frequency recovery stage, continuous and gradual power regulation is needed. To compensate for the insufficient primary frequency regulation capability of wind power, related technologies introduce energy storage systems into wind farms, improving the system's frequency support capability through wind-storage synergy, resulting in various frequency regulation control schemes.

[0003] Existing technical solutions mainly include centralized optimization control methods based on a single time scale, simple hierarchical or two-layer control architectures, and frequency regulation control methods based on data-driven or reinforcement learning. These methods can improve frequency response performance to some extent under small to medium disturbance conditions, but they are still based on the fundamental assumption of "single-rate control" and fail to uniformly characterize the inherent time scale differences between optimization decision-making, power allocation, and execution control in wind power-storage systems, resulting in significant drawbacks. For example, ignoring the multi-rate characteristics of the control loop leads to a mismatch between decision-making and execution. In actual systems, system-level frequency regulation optimization typically operates on a second-level or even slower time scale, while energy storage power, current, and electromagnetic control loops operate on a millisecond-level or faster time scale. Existing technologies perform control design on a uniform time scale, making it impossible to accurately track the optimal power command generated at the upper level during execution at the lower level, easily leading to frequency oscillations or command saturation. The lack of a multi-rate state evolution model also results in insufficient attainability of optimization results. Most existing optimization models are based on a single-rate state-space description, failing to explicitly consider the coupling relationship between fast and slow dynamic states. This results in the theoretically optimal solution being unattainable in practical systems, creating a problem of "mathematically optimal but physically unattainable." The basic loop and the optimization loop are disconnected, and system stability relies on empirical parameter tuning. In most existing schemes, the basic loop controller exists only as an execution unit; its dynamic characteristics are not incorporated into the upper-level optimization and decision-making processes. The overall system stability highly depends on empirical parameter tuning, resulting in insufficient robustness under large or continuous disturbances.

[0004] Therefore, how to structurally divide frequency regulation decisions, power allocation and execution control according to their intrinsic dynamic rates in the wind power-energy storage joint frequency regulation process, and establish coordination and constraint mechanisms between each level, is a key technical problem that has not yet been effectively solved by existing technologies. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a multi-timescale hierarchical frequency regulation control method and system for wind storage systems. Through multi-rate hierarchical control structure reconstruction, operational layer optimization decision-making, and fast and slow layer coordinated execution, a systematic balance is achieved between frequency regulation performance, energy storage efficiency, and control stability.

[0006] In a first aspect, this application provides a multi-timescale hierarchical frequency regulation control method for a wind-storage system, the method comprising:

[0007] Step S1: Based on the dynamic evolution characteristics of power grid frequency, introduce dual state variables of frequency change rate and frequency deviation to divide the frequency regulation stage, and construct a multi-rate hierarchical control structure, which includes a fast execution layer and a slow operation coordination layer.

[0008] Step S2: Based on the multi-rate hierarchical control structure, the multi-rate discretization and state augmentation methods are used to model the wind power-flywheel energy storage joint frequency regulation system under both fast and slow time scales. An equivalent discrete state space model is constructed that includes system frequency deviation, flywheel energy storage power output and flywheel energy state, and the coupling relationship between fast dynamic state and slow dynamic state is obtained.

[0009] Step S3: Based on the equivalent discrete state space model, construct the operation layer optimization control framework in combination with the Bellman optimality principle, introduce the Q function to provide an equivalent description of the frequency regulation decision process, and transform the flywheel energy storage reference power generation problem into an infinite time domain discrete optimal control problem to generate the optimal frequency regulation reference power.

[0010] Step S4: Based on the real-time collected data of the optimal frequency regulation reference power and system operating status, construct the time-series difference error according to the Bellman optimality equation, and use the recursive least squares method to update the Q function parameters online, so that the updated control strategy adapts to the randomness of wind power output and changes in system parameters.

[0011] Step S5: Based on the optimal frequency modulation reference power, under the fast sampling period of the basic loop layer, design a controller to realize the rapid and stable tracking of the reference power of the operating layer by the basic loop layer. The dynamic response speed of the basic loop layer is guaranteed by the controller design. For slow dynamic objects, model predictive control is used to improve the response speed.

[0012] In conjunction with the first aspect, step S1 includes:

[0013] Based on the multi-rate hierarchical control structure, a constraint mapping relationship is established between the fast execution layer and the slow operation coordination layer. The frequency modulation command output by the slow operation coordination layer satisfies the executable conditions of the fast execution layer and the physical system in terms of amplitude, rate of change and duration.

[0014] The fast execution layer uses the frequency change rate and frequency prediction as the main inputs, adopts a preset sampling period, generates a power compensation command, and the flywheel energy storage module executes the power compensation command to suppress the frequency change rate in the early stage of the disturbance.

[0015] The slow-speed operation coordination layer takes frequency deviation and system energy state as the main inputs, adopts a preset time scale, generates a smooth power reference trajectory, and the wind turbine and operation layer optimization module execute the power reference trajectory to complete frequency recovery.

[0016] Combining the first aspect, constructing an equivalent discrete state-space model includes:

[0017] Fast timescales are located in the first time interval, ranging from milliseconds to seconds, while slow timescales are located in the second time interval, ranging from seconds to minutes.

[0018] The state vector of the equivalent discrete state-space model is defined as follows: ,in, This represents the state vector of the wind power-flywheel energy storage joint frequency regulation system in the k-th sampling period of the operating layer. This represents the discrete state of the system frequency deviation. The discrete state of energy deviation for flywheel energy storage. The power output discrete state of flywheel energy storage participating in frequency regulation is defined as follows: the wind power-flywheel energy storage joint frequency regulation system is a coordinated frequency regulation system composed of wind turbine, flywheel energy storage device and associated control module.

[0019] The continuous-time dynamics of the system frequency deviation satisfy the first formula, which is: ,in, Let M be the rate of change of the system frequency deviation, and M be the system's equivalent inertia constant. The instantaneous frequency deviation of the system is given by D, where D is the damping coefficient. The active power of wind power participating in frequency regulation. To store energy and output power for the flywheel. The load disturbance power is denoted by t, and the frequency modulation time scale is t.

[0020] The flywheel energy storage execution layer satisfies the second formula on the fast timescale, and the second formula is: Where k is the sampling period, To quickly execute the instantaneous power control command for the (k+1)th sampling period of the layer, The flywheel energy storage output power in the (k+1)th sampling period. The instantaneous power control command drives the flywheel energy storage to rapidly respond to frequency changes within a fast timescale, representing the flywheel energy storage output power for the kth sampling period.

[0021] The runtime layer obtains the frequency deviation and energy state evolution based on the third formula at a slow time scale. The third formula is: ,in, This is the state vector of the (k+1)th sampling period of the runtime layer. Here is the state transition matrix. To control the input matrix, This is the operational layer control input, corresponding to the instantaneous power control command. This is a power reference value for flywheel energy storage during a single frequency regulation process. This represents the external uncertain input caused by load disturbances and wind power fluctuations. This is the perturbation input matrix.

[0022] In conjunction with the first aspect, a runtime layer optimization control framework is constructed, including:

[0023] The performance index function of the runtime layer optimization control framework is defined as follows: ,in, This represents the objective function corresponding to the performance metric. Indicates frequency regulation performance. The flywheel power usage cost is represented by Q, the state weighting matrix, and the control weighting matrix. This is the state vector of the k-th sampling period of the runtime layer. This is the control input for the k-th sampling period of the runtime layer. The discount factor for the k-th sampling period;

[0024] The Bellman optimality equation corresponding to the runtime layer optimization control framework is: ,in, For state The value function represents the future cumulative frequency modulation performance. This represents the state vector of the runtime layer in the (k+1)th sampling period corresponding to the next state of the system. This is the discount factor.

[0025] In conjunction with the first aspect, the Q-function parameters are updated online, enabling the updated control strategy to adapt to the randomness of wind power output and changes in system parameters, including:

[0026] The Q function is in parametric quadratic form: ,in, The eigenvectors of the Q function, The parameter vector to be identified, This represents the Kronecker product, where T is the transpose operator. This is the control input for the k-th sampling period of the runtime layer. This is the state vector of the k-th sampling period of the runtime layer. The predicted value of the Q function. and Both represent concatenated vectors of state variables and control variables;

[0027] The optimal frequency modulation reference power is obtained through the fourth formula, and the eigenvector of the Q function includes a second-order coupling term between the system frequency deviation, flywheel energy state, and frequency modulation power, thus realizing a linear parameterized expression of the quadratic Q function. The fourth formula is: ,in, This represents the optimal frequency modulation reference power. To find the minimum point operator;

[0028] Specifically, the structure includes elements of the state and control quantities, along with their quadratic interaction terms, used to accurately approximate the quadratic value function, such that the optimal frequency modulation reference power... It can be obtained analytically by solving a system of linear equations.

[0029] Combining the first aspect, a time-series differential error is constructed, including:

[0030] The timing difference error Defined as: , Discount factor;

[0031] The parameter update uses the recursive least squares method, and the update formula is: ,in, For the updated Q-function parameter vector, The Q-function parameter vector before the update. The covariance matrix is ​​inherent to the recursive least squares method, and its initial value is set to the identity matrix.

[0032] In conjunction with the first aspect, step S5 includes:

[0033] To improve the dynamic response rate of the base loop layer, the control objective is designed to minimize the tracking error of the reference power and its corresponding rate of change, while satisfying the dynamic constraints of the base loop. The definition is as follows: ,in, The power reference value given to the operating layer, The actual output power of flywheel energy storage. for The corresponding rate of change, for The corresponding rate of change, and These are weighting coefficients, used to balance tracking accuracy and response speed;

[0034] The dynamic constraints of the basic loop are: For, among which, , These are the flywheel energy storage power output limits. The maximum power change rate threshold;

[0035] By employing a multi-rate boosting modeling method, the optimal frequency modulation reference power generated by the operating layer at a slow sampling time scale is equivalently mapped to the power command sequence or control parameters executable by the base loop layer at a fast sampling time scale.

[0036] In conjunction with the first aspect, the basic loop layer employs a model predictive controller to achieve fast, constrained optimal tracking of the given power reference of the operating layer. The model predictive controller executes the following steps in each control cycle:

[0037] Step S51: Based on the current measured system state, use the boosting model to predict the trajectory of the system output power change in the next prediction time domain;

[0038] Step S52: Solve a finite-time open-loop optimal control problem. The corresponding objective function is: ,in, To predict the time domain, To control the time domain, And defined as an optimization variable sequence, The power reference value given by the runtime layer for the current k-th cycle. This is the predicted output power for the i-th step based on the aforementioned enhancement model. , and defined as the control increment;

[0039] Step S53: Solve the optimal control sequence The first element in The energy storage converter is applied, and steps S51 to S53 are repeated in the next sampling cycle to achieve rolling time-domain optimization control.

[0040] The model predictive controller solves online under the conditions of satisfying the basic loop dynamic constraints to obtain the optimal control input sequence of the basic loop layer.

[0041] In conjunction with the first aspect, the real-time acquired data includes grid frequency, wind power output, flywheel energy storage energy status, load disturbance power, frequency regulation reference power of the previous cycle, and the corresponding frequency response effect. The acquisition frequency is matched with the time scale of the corresponding level. The real-time acquired data is filtered and denoised before being used as state-action sample input. The online update process adopts the Q-learning reinforcement learning algorithm. Through the constructed time-series difference error, the recursive least squares method is used to iteratively update the Q function parameter vector online, thereby realizing the adaptive optimization of the operation layer control strategy.

[0042] Secondly, this application provides a multi-timescale hierarchical frequency regulation and control system for a wind-storage system, the system comprising:

[0043] The status acquisition unit is used to divide the frequency regulation stage by introducing dual status quantities of frequency change rate and frequency deviation according to the dynamic evolution characteristics of the power grid frequency, and to construct a multi-rate hierarchical control structure, which includes a fast execution layer and a slow operation coordination layer.

[0044] The hierarchical control unit is used to model the wind power-flywheel energy storage joint frequency regulation system at both fast and slow time scales based on the multi-rate hierarchical control structure and using multi-rate discretization and state augmentation methods. It constructs an equivalent discrete state space model that includes system frequency deviation, flywheel energy storage power output and flywheel energy state, and obtains the coupling relationship between fast dynamic state and slow dynamic state.

[0045] The operation layer optimization unit constructs an operation layer optimization control framework based on the equivalent discrete state space model and the Bellman optimality principle. It introduces the Q function to provide an equivalent description of the frequency regulation decision process, transforming the flywheel energy storage reference power generation problem into an infinite time domain discrete optimal control problem, and generating the optimal frequency regulation reference power.

[0046] The online learning unit constructs a time-series difference error based on the real-time data collected from the optimal frequency regulation reference power and system operating status, according to the Bellman optimality equation, and updates the Q function parameters online using the recursive least squares method, so that the updated control strategy adapts to the randomness of wind power output and changes in system parameters.

[0047] The basic loop control unit, based on the optimal frequency modulation reference power, designs a controller to achieve rapid and stable tracking of the reference power of the operating layer by the basic loop layer under the fast sampling period of the basic loop layer. The dynamic response speed of the basic loop layer is guaranteed by the controller design, and model predictive control is used to improve the response speed for slow dynamic objects.

[0048] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0049] First, a multi-rate hierarchical control structure is adopted to adapt to the multi-stage requirements of frequency regulation. The fast execution layer relies on the millisecond-level response of flywheel energy storage to suppress frequency abrupt changes, while the slow operation coordination layer generates a smooth power reference trajectory to guide the stable recovery of frequency, thus solving the problem of decision-making and execution mismatch. Next, multi-rate discretization modeling explicitly characterizes the dynamic coupling relationship between fast and slow operations, and state augmentation is used to make the model take into account both frequency and energy storage state, avoiding "mathematically optimal but physically unattainable". Then, an optimization framework built using the Bellman optimality principle is employed, and the long-term state-action benefits are quantified through the Q-function, generating an optimal frequency regulation reference power that balances frequency regulation performance and energy storage losses. The recursive least squares method is used to update the Q-function parameters online, adapting to the randomness of wind power output and changes in system parameters, improving robustness under complex operating conditions. Finally, multi-rate boosting modeling achieves dynamic synchronization between the fast and slow layers, and the basic loop layer accurately tracks the reference power, reducing command tracking errors and ensuring stable system operation. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 is a flowchart of a multi-timescale hierarchical frequency regulation control method for a wind storage system according to an embodiment of this application;

[0052] Figure 2 is a schematic diagram of the multi-rate hierarchical control architecture of an embodiment of this application;

[0053] Figure 3 is a schematic diagram of multi-rate discretization modeling in an embodiment of this application;

[0054] Figure 4 is a schematic diagram of the structure of a multi-timescale hierarchical frequency regulation and control system for a wind storage system according to an embodiment of this application. Detailed Implementation

[0055] This application provides a multi-timescale hierarchical frequency regulation control method and system for a wind-storage system. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0056] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to Figure 1. An embodiment of a multi-time-scale hierarchical frequency regulation control method for a wind storage system in this application includes:

[0057] Step S1: Based on the dynamic evolution characteristics of the power grid frequency, introduce dual state variables of frequency change rate and frequency deviation to divide the frequency regulation stage, and construct a multi-rate hierarchical control structure, which includes a fast execution layer and a slow operation coordination layer.

[0058] Step S1 includes:

[0059] Based on a multi-rate hierarchical control structure, a constraint mapping relationship is established between the fast execution layer and the slow operation coordination layer. The frequency modulation command output by the slow operation coordination layer satisfies the executable conditions of the fast execution layer and the physical system in terms of amplitude, rate of change, and duration. The fast execution layer uses the frequency change rate and frequency prediction as the main inputs, adopts a preset sampling period, generates a power compensation command, and executes the power compensation command by the flywheel energy storage module to suppress the frequency change rate in the initial stage of the disturbance.

[0060] The slow-speed operation coordination layer takes frequency deviation and system energy state as the main inputs, uses a preset time scale to generate a smooth power reference trajectory, and the wind turbine and operation layer optimization module execute the power reference trajectory to complete frequency recovery.

[0061] Specifically, the dynamic evolution characteristics of power grid frequency refer to the two-stage characteristics of frequency response—"rapid change - gradual recovery"—when the power grid is subjected to load disturbances or fluctuations in renewable energy output. In the initial stage of the disturbance, the frequency change rate is large, requiring instantaneous power support; subsequently, in the recovery stage, continuous smooth adjustment is needed. Therefore, two state quantities are introduced: frequency change rate and frequency deviation. The frequency change rate refers to the change in power grid frequency per unit time, reflecting the speed of dynamic frequency change, usually measured in Hz / s. The frequency deviation refers to the difference between the current actual frequency and the rated frequency, reflecting the degree of deviation between the current frequency and the rated frequency, also measured in Hz. Both serve as the basis for dividing the frequency regulation stage.

[0062] The constructed multi-rate hierarchical control structure includes a fast execution layer and a slow operation coordination layer. The fast execution layer is the control level that responds to frequency regulation demands on a millisecond to second timescale. It employs a preset sampling period of milliseconds to seconds, matching the fast response capability of the flywheel energy storage. It uses the frequency change rate and frequency prediction as the main inputs to generate power compensation commands. The flywheel energy storage module, as the execution entity of the fast execution layer, features fast response speed and high power density, enabling rapid output of high-bandwidth, short-duration continuous power compensation. This effectively suppresses the problem of excessive frequency change rate in the initial stage of disturbances, achieving inertia support. Inertia support refers to the process of releasing or absorbing energy through energy storage devices to simulate the rotational inertia characteristics of a traditional synchronous generator, resisting sudden changes in grid frequency. The slow operation coordination layer is the control level that coordinates frequency regulation decisions on a second to minute timescale. It employs a preset timescale of seconds to minutes, adapted to the regulation characteristics of wind turbines and the system frequency recovery requirements. It uses frequency deviation and system energy state as the main inputs to generate a smooth power reference trajectory. The power reference trajectory refers to a continuously changing power target curve over time, which includes power setpoints at different times. It guides the execution entity to gradually adjust the output power, avoiding system fluctuations caused by sudden power changes. The wind turbine generator and the operation layer optimization module act as the execution entity. The wind turbine generator is the device that converts wind energy into electrical energy, providing continuous power regulation capabilities. The operation layer optimization module is the core computing unit responsible for coordinating power allocation and dynamically generating frequency regulation commands. Together, they complete frequency restoration and power reconstruction, enabling the system frequency to gradually return to its rated value.

[0063] To avoid conflicts or execution failures between fast and slow execution layers, a constraint mapping relationship needs to be established between the two layers. This constraint mapping relationship refers to a mechanism, through mathematical modeling and parameter constraints, that ensures the commands output by the slow-running coordination layer can be physically implemented by the fast execution layer. The frequency regulation commands output by the slow-running coordination layer must not exceed the maximum power output limit of the flywheel energy storage in amplitude, not exceed the maximum power change rate threshold of the flywheel energy storage in rate of change, and match the energy storage capacity of the flywheel energy storage in duration. This ensures that the commands can be accurately responded to by the fast execution layer and the physical system, achieving a synergistic effect of "decision-executable and execution-implementable." For example, Figure 2 shows a schematic diagram of a multi-rate hierarchical control architecture. This architecture, targeting the multi-timescale characteristics of the wind power-flywheel energy storage joint frequency regulation system, decouples the traditional single-rate centralized control into two collaborative control layers, achieving an organic combination of rapid response and smooth recovery.

[0064] Step S2: Based on the multi-rate hierarchical control structure, the multi-rate discretization and state augmentation methods are used to model the wind power-flywheel energy storage joint frequency regulation system under both fast and slow time scales. An equivalent discrete state space model containing the system frequency deviation and flywheel energy state is constructed to obtain the coupling relationship between the fast dynamic state and the slow dynamic state.

[0065] The construction of the equivalent discrete state-space model includes:

[0066] Fast timescales are located in the first time interval, ranging from milliseconds to seconds, while slow timescales are located in the second time interval, ranging from seconds to minutes.

[0067] The state vector of the equivalent discrete state-space model is defined as: ,in, This represents the state vector of the wind power-flywheel energy storage joint frequency regulation system in the k-th sampling period of the operating layer. This represents the discrete state of the system frequency deviation. The discrete state of energy deviation for flywheel energy storage. For the discrete power output state of flywheel energy storage participating in frequency regulation, the wind power-flywheel energy storage joint frequency regulation system is a coordinated frequency regulation system composed of wind turbine generators, flywheel energy storage devices, and associated control modules; the continuous-time dynamics of the system frequency deviation satisfies the first formula, which is: ,in, Let M be the rate of change of the system frequency deviation, and M be the system's equivalent inertia constant. The instantaneous frequency deviation of the system is given by D, where D is the damping coefficient. The active power of wind power participating in frequency regulation. To store energy and output power for the flywheel. Let t be the load disturbance power and t be the frequency regulation time scale; the flywheel energy storage execution layer satisfies the second formula in the fast time scale, which is: Where k is the sampling period, To quickly execute the instantaneous power control command for the (k+1)th sampling period of the layer, The flywheel energy storage output power in the (k+1)th sampling period. For the flywheel energy storage output power in the kth sampling period, the instantaneous power control command drives the flywheel energy storage to quickly respond to frequency changes on a fast timescale; the operating layer obtains the frequency deviation and energy state evolution based on the third formula on a slow timescale, the third formula being: ,in, This is the state vector of the (k+1)th sampling period of the runtime layer. Here is the state transition matrix. To control the input matrix, This is the operational layer control input, corresponding to the instantaneous power control command. This is a power reference value for flywheel energy storage during a single frequency regulation process. This represents the external uncertain input caused by load disturbances and wind power fluctuations. This is the perturbation input matrix.

[0068] Specifically, multi-rate discretization and state augmentation methods are employed to characterize the coupling relationship between fast and slow dynamic states, providing a model foundation for subsequent optimization decisions. Multi-rate discretization refers to a method of modeling the system separately for different time scales of the fast execution layer and the slow operation coordination layer. This method can accurately capture the dynamic response characteristics of different levels. State augmentation refers to adding key state variables, such as flywheel energy storage energy deviation, to the original state variables, enabling the model to more comprehensively reflect the system's operating state. The fast time scale refers to the first time interval, ranging from milliseconds to seconds, corresponding to rapid response processes such as flywheel energy storage power regulation and electromagnetic control. The slow time scale refers to the second time interval, ranging from seconds to minutes, corresponding to slow changes such as grid frequency evolution and wind power output fluctuations. The fast time scale is much smaller than the slow time scale, ensuring that the dynamic characteristics of different levels are accurately described.

[0069] The modeling object is a wind power-flywheel energy storage joint frequency regulation system. This system consists of wind turbines, flywheel energy storage devices, and associated control modules. Its core function is to achieve frequency stability through coordinated wind and energy storage to respond to grid frequency fluctuations. The equivalent discrete state-space model transforms the dynamic characteristics of a continuous-time system into a discrete-time mathematical model. Its core function is to describe the evolution of the system state over the sampling period. Its input data includes real-time operating data such as wind power output, load disturbance power, and flywheel energy storage output power. The output data consists of a state vector composed of system frequency deviation, flywheel energy storage energy deviation, and flywheel energy storage power output, along with the state evolution trend.

[0070] In the state variable definition formula, the sampling period refers to the time interval between the system acquiring data, updating the state, and generating control commands. The sampling period of different levels is adapted according to their dynamic characteristics. The system frequency deviation discrete state refers to the value of the current grid frequency offset relative to the rated frequency after discretization. The flywheel energy storage energy deviation discrete state refers to the value of the difference between the current actual energy state of the flywheel and the preset reference energy state after discretization. The power output discrete state refers to the value of the actual frequency regulation power output by the flywheel energy storage after discretization. This state variable can simultaneously characterize the system frequency state and the energy storage energy state.

[0071] In the first formula, the rate of change of system frequency deviation refers to the derivative of frequency deviation with time, intuitively reflecting the dynamic trend of frequency deviation; the system equivalent inertia constant characterizes the power system's ability to resist frequency changes, which decreases when new energy sources are connected to the grid; the damping coefficient reflects the damping effect when the system frequency changes, and the larger its value, the faster the frequency change decays; the active power of wind power participating in frequency regulation refers to the portion of the wind turbine's active power exceeding its own baseline value used for frequency regulation; the flywheel energy storage output power refers to the power released or absorbed by the flywheel energy storage, which is negative during charging and positive during discharging; the load disturbance power refers to the difference between the actual load and the rated load, which is the main source of disturbance causing frequency fluctuations; the frequency regulation time scale describes the time range of the frequency adjustment process. The first formula clearly depicts the dynamic relationship between system frequency deviation and wind power, flywheel energy storage, and load disturbance, providing a theoretical basis for subsequent control strategy design.

[0072] In the second formula, the instantaneous power control command is generated by the fast execution layer based on the frequency change rate and the prediction amount, which can drive the flywheel energy storage to respond quickly to the frequency change amount in a fast time scale and achieve instantaneous power support; the flywheel energy storage output power in the (k+1)th sampling period represents the actual frequency modulation power output by the flywheel energy storage at the next discrete time point; the flywheel energy storage output power in the kth sampling period represents the actual frequency modulation power output by the flywheel energy storage at the current discrete time point.

[0073] In the third formula, the state transition matrix is ​​a 3×3 square matrix, representing the evolution relationship from the current state to the next state. Its element values ​​are determined by the inherent characteristics of the system, and the data are diagonally dominant. The off-diagonal elements are coupling coefficients, derived from the influence characteristics of energy storage power output on frequency. The diagonal elements are attenuation coefficients (absolute values ​​between 0 and 1 to ensure system stability). The first diagonal element is derived from the system's equivalent inertia M and damping coefficient D. The second diagonal element corresponds to the attenuation characteristics of flywheel energy storage energy deviation, determined by the energy loss coefficient of flywheel energy storage. The third diagonal element corresponds to the attenuation characteristics of flywheel energy storage power output deviation, derived from the power response inertia of flywheel energy storage. The control input matrix is ​​a 3×1 column matrix, representing the degree of influence of control decisions on state evolution. The first element corresponds to the influence coefficient of control decisions on frequency deviation, derived from the system's equivalent inertia M and slow time scale. The sampling period is derived from the following: the second element corresponds to the influence coefficient of control decision on energy storage deviation, derived from the capacity parameters of flywheel energy storage; the third element corresponds to the influence coefficient of control input on flywheel energy storage power output deviation, derived from the power response speed of flywheel energy storage; the operation layer control input refers to the power reference trajectory parameters generated by the slow-speed operation coordination layer; external uncertain input refers to external disturbances that cannot be accurately predicted in advance; the disturbance input matrix is ​​a 3×1 column matrix, and its dimension must be consistent with the dimension of the state vector. It is used to characterize the influence of external uncertain inputs (load disturbance, wind power fluctuation) on the three state quantities. The three elements correspond to the influence coefficients of external disturbance on system frequency deviation, flywheel energy storage deviation, and flywheel energy storage power output deviation, respectively, which are determined according to the transmission path and intensity of disturbance in the actual system. For example, the influence coefficient of load disturbance on frequency deviation is derived from the equivalent inertia of the system.

[0074] The third formula can predict the changing trend of the system state, providing support for optimization decisions at the operational level.

[0075] For example, Figure 3 is a schematic diagram of multi-rate discretization modeling. Figure 3 shows a comparison of dynamic characteristics at two time scales. The fast time scale dynamics exhibit rapid changes and high-frequency fluctuations, reflecting the flywheel energy storage's ability to quickly suppress the rate of frequency change in the early stages of disturbances. The multiple peaks of the curves correspond to different disturbance events, demonstrating the agility of the flywheel energy storage's millisecond-level response. The slow time scale dynamics are smoother and slower, reflecting the macroscopic dynamics of the system at the second-level time scale.

[0076] Step S3: Based on the equivalent discrete state-space model, construct the operation layer optimization control framework in combination with the Bellman optimality principle, introduce the Q function to provide an equivalent description of the frequency regulation decision process, and transform the flywheel energy storage reference power generation problem into an infinite time-domain discrete optimal control problem to generate the optimal frequency regulation reference power.

[0077] The construction of the runtime layer optimization control framework includes:

[0078] The performance metric function of the runtime layer optimization control framework is defined as follows: ,in, This represents the objective function corresponding to the performance metric. Indicates frequency regulation performance. The flywheel power usage cost is represented by Q, the state weighting matrix, and the control weighting matrix. This is the state vector of the k-th sampling period of the runtime layer. This is the control input for the k-th sampling period of the runtime layer. The discount factor for the k-th sampling period; the Bellman optimality equation corresponding to the runtime optimization control framework is: ,in, For state The value function represents the future cumulative frequency modulation performance. This represents the state vector of the runtime layer in the (k+1)th sampling period corresponding to the next state of the system. This is the discount factor.

[0079] This includes updating the Q-function parameters online to make the updated control strategy adaptable to the randomness of wind power output and changes in system parameters, including:

[0080] The Q function takes a parameterized quadratic form: ,in, The eigenvectors of the Q function, The parameter vector to be identified, This represents the Kronecker product, where T is the transpose operator. This is the control input for the k-th sampling period of the runtime layer. This is the state vector of the k-th sampling period of the runtime layer. The predicted value of the Q function. and Both represent the concatenated vector of state and control variables; the optimal frequency modulation reference power is obtained through the fourth formula, and the eigenvector of the Q function contains a second-order coupling term between the system frequency deviation, flywheel energy state, and frequency modulation power, realizing a linear parameterized expression of the quadratic Q function. The fourth formula is: ,in, This represents the optimal frequency modulation reference power. To find the minimum point operator; The specific structure includes elements of the state and control quantities, along with their quadratic interaction terms, used to accurately approximate the quadratic value function, thus achieving the optimal frequency modulation reference power. It can be obtained analytically by solving a system of linear equations.

[0081] Specifically, the optimal frequency regulation reference power refers to the power command value that minimizes the overall performance index and balances frequency regulation effect and energy storage loss under the current system state, and serves as the control basis for the subsequent execution layer.

[0082] The core of the operational-layer optimization control framework is to quantify the system's frequency regulation performance and energy storage power utilization costs through performance index functions. In the performance index function definition formula of the operational-layer optimization control framework, the objective function corresponding to the performance index is used to quantify the comprehensive level of the system's frequency regulation performance and energy storage power utilization costs; the smaller the value, the better the comprehensive performance. Frequency regulation performance is an index that quantifies the system's frequency stability; the smaller the value, the better the frequency regulation effect. Flywheel power utilization costs are an index that quantifies the losses and resource consumption generated by the flywheel's energy storage output power, including energy loss, equipment wear and tear costs; the smaller the value, the more economical the energy storage utilization. Q is a 3×3 diagonal state weighting matrix used to adjust the importance of different state variables in the performance index. The first pair... The first diagonal element corresponds to the weighting coefficient of frequency deviation, set according to the power grid frequency regulation standard. The second diagonal element corresponds to the weighting coefficient of energy storage deviation, set according to energy storage utilization efficiency requirements. Adjusting these element values ​​changes the weight of frequency deviation and energy storage deviation in the performance index; a larger weight indicates a higher priority for system regulation of that state. R is a 1×1 dimensional control weighting matrix, representing the weighting coefficients of control input decisions (power control parameters), used to limit the power output amplitude of flywheel energy storage and prevent overuse leading to rapid energy depletion or equipment damage. The discount factor typically ranges from 0.9 to 0.99, used to balance the weights of current and future performance indicators; the closer to 1, the more the system prioritizes long-term performance. This performance index function achieves a trade-off between frequency regulation effectiveness and energy storage utilization efficiency, ensuring that the system rationally utilizes energy storage resources while meeting frequency regulation requirements.

[0083] The Bellman optimality principle, corresponding to the runtime optimization control framework, is the core principle of dynamic programming. Its core idea is that "the sub-policies of the optimal policy are also optimal," meaning that the optimal control decision in the current state must consider not only the current performance indicators but also the impact of that decision on the performance indicators of future states. In the Bellman optimality equation, the value function represents the value of the decision from the current state... Initially, the cumulative performance index after adopting the optimal control strategy is used; the smaller the value, the better the long-term overall performance. This equation transforms the infinite-time domain optimization problem into a finite-time domain optimization problem with recursive solutions, reducing the solution complexity.

[0084] To avoid relying on an accurate model of the system, a Q-function is introduced to provide an equivalent description of the frequency modulation decision-making process. The Q-function, also known as the state-action value function, is a core concept in reinforcement learning. It is used to quantify the cumulative performance index after taking a certain control action in the current state. The larger the value, the better the long-term benefit of the state-action pair.

[0085] In the parametric quadratic form formula, the eigenvector of the Q function consists of the transpose of the state vector, the transpose of the control decision, and their Kronecker product. It includes second-order coupling terms between frequency deviation, flywheel energy state, and frequency modulation power, accurately characterizing the complex nonlinear relationship between system state and control decision. The Kronecker product is a matrix multiplication operation used to generate high-dimensional eigenvalues, capturing the complex second-order coupling relationship between state and action. The parameter vector to be identified is a set of parameters that needs to be updated through subsequent online learning processes, and its value directly determines the prediction accuracy of the Q function. The predicted value of the Q function is the prediction result of the comprehensive performance of the future based on the current state and action. The concatenated vector of state and control variables is a high-dimensional vector formed by sequentially combining the state vector and control vector, used to comprehensively describe the comprehensive information of system state and control action.

[0086] The meaning of the fourth formula is in the current state Next, we seek a control decision that minimizes the Q function value. This is the optimal frequency regulation reference power. The minimum point operator is used to find the variable values ​​that minimize the objective function. Since the Q function has quantified the long-term performance indicators of the control decision, this optimal frequency regulation reference power can balance current frequency regulation needs with future energy storage utilization efficiency, achieving global optimization.

[0087] Step S4: Based on the real-time acquired data of the optimal frequency regulation reference power and system operating status, construct the time-series difference error according to the Bellman optimality equation, and use the recursive least squares method to update the Q function parameters online, so that the updated control strategy adapts to the randomness of wind power output and changes in system parameters.

[0088] The construction of time-series difference error includes:

[0089] Timing Differential Error Defined as: , The discount factor is used; the parameter update employs the recursive least squares method, and the update formula is: ,in, For the updated Q-function parameter vector, The Q-function parameter vector before the update. The covariance matrix is ​​inherent to the recursive least squares method, and its initial value is set to the identity matrix.

[0090] Real-time data acquisition includes grid frequency, wind power output, flywheel energy storage status, load disturbance power, previous cycle frequency regulation reference power and corresponding frequency response effect. The acquisition frequency is matched with the time scale of the corresponding level. The real-time data is filtered and denoised before being used as state-action sample input. The online update process adopts the Q-learning reinforcement learning algorithm. Through the constructed time-series difference error, the recursive least squares method is used to iteratively update the Q function parameter vector online, realizing the adaptive optimization of the operation layer control strategy.

[0091] Specifically, based on the optimal frequency regulation reference power generated in step S3 and the real-time data of the system operating status, a time-series difference error is constructed according to the Bellman optimality equation, and the Q function parameters are updated online using the recursive least squares method. This enables the control strategy to adapt to the randomness of wind power output and changes in system parameters, thereby improving the system robustness—that is, the system's ability to maintain stable performance when faced with external disturbances or changes in internal parameters.

[0092] Real-time data acquisition includes grid frequency, wind power output, flywheel energy storage status, load disturbance power, previous cycle frequency regulation reference power, and corresponding frequency response. The acquisition frequency is matched to the time scale of the corresponding level; high-frequency acquisition is used for the fast execution layer, and low-frequency acquisition is used for the slow operation coordination layer, ensuring that the data accurately reflects the dynamic characteristics of the corresponding level. The real-time acquired data is filtered and denoised before being used as state-action sample input. A first-order low-pass filter can be used for denoising to eliminate the impact of measurement noise on data accuracy.

[0093] Q-learning is a reinforcement learning method based on the Bellman optimality principle. It constructs a Q-function and minimizes the time difference error to achieve online learning and optimization of the control strategy. Its core advantage is that it does not rely on an accurate system model. It can learn the optimal control strategy only through interaction data with the environment, and can effectively adapt to the randomness of wind power output and changes in system parameters.

[0094] Temporal difference error is a core error metric in reinforcement learning, used to measure the difference between the current Q-function prediction and the target value based on the Bellman optimality principle. A smaller value indicates that the Q-function is closer to its optimal state. In the definition formula of temporal difference error, ... The predicted Q-function value corresponds to the current state-action sequence, with the part within parentheses representing the theoretical target value, derived from the current performance metrics. With the future optimal Q function value Composition. When the timing difference error is 0, it indicates that the current Q function satisfies the Bellman optimality equation, and the control strategy at this time is the optimal strategy.

[0095] Recursive least squares is a classic parameter identification method characterized by fast convergence, low computational cost, and suitability for online updates. Its core idea is to gradually correct the parameter estimates by minimizing the sum of squared errors, making the estimated values ​​approximate the true values. In the update formula, the initial value of the covariance matrix is ​​set as the identity matrix, used to characterize the uncertainty of the parameter estimates. The larger the value, the greater the uncertainty of the parameter estimation and the greater the correction magnitude for new errors. The diagonal elements in the updated data are the variances of the estimated values ​​of each Q function parameter. The smaller the value, the more accurate the parameter estimation. The off-diagonal elements are the covariances between parameters. Approaching 0 indicates that the parameter estimates are independent of each other and there is no obvious coupling interference.

[0096] Through the above online learning process, the Q function parameters can be dynamically adjusted according to the system's operating status, enabling the generation of the optimal frequency regulation reference power to adapt in real time to uncertain factors such as wind power output fluctuations and load disturbance changes, thereby improving the system's frequency regulation performance and stability under complex operating conditions.

[0097] Step S5: Based on the optimal frequency modulation reference power, under the fast sampling period of the basic loop layer, design a controller to realize the rapid and stable tracking of the reference power of the operating layer by the basic loop layer. The dynamic response speed of the basic loop layer is guaranteed by the controller design. For slow dynamic objects, model predictive control is used to improve the response speed.

[0098] Step S5 includes:

[0099] To improve the dynamic response rate of the base loop layer, the control objective is designed to minimize the tracking error of the reference power and its corresponding rate of change, while satisfying the dynamic constraints of the base loop. The definition is as follows: ,in, The power reference value given to the operating layer, The actual output power of flywheel energy storage. for The corresponding rate of change, for The corresponding rate of change, and These are weighting coefficients, used to balance tracking accuracy and response speed; the dynamic constraints of the basic loop are: For, among which, , These are the flywheel energy storage power output limits. The maximum power change rate threshold is used; a multi-rate boosting modeling method is adopted to map the optimal frequency modulation reference power generated by the operating layer under the slow sampling time scale to the power command sequence or control parameters that can be executed by the basic loop layer under the fast sampling time scale.

[0100] In this system, the basic loop layer employs a model predictive controller to achieve fast, constrained optimal tracking of the given power reference in the operating layer. The model predictive controller executes the following steps in each control cycle:

[0101] Step S51: Based on the currently measured system state, use the boosting model to predict the trajectory of the system output power change in the future prediction time domain; Step S52: Solve a finite-time open-loop optimal control problem, with the corresponding optimization objective function being: ,in, To predict the time domain, To control the time domain, And defined as an optimization variable sequence, The power reference value given by the runtime layer for the current k-th cycle. To improve the model's prediction of the output power in the i-th step, And define it as the control increment; Step S53, the optimal control sequence obtained by solving The first element in The control is applied to the flywheel energy storage converter, and steps S51 to S53 are repeated in the next sampling period to achieve rolling time-domain optimization control; in step S54, the model predictive controller solves online under the dynamic constraints of the basic loop to obtain the optimal control input sequence of the basic loop layer.

[0102] Specifically, based on the optimal frequency modulation reference power generated in step S3, a controller is designed to achieve fast and stable tracking of the reference power of the operating layer by the basic loop layer under the sampling period of the basic loop layer. The dynamic response speed of the basic loop layer is adapted according to the characteristics of the execution unit. For execution units with slow dynamic response, model predictive control (MPC) is used to improve the response speed. For execution units such as flywheel energy storage that have millisecond-level fast response characteristics, their fast sampling period advantage is directly used to achieve accurate tracking. The core is to adapt the multi-rate modeling method to execution units with different dynamic characteristics to ensure the speed and stability of reference power tracking.

[0103] The basic loop layer is the lowest control level that directly drives the execution units, such as flywheel energy storage and temperature control devices, to perform power output or regulation actions. Its sampling period needs to match the dynamic response characteristics of the execution unit. For execution units with high power density and fast response speed, such as flywheel energy storage, the basic loop layer naturally has a fast sampling period of milliseconds, which can meet the fast tracking requirements without additional rate boosting. However, for slow dynamic objects such as temperature control and some slow-response energy storage, the original sampling period of the basic loop layer is relatively long. It is necessary to improve the response speed through multi-rate boosting modeling methods to avoid tracking errors caused by response lag.

[0104] The multi-rate enhancement modeling method is a rate optimization technique for slow-dynamic execution units. Its core principle is to improve the response rate of slow-dynamic objects through model optimization and control algorithm design. This allows the base loop layer, which originally had a long sampling period, to adapt to the scheduling cycle of the runtime layer, ensuring a rapid response to the runtime layer's reference power, rather than equating a fast-timescale model with a slow-timescale model. The input data for this method includes the original response rate, original sampling period, power change rate limit, runtime layer scheduling cycle, and optimal frequency modulation reference power corresponding to the base loop parameters of the slow-dynamic execution unit. The output is the enhanced base loop sampling period, optimized control parameters, and an executable power command sequence. This method improves the base loop response rate of the slow-dynamic execution unit to a level that matches the runtime layer's scheduling requirements, thereby achieving rapid tracking of the optimal frequency modulation reference power.

[0105] In the definition of the control objective, the larger the value of τ, the more the system emphasizes tracking accuracy; the larger the value of ω, the more the system emphasizes the smoothness of the response speed. The power reference value given by the operating layer refers to the optimal frequency regulation reference power generated by the operating layer; the actual output power of the flywheel energy storage refers to the power output by the flywheel energy storage device in real time.

[0106] The dynamic constraint formula for the basic circuit includes two aspects: one is the power output range constraint corresponding to... Among them, the flywheel energy storage power output limit is determined by the equipment parameters of the flywheel energy storage, and is used to avoid damage caused by the power output exceeding the equipment's tolerance range; the second is the power change rate constraint. Among them, the rate of change of the actual output power of flywheel energy storage refers to the derivative of the actual output power with time; the maximum power change rate threshold is used to limit the speed of power change and avoid drastic power fluctuations from impacting the power grid and flywheel energy storage equipment.

[0107] For flywheel energy storage and other execution units with inherently fast sampling periods (milliseconds), their basic loop layer does not require multi-rate boosting modeling to increase speed. They can directly rely on their fast response characteristics, designing a controller within the fast sampling period to achieve rapid and stable tracking of the operating layer's reference power. However, for slow-dynamic objects (such as temperature control and slow-response energy storage systems), their basic loop layer needs to utilize multi-rate boosting modeling combined with model predictive control (MPC) to improve response speed. MPC, as an advanced rolling time-domain optimization control method, can predict future output trajectories based on the current system state. By solving finite-time optimization problems online, it generates optimal control sequences, effectively compensating for the response lag of slow-dynamic objects. This allows the boosted basic loop layer to possess a fast response capability adapted to the operating layer's scheduling cycle.

[0108] The control objective of the basic loop layer is to minimize the tracking error between the reference power of the operating layer and the actual output of the execution unit while satisfying dynamic constraints. For fast-response units such as flywheel energy storage, the constraints mainly include power output limits and maximum power change rate thresholds to ensure that the output within the fast sampling period meets the frequency regulation requirements without exceeding the hardware limitations of the equipment. For slow-dynamic objects optimized by multi-rate boosting modeling, the constraints also need to include the dynamic response boundary under the boosted sampling period to avoid control oscillations or equipment losses caused by rate boosting.

[0109] Specifically, the system status includes real-time data such as the current power output of the flywheel energy storage, energy status, and grid frequency; the prediction time domain refers to the time length during which the model predicts future outputs, set according to the system's dynamic response speed and control accuracy requirements; the control time domain refers to the time length during the optimization process to determine the control input, which is usually shorter than the prediction time domain; the optimization variable sequence is a vector composed of control inputs from multiple future control cycles; the control increment refers to the difference between the control inputs of two adjacent control cycles, used to limit the rate of change of the control input and avoid sudden power changes.

[0110] A flywheel energy storage converter is a power electronic device used to drive the output power of a flywheel energy storage device; rolling time-domain optimization control is a real-time optimization control method that achieves long-term optimal control by continuously updating the current state and resolving the finite time-domain optimization problem.

[0111] The controller design of the basic loop layer needs to be configured differently according to the characteristics of the execution unit: the basic loop layer of flywheel energy storage can adopt fast tracking algorithms such as PI / PID control and sliding mode control, which can directly respond to the reference power of the operating layer at the millisecond sampling period without additional rate boosting operation; after multi-rate boosting modeling, the basic loop layer of slow dynamic objects adopts a model predictive controller. Through reasonable configuration of prediction time domain and control time domain, the optimal control input is generated in each control cycle, and only the control command at the current moment is executed. The prediction-optimization process is repeated in the next cycle to realize rolling time domain optimization control, ensuring that the boosted response speed can accurately track the reference power of the operating layer.

[0112] The Q-learning reinforcement learning algorithm used in the runtime layer has the core function of learning the optimal control strategy and generating the optimal frequency regulation reference power under the condition of an unknown precise model of the system. It belongs to a different control level and performs different functions from the multi-rate boosting modeling method in the basic loop layer: Q-learning focuses on decision optimization in the runtime layer and does not involve rate adaptation in the basic loop layer; the multi-rate boosting modeling method focuses on response speed optimization in the basic loop layer, only applies to slow dynamic objects, and is fully compatible with the native fast sampling period characteristics of fast response units such as flywheel energy storage. Through differentiated adaptation, it realizes the coordinated linkage between each execution unit and the runtime layer, ensuring the coordinated and stable operation of the entire control system.

[0113] Through the above process, the basic loop layer can accurately track the reference power of the operating layer at a fast time scale, avoiding command failure or tracking deviation caused by time scale mismatch, and ensuring the coordinated and stable operation of the entire control system.

[0114] The above describes a multi-timescale hierarchical frequency regulation control method for a wind storage system according to an embodiment of this application. The following describes a multi-timescale hierarchical frequency regulation control system for a wind storage system according to an embodiment of this application. Referring to Figure 4, one embodiment of the multi-timescale hierarchical frequency regulation control system for a wind storage system according to an embodiment of this application includes:

[0115] The state acquisition unit is used to divide the frequency regulation stage by introducing dual state quantities of frequency change rate and frequency deviation according to the dynamic evolution characteristics of the power grid frequency, and to construct a multi-rate hierarchical control structure, which includes a fast execution layer and a slow operation coordination layer.

[0116] The hierarchical control unit is used to model the wind power-flywheel energy storage joint frequency regulation system at both fast and slow time scales based on the multi-rate hierarchical control structure and employing multi-rate discretization and state augmentation methods. It constructs an equivalent discrete state space model that includes system frequency deviation, flywheel energy storage power output, and flywheel energy state, and obtains the coupling relationship between fast dynamic state and slow dynamic state.

[0117] The operation layer optimization unit constructs an operation layer optimization control framework based on the equivalent discrete state space model and the Bellman optimality principle. It introduces the Q function to provide an equivalent description of the frequency regulation decision process, transforming the flywheel energy storage reference power generation problem into an infinite time domain discrete optimal control problem, and generating the optimal frequency regulation reference power.

[0118] The online learning unit constructs a time-series difference error based on real-time data collected from the optimal frequency regulation reference power and system operating status, according to the Bellman optimality equation, and uses the recursive least squares method to update the Q function parameters online, so that the updated control strategy adapts to the randomness of wind power output and changes in system parameters.

[0119] The basic loop control unit, based on the optimal frequency modulation reference power, designs a controller to achieve rapid and stable tracking of the reference power of the operating layer by the basic loop layer under the fast sampling period of the basic loop layer. The dynamic response speed of the basic loop layer is guaranteed by the controller design, and model predictive control is used to improve the response speed for slow dynamic objects.

[0120] Specifically, the units synchronize data and transmit instructions through a high-speed communication network to ensure that the entire system can operate in a coordinated and efficient manner, jointly complete the grid frequency regulation task, and improve the frequency stability and renewable energy absorption capacity of the power system. Renewable energy absorption capacity refers to the power system's ability to accept renewable energy generation (such as wind power). The frequency regulation control method of this application can improve the system's adaptability to renewable energy fluctuations, thereby improving the renewable energy absorption level.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0122] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-time-scale stratified frequency regulation control method for a wind-storage system, characterized in that, The method includes: Step S1, introducing frequency change rate and frequency deviation as dual state variables based on the dynamic evolution characteristics of the power grid frequency to divide the frequency regulation stage, and constructing a multi-rate hierarchical control structure, which includes a fast execution layer and a slow operation coordination layer; Step S2, based on the multi-rate hierarchical control structure, using multi-rate discretization and state augmentation methods, modeling the wind power-flywheel energy storage joint frequency regulation system at both fast and slow time scales, constructing an equivalent discrete state space model including system frequency deviation, flywheel energy storage power output, and flywheel energy state, and obtaining the coupling relationship between the fast dynamic state and the slow dynamic state; Step S3, based on the equivalent discrete state space model, constructing an operation layer optimization control framework in conjunction with the Bellman optimality principle, and introducing Q-function modulation. The frequency decision-making process is equivalently described, transforming the flywheel energy storage reference power generation problem into an infinite time-domain discrete optimal control problem, generating the optimal frequency regulation reference power; Step S4: Based on the optimal frequency regulation reference power and real-time acquired data of system operating status, a time-series difference error is constructed according to the Bellman optimality equation, and the Q-function parameters are updated online using the recursive least squares method, so that the updated control strategy adapts to the randomness of wind power output and changes in system parameters; Step S5: Based on the optimal frequency regulation reference power, under the fast sampling period of the base loop layer, a controller is designed to achieve fast and stable tracking of the reference power of the operating layer by the base loop layer. The dynamic response speed of the base loop layer is guaranteed by the controller design, and model predictive control is used to improve the response speed for slow dynamic objects.

2. The multi-timescale stratified frequency regulation control method for a wind-storage system according to claim 1, characterized in that, Step S1 includes: establishing a constraint mapping relationship between the fast execution layer and the slow operation coordination layer based on a multi-rate hierarchical control structure. The frequency modulation command output by the slow operation coordination layer satisfies the executable conditions of the fast execution layer and the physical system in terms of amplitude, rate of change, and duration. The fast execution layer uses the frequency change rate and frequency prediction as the main inputs, adopts a preset sampling period, generates a power compensation command, and executes the power compensation command by the flywheel energy storage module to suppress the frequency change rate in the initial stage of the disturbance. The slow operation coordination layer uses the frequency deviation and system energy state as the main inputs, adopts a preset time scale, generates a smooth power reference trajectory, and executes the power reference trajectory by the wind turbine and the operation layer optimization module to complete the frequency recovery.

3. The multi-timescale stratified frequency regulation control method for a wind-storage system according to claim 1, characterized in that, Constructing an equivalent discrete state-space model includes: a first time interval with a fast time scale ranging from milliseconds to seconds, and a second time interval with a slow time scale ranging from seconds to minutes; the state vector of the equivalent discrete state-space model is defined as: ,in, This represents the state vector of the wind power-flywheel energy storage joint frequency regulation system in the k-th sampling period of the operating layer. This represents the discrete state of the system frequency deviation. The discrete state of energy deviation for flywheel energy storage. For the discrete power output state of flywheel energy storage participating in frequency regulation, the wind power-flywheel energy storage joint frequency regulation system is a coordinated frequency regulation system composed of wind turbine generators, flywheel energy storage devices, and associated control modules; the continuous-time dynamics of the system frequency deviation satisfies the first formula, which is: ,in, Let M be the rate of change of the system frequency deviation, and M be the system's equivalent inertia constant. The instantaneous frequency deviation of the system is given by D, where D is the damping coefficient. The active power of wind power participating in frequency regulation. To store energy and output power for the flywheel. Let t be the load disturbance power and t be the frequency modulation time scale; the flywheel energy storage execution layer satisfies the second formula under the fast time scale, and the second formula is: Where k is the sampling period, To quickly execute the instantaneous power control command for the (k+1)th sampling period of the layer, The flywheel energy storage output power in the (k+1)th sampling period. For the flywheel energy storage output power in the kth sampling period, the instantaneous power control command drives the flywheel energy storage to quickly respond to frequency changes within a fast timescale; the operating layer obtains the frequency deviation and energy state evolution based on the third formula in a slow timescale, the third formula being: ,in, This is the state vector of the (k+1)th sampling period of the runtime layer. Here is the state transition matrix. To control the input matrix, This is the operational layer control input, corresponding to the instantaneous power control command. This is a power reference value for flywheel energy storage during a single frequency regulation process. This represents the external uncertain input caused by load disturbances and wind power fluctuations. This is the perturbation input matrix.

4. The multi-timescale stratified frequency regulation control method for a wind-storage system according to claim 1, characterized in that, Constructing a runtime layer optimization control framework, including: the performance index function of the runtime layer optimization control framework is defined as: ,in, This represents the objective function corresponding to the performance metric. Indicates frequency regulation performance. The flywheel power usage cost is represented by Q, the state weighting matrix, and the control weighting matrix. This is the state vector of the k-th sampling period of the runtime layer. This is the control input for the k-th sampling period of the runtime layer. The discount factor is the value for the k-th sampling period; the Bellman optimality equation corresponding to the runtime optimization control framework is: ,in, For state The value function represents the future cumulative frequency modulation performance. This represents the state vector of the runtime layer in the (k+1)th sampling period corresponding to the next state of the system. This is the discount factor.

5. The multi-timescale stratified frequency regulation control method for a wind-storage system according to claim 1, characterized in that, Online updates to the Q-function parameters enable the updated control strategy to adapt to the randomness of wind power output and changes in system parameters. This includes: the Q-function adopting a parameterized quadratic form. ,in, The eigenvectors of the Q function, The parameter vector to be identified, This represents the Kronecker product, where T is the transpose operator. This is the control input for the k-th sampling period of the runtime layer. This is the state vector of the k-th sampling period of the runtime layer. The predicted value of the Q function. and Both represent the concatenated vector of state and control variables; the optimal frequency modulation reference power is obtained through the fourth formula, and the eigenvector of the Q function contains a second-order coupling term between the system frequency deviation, flywheel energy state, and frequency modulation power, realizing a linear parameterized expression of the quadratic Q function. The fourth formula is: ,in, This represents the optimal frequency modulation reference power. To find the minimum point operator; Specifically, the structure includes elements of the state and control quantities, along with their quadratic interaction terms, used to accurately approximate the quadratic value function, such that the optimal frequency modulation reference power... It can be obtained analytically by solving a system of linear equations.

6. The multi-time-scale stratified frequency regulation control method for a wind-storage system according to claim 1, characterized in that, Constructing a timing difference error includes: the timing difference error Defined as: ， The discount factor is used; the parameter update employs the recursive least squares method, and the update formula is: ,in, For the updated Q-function parameter vector, The Q-function parameter vector before the update. The covariance matrix is ​​inherent to the recursive least squares method, and its initial value is set to the identity matrix.

7. The multi-timescale stratified frequency regulation control method for a wind-storage system according to claim 1, characterized in that, Step S5 includes: To improve the dynamic response rate of the basic loop layer, the control objective is designed to minimize the tracking error of the reference power and the corresponding rate of change while satisfying the dynamic constraints of the basic loop, defined as: ,in, The power reference value given to the operating layer, The actual output power of flywheel energy storage. for The corresponding rate of change, for The corresponding rate of change, and These are weighting coefficients, used to balance tracking accuracy and response speed; the dynamic constraints of the basic loop are: For, among which, 、 These are the flywheel energy storage power output limits. The maximum power change rate threshold is used; a multi-rate boosting modeling method is adopted to map the optimal frequency modulation reference power generated by the operating layer at a slow sampling time scale to an equivalent power command sequence or control parameter that can be executed by the basic loop layer at a fast sampling time scale.

8. The multi-timescale stratified frequency regulation control method for a wind-storage system according to claim 7, characterized in that, The basic loop layer employs a model predictive controller to achieve fast, constrained optimal tracking of the given power reference in the operating layer. The model predictive controller executes the following steps in each control cycle: Step S51: Based on the currently measured system state, use the boosting model to predict the trajectory of the system output power change within a future prediction time domain; Step S52: Solve a finite-time open-loop optimal control problem, with the corresponding objective function being: ,in, To predict the time domain, To control the time domain, And defined as an optimization variable sequence, The power reference value given by the runtime layer for the current k-th cycle. This is the predicted output power for the i-th step based on the aforementioned enhancement model. And define it as the control increment; Step S53, the optimal control sequence obtained by solving The first element in The control is applied to the flywheel energy storage converter, and steps S51 to S53 are repeated in the next sampling period to achieve rolling time-domain optimization control; in step S54, the model predictive controller solves online under the condition of satisfying the basic loop dynamic constraints to obtain the optimal control input sequence of the basic loop layer.

9. The multi-time-scale stratified frequency regulation control method for a wind-storage system according to claim 1, characterized in that, The real-time data collected includes grid frequency, wind power output, flywheel energy storage energy status, load disturbance power, frequency regulation reference power of the previous cycle and the corresponding frequency response effect. The collection frequency is matched with the time scale of the corresponding level. The real-time data is filtered and denoised before being used as state-action sample input. The online update process employs the Q-learning reinforcement learning algorithm. By constructing a temporal difference error, the recursive least squares method is used to iteratively update the Q-function parameter vector online, thereby achieving adaptive optimization of the runtime control strategy.

10. A multi-time-scale hierarchical frequency regulation and control system for a wind-storage system, used to implement the multi-time-scale hierarchical frequency regulation and control method for a wind-storage system as described in any one of claims 1-9, characterized in that, The system includes: a state acquisition unit, used to divide the frequency regulation stage by introducing dual state variables of frequency change rate and frequency deviation based on the dynamic evolution characteristics of the power grid frequency, and construct a multi-rate hierarchical control structure, which includes a fast execution layer and a slow operation coordination layer; a hierarchical control unit, used to model the wind power-flywheel energy storage joint frequency regulation system at both fast and slow time scales based on the multi-rate hierarchical control structure and employing multi-rate discretization and state augmentation methods, constructing an equivalent discrete state space model including system frequency deviation, flywheel energy storage power output, and flywheel energy state, and obtaining the coupling relationship between fast and slow dynamic states; and an operation layer optimization unit, which constructs an operation layer optimization control framework based on the equivalent discrete state space model and the Bellman optimality principle, and introduces a Q function. The frequency regulation decision-making process is equivalently described, transforming the flywheel energy storage reference power generation problem into an infinite time-domain discrete optimal control problem, generating the optimal frequency regulation reference power. An online learning unit, based on the optimal frequency regulation reference power and real-time acquired data of the system's operating status, constructs a time-series difference error according to the Bellman optimality equation and uses the recursive least squares method to update the Q-function parameters online, enabling the updated control strategy to adapt to the randomness of wind power output and changes in system parameters. A base loop control unit, based on the optimal frequency regulation reference power and under the fast sampling period of the base loop layer, designs a controller to achieve rapid and stable tracking of the operating layer reference power by the base loop layer. The dynamic response speed of the base loop layer is guaranteed through controller design; for slow-dynamic objects, model predictive control is used to improve the response speed.

Images