Flywheel array auxiliary frequency modulation method for thermal power generating unit based on TD3 algorithm

Abstract

The present application belongs to the technical field of power system auxiliary frequency modulation, and particularly relates to a flywheel array auxiliary thermal power unit frequency modulation method based on a TD3 algorithm, comprising the following steps: S1, modeling a flywheel array auxiliary thermal power unit frequency modulation system as a Markov decision process; S2, constructing a deep reinforcement learning model based on the TD3 algorithm, describing the control action of the flywheel array as a continuously changing action space, training the model to learn the state space of the flywheel array, and finding the optimal control strategy; S3, introducing a global self-attention mechanism into the TD3 model; S4, inputting real-time power grid operation data into the trained TD3 model, outputting the continuous power instructions of each flywheel unit from the model, and realizing the dynamic assistance of the flywheel array to the frequency modulation of the thermal power unit. The present scheme can significantly improve the response speed, training stability and power distribution flexibility of the thermal power-flywheel combined frequency modulation system.

Description

Technical Field

[0001] This invention belongs to the field of power system auxiliary frequency regulation technology, and particularly relates to a flywheel array-assisted frequency regulation method for thermal power units based on the TD3 algorithm. Background Technology

[0002] The frequency regulation task of traditional power systems mainly relies on the regulation capabilities of thermal power units themselves. However, thermal power units are limited by physical factors such as boiler thermal inertia, fuel combustion lag, and the inability to change steam flow abruptly. They have inherent defects such as long response time, low regulation rate, and inability to operate frequently. In the context of new power systems with a high proportion of new energy sources, grid load disturbances are frequent and have variable amplitudes. Relying solely on the frequency regulation of thermal power units is no longer sufficient to suppress frequency fluctuations, which seriously threatens the safe and stable operation of the power grid.

[0003] Flywheel energy storage, as a physical energy storage method with fast response, high power density, and high-frequency charging and discharging capabilities, can effectively compensate for the lack of dynamic response capability of thermal power units. In order to overcome the shortcomings of limited power and energy storage of a single flywheel, multiple flywheels are usually used in engineering to form a flywheel array to assist thermal power units in participating in frequency regulation, thereby improving the dynamic response capability of the frequency regulation system.

[0004] Currently, flywheel arrays generally adopt a two-layer control architecture: the upper layer control uses strategies such as equal time and equal power to manage the energy of the entire array; the lower layer control uses algorithms such as droop control and adaptive control to adjust the dynamic frequency response characteristics of each flywheel unit.

[0005] However, in real-world complex power grid load disturbance scenarios, traditional control strategies have the following problems: 1) When faced with complex power grid load disturbance scenarios, the response speed is insufficient to fully utilize the rapid response characteristics of flywheel energy storage; 2) To meet the SOC consistency principle, flywheel scheduling and power allocation lack flexibility; 3) There is insufficient coordination among individual flywheels within the array, and their actions lack synergy.

[0006] In recent years, deep reinforcement learning has shown great potential in complex continuous control problems. Among them, deep Q-learning (DQN) is difficult to handle continuous action spaces, and deep deterministic policy gradient (DDPG) is suitable for continuous control, but it is prone to Q-value overestimation during training, which leads to violent oscillations when the grid parameters change, making it difficult to be directly applied to power frequency regulation scenarios with high reliability requirements.

[0007] Therefore, there is an urgent need for a flywheel array frequency modulation control method that can adapt to high-dimensional state space, provide continuous action output, and has stable training and strong coordination, so as to improve the dynamic performance and robustness of the flywheel-thermal power plant joint frequency modulation system. Summary of the Invention

[0008] In view of this, the present invention aims to provide a flywheel array-assisted frequency regulation method for thermal power units based on the TD3 algorithm, so as to solve the technical problems of slow response speed, inflexible power allocation, and poor coordination between flywheels in traditional flywheel array control strategies.

[0009] To achieve the above objectives, the technical solution created by this invention is implemented as follows: A method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm includes the following steps: S1. Model the flywheel array-assisted thermal power unit frequency regulation system as a Markov decision process, and define the state space, action space and reward function. S2. Construct a deep reinforcement learning model based on the TD3 algorithm, describe the control action of the flywheel array as a continuously changing action space, train the model to perform perceptual learning on the state space of the flywheel array, and find the optimal control strategy. S3. Introduce a global self-attention mechanism in the TD3 model to enable each flywheel unit and the flywheel units to perceive each other's status, as well as the array as a whole and the flywheel units, so as to achieve coordinated action. S4. Input the real-time operation data of the power grid into the trained TD3 model, and the model outputs continuous power commands for each flywheel unit to realize the dynamic assistance of the flywheel array for the frequency regulation of thermal power units.

[0010] Furthermore, deep reinforcement learning is performed in S2 according to the following sub-steps: S21. Input the current system state containing global self-attention features into the Actor network, and the Actor network outputs a continuously changing action vector; this action vector is used to generate the active power command for each flywheel unit and drive the frequency modulation system state space to transition to the next moment. S22. Input the current state, the action vector output in S21, and the state at the next moment into the Critic network. The Critic network calculates the Q value of the action. The Q value is a weighted evaluation based on the frequency control effect and the consistency of the charge state of each flywheel unit. S23. Generate a reward value based on the Q value calculated in S22 and the preset reward function to guide the Actor network parameters to update in the direction of maximizing the cumulative reward.

[0011] Furthermore, in S22, two independent, structurally identical Critic networks were constructed to suppress Q-value overestimation. The two Critic networks are respectively... When calculating the target Q-value, the smaller value between the two Critic networks is selected as the evaluation criterion, specifically defined by the following formula: ;in, Value function for expected future actions; The target action after smoothing; This indicates that the action was completed at time t. The instant reward obtained afterward; This is the discount factor.

[0012] Furthermore, the construction of the state space and feature extraction in S1 include: System status Decomposed into global power grid state and local flywheel assembly status Two subspaces; Including power grid indicators: frequency deviation, current output of thermal power units, output change rate of thermal power units, current load disturbance amplitude, power imbalance, frequency change rate, load change rate, historical frequency within 1 second, historical frequency within 5 seconds, historical frequency within 10 seconds, and historical frequency within 20 seconds. It includes 10 flywheel sets, and the status of each flywheel set includes: the state of charge (SOC) of the set, the average speed, and the consistency of SOC within the set; and Two independent multilayer perceptrons are input separately, and each is mapped to a high-dimensional state space before feature fusion is performed.

[0013] Furthermore, and The inputs are fed into the multilayer perceptron and mapped to a high-dimensional state space, as expressed by the following formula: ;in, For the overall goal, Here, N represents the local state of each flywheel assembly, and N is the number of flywheel assemblies. As a hidden layer, and These are the global grid status encoder and the flywheel local status encoder, respectively.

[0014] Furthermore, the action space in S1 is defined as the continuous active power command of each flywheel unit or flywheel cluster; the reward function includes at least a frequency deviation penalty term and a flywheel SOC equalization term.

[0015] Furthermore, the global self-attention mechanism in S3 calculates attention weights and generates cooperative actions according to the following steps: S31. Based on the Transformer attention mechanism, define the query vector Q, the key vector K, and the value vector V; S32. Calculate attention weights; S33. Perform feature fusion; S34. Generate the original action; S35, Final Action Output.

[0016] Furthermore, S31, define the vector according to the following formula: Where Q represents the overall power grid characteristics; K records whether the flywheel assembly is suitable for frequency regulation; V records the actual operating characteristics of each flywheel assembly; and W represents the weight. , , The weight matrix is ​​a learnable matrix; This is the global feature vector output by the global power grid state encoder; This is the local state feature matrix of each flywheel assembly output by the local state encoder.

[0017] Furthermore, in S32, the attention weight A of each flywheel group is dynamically allocated by calculating the similarity between the query vector Q and the key vector K. The attention weight A is defined as: Where N is the number of flywheel assemblies; For the hidden layer dimension.

[0018] Furthermore, in S33, global information is injected into each group of features using broadcast addition to obtain the fused feature F, which is expressed by the formula: ; In S34, through the policy head network Generate original actions , Expressed as a formula: ; In S35, the original action is multiplied by the amplified attention weight to obtain the final control action a, which is expressed by the formula: ;in, This indicates element-wise multiplication. This is a scaling factor used to adjust the magnification of attention weights.

[0019] Compared with the prior art, the present invention can achieve the following beneficial effects: 1. This invention directly learns the continuous action space through the TD3 algorithm, which significantly shortens the power response time of the flywheel array, fully utilizes the rapid charging and discharging characteristics of flywheel energy storage, and greatly improves the frequency regulation response speed. At the same time, by adopting the dual-Q network, target policy smoothing regularization and delayed update mechanism in the TD3 algorithm, the overestimation of Q value is effectively suppressed, and the training stability and robustness under changes in grid parameters are improved.

[0020] 2. This invention introduces a global self-attention mechanism, enabling each flywheel unit to sense the status of other units and the array as a whole, achieving coordinated actions and avoiding power command conflicts or excessive SOC dispersion. At the same time, compared with the traditional equal time and equal power strategy, this method can dynamically optimize power allocation based on real-time grid frequency deviation and the SOC of each flywheel, no longer limited by fixed rules, thus improving array utilization efficiency.

[0021] 3. This invention models the control of the flywheel array as a Markov decision process, which is applicable to large-scale flywheel arrays (high-dimensional state space) and continuously changing load disturbance scenarios, and has good generalization ability. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 The algorithm principle diagram of the flywheel array-assisted frequency modulation method for thermal power units based on the TD3 algorithm described in the embodiment of the present invention is shown below; Figure 2 The diagram shows a comparison of the frequency modulation effect of the method of this invention with other methods under an 8MW step disturbance. Line 1 represents the pure thermal power frequency modulation method, line 2 represents the traditional flywheel array frequency modulation method with droop control, line 3 represents the fuzzy adaptive control flywheel array frequency modulation method, and line 4 represents the method in this solution. Figure 3 This is a comparison chart of the frequency modulation effect of the method of the present invention with other methods under real frequency fluctuation data; where line 1 is the pure thermal power frequency modulation method, line 2 is the traditional flywheel array frequency modulation method with droop control, line 3 is the fuzzy adaptive control flywheel array frequency modulation method, and line 4 is the method in this scheme; Figure 4 Attention weight heatmap; Figure 5 This is a graph showing the relationship between attention weight and SOC deviation; line a represents the relationship between the absolute value of SOC deviation from the average and attention weight. Detailed Implementation

[0023] To make the purpose, technical solution, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1-5 The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and do not constitute a limitation thereof.

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0025] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0026] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0027] The following will refer to the appendix. Figure 1-5 The invention will be described in detail with reference to the embodiments.

[0028] A flywheel array-assisted frequency regulation method for thermal power units based on the TD3 algorithm is proposed. By transforming the control of the flywheel array into a Markov stochastic decision process, the TD3 algorithm is used to describe the action of the flywheel array as a continuously changing state space. The training model performs perceptual learning on the state space of the flywheel array to find the optimal control strategy. At the same time, a global self-attention mechanism is introduced to achieve coordinated action. The method in this embodiment has fast response speed, flexible power allocation, and good coordination between flywheels.

[0029] The algorithm in this embodiment operates in the following environment: 100 flywheel units with a capacity of 500kW / 30kWh are used to form an array of 100 flywheels. The array is divided into 10 groups, with 10 units in each group. The total power of the array is 50MW. The thermal power units used are 600MW units. Real grid frequency change data is used as the dataset for training. During the training process, there is a 50% probability of adding a 25-30MW step load disturbance to enhance the model's response to large disturbances.

[0030] In this embodiment, the flywheel array-assisted frequency regulation method for thermal power units based on the TD3 algorithm specifically includes the following steps: S1. Model the flywheel array-assisted frequency regulation system of the thermal power unit as a Markov decision process, defining the state space, action space, and reward function. The action space is defined as the continuous active power command of each flywheel unit or flywheel cluster. The reward function includes at least a frequency deviation penalty term and a flywheel SOC equalization term. The construction and feature extraction of the state space include: System status Decomposed into global power grid state and local flywheel assembly status Two subspaces, of which, It includes 11 power grid indicators, specifically: frequency deviation, current output of thermal power units, output change rate of thermal power units, current load disturbance amplitude, power imbalance, frequency change rate, load change rate, historical frequency within 1 second, historical frequency within 5 seconds, historical frequency within 10 seconds, and historical frequency within 20 seconds.

[0031] It includes 10 flywheel sets, specifically G0-G9. The status of each flywheel set includes: the state of charge of the set, the average speed, and the consistency of the state of charge within the set.

[0032] Will and The data are input into two independent multilayer perceptrons, each mapped to a high-dimensional state space, and then feature fusion is performed. and The inputs to the multilayer perceptron are then mapped to a high-dimensional state space, as expressed by the following formula: .in, For the overall goal, Here, N represents the local state of each flywheel assembly, and N is the number of flywheel assemblies. This is a hidden layer (set to 256 dimensions in this embodiment). and These are a global grid state encoder and a flywheel assembly local state encoder, respectively, responsible for extracting features and mapping the grid state and flywheel assembly state to the same high-dimensional state space for semantic space alignment.

[0033] S2. Construct a deep reinforcement learning model based on the TD3 algorithm, describing the control actions of the flywheel array as a continuously changing action space. Train the model to learn the state space of the flywheel array and find the optimal control strategy. Perform deep reinforcement learning according to the following sub-steps: S21. Input the current system state containing global self-attention features into the Actor network. The Actor network outputs a continuously changing action vector, which is used to generate the active power command for each flywheel unit and drive the frequency modulation system state space to the next moment.

[0034] S22. Input the current state, the action vector output in S21, and the state at the next moment into the Critic network. The Critic network calculates the Q value of the action. The Q value is a weighted evaluation based on the frequency control effect and the consistency of the charge state of each flywheel unit.

[0035] In S22, two independent Critic networks with identical structures are constructed to suppress Q-value overestimation. The two Critic networks are respectively... When calculating the target Q-value, the smaller value between the two Critic networks is selected as the evaluation criterion to avoid the agent blindly and optimistically exploring dangerous actions. This is specifically defined by the Bellman equation. ;in, Let be the expected future action value function, representing the state at time t+1 (including future grid inertia, flywheel power, SOC, and other state information). Take action below The cumulative reward value that can be obtained, in this embodiment, represents a comprehensive evaluation score of the frequency and SOC control effect; The target action after smoothing; This indicates that the action was completed at time t. The instant reward obtained afterward; This is a discount factor used to weight and decay future rewards, representing the agent's level of attention to and foresight regarding future states.

[0036] S23. Generate a reward value based on the Q value calculated in S22 and the preset reward function to guide the Actor network parameters to update in the direction of maximizing the cumulative reward, so as to quickly suppress the grid frequency deviation, maintain the SOC balance of the flywheel array, and reduce the drastic fluctuations in power command.

[0037] S3. A global self-attention mechanism is introduced into the TD3 model to enable mutual awareness of the states of individual flywheels and between individual flywheels, as well as between the array as a whole and individual flywheels, thus achieving coordinated actions. The global self-attention mechanism calculates attention weights and generates coordinated actions according to the following steps: S31. Based on the Transformer attention mechanism, define the query vector Q, key vector K, and value vector V. Define the vectors according to the following formula: Where Q represents the overall power grid characteristics; K records whether the flywheel assembly is suitable for frequency regulation; V records the actual operating characteristics of each flywheel assembly; and W represents the weight. , , The weight matrix is ​​a learnable matrix; This is the global feature vector output by the global power grid state encoder; This is the local state feature matrix of each flywheel assembly output by the local state encoder.

[0038] S32. Calculate attention weights. Dynamically assign attention weights A to each flywheel group by calculating the similarity between the query vector Q and the key vector K. Define the attention weight A as: Where N is the number of flywheel assemblies; For the hidden layer dimension.

[0039] S33. Perform feature fusion. Use broadcast addition to inject global information into each group of features to obtain the fused feature F, which is expressed by the formula: .

[0040] S34. Generate the original action. This is done through the policy head network. Generate original actions , Expressed as a formula: .

[0041] S35. Final Action Output. Multiply the original action by the amplified attention weight to obtain the final control action 'a', which is expressed by the formula: .in, This indicates element-wise multiplication. This is a scaling factor used to adjust the magnification of attention weights.

[0042] S4. Input the real-time operation data of the power grid into the trained TD3 model, and the model outputs continuous power commands for each flywheel unit to realize the dynamic assistance of the flywheel array for the frequency regulation of thermal power units.

[0043] The flywheel array-assisted frequency modulation algorithm in this embodiment can effectively suppress frequency fluctuations in complex power grid disturbance environments. Even when facing large disturbances, it can effectively suppress frequency fluctuations and achieve rapid response. It can also effectively control SOC consistency and SOC recovery, which helps the flywheel array to operate stably for a long time. At the same time, it can use an attention-based agent to dynamically and intelligently adjust the state of the array in real time, achieving a global optimal balance between frequency control and SOC control.

[0044] Figure 2The graph shows a comparison of the frequency modulation effect of the method in this embodiment with other methods under an 8MW step disturbance, as shown in the figure. Figure 2 As shown, the step test was set to show a positive load step at t=15s, which disappeared at t=65s, and a negative load step at t=125s, which disappeared at t=175s. The results show that the frequency control effect of the scheme shown in line 4 is the best. The maximum frequency fluctuation under an 8MW step is maintained at around 0.5Hz. For an array with a total power of 50MW, 8MW is already a significant load change. Under the algorithm of this scheme, the flywheel response time is 0.25s, suppressing the frequency fluctuation to 0 in 0.5s, with a maximum frequency fluctuation of only 0.5Hz. In contrast, the pure thermal power frequency regulation method shown in line 1 is close to frequency collapse at this point. Although the fuzzy adaptive frequency regulation method shown in line 3 and the traditional flywheel control frequency regulation method shown in line 2 have a maximum frequency fluctuation of only 0.25Hz, their repair time exceeds 2s, making their effects significantly inferior to the method in this scheme.

[0045] Figure 3 This is a comparison chart showing the frequency modulation effect of the method in this embodiment compared with other methods under real frequency fluctuation data, as shown below. Figure 3 As shown, the frequency fluctuation of the algorithm shown in this scheme is the smallest for line 4. Most of the time, the maximum frequency fluctuation does not exceed 0.01Hz (in order to prevent mechanical and electrical losses caused by frequent flywheel operation, the flywheel operation dead zone is set to 0.01Hz), which is much better than the effect of other algorithms.

[0046] Figure 4 For attention weight heatmap, Figure 4 In the heatmap, the vertical axis represents 10 flywheel groups, numbered G0-G9, with color intensity indicating the attention weight of each group. The entire graph shows the attention distribution of each flywheel group over time. Figure 4 It can be seen that when a disturbance occurs, the agent increases the attention weight (light-colored bright area) and decreases the weight when the power grid is stable (dark-colored dark area). Furthermore, the attention weight is unevenly distributed between groups and on the time scale. This proves that the agent dynamically allocates power commands according to the state of each group, which is efficient in the time dimension.

[0047] Figure 5This is a scatter plot showing the relationship between attention weights and SOC deviation. The red trend line in the scatter plot, line 'a', represents the relationship between the absolute value of the SOC deviation from the average (x-axis) and the attention weights (y-axis). This indicates the soft equilibrium control law learned by the agent, which dynamically allocates weights based on the SOC deviation. The purple scatter points represent the actual state of a certain flywheel group at a certain moment, representing the agent's environmental awareness. Points above line 'a' indicate that the flywheels in that area are assigned higher weights due to large fluctuations in the power grid or large SOC deviations, and will receive focused attention and action. Points below line 'a' indicate that the flywheels in that area have lower weights due to small frequency fluctuations or small SOC deviations, and are in a relatively quiet state. The intercept of line 'a' is 0.08, representing the base weight, which ensures that the flywheel group is always ready to participate in frequency regulation.

[0048] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0049] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm, characterized in that: Includes the following steps: S1. Model the flywheel array-assisted thermal power unit frequency regulation system as a Markov decision process, and define the state space, action space and reward function. S2. Construct a deep reinforcement learning model based on the TD3 algorithm, describe the control action of the flywheel array as a continuously changing action space, train the model to perform perceptual learning on the state space of the flywheel array, and find the optimal control strategy. S3. Introduce a global self-attention mechanism in the TD3 model to enable each flywheel unit and the flywheel units to perceive each other's status, as well as the array as a whole and the flywheel units, so as to achieve coordinated action. S4. Input the real-time operation data of the power grid into the trained TD3 model, and the model outputs continuous power commands for each flywheel unit to realize the dynamic assistance of the flywheel array for the frequency regulation of thermal power units.

2. The method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm according to claim 1, characterized in that: Deep reinforcement learning is performed in S2 using the following sub-steps: S21. Input the current system state containing global self-attention features into the Actor network, and the Actor network outputs a continuously changing action vector; this action vector is used to generate the active power command for each flywheel unit and drive the frequency modulation system state space to transition to the next moment. S22. Input the current state, the action vector output in S21, and the state at the next moment into the Critic network. The Critic network calculates the Q value of the action. The Q value is a weighted evaluation based on the frequency control effect and the consistency of the charge state of each flywheel unit. S23. Generate a reward value based on the Q value calculated in S22 and the preset reward function to guide the Actor network parameters to update in the direction of maximizing the cumulative reward.

3. The method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm according to claim 2, characterized in that: In S22, two independent Critic networks with identical structures are constructed to suppress Q-value overestimation. The two Critic networks are respectively... When calculating the target Q-value, the smaller value between the two Critic networks is selected as the evaluation criterion; specifically defined by the following formula: ;in, The expected value function for future actions; The target action after smoothing; This indicates that the action was completed at time t. The instant reward obtained afterward; This is the discount factor.

4. The method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm according to claim 1, characterized in that: The construction of the state space and feature extraction in S1 include: System status Decomposed into global power grid state and local flywheel assembly status Two subspaces; Including power grid indicators: frequency deviation, current output of thermal power units, output change rate of thermal power units, current load disturbance amplitude, power imbalance, frequency change rate, load change rate, historical frequency within 1 second, historical frequency within 5 seconds, historical frequency within 10 seconds, and historical frequency within 20 seconds. It includes 10 flywheel sets, and the status of each flywheel set includes: the state of charge (SOC) of the set, the average speed, and the consistency of SOC within the set; and Two independent multilayer perceptrons are input separately, and each is mapped to a high-dimensional state space before feature fusion is performed.

5. The flywheel array-assisted frequency regulation method for thermal power units based on the TD3 algorithm according to claim 4, characterized in that: and The inputs are fed into the multilayer perceptron and mapped to a high-dimensional state space, as expressed by the following formula: ;in, For the overall goal, Here, N represents the local state of each flywheel assembly, and N is the number of flywheel assemblies. As a hidden layer, and These are the global grid status encoder and the flywheel local status encoder, respectively.

6. The method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm according to claim 1, characterized in that: The action space in S1 is defined as the continuous active power command of each flywheel unit or flywheel cluster; the reward function includes at least a frequency deviation penalty term and a flywheel SOC equalization term.

7. The method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm according to claim 1, characterized in that: The global self-attention mechanism in S3 calculates attention weights and generates cooperative actions according to the following steps: S31. Based on the Transformer attention mechanism, define the query vector Q, the key vector K, and the value vector V; S32. Calculate attention weights; S33. Perform feature fusion; S34. Generate the original action; S35, Final Action Output.

8. The method for frequency regulation of thermal power units assisted by flywheel array based on TD3 algorithm according to claim 7, characterized in that: S31. Define a vector according to the following formula: Where Q represents the overall power grid characteristics; K records whether the flywheel assembly is suitable for frequency regulation; V records the actual operating characteristics of each flywheel assembly; and W represents the weight. , , The weight matrix is ​​a learnable matrix; This is the global feature vector output by the global power grid state encoder; This is the local state feature matrix of each flywheel assembly output by the local state encoder.

9. The flywheel array-assisted frequency modulation method for thermal power units based on the TD3 algorithm according to claim 7, characterized in that: In S32, the attention weight A of each flywheel group is dynamically assigned by calculating the similarity between the query vector Q and the key vector K. The attention weight A is defined as: Where N is the number of flywheel assemblies; For the hidden layer dimension.

10. The flywheel array-assisted frequency modulation method for thermal power units based on the TD3 algorithm according to claim 7, characterized in that: In S33, global information is injected into each group of features using broadcast addition to obtain the fused feature F, which is expressed by the formula: ; In S34, through the policy head network Generate original actions , Expressed as a formula: ; In S35, the original action is multiplied by the amplified attention weight to obtain the final control action a, which is expressed by the formula: ;in, This indicates element-wise multiplication. This is a scaling factor used to adjust the magnification of attention weights.

Images