Multi-node power system frequency online control method and device based on SAC algorithm

Abstract

The present invention discloses a multi-node power system frequency online control method and device based on the SAC algorithm. Specifically, the method comprises the following steps: collecting the frequency and power of each node in the power system and calculating the frequency deviation; designing a frequency controller based on the SAC algorithm, determining the state-action set, receiving local state information in real time, selecting the optimal strategy with the goal of maximizing the reward and policy entropy, and performing frequency online control; determining the control node according to the system network topology, and synchronizing the non-control node to the control node according to the control matrix to achieve frequency consistency regulation; then calculating the reward value of the current round of control and updating the action strategy; determining whether the current system frequency is stable, terminating the control process if it is stable, and returning to execute the next round of frequency online control if it is not stable until the system is stable. The present invention does not require offline pre-training, reduces the cost of multi-node power system frequency control, improves control accuracy, and improves the stability of the power system.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of power systems, and in particular to a method and device for online frequency control of a multi-node power system based on a SAC algorithm. Background Art

[0002] With the continued increase in the proportion of renewable energy access, the frequency stability of the power system is facing unprecedented challenges. Renewable energy sources such as wind and photovoltaics are highly volatile and uncontrollable, making grid power balancing more difficult. Traditional centralized frequency control mechanisms struggle to meet the system's requirements for real-time performance, adaptability, and local responsiveness. Especially with the widespread deployment of distributed energy resources and the diverse distribution of sources and loads, the complexity of frequency control in multi-node systems has increased significantly, and the risk of frequency instability is increasing.

[0003] Currently, frequency control methods mainly include traditional linear control strategies based on droop control and PI control. These methods rely on precise system modeling and parameter tuning, have poor robustness to changes in the operating environment, and cannot fully adapt to the dynamic uncertainty and nonlinear characteristics caused by new energy.

[0004] With the development of artificial intelligence technology, reinforcement learning has been gradually applied to frequency control problems, especially deep reinforcement learning, which has powerful state space expression capabilities and strategy optimization capabilities and has become a research hotspot. However, existing frequency control methods based on reinforcement learning generally rely on offline pre-training, require a large amount of prior data or simulation environment, and lack the ability to respond quickly to environmental changes. In complex operating scenarios, offline training strategies often cannot guarantee effectiveness and stability during actual deployment, which limits their engineering application in power systems. In addition, existing methods often ignore the collaborative relationship between nodes and lack a mechanism to synchronously propagate local control strategies through the network structure, resulting in uncoordinated responses between multiple nodes, prone to frequency oscillations or regulation conflicts, and affecting the overall stability of the system. Summary of the Invention

[0005] The purpose of the present invention is to provide a multi-node power system frequency online control method and device that does not require pre-training, has real-time adaptive capabilities and can coordinate multi-node frequency control behaviors, and has high frequency control stability, low cost and high economy.

[0006] The technical solution to achieve the purpose of the present invention is: a multi-node power system frequency online control method based on the SAC algorithm, comprising the following steps:

[0007] Step 1: Collect real-time operating status information of each node in the multi-node power system and calculate the frequency deviation;

[0008] Step 2: Design a frequency controller based on the SAC algorithm, determine the state-action set, receive local state information in real time, select the optimal strategy with the goal of maximizing the reward and policy entropy, perform frequency online control, and output the control action value;

[0009] Step 3: Determine the pinning node based on the network topology of the multi-node power system, and synchronize the non-pinning nodes to the pinning node according to the pinning matrix to achieve frequency consistency control;

[0010] Step 4: Calculate the reward value of this round of frequency online control and update the action strategy;

[0011] Step 5: Determine whether the current system frequency has entered the set stable range:

[0012] If it does not enter the stable range, return to step 2 to continue the next round of frequency online control;

[0013] If it has entered the stable range, the control process is terminated.

[0014] A multi-node power system frequency online control device based on the SAC algorithm is used to implement the multi-node power system frequency online control method. The device includes a first module to a fifth module, wherein the function of each module is as follows:

[0015] The first module collects real-time operating status information of each node in the multi-node power system and calculates the frequency deviation;

[0016] The second module designs a frequency controller based on the SAC algorithm, determines the state-action set, receives local state information in real time, selects the optimal strategy with the goal of maximizing the reward and policy entropy, performs online frequency control, and outputs the control action value;

[0017] The third module determines the pinning node based on the network topology of the multi-node power system. The non-pinning nodes synchronize to the pinning node according to the pinning matrix to achieve frequency consistency control.

[0018] The fourth module calculates the reward value of the current round of frequency online control and updates the action strategy;

[0019] The fifth module determines whether the current system frequency has entered the set stable range: if it has not entered the stable range, it returns to the second module to continue to execute the next round of frequency online control; if it has entered the stable range, the control process is terminated.

[0020] A mobile terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the multi-node power system frequency online control method is implemented.

[0021] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-node power system frequency online control method.

[0022] Compared with the prior art, the present invention has the following significant advantages:

[0023] (1) No need for pre-learning, ready for deployment: The control strategy is based on the maximum entropy reinforcement learning (SAC) algorithm. The controller learns and controls in real time during operation, without requiring a large amount of historical data and training time, thus improving the deployment efficiency and versatility of the control system.

[0024] (2) Pinning consistency modeling to enhance multi-node coordination capabilities: By constructing a pinning matrix and selecting pinning points, frequency synchronization of non-pinning nodes to key nodes is achieved, ensuring the system frequency coordinated control effect and adapting to dynamic topology and multi-source heterogeneous structure;

[0025] (3) Strong adaptability and robustness: The controller can update its strategy according to the real-time status and maintain frequency control stability in dynamic environments such as different load disturbances and distributed energy fluctuations;

[0026] (4) Reduce control costs: System-level coordinated control can be achieved by simply deploying intelligent controllers at key control points, reducing the communication and computing resource overhead of large-scale deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the multi-node power system frequency online control method based on the SAC algorithm of the present invention.

[0028] Figure 2 Schematic diagram of the communication structure between the microgrid system simulation model and distributed power sources in an embodiment of the present invention.

[0029] Figure 3 It is a frequency waveform diagram based on enhanced containment in an embodiment of the present invention.

[0030] Figure 4 It is a frequency waveform diagram based on maximum entropy reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The SAC algorithm (Soft Actor-Critic) is an algorithm based on the maximum entropy reinforcement learning framework. By introducing entropy maximization during policy optimization and balancing exploration and exploitation, it enables intelligent agents to explore and learn more effectively in complex environments. The SAC algorithm is well suited for solving problems with high-dimensional continuous action spaces. In such cases, discretizing the action space often leads to the curse of dimensionality. However, the probability distribution-based SAC algorithm can effectively overcome this problem by directly sampling continuous variables. Therefore, using the SAC algorithm for power system frequency control can adaptively coordinate multi-node frequency control behaviors in real time, achieve global frequency coordinated control, reduce power system control costs, and improve power system frequency stability.

[0032] Combine Figure 1 The present invention provides a multi-node power system frequency online control method based on the SAC algorithm, comprising the following steps:

[0033] Step 1: Collect real-time operating status information of each node in the multi-node power system and calculate the frequency deviation;

[0034] Step 2: Design a frequency controller based on the SAC algorithm, determine the state-action set, receive local state information in real time, select the optimal strategy with the goal of maximizing the reward and policy entropy, perform frequency online control, and output the control action value;

[0035] Step 3: Determine the pinning node based on the network topology of the multi-node power system, and synchronize the non-pinning nodes to the pinning node according to the pinning matrix to achieve frequency consistency control;

[0036] Step 4: Calculate the reward value of this round of frequency online control and update the action strategy;

[0037] Step 5: Determine whether the current system frequency has entered the set stable range:

[0038] If it does not enter the stable range, return to step 2 to continue the next round of frequency online control;

[0039] If it has entered the stable range, the control process is terminated.

[0040] As a specific example, the real-time operating status information of each node in the multi-node power system is collected and the frequency deviation is calculated as follows:

[0041] Step 1.1: Collect the real-time operating status information of each node in the multi-node power system, including The actual frequency of the nodes , No. Active power of nodes , , is the total number of nodes in the system;

[0042] Step 1.2, calculate the Frequency deviation of nodes :

[0043] (1)

[0044] Where, The reference frequency value set for the system.

[0045] As a specific example, the design of a frequency controller based on the SAC algorithm described in step 2 determines the state-action set, receives local state information in real time, selects the optimal strategy with the goal of maximizing the reward and policy entropy, performs online frequency control, and outputs the control action value. The details are as follows:

[0046] Step 2.1, select actual frequency, active power and frequency deviation as the first The status of the node :

[0047] (2)

[0048] Step 2.2, select Frequency action value of each node , active power increment As the first Action information of each node :

[0049] (3)

[0050] Step 2.3, the objective function of the SAC algorithm is to maximize the weighted sum of the reward and the policy entropy:

[0051] (4)

[0052] Where, is the optimal strategy; is the mathematical expectation; Represents a state-action pair In strategy The distribution under 、 The intelligent agents are The state at the moment and the actions taken; For the intelligent agent Next Collection The return value obtained; is the temperature coefficient, which is used to adjust the entropy weight; It is a calculation function of the policy entropy value, which is used to measure the randomness of action selection; Indicates that the status The strategy under the given state The probability of selecting different actions under the condition of ;

[0053] in, Obtained by reparameterization method:

[0054] (5)

[0055] Where, is the average value of the policy network output; is the variance of the value network output; is random noise, sampled from a standard normal distribution; represents the hyperbolic tangent function, which is used to convert Limited to (-1, 1);

[0056] For continuous action spaces, set the strategy Is in state Next action The probability density function of The entropy of is defined as:

[0057] (6)

[0058] Where, Indicates action According to the strategy Sampling obtained; Representation Strategy In state The uncertainty of action selection is eliminated. When the strategy is evenly distributed over all actions, the entropy is maximized, indicating completely random action selection. This ensures that the strategy has a certain degree of randomness and can better adapt to changes in the environment.

[0059] As a specific example, in step 3, the pinning node is determined according to the network topology of the multi-node power system, and the non-pinning nodes are synchronized to the pinning node according to the pinning matrix to achieve frequency consistency control, as follows:

[0060] Step 3.1: Determine the control node based on the multi-node power system topology. The control consistency algorithm is as follows:

[0061] (7)

[0062] Where, For the nodes and The connection weight coefficient between nodes; For the The control gain of the node is When a node is a pinning node ,otherwise ; is the reference value for containment; For the The actual frequency of each node; For the The set of adjacent nodes of a node; For the The frequency state quantity of each node changes over time;

[0063] Step 3.2: Define the deviation matrix :

[0064] (8)

[0065] Where, It is The frequency deviation matrix at the iteration; 、 、 They are 1st, 2nd, The node in Frequency deviation at iterations; is the frequency action value output by the SAC controller under the optimal strategy;

[0066] Step 3.3: According to the pinning iteration formula, calculate the Action value of secondary control :

[0067] (9)

[0068] (10)

[0069] Where, Update the matrix for pinning information; For the The frequency deviation matrix at the iteration; For the The node in Frequency deviation at iterations; is the first frequency deviation vector The components corresponding to the nodes;

[0070] Step 3.4: According to the restraining action value, calculate the first node selected as the restraining node. Secondary frequency adjustment value of each node as follows:

[0071] (11)

[0072] in, It is The updated reference frequency value of each node; It is The original frequency reference value of each node; It is The frequency deviation of nodes at the th iteration.

[0073] As a specific example, the calculation of the reward value of the current round of frequency online control and the update of the action strategy described in step 4 are as follows:

[0074] Step 4.1. Calculate the reward value obtained after this round of control action:

[0075] (12)

[0076] Where, is the reward function; is the reward coefficient corresponding to four different frequency offset intervals; To select When a node is a pinning node, the frequency deviation of the pinning point; Indicates calculation of absolute value;

[0077] Step 4.2: Update the SAC network parameters and temperature coefficients, including the parameters of the strategy network. , parameters of the value network and the parameters of the target value network :

[0078] Update the parameters of the policy network by minimizing the policy loss function :

[0079] (13)

[0080] Where, is the network parameter Minimize the loss function; An experience replay pool for storing samples; For the value network Next Collection The output value of For the policy network in state Output Action probability; Indicates status From the experience replay pool The sampling is obtained; Indicates action From the policy network The sampling is obtained;

[0081] Update the parameters of the value network by minimizing the Bellman error :

[0082] (14)

[0083] (15)

[0084] Where, and For network parameters Updates, is the loss function of the value network, is the target Q value; is the state transition triple sampled from the experience replay pool; Indicates action Sampled from the policy network; For the policy network in state Output Action probability; 、 The intelligent agents are The state at the moment and the actions taken; For the agent in state Output Action Rewards received after For the target value network Next Collection The output value after is the discount factor;

[0085] Parameters of the target value network The update formula is:

[0086] (16)

[0087] Where, is the update coefficient;

[0088] Dynamically adjust the temperature coefficient to make the actual entropy close to the target entropy:

[0089] (17)

[0090] Where, is the loss function of the temperature coefficient; For the intelligent agent strategy at every moment; is the target entropy; For the policy network in state Output Action probability.

[0091] As a specific example, the determination of whether the current system frequency enters the set stable range in step 5 is as follows:

[0092] Step 5.1, calculate the frequency deviation of each node in the current system. If , then the control process ends; Indicates the allowable frequency fluctuation range, Indicates calculation of absolute value;

[0093] Step 5.2: If , then return to step 2 to continue the next round of frequency online control until the system is stable.

[0094] The present invention also provides a multi-node power system frequency online control device based on the SAC algorithm, which is used to implement the multi-node power system frequency online control method. The device includes a first module to a fifth module, wherein the function of each module is as follows:

[0095] The first module collects real-time operating status information of each node in the multi-node power system and calculates the frequency deviation;

[0096] The second module designs a frequency controller based on the SAC algorithm, determines the state-action set, receives local state information in real time, selects the optimal strategy with the goal of maximizing the reward and policy entropy, performs online frequency control, and outputs the control action value;

[0097] The third module determines the pinning node based on the network topology of the multi-node power system. The non-pinning nodes synchronize to the pinning node according to the pinning matrix to achieve frequency consistency control.

[0098] The fourth module calculates the reward value of the current round of frequency online control and updates the action strategy;

[0099] The fifth module determines whether the current system frequency has entered the set stable range: if it has not entered the stable range, it returns to the second module to continue to execute the next round of frequency online control; if it has entered the stable range, the control process is terminated.

[0100] The present invention also provides a mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-node power system frequency online control method when executing the program.

[0101] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps in the multi-node power system frequency online control method are implemented.

[0102] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0103] Example

[0104] This embodiment constructs a microgrid load disturbance simulation model, performs frequency stability control under enhanced containment and the frequency online control method based on the SAC algorithm of the present invention, and compares the dynamic response speed and steady-state control accuracy.

[0105] Example data: Based on the typical low-voltage microgrid system proposed by the EU microgrid project, a microgrid simulation model was constructed on the MATLAB / Simulink simulation platform. The system frequency and voltage levels are 50Hz and 380V respectively. The four distributed power sources are located at nodes 10, 14, 16, and 17 respectively. DG1 is used as the restraining intelligent agent. The simulation topology and the communication structure between the distributed power sources are as follows: Figure 2 As shown in Figure 2, at t = 1s, an overload disturbance occurs in the microgrid, and the active load at the five load points increases by 40kW, causing the system frequency to drop. At t = 2s, some loads are removed, of which 20kW of active load is removed.

[0106] Algorithm data: The hidden layer of the policy network is set to [256, 256], and the learning rate is 0.0003; the hidden layer of the value network and the target value network are both set to [256, 256], and the learning rate is 0.003; the training batch size is set to 128, and the experience pool capacity is set to 10 6 ; Temperature coefficient The initial value is set to 0.2, and the target entropy Set to -1, discount factor Set to 0.99, update coefficient Set to 0.01 to allow frequency fluctuation range Set to 0.02Hz.

[0107] In the load disturbance scenario, the frequency changes of the microgrid based on the enhanced control and the method of the present invention are as follows: Figure 3 、 Figure 4 shown. Figure 3 The frequency waveform diagram based on enhanced pinning shows that the traditional enhanced pinning algorithm, without pre-learning, has a frequency fluctuation range of [49.5, 50.3] Hz, exceeding the frequency range of [49.8, 50.2] Hz during normal system operation. Furthermore, the dynamic response speed is slow, and after a load disturbance, the system frequency cannot be quickly restored to the allowable frequency fluctuation range. Figure 4The frequency waveform diagram of the frequency online control method based on the SAC algorithm of the present invention shows that the frequency fluctuation range of the frequency online control method based on the SAC algorithm is [49.8, 50.1] Hz, which always meets the frequency requirements during normal operation of the system, and can quickly respond to load disturbances. It has good robustness and anti-disturbance performance, and the final control accuracy reaches ±0.015 Hz.

[0108] The comparison of the dynamic response speed, maximum frequency deviation and steady-state control accuracy of the two control algorithms is shown in Table 1. It can be seen that the multi-node power system frequency online control method based on the SAC algorithm proposed in this invention has obvious advantages over the traditional enhanced control algorithm.

[0109] Table 1 Comparison of control effects of two algorithms

[0110]

[0111] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-node power system frequency online control method based on SAC algorithm, characterized in that: The following steps are involved: Step 1: Collect real-time operating status information of each node in the multi-node power system and calculate the frequency deviation; Step 2: Design a frequency controller based on the SAC algorithm, determine the state-action set, receive local state information in real time, select the optimal strategy with the goal of maximizing the reward and policy entropy, perform frequency online control, and output the control action value; Step 3: Determine the pinning node based on the network topology of the multi-node power system, and synchronize the non-pinning nodes to the pinning node according to the pinning matrix to achieve frequency consistency control; Step 4: Calculate the reward value of this round of frequency online control and update the action strategy; Step 5: Determine whether the current system frequency has entered the set stable range: If it does not enter the stable range, return to step 2 to continue the next round of frequency online control; If it has entered the stable range, the control process is terminated.

2. The multi-node power system frequency online control method based on the SAC algorithm according to claim 1 is characterized in that: The real-time operating status information of each node in the multi-node power system is collected in step 1, and the frequency deviation is calculated as follows: Step 1.1: Collect the real-time operating status information of each node in the multi-node power system, including The actual frequency of the nodes , No. Active power of nodes ,in , is the total number of nodes in the system; Step 1.2, calculate the Frequency deviation of nodes : （1） Where, The reference frequency value set for the system.

3. The multi-node power system frequency online control method based on the SAC algorithm according to claim 2 is characterized in that: The design of the frequency controller based on the SAC algorithm described in step 2 determines the state-action set, receives local state information in real time, selects the optimal strategy with the goal of maximizing the reward and policy entropy, performs online frequency control, and outputs the control action value. The details are as follows: Step 2.1, select actual frequency, active power and frequency deviation as the first The status of the node : （2） Step 2.2, select Frequency action value of each node , active power increment As the first Action information of each node : （3） Step 2.3, the objective function of the SAC algorithm is to maximize the weighted sum of the reward and the policy entropy: （4） Where, is the optimal strategy; is the mathematical expectation; Represents a state-action pair In strategy The distribution under 、 The intelligent agents are The state at the moment and the actions taken; For the intelligent agent Next Collection The return value obtained; is the temperature coefficient, which is used to adjust the entropy weight; It is a calculation function of the policy entropy value, which is used to measure the randomness of action selection; Indicates that the status The strategy under the given state The probability of selecting different actions under the condition of ; in, Obtained by reparameterization method: （5） Where, is the average value of the policy network output; is the variance of the value network output; is random noise, sampled from a standard normal distribution; represents the hyperbolic tangent function, which is used to convert Limited to (-1, 1); For continuous action spaces, set the strategy Is in state Next action The probability density function of The entropy of is defined as: （6） Where, Indicates action According to the strategy Sampling obtained; Representation Strategy In state The uncertainty of action selection is maximized when the policy is evenly distributed over all actions, indicating completely random action selection.

4. The multi-node power system frequency online control method based on the SAC algorithm according to claim 3 is characterized in that: In step 3, the pinning node is determined based on the network topology of the multi-node power system, and the non-pinning nodes are synchronized to the pinning node according to the pinning matrix to achieve frequency consistency control, as follows: Step 3.1: Determine the control node based on the multi-node power system topology. The control consistency algorithm is as follows: （7） Where, For the nodes and The connection weight coefficient between nodes; For the The control gain of the node is When a node is a pinning node ,otherwise ; is the reference value for containment; For the The actual frequency of each node; For the The set of adjacent nodes of a node; For the The frequency state quantity of each node changes over time; Step 3.2: Define the deviation matrix : （8） Where, It is The frequency deviation matrix at the iteration; 、 、 They are 1st, 2nd, The node in Frequency deviation at iterations; is the frequency action value output by the SAC controller under the optimal strategy; Step 3.3: According to the pinning iteration formula, calculate the Action value of secondary control : （9） （10） Where, Update the matrix for pinning information; For the The frequency deviation matrix at the iteration; For the The node in Frequency deviation at iterations; is the first frequency deviation vector The components corresponding to the nodes; Step 3.4: According to the restraining action value, calculate the first node selected as the restraining node. Secondary frequency adjustment value of each node as follows: （11） in, It is The updated reference frequency value of each node; It is The original frequency reference value of each node; It is The frequency deviation of nodes at the th iteration.

5. The multi-node power system frequency online control method based on the SAC algorithm according to claim 4 is characterized in that: Calculate the reward value of the current round of frequency online control as described in step 4 and update the action strategy as follows: Step 4.

1. Calculate the reward value obtained after this round of control action: （12） Where, is the reward function; is the reward coefficient corresponding to four different frequency offset intervals; To select When a node is a pinning node, the frequency deviation of the pinning point; Indicates calculation of absolute value; Step 4.2: Update the SAC network parameters and temperature coefficients, including the parameters of the strategy network. , parameters of the value network and the parameters of the target value network : Update the parameters of the policy network by minimizing the policy loss function : （13） Where, is the network parameter Minimize the loss function; An experience replay pool for storing samples; For the value network Next Collection The output value of For the policy network in state Output Action probability; Indicates status From the experience replay pool The sampling is obtained; Indicates action From the policy network The sampling is obtained; Update the parameters of the value network by minimizing the Bellman error : （14） （15） Where, and For network parameters Updates, is the loss function of the value network, is the target Q value; is the state transition triple sampled from the experience replay pool; Indicates action Sampled from the policy network; For the policy network in state Output Action probability; 、 The intelligent agents are The state at the moment and the actions taken; For the agent in state Output Action Rewards received after For the target value network Next Collection The output value after is the discount factor; Parameters of the target value network The update formula is: （16） Where, is the update coefficient; Dynamically adjust the temperature coefficient to make the actual entropy close to the target entropy: （17） Where, is the loss function of the temperature coefficient; For the intelligent agent strategy at every moment; is the target entropy; For the policy network in state Output Action probability.

6. The multi-node power system frequency online control method based on the SAC algorithm according to claim 5 is characterized in that: Step 5 determines whether the current system frequency has entered the set stable range, as follows: Step 5.1, calculate the frequency deviation of each node in the current system. If , then the control process ends; Indicates the allowable frequency fluctuation range, Indicates calculation of absolute value; Step 5.2: If , then return to step 2 to continue the next round of frequency online control until the system is stable.

7. The multi-node power system frequency online control method based on the SAC algorithm according to claim 6 is characterized in that: Temperature coefficient The initial value is set to 0.2, and the target entropy Set to -1, discount factor Set to 0.99, update coefficient Set to 0.01 to allow frequency fluctuation range Set to 0.02Hz.

8. A multi-node power system frequency online control device based on SAC algorithm, characterized in that: The device is used to implement the multi-node power system frequency online control method according to any one of claims 1 to 7. The device includes a first module to a fifth module, wherein the function of each module is as follows: The first module collects real-time operating status information of each node in the multi-node power system and calculates the frequency deviation; The second module designs a frequency controller based on the SAC algorithm, determines the state-action set, receives local state information in real time, selects the optimal strategy with the goal of maximizing the reward and policy entropy, performs online frequency control, and outputs the control action value; The third module determines the pinning node based on the network topology of the multi-node power system. The non-pinning nodes synchronize to the pinning node according to the pinning matrix to achieve frequency consistency control. The fourth module calculates the reward value of the current round of frequency online control and updates the action strategy; The fifth module determines whether the current system frequency has entered the set stable range: if it has not entered the stable range, it returns to the second module to continue to execute the next round of frequency online control; if it has entered the stable range, the control process is terminated.

9. A mobile terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-node power system frequency online control method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the multi-node power system frequency online control method according to any one of claims 1 to 7 are implemented.

Images