A method for fast regulation and stability control of grid voltage

Abstract

This invention discloses a method for rapid regulation and stable control of power grid voltage, relating to the field of power system technology. It addresses the technical problem that existing technologies often cannot quickly respond to and adjust control strategies when facing fluctuations from large-scale renewable energy sources such as wind power. The invention includes: collecting information to construct a power grid state vector and calculating the voltage stability margin of power grid nodes; representing the power grid system state using a node feature matrix H; calculating the mutual information information between each pair of nodes to determine the information gain between nodes; constructing a spatiotemporal relationship matrix based on spatiotemporal information gain; constructing a node feature matrix based on the node feature matrix and the spatiotemporal relationship matrix; building a voltage stability optimization control model using a reinforcement learning algorithm to output the optimal action for this round of power grid optimization control; and performing power grid voltage stability control based on the optimal action. This invention accurately extracts the spatiotemporal dependency features between power grid nodes, improving the accuracy of voltage stability margin control.

Description

Technical Field

[0001] This invention relates to the field of power system technology, and more specifically to a method for rapid regulation and stable control of power grid voltage. Background Technology

[0002] With the large-scale integration of renewable energy into the grid, the voltage stability problem of the grid has become increasingly complex. Traditional voltage stability margin analysis methods mainly rely on static models, which cannot effectively cope with the impact of dynamic disturbances in the grid, especially when wind power generation fluctuates significantly. These methods often exhibit poor adaptability to rapidly changing loads and renewable energy fluctuations, and cannot provide real-time, accurate grid stability optimization control strategies.

[0003] While existing methods based on classical optimization algorithms and model control can optimize power grid control, these methods suffer from high computational complexity and limited real-time computation and control capabilities for large-scale power grid systems. More importantly, traditional algorithms often fail to respond quickly and adjust control strategies in the face of fluctuations from large-scale renewable energy sources such as wind power, resulting in ineffective guarantees of grid voltage stability. Summary of the Invention

[0004] To address the technical challenge of existing technologies' inability to quickly respond to and adjust control strategies in the face of fluctuations from large-scale renewable energy sources such as wind power, this invention constructs a spatiotemporal dependency graph between grid nodes based on spatiotemporal information gain and extracts the spatiotemporal features of grid nodes using a graph attention network. Then, a long short-term memory network is employed to perform time-series optimization of the voltage stability index (VSI). Finally, a reinforcement learning-based optimization control algorithm is used to adjust the reactive power compensation of the grid in real time to ensure grid voltage stability, achieving precise optimized control of grid voltage stability under dynamic conditions such as grid disturbances.

[0005] A method for rapid regulation and stable control of power grid voltage includes:

[0006] S1: Collect data in real time, construct the power grid state vector, and calculate the voltage stability margin of the power grid nodes;

[0007] S2: Design an initial node feature matrix to represent the state of the power grid system, where each row represents the physical state of each node; calculate the mutual information information between each pair of nodes to determine the information gain between nodes, and construct a spatiotemporal relationship matrix based on spatiotemporal information gain;

[0008] S3: Construct a node feature matrix based on spatiotemporal information gain based on the initial node feature matrix and the spatiotemporal relationship matrix;

[0009] S4: Build a voltage stability optimization control model based on reinforcement learning algorithm. The state space of the model includes a node feature matrix based on spatiotemporal information gain. Use the model to output the optimal action of the current round of power grid optimization control.

[0010] S5: Perform grid voltage stability control based on the optimal action.

[0011] Furthermore, S2 includes: calculating the feature vector and mean delay voltage of each node based on the physical state of the node; calculating the mutual correlation information between each pair of nodes based on the feature vector and mean delay voltage; selecting the maximum mutual correlation information value between each pair of nodes to determine the information gain between the nodes; and constructing an adjacency matrix as a spatiotemporal relation matrix based on the information gain between the nodes to determine the matrix elements.

[0012] Furthermore, S3 includes: inputting the spatiotemporal relationship matrix and the node feature matrix into the graph neural network, calculating the similarity of each pair of nodes, and obtaining the attention coefficient of each pair of nodes; using the attention coefficient to perform a weighted summation of the neighboring nodes of each node to obtain the new feature vector of that node; and combining the new feature vectors of all nodes into a feature matrix based on spatiotemporal information gain.

[0013] Further, S4 includes:

[0014] Construct a Markov decision process, including state space, action space, and reward function;

[0015] A main execution network and a target execution network are constructed to realize the transformation of node states into deterministic actions; a main evaluation network and a target evaluation network are constructed to evaluate the reactive power compensation function for performing a deterministic action in a deterministic node state.

[0016] Set and initialize the training hyperparameters, train the main execution network and the target execution network, output and save the optimal control action after training, use it as the optimal action for this round of power grid optimization control, and update the node state.

[0017] Furthermore, the training includes: starting a new training round, obtaining the optimal action strategy from the action space as the initial action for this training round; determining whether the current time step is less than the maximum number of time steps, if not, ending this training round, if so, creating training samples and placing them into the experience replay buffer.

[0018] Randomly select a small batch of B samples from the experience replay buffer and calculate the target value; update the parameter combination of the main evaluation network by minimizing the loss function; then determine whether the current time step meets the delayed update condition. If so, execute the update of the main execution network and the target network; otherwise, increment the time step by 1 and return to the time step for judgment.

[0019] At the end of this training round, the round counter is incremented by 1. It is then determined whether the current round number is greater than the total number of training rounds. If so, the training process ends and the iteration stops. Otherwise, the time step is reset to 1 and a new round of training begins.

[0020] Furthermore, the creation of training samples includes:

[0021] Based on the current state, the basic action is selected by the main execution network and Gaussian noise is added to obtain the exploration action;

[0022] Perform exploration actions in the power grid environment and observe the immediate rewards and transitions to the next time-instance state in the environmental feedback.

[0023] The exploration actions and the related information obtained are used to construct the raw information, which is stored as a sample in the experience replay buffer, and then the current state is updated.

[0024] Determine if the number of samples in the experience replay buffer is greater than the batch size. If so, start network training; otherwise, increment the time step by 1 and return to the previous time step for judgment.

[0025] Furthermore, S5 includes: reconstructing a new node feature matrix of the power grid based on the optimal action of the power grid optimization control and the power grid state vector; comparing the new node feature matrix with the node feature matrix based on spatiotemporal information gain determined in S3; keeping the node voltage and active power unchanged; adjusting the reactive power; returning to S1 to calculate the power grid voltage stability margin; thereby obtaining the voltage stability margin result after real-time output action control, and realizing the optimized control of power grid voltage stability.

[0026] The beneficial effects of this invention include:

[0027] (1) Improve voltage stability accuracy: The combination of the spatiotemporal dependency graph of the power grid nodes constructed by the spatiotemporal information gain and the graph attention network can accurately extract the spatiotemporal dependency features between the power grid nodes, thereby improving the control accuracy of voltage stability margin (VSI).

[0028] (2) Real-time optimization control: Based on reinforcement learning algorithms, reactive power compensation can be adaptively adjusted according to the real-time grid status and voltage value. The agent continuously optimizes the control strategy through interaction with the grid environment to ensure that the grid maintains voltage stability under different loads and disturbances.

[0029] (3) Adaptability to complex dynamic environments: The system of this invention can adapt to real-time changes in complex dynamic environments such as power grid disturbances and load changes. The introduction of reinforcement learning algorithms enables the control system to have strong adaptive capabilities and cope with voltage stability challenges in different scenarios.

[0030] (4) Reduced human intervention: This invention reduces the reliance on manual adjustment in traditional power grid control through the automatic learning and optimization control of the intelligent agent. By learning historical data of the power grid, the intelligent agent can complete voltage stability control tasks without human intervention, thereby improving the level of automated management of the power grid.

[0031] (5) Improve the safety and reliability of the power grid: Based on real-time regulation of the reactive power state of the power grid and voltage optimization control, this invention can ensure the stable operation of the power grid, avoid power system failures caused by voltage fluctuations, and improve the safety and reliability of the power grid. Attached Figure Description

[0032] Figure 1 This is a flowchart of a method for rapid regulation and stabilization control of power grid voltage according to an embodiment of this application.

[0033] Figure 2 This is a flowchart illustrating the construction of a voltage stability optimization control model based on a reinforcement learning algorithm, as described in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0035] The following is combined Figures 1-2 Specific embodiments of the present invention will be described in detail;

[0036] In this embodiment, as Figure 1 As shown, a method for rapid regulation and stable control of power grid voltage includes the following steps:

[0037] S1, Real-time acquisition of node voltage Active power reactive power Constructing the power grid state vector ,node exist Voltage stability margin at time:

[0038]

[0039] in, Represents a node Self-guided absorbance, Represents a node exist Active power at any given time Represents a node The self-induced phase angle, Represents a node exist Reactive power at any given moment Represents a node With nodes Mutual admittance, Represents a node exist The voltage phase angle at any given time; the system collects real-time data on voltage, current, active power, and reactive power at each node of the power grid through sensors. The power fluctuation of the wind turbine follows a normal distribution, and the output power range of the wind turbine is... These data are collected by sensors to form time series data, which are used for subsequent processing and model input, and undergo data preprocessing. Data preprocessing includes removing outliers and normalization to ensure that features of different units and magnitudes have the same scale.

[0040] S2. Construct a power grid spatiotemporal relationship matrix based on spatiotemporal information gain:

[0041] S2.1, Assume the power grid system consists of... It consists of nodes, each node has Each observable physical variable (voltage, active power, reactive power, etc.); at each time step The system state is determined by the node characteristic matrix. This represents the physical state of a node, where each row represents the physical state of the node. :

[0042]

[0043] S2.2, Define the feature vector of node i at time step:

[0044]

[0045] in, is the voltage value of node i at time step t, and w is the sampling window size, which takes the value of 12 steps (12 hours).

[0046] S2.3 Define the average delay voltage of node j at time step t. :

[0047]

[0048] in, This represents the time step offset of the delay, with a value of 3 steps (3 hours).

[0049] S2.4 For each node x and y, calculate their cross-correlation value. :

[0050]

[0051] in, This represents the joint probability density of x and y. and Let x and y represent the marginal probability densities, respectively.

[0052] S2.5. By calculating the cross-correlation values ​​for different time delays, select the maximum cross-correlation value. As the strongest spacetime dependency:

[0053]

[0054] S2.6. Based on the calculated maximum cross-correlation value, construct a spatiotemporal dependency graph, including the adjacency matrix. and node feature matrix Adjacency matrix This represents the dependency relationship between nodes, each element. Indicate the information gain of node i and node j;

[0055]

[0056] S3. Construct a node feature matrix based on spatiotemporal information gain:

[0057] S3.1, Adjacency matrix and node feature matrix The input is fed into the graph neural network. The Leaky ReLU function in the graph neural network is called to compute each pair of nodes. and similarity between :

[0058]

[0059] In the Leaky ReLU function, This is the weight matrix;

[0060] S3.2. Softmax normalize the similarity between nodes to obtain the attention coefficient for each pair of nodes. :

[0061]

[0062] S3.3 For each node, the calculated attention coefficients are used to perform a weighted summation of the features of the neighboring nodes to obtain a new feature vector for the node. :

[0063]

[0064] S3.4, Represent the new features of all nodes. Combined into a new feature matrix :

[0065]

[0066] S3.5, Extract the node feature matrix from GAT The data is input into an LSTM network to process the time series data and predict the voltage stability margin at future time points.

[0067]

[0068] S4. Build a voltage stability optimization control model using reinforcement learning algorithm;

[0069] S4.1 Constructing a Markov Decision Process: Including a state space Action space and reward function ;

[0070] state space Defined as the real-time operation feature vector of the power grid ,include Matrix, real-time node voltage Active power reactive power and voltage stability margin State vector Represented as:

[0071] ;

[0072] reward function The secondary penalty method is as follows:

[0073]

[0074] Where sigmoid represents the sigmoid function. This is the proportionality constant, with a value of 0.2. This is the scaling factor, with a value of 0.5. This is the proportionality coefficient, with a value of 0.6;

[0075] Action space Let be a continuous space, where the action vector is represented as: ; This indicates the adjustment amount for reactive power compensation in the power grid;

[0076] S4.2, Construct the main execution network and target execution network Their parameter sets are denoted as follows: , Used to implement state To deterministic actions ;

[0077] Construct two sets of main evaluation networks and two sets of target evaluation networks Their parameter sets are respectively Used to evaluate in state Next action value function , The mapping strategy is as follows. All networks employ a fully connected structure with two hidden layers (256 neurons each).

[0078] S4.3 Setting training parameters and initialization; setting the total number of training rounds. =10000, maximum number of rounds per round =48; Set the experience playback buffer. The capacity is =100000 and initialize to empty; set the learning rate for the execution network and the comment network. =0.001, discount factor =0.99, soft update coefficient =0.005, batch size =256, policy delay update frequency =2; Initialize all main network parameters using orthogonal initialization method; Initialize target network parameters; Initialize round counter. =1, initialize the time step counter. =1;

[0079] S4.4. Begin a new training round, reset the power grid simulation environment, and obtain the initial standardized state. Simultaneously initialize the normal noise process. This is used to generate time-dependent exploration noise to adapt to the dynamic characteristics of the power grid; its mean is set to 0 and its standard deviation is set to... Set to 0.02; obtain the optimal action strategy from step S4.15. This serves as the initial movement for this training round;

[0080] S4.5 Determine the current time step Is it less than or equal to the maximum number of time steps? ,like < If yes, proceed to step S4.6; otherwise, the current round ends and proceed to step S4.14.

[0081] S4.6 Initial actions of the main execution network as determined in S4.4 Add Gaussian noise to the base , obtain exploration action ,in, It is Gaussian noise;

[0082] S4.7 Perform exploration actions in the power grid environment Immediate rewards for observing environmental feedback and the state transitioned to in the next moment ;

[0083] S4.8 Constructing the original information And store it in the experience replay buffer. Then update the current state. ;

[0084] S4.9 Determine the experience replay buffer Is the number of samples greater than the batch size? If yes, proceed to step S4.10 to begin network training; otherwise, skip training and proceed to step S4.13.

[0085] S4.10, from the experience replay buffer A small batch is randomly selected from the middle. For each sample, calculate the target Q value.

[0086] First, target policy smoothing techniques are applied to calculate the noisy target actions. :

[0087]

[0088] in, This is a truncation function. To smooth out the noise, it follows a Gaussian distribution with a standard deviation of 0.2. This is the noise cutoff value, which is 0.5.

[0089] Then, the target evaluation network is computed using truncated double-Q learning. Current target value :

[0090]

[0091] S4.11 Update the main evaluation network by minimizing the loss function parameter set ;

[0092]

[0093]

[0094] in, For the evaluation network of the target The calculated target value, Main evaluation network The loss function; Indicates the state and actions The main evaluation network Output action value function value; Indicates the state and actions The main evaluation network Output action value function value; As a discount factor, Indicates the main evaluation network The gradient of the loss function, Indicates the main evaluation network The gradient of the loss function;

[0095] S4.12 Determine whether the current time step t meets the delayed update condition. If so, then perform updates to the main execution network and the target network: update the main execution network. Maximize the reward through a deterministic policy gradient:

[0096]

[0097] in, Represents the policy gradient;

[0098] Then, all target network parameters are updated using a soft update method:

[0099]

[0100] If the delayed update conditions are not met, skip this step;

[0101] S4.13, Time Step Counter Increment by one, then return to step S4.5;

[0102] S4.14, End of this round, turn counter. Add one. Determine the current round number. Is it greater than the total number of training rounds? If yes, proceed to step S4.15; otherwise, reset the time step counter. =1, return to step S4.4 to start the next round of training;

[0103] S4.15. Training process ends, iteration stops; output and save the optimal control action. As the optimal action in this round of power grid optimization control Return to S4.1 and restart training to further optimize the voltage stability margin of the power grid.

[0104] S5, Power grid voltage stability control;

[0105] Based on the optimal action of power grid optimization control , power grid state vector Reconstruct the node feature matrix of the power grid ,Compare The node feature matrix based on spatiotemporal information gain determined in step S3 Maintaining the node voltage, active power remains constant, reactive power is adjusted, and (1) the grid voltage stability margin is calculated to obtain the real-time output action of the main execution network. The voltage stability margin results after control This enables optimized control of grid voltage stability.

[0106] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A method for rapid regulation and stable control of power grid voltage, characterized in that, Includes the following steps: S1: Collect data in real time, construct the power grid state vector, and calculate the voltage stability margin of the power grid nodes; S2: Design an initial node feature matrix to represent the state of the power grid system, where each row represents the physical state of each node; calculate the mutual information information between each pair of nodes to determine the information gain between nodes, and construct a spatiotemporal relationship matrix based on spatiotemporal information gain; S3: Construct a node feature matrix based on spatiotemporal information gain based on the initial node feature matrix and the spatiotemporal relationship matrix; S4: Build a voltage stability optimization control model based on reinforcement learning algorithm. The state space of the model includes a node feature matrix based on spatiotemporal information gain. Use the model to output the optimal action of the current round of power grid optimization control. S5: Perform grid voltage stability control based on the optimal action.

2. The method for rapid regulation and stable control of power grid voltage according to claim 1, characterized in that, S2 includes: calculating the feature vector and mean delay voltage of each node based on the physical state of the node; calculating the mutual correlation between each pair of nodes based on the feature vector and mean delay voltage; selecting the maximum mutual correlation value between each pair of nodes to determine the information gain between the nodes; and constructing an adjacency matrix as a spatiotemporal relation matrix based on the information gain between the nodes.

3. The method for rapid regulation and stable control of power grid voltage according to claim 1, characterized in that, S3 includes: inputting the spatiotemporal relationship matrix and the node feature matrix into the graph neural network, calculating the similarity of each pair of nodes, and obtaining the attention coefficient of each pair of nodes; using the attention coefficient to perform a weighted summation of the neighboring nodes of each node to obtain the new feature vector of that node; and combining the new feature vectors of all nodes into a node feature matrix based on spatiotemporal information gain.

4. The method for rapid regulation and stable control of power grid voltage according to claim 1, characterized in that, S4 includes: Construct a Markov decision process, including state space, action space, and reward function; A main execution network and a target execution network are constructed to realize the transformation of node states into deterministic actions; a main evaluation network and a target evaluation network are constructed to evaluate the reactive power compensation function for performing a deterministic action in a deterministic node state. Set and initialize the training hyperparameters, train the main execution network and the target execution network, output and save the optimal control action after training, use it as the optimal action for this round of power grid optimization control, and update the node state.

5. The method for rapid regulation and stable control of power grid voltage according to claim 4, characterized in that, state space Defined as the real-time operation feature vector of the power grid Including node feature matrices based on spatiotemporal information gain Real-time node voltage Active power reactive power and voltage stability margin State vector Represented as: ； reward function The secondary penalty method is as follows: Where sigmoid represents the sigmoid function. This is the proportionality coefficient. Scaling factor This is the proportionality coefficient; Action space Let be a continuous space, where the action vector is represented as: ; The adjustment amount for reactive power compensation in the power grid is determined by the optimal control action.

6. The method for rapid regulation and stable control of power grid voltage according to claim 5, characterized in that, The training includes: starting a new training round, obtaining the optimal action strategy from the action space as the initial action for this training round; determining whether the current time step is less than the maximum time step; if not, the training round ends; if so, creating training samples and placing them into the experience replay buffer. B samples are randomly selected from the experience replay buffer, and the target value is calculated. The parameters of the main evaluation network are updated by minimizing the loss function. Then it is determined whether the current time step meets the delayed update condition. If so, the main execution network and the target network are updated. Otherwise, the time step is incremented by 1 and the time step is returned for judgment. At the end of this training round, the round counter is incremented by 1. It is then determined whether the current round number is greater than the total number of training rounds. If so, the training process ends and the iteration stops. Otherwise, the time step is reset to 1 and a new round of training begins.

7. The method for rapid regulation and stable control of power grid voltage according to claim 6, characterized in that, The process of creating training samples includes: Based on the current state, the basic action is selected by the main execution network and Gaussian noise is added to obtain the exploration action; Perform exploration actions in the power grid environment and observe the immediate rewards and transitions to the next time-instance state in the environmental feedback. The exploration actions and the related information obtained are used to construct the raw information, which is stored as a sample in the experience replay buffer, and then the current state is updated. Determine if the number of samples in the experience replay buffer is greater than the batch size. If so, start network training; otherwise, increment the time step by 1 and return to the previous time step for judgment.

8. The method for rapid regulation and stable control of power grid voltage according to claim 1, characterized in that, S5 includes: based on the optimal action of the power grid optimization control and the power grid state vector, reconstructing the new node feature matrix of the power grid, comparing the new node feature matrix with the original node feature matrix, keeping the node voltage and active power unchanged, adjusting the reactive power, returning to S1 to calculate the power grid voltage stability margin, thereby obtaining the voltage stability margin result after real-time output action control, and realizing the optimized control of power grid voltage stability.

Images