Power system control method and device based on data imbalance, equipment and medium

Abstract

The invention provides a power system control method, device and equipment based on data imbalance and a medium, and relates to the technical field of power system safe operation and control. According to the method, a topological feature matrix is used as an observation state to be input into a D3QN network, an optimal emergency control strategy is learned through competition of a target network and a main network and value flow and advantage flow network interactive learning, unbalanced data in an experience pool is processed through an adaptive generative adversarial network enhancement algorithm in the training process, and an experience playback mechanism is assisted, so that the optimal emergency control strategy is obtained. And the power system control method is formed. According to the method, adaptive data expansion can be performed on the reinforcement learning experience pool, and the training process of reinforcement learning is accelerated by matching with an experience playback mechanism. Besides, association of the topological structure of the power system in the space-time dimension can be captured, key characteristics can be extracted, the method can adapt to various fault scenes in the power system, high adaptability is achieved, and an efficient and economical power system emergency control low-voltage load shedding strategy can be given.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of safe operation and control of power systems, and particularly relates to a power system control method and device based on data imbalance, equipment and medium. BACKGROUND

[0002] With the expansion of new power system scale and the increase of new energy penetration rate, the complexity and uncertainty of power grid operation conditions have significantly increased. Traditional emergency control strategies mostly rely on offline simulation and artificial rule base, which have response lag, high risk of over-control, and difficulty in adapting to topology changes. In recent years, deep reinforcement learning (DRL) has been introduced into the field of power grid control due to its environmental adaptability, but it faces two major technical problems: the proportion of severe voltage instability scenarios in actual power grids is less than 1%, resulting in a lack of high-value samples in the DRL experience pool, and the agent strategy tends to focus on high-frequency low-risk actions, ignoring critical emergency control needs; existing DRL methods use fully connected neural networks to process power grid states, which fail to effectively extract the electrical coupling relationship between nodes, leading to a mismatch between control actions and network dynamic responses. SUMMARY

[0003] The present application aims to provide a power system control method and device based on data imbalance, equipment and medium, which can solve the problems of lack of high-value samples in the deep learning experience pool, tendency of agent strategy to focus on high-frequency low-risk actions, and ignoring critical emergency control needs, and fail to effectively extract the electrical coupling relationship between nodes, leading to a mismatch between control actions and network dynamic responses. The present application not only increases the imbalance data in the experience pool and speeds up the training process, but also ensures the safe operation of the system while maintaining economic efficiency and high efficiency.

[0004] To achieve the above-mentioned purpose, the present application provides a power system control method based on data imbalance, comprising: setting the data imbalance condition of the power system, the fault scenario, and the action strategy, constraint condition and objective function of the power system emergency control considering the data imbalance condition; initializing the system operation state, real-time collecting the observation values of multiple nodes of the power grid to be tested at each time; randomly disconnecting two busbars to simulate relay maintenance, randomly selecting a fault busbar, and cutting off the fault after a certain duration; obtaining the node feature matrix and performing preprocessing; extracting the power system topology information through the graph convolutional neural network and generating the topology feature matrix from the preprocessed node feature matrix; The topological feature matrix is ​​input into the D3QN network as the observation state. The optimal emergency control strategy is learned through competition between the target network and the main network, and interactive learning between the value stream and advantage stream networks. During the training process, the unbalanced data in the experience pool is processed by an adaptive generative adversarial network enhancement algorithm, supplemented by an experience replay mechanism, to form a power system control method.

[0005] According to a power system control method based on data imbalance provided by the present invention, the action strategy of power system emergency control is low voltage load reduction; Constraints include: Dynamic behavior constraints of the generator and its controller,

[0006] Network coupling constraint relationships between generators, transmission branches and loads,

[0007] Inequality constraints on node voltage, load ratio, frequency, and power angle,

[0008] Where, x t are the algebraic variables in the operation of the power grid, including node voltage amplitude, frequency, and power angle; y t Represents the power grid system parameters, including system topology and given node power; d t Indicates disturbances in the power grid; a t Provides low voltage load shedding action for the power system; The objective function includes:

[0009] Where, is the total cost function of low voltage load shedding control, including the load shedding cost function , voltage cost function , other cost functions ; 、 、 They are load reduction factor, voltage factor and other factors respectively; Corresponding node i The apparent power, For nodes i The bus voltage, is the line impedance, L is the total load demand; Busbar node i The minimum voltage value, 1p.u. It is the per-unit reference value.

[0010] According to a power system control method based on data imbalance provided by the present invention, the observation values ​​of multiple nodes of the power grid to be tested at each moment are:

[0011] If and only if the node is configured with a generator set, the power angle information of the node is collected; to Corresponding to the selected system topology i Node, node voltage amplitude from time n-1 before sampling time t to sampling time t; to Corresponding to the selected system topology i The node load ratio from n-1 moments before sampling time t to sampling time t; to Corresponding to the selected system topology j The power angle of a generator node from the time n-1 before the sampling time t to the sampling time t; to Corresponding to the selected system topology i The node frequency from n-1 moments before sampling time t to sampling time t.

[0012] According to a power system control method based on data imbalance provided by the present invention, the node characteristic matrix formed by the observation values ​​of multiple nodes of the power grid to be tested at each moment is:

[0013] in,

[0014] Where, They represent the node voltage amplitude set, node load proportion set, power angle set and node frequency set of all nodes in the selected power system at sampling time t respectively; 、 、 They represent the node voltage amplitude, node load proportion, and node frequency of the mth node in the selected power system at sampling time t respectively; Indicates the first x The power angle of each generator set at sampling time t.

[0015] According to a data imbalance-based power system control method provided by the present invention, a node characteristic matrix is ​​obtained and preprocessed, including: The power system topology relationship is constructed based on the node adjacency matrix A of the power system topology structure; in the node feature matrix,v i 、 v j Respectively represent the selected power system i nodes and j nodes, i =1, 2, ..., n, j =1, 2, ..., n; node adjacency matrix A Elements A ij Used to indicate v i 、 v j The connection status between nodes i and nodes j If there is a connection relationship, A ij =1, otherwise, A ij =0; Perform Z-Score normalization on the node feature matrix to obtain the normalized node feature matrix ,in is the normalized node feature matrix The element at row i and column j in .

[0016] According to a power system control method based on data imbalance provided by the present invention, the layer-by-layer propagation rule of the graph convolutional neural network is:

[0017] Initially, ; Where, is the normalized Laplace matrix, and the calculation process is:

[0018] Where, A is the node adjacency matrix; D is the degree matrix, which represents the number of edges connected to the node; I is the identity matrix; Representative l The feature matrix of the layer neural network; It is l The weight matrix of the layer neural network; is the activation function.

[0019] The application provides a power system control method based on data imbalance, which inputs a topological characteristic matrix as an observation state into a D3QN network, competes through a target network and a main network, and learns an optimal emergency control strategy through value flow and advantage flow network interaction, and the method comprises the following steps: The reward function is set according to the constraint condition and the target function:

[0020] In the formula, is a reward function of a low-voltage load shedding control process, including a steady-state reward function and an instability reward function ; , respectively represent system instability and steady-state reward coefficients; a, b, c, d and e are reward function coefficients set; V ref is a node voltage reference value; if the system low-voltage load shedding control fails, a maximum penalty -g is given; is a deviation of the node i voltage at the time t from the target voltage, is a voltage amplitude of the node i at the time t, is a maximum deviation value of the node voltage and the target voltage; is a maximum deviation value of the node frequency and the reference frequency; is a node load shedding amount; is a node frequency deviation at the time t, is a frequency value of the node i at the time t; f nom is a reference frequency; is a node power angle deviation at the time t; For each time step, according to the current state s , a greedy strategy is used to select a power system low-voltage load shedding action a, and a reward r and a next state s’ are obtained after the action is performed; these experiences (s, a, r, s’ ) are stored in an experience pool for subsequent training; the D3QN network performs Q value estimation through a target network and a main network; during training, a small batch of experiences is sampled from the experience pool, and the main network θ selects an optimal action, and the target network θ' evaluates the value of the selected low-voltage load shedding action, and the target function calculation formula is:

[0021] Then, the value flow V ( s ) and the advantage flow A ( s , a ) double networks decompose the Q value, and the expression is:

[0022] The loss function used by the D3QN network is:

[0023] The Q value is updated by the loss function, and the target network is periodically synchronized by soft update. In a second aspect, the present application provides a power system control device based on data imbalance, comprising: A setting unit is configured to set the data imbalance condition of the power system, the fault scenario, and the action strategy, the constraint condition and the objective function of the emergency control of the power system considering the data imbalance condition; A preprocessing unit is configured to initialize the system operation state, collect the observation values of each time of multiple nodes of the power grid to be tested in real time, randomly disconnect two busbars to simulate relay maintenance, randomly select a fault busbar, cut off the fault after a certain duration, obtain a node feature matrix, and perform preprocessing; An extraction unit is configured to extract power system topology information from the preprocessed node feature matrix through a graph convolutional neural network, and generate a topology feature matrix; An output unit is configured to input the topology feature matrix as an observation state into the D3QN network, compete between the target network and the main network, and interactively learn through the value flow and advantage flow networks to learn the best emergency control strategy, process the unbalanced data in the experience pool through the self-adaptive generative adversarial network enhancement algorithm during the training process, and assist with the experience replay mechanism to form a power system control method.

[0024] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the power system control method based on data imbalance of the first aspect when executing the computer program.

[0025] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the power system control method based on data imbalance of the first aspect.

[0026] The present application at least has the following technical effects: The application provides a power system control method and device based on data imbalance, a equipment and a medium. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0028] In the drawings: Figure 1 The flowchart of the power system control method based on data imbalance of the application is shown in the figure. Figure 2 The flowchart of the adaptive generative adversarial network enhancement algorithm of the application is shown in the figure. Figure 3 The structural block diagram of the electronic equipment of the application is shown in the figure. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the application more clear, the technical solutions in the application will be described clearly and completely in combination with the drawings in the application. Obviously, the described embodiments are some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without any creative effort belong to the protection scope of the application.

[0030] The embodiments of the application will be described in detail in combination with the drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0031] To solve the above problems, the application provides a deep reinforcement learning control framework fusing a graph convolutional neural network (GCN) and an adversarial generative data enhancement (AGAA), dynamically extracts bus-branch correlation features through the GCN, constructs a power grid topology-aware state space, and improves the matching accuracy of the control strategy and the network structure; the AGAA algorithm is designed to generate voltage instability scene samples in a directional manner, simultaneously introduces a power flow equation constraint to ensure the physical rationality and control operability of the synthesized data, constructs a hybrid priority index based on a TD error and action scarcity, realizes efficient reuse of high-value experience by using a SumTree structure, accelerates strategy convergence, improves a traditional deep reinforcement learning emergency control strategy, and trains a power system emergency control strategy with economy and high efficiency.

[0032] Embodiment 1 Please refer to Figure 1 The embodiment of the application provides a power system emergency control method considering data imbalance, which comprises the following steps: Step 1, setting the data imbalance of the power system, the fault scene, and the action strategy, the constraint condition and the objective function of the power system emergency control considering the data imbalance; Step 2, initializing the system running state, collecting the observation values of each time of the nodes of the power grid to be tested in real time; randomly disconnecting two buses to simulate the maintenance of the relay protection, randomly selecting a fault bus, and cutting off the fault after a certain time; obtaining a node feature matrix and performing pretreatment; Step 3, extracting the power system topology information through a graph convolutional neural network and generating a topology feature matrix by using the pretreated node feature matrix; Step 4, inputting the topology feature matrix as an observation state into a D3QN network, competing between a target network and a main network, and learning a best emergency control strategy through the interaction learning of a value flow network and an advantage flow network, processing the unbalanced data in the experience pool through an adaptive generative adversarial network enhancement algorithm, and forming a power system control method by using an experience replay mechanism.

[0033] Specifically, in step 1, the scale of the new power system example is m, and the new energy penetration rate is 50%. The action strategy of the power system emergency control set in step 1 is low-voltage load shedding, and the action bus selected is a bus with more load shedding in the system. The designed control constraints include: The dynamic behavior constraint relationship of the generator and the controller thereof:

[0034] The network coupling constraint relationship among the generator, the power transmission branch and the load:

[0035] Undervoltage load reduction control requires that the node voltage, load ratio, frequency, power angle, etc. cannot exceed the limit value. Its inequality constraint is:

[0036] in, x t are the algebraic variables in the operation of the power grid, including node voltage amplitude, frequency, power angle, etc. y t Represents the grid system parameters, including system topology, given node power, etc. d t represents the disturbance in the power grid, a t Provides low voltage load shedding action for the power system; The objective function of the low-voltage load reduction control strategy designed in step 1 is:

[0037] in, is the total cost function of low voltage load shedding control, including the load shedding cost function , voltage cost function , other cost functions ; 、 、 is the load reduction factor, voltage factor and other factors; Corresponding node i The apparent power, For nodes i The bus voltage, is the line impedance, L is the total load demand; Busbar node i The minimum voltage value, 1 p.u. The per-unit value is the benchmark value. Using this as a guide, subsequent reward functions are designed to achieve effective and economical control.

[0038] Specifically, in step 2, the observation values ​​of multiple nodes of the power grid to be tested at each moment are: the observation sample set collected based on the PMU includes the node voltage amplitude, node load ratio, frequency and power angle value; it includes the observation value at time t and all observation values ​​at the previous n-1 moments:

[0039] If and only if the node is configured with a generator set, the node power angle information is collected. to Corresponding to the selected system topology i The node voltage amplitude from time n-1 before sampling time t to sampling time t is to corresponding to the selected system topology, the node load proportion from the time point t-n+1 to the time point t; i corresponding to the selected system topology, the node load proportion from the time point t-n+1 to the time point t; j corresponding to the selected system topology, the node load proportion from the time point t-n+1 to the time point t; i Step 3 needs to observe the state of all nodes in the power system topology structure to form a sample set, that is, the node feature matrix composed of the observation values of each time point of the multiple nodes of the power grid to be tested is:

[0040] wherein,

[0041] wherein, respectively represent the node voltage amplitude set, the node load proportion set, the power angle set and the node frequency set of all nodes in the selected power system at the sampling time t, and the superscripts t-n+1 and t-n+2 respectively represent the parameter values at the time points t-n-1 and t-n-2 before the sampling time t.

[0042] , , respectively represent the node voltage amplitude, the node load proportion and the node frequency of the mth node in the selected power system at the sampling time t; represent the power angle of the mth generator set in the selected power system at the sampling time t. x

[0043] In step 2, the node feature matrix is obtained and preprocessed, including: According to the node adjacency matrix A of the power system topology structure, the power system topology relationship is constructed, and the node feature matrix is processed in the node feature matrix v i , v j respectively represent the mth node and the nth node of the selected power system, i =1, 2, …, n; j =1, 2, …, n; the element i j A A ij for representing v i , v j ​​​​​​​​​The connection status between nodes i and nodes j If there is a connection relationship, A ij =1, otherwise, A ij =0.

[0044] Perform Z-Score normalization on the obtained node feature matrix: According to the node feature matrix obtained previously X ,in i Its row index, corresponding to different samples; j Its column index corresponds to different features; for the node feature matrix X Each element in x i,j In terms of:

[0045] in, is the node feature matrix X Middle j The standard deviation of the column elements is calculated as:

[0046] m is the number of samples, corresponding to the size of the power system. is the node feature matrix X Middle j The standard deviation of the column elements is calculated as:

[0047] The normalized node feature matrix is ​​obtained through the above process ,in is the normalized node feature matrix The element in row i and column j; In step 3, based on the selected new power system, a graph convolutional neural network is used to extract topological features. Its layer-by-layer propagation rules are as follows:

[0048] Initially, In the above formula is the normalized Laplace matrix, and its specific calculation process is as follows:

[0049] in, A is the node adjacency matrix; D is the degree matrix, which represents the number of edges connected to the node; I is the identity matrix. represent the first l characteristic matrix of the layer, which is constantly updated to learn deeper topological features as the number of network layers increases. is the weight matrix of the first l layer neural network, which is constantly adjusted and optimized during the training process; is the activation function, which introduces a nonlinear factor to facilitate learning; After the observation sample set extracts key topological information through the graph convolutional neural network in step 3, the topological feature matrix is obtained, and the reinforcement learning process is performed using the D3QN network, i.e., step 4 specifically includes: According to the emergency control constraint conditions and low-voltage load shedding control objectives established in step 1, the reward function is designed:

[0050] wherein, is the reward function of the low-voltage load shedding control process; it is composed of a steady-state reward function and an instability reward function , and , represent the system instability and steady-state reward coefficients, respectively; a, b, c, d, and e are the set reward function coefficients; V ref is the node voltage reference value; if the system low-voltage load shedding control fails, for example, the system voltage does not recover to 0.8 p.u. after 10 s of simulation, or the frequency difference is greater than ±0.2 UHz during the simulation process, or the power angle difference is greater than 180°, it is determined that the control has failed, and a maximum penalty of -g is given; is the deviation of the target voltage value of node i at time t, is the voltage amplitude of node i at time t, is the maximum deviation value of the node voltage and the target voltage during the simulation process; is the maximum deviation value of the node frequency and the reference frequency during the simulation process; is the node load shedding amount; is the node frequency deviation at time t, is the frequency value of node i at time t; f nom is the reference frequency, which is 50 Hz; is the node power angle deviation at time t; When the power system has the following conditions, it is considered to be system collapse, and the agent is given a high penalty of -g: first, the system operating frequency deviates from the established standard range within a certain period of time; second, the power angle difference between any two generator units exceeds the control threshold within the specified time; third, the system voltage fails to recover to the target voltage level within the specified time.

[0051] For each time step, the agent selects an action a according to the current state s s The power system low voltage load shedding action a is selected using a greedy policy, and the reward r and next state s' are obtained after performing the action s’ These experiences (s, a, r, s’ ) are stored in the experience pool for subsequent training. The D3QN network estimates the Q value using the target network and the main network; during its training, a small batch of experiences is sampled from the experience pool, and the main network θ The optimal action is selected, and the target network θ ’ The value of the selected low voltage load shedding action is evaluated, and its objective function is calculated as:

[0052] After that, the value stream V ( s ) and the advantage stream A ( s , a ) double network decompose the Q value, and its expression is:

[0053] The loss function used by the D3QN network is:

[0054] The Q network parameters are updated by the loss function, and the target network is periodically synchronized by soft update.

[0055] In step 4, as shown in Figure 2 , the adaptive generative adversarial network enhancement algorithm includes: A dynamic priority experience pool is constructed and experiences are stored, and the experience pool is defined as:

[0056]

[0057] Its multi-level storage architecture includes: constructing a real-time cache area to store the original interaction data of the last 10 episodes , with an upper limit of capacity of 30000; constructing a synthetic enhancement area to store the synthetic experiences generated by AGAA , with a dynamic capacity adjustment :

[0058] A priority core pool is constructed to store high-value experiences , and high-value experiences in the reinforcement learning experience pool refer to unstable samples:

[0059] is the TD error, and its expression is:

[0060] Use SumTree structure to manage sample priority The priority calculation rules are as follows: Initial priority, raw experience :

[0061] Where r is the immediate reward value; is the discount factor, which represents the rate of decay of future returns; is the target network Q value, is the Q value of the main network; is the lowest voltage value of all nodes in the corresponding state s; Every time you pass After that, the priority is updated to:

[0062] Build adversarial generator and discriminator, generator Input noise Splicing category labels y ,in y= 1 is the voltage collapse sample (definition), y =0 is a normal sample; discriminator Output the sample authenticity probability and category probability ; The loss function of the adversarial generation model is designed as:

[0063] The constraint parameters satisfy:

[0064] in is the node admittance matrix; is the generator node voltage vector; Inject power vectors into nodes; is the power imbalance; J is the Jacobian matrix; For the generated samples , perform fast transient simulation judgment. If the voltage is unstable during the simulation, reject the sample and update the generator:

[0065] The samples that pass the verification are added to and give it the initial priority:

[0066] To prevent the generated reinforcement learning policy from violating the power system security constraints, a penalty is introduced in the priority:

[0067] in For actionable collections, is the violation penalty coefficient, I ( x ) is the indicator function, which takes 1 if there is a violation and 0 if there is no violation; Calculate the sampling probability of each experience according to the priority:

[0068] Among them, the priority adjustment factor ; To eliminate the bias introduced by priority sampling, a weighted correction is required for the gradient update:

[0069]

[0070] After normalization:

[0071] The experience pool adopts a dynamic update mechanism, and its synthetic data ratio is controlled as follows:

[0072] After every 100 training steps, the growth rate The capacity of the amplified and synthesized sample should not exceed 30% of the basic capacity; Through the above series of steps, the output value of the main network is made close to the target Q Then, select the value corresponding to the maximum Q The action of the value is used to obtain the optimal low-voltage load reduction control solution. Compared with the defects of current reinforcement learning technology in emergency control when dealing with power system data imbalance, the present invention can adaptively expand the data of the reinforcement learning experience pool, and at the same time, with the experience replay mechanism, accelerate the training process of reinforcement learning. In addition, it can also capture the correlation of the power system topology in the time and space dimensions, extract key characteristics, and can adapt to various fault scenarios in the power system. It has strong adaptability and can provide an efficient and economical power system emergency control low-voltage load reduction strategy.

[0073] Example 2: In the daily operation of power systems, the stability of the system is challenged by various complex factors, such as equipment aging, extreme weather, and intermittent fluctuations in new energy access. Once a fault occurs, low-voltage load shedding, as a key emergency control measure, plays a decisive role in maintaining the stable operation of the power system. However, in the practice of using reinforcement learning to optimize the low-voltage load shedding strategy, the problem of data imbalance is very prominent. Specifically, the probability of severe instability is low in actual practice, and the corresponding sample data is extremely scarce. For example, due to the sudden failure of large-scale power generation equipment, the system frequency drops sharply, and the voltage collapses dramatically. Such severe instability events occur infrequently, and the relevant state data, low-voltage load shedding action data, and reward data corresponding to the system recovery stability account for a very small proportion in the experience pool. On the contrary, there are a large number of mild instability samples and normal samples. Seasonal load fluctuations, local small power equipment entering or exiting operation, and other factors cause mild voltage fluctuations, and the power system runs smoothly in normal conditions, generating a large amount of data and dominating the experience pool. This uneven distribution of data causes the model parameters to be biased towards the characteristics of a large number of mild instability and normal samples during training, resulting in insufficient feature learning for severe instability scenarios, and ultimately leading to the low-voltage load shedding strategy being ineffective in the face of severe instability.

[0074] To address this issue, an AAGA adaptive adversarial generative neural network is introduced. It is composed of a generator and a discriminator. The generator focuses on the severely deficient severe instability sample category in the experience pool and uses a deep neural network to deeply mine and learn the characteristics of the existing severe instability sample data. Based on the voltage drop and current distortion of the bus during severe instability, the generator generates new power system state data under severe instability conditions through complex mathematical transformations and nonlinear activation functions, while matching the corresponding low-voltage load shedding action data and reward data reflecting system recovery. The discriminator distinguishes between the generated samples and the real data samples. During training, the generator and the discriminator are in a state of antagonism. The generator continuously improves the accuracy of data generation to obtain severe instability samples close to real fault conditions, while the discriminator continuously improves its discrimination ability to identify generated data and real data. Through this antagonistic training mechanism, the generator gradually generates high-quality new samples similar to real severe instability samples, expanding the number of severe instability samples in the experience pool. In addition, AAGA has adaptive adjustment capabilities and can sensitively perceive the distribution dynamics of different categories of data in the experience pool. Once it detects that a certain type of severe instability sample data is extremely scarce, AAGA will automatically increase the generation weight of the generator for that type of sample, increasing the intensity of generating new samples, and further promoting the balance of the experience pool data.

[0075] In the process of constructing the low-voltage load shedding optimization model of the power system, the GCN-D3QN deep reinforcement learning algorithm is adopted (i.e., the topology information of the power system is extracted by the GCN in the foregoing embodiment, and the topology feature matrix is input into the D3QN network as an observation state). The GCN processes the topology structure data of the power system through a graph convolutional neural network, takes the power elements such as substations and transmission lines as nodes, and takes the electrical connection relationship between the elements as edges. By constructing an adjacency matrix, the topology relationship of the power system is represented. The GCN can efficiently extract the topology features of the power system, such as the complex power transmission correlation characteristics between different substations, the cascading influence on the surrounding area when the transmission line fails, and other key information. The features processed by the GCN are input into the D3QN reinforcement learning network as an observation state. Different levels of low-voltage load shedding control strategies implemented at different buses are taken as the action space of the D3QN network, which includes determining the accurate load shedding amount of different areas, reasonably planning the sequence of load shedding, and other key operations. Based on the topology feature state, the D3QN network is trained through a main network and a target network. Each network uses a value stream network and an advantage stream network to evaluate the action strategy, and then selects the action with the highest value as the decision output to obtain the optimal low-voltage load shedding strategy.

[0076] In the model training phase, a reward function is designed to evaluate the actual effect of the low-voltage load shedding action taken. The reward function takes into account the influence of data imbalance on the training results, and divides the reward function into a steady-state reward function and an unstable reward function, as well as a misaction penalty coefficient, and sets appropriate weights. On the one hand, the steady-state reward function is set to guide the agent to correctly handle the steady-state power system, and a misaction penalty is set. On the other hand, the unstable reward function is set to monitor the voltage amplitude of each bus in real time, determine whether the system voltage quickly rises to the safe range after the low-voltage load shedding strategy is taken, pay attention to the frequency fluctuation of the system to confirm whether it is within the allowed range, measure whether the system power has recovered to a balanced state, and consider the cost of load loss caused during the load shedding process. If the system voltage quickly stabilizes in the safe range after the implementation of the low-voltage load shedding strategy, the frequency fluctuation is effectively controlled, the power is restored to balance, and the cost of load loss is within an acceptable range, a higher reward is given. Conversely, if the system voltage is continuously low, the frequency fluctuation exceeds the allowed range, the power is severely unbalanced, or the cost of load loss is too high, a lower reward or even severe punishment is given. The GCN-D3QN model is further trained using the experience pool data balanced by the AAGA. Such data includes the state changes of the power system under different fault types and different severity levels, and are processed by the AAGA. In the training process, the network parameters are continuously updated by means of the reward feedback mechanism in the D3QN algorithm, and the decision-making ability of the model is gradually optimized.

[0077] Based on this, this embodiment provides a power system control device based on data imbalance, including: A setting unit, used to set data imbalance conditions, fault scenarios, and action strategies, constraints, and objective functions for emergency control of the power system considering the data imbalance conditions; The preprocessing unit is used to initialize the system operating status and collect observation values ​​of multiple nodes of the power grid to be tested at each moment in real time; randomly disconnect two buses to simulate relay maintenance, randomly select the faulty bus, and remove the fault after the fault persists for a certain period of time; obtain the node feature matrix and perform preprocessing; An extraction unit is used to extract power system topology information from the preprocessed node feature matrix through a graph convolutional neural network and generate a topology feature matrix; The output unit is used to input the topological feature matrix as the observation state into the D3QN network, learn the optimal emergency control strategy through competition between the target network and the main network, and interactive learning between the value stream and the advantage stream network, and process the unbalanced data in the experience pool through the adaptive generative adversarial network enhancement algorithm during the training process, supplemented by the experience replay mechanism, to form a power system control method.

[0078] Example 3 Figure 3 The following is a schematic diagram of the structure of an electronic device, such as Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The memory 830 stores a computer program that can be executed on the processor 810. When the processor 810 executes the computer program, the power system control method based on data imbalance of Example 1 is implemented.

[0079] Example 4 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the power system control method based on data imbalance of embodiment 1 is implemented.

[0080] It should be noted that the computer readable storage medium in this embodiment can be a computer readable signal medium or a computer readable storage medium or any combination of the above two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0081] In this embodiment, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In this embodiment, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable programs. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can send, propagate or transmit programs for use by or in connection with an instruction execution system, apparatus or device. The computer program contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0082] The computer readable storage medium described above can be written in one or more programming languages or a combination of the above for executing the computer program of this embodiment, including object-oriented programming languages such as Java, Python, C++, and conventional procedural programming languages such as C language or similar programming languages. The program can be executed entirely on a user computer, partially on a user computer, as a separate software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any kind of network, including a LAN (local area network) or a WAN (wide area network), or can be connected to an external computer (for example, through the Internet using an Internet service provider).

[0083] The following is a specific embodiment of the present application.

[0084] The scale of the new power system example selected in Step 1 was set to m = 300. This example covers 56 generators, with a renewable energy penetration rate of 60%. Regarding the emergency control action strategy, a low-voltage load shedding method was adopted. After a fault occurs, an action is executed every 0.1 seconds, depending on the actual situation. There are five types of load shedding: no load shedding; 2.5% load shedding; 5% load shedding; 7.5% load shedding; and 10% load shedding. Furthermore, the load shedding of each action node must not exceed 30% of its initial load, and all nodes must participate in the load shedding process.

[0085] Selected power system t Moment i The observation samples of each node include the node voltage amplitude, load proportion, and frequency value at the first 9 moments; j The power angle of a generator set at time t including the first 9 moments can be expressed as:

[0086] Among them, the sampling set contains power angle information only when there is a generator set at the node; 、 、 They represent the new power system topology. i The node voltage amplitude of a node at 9 moments before sampling time t, 8 moments before sampling time t, and sampling time t, 、 、 They represent the new power system topology. i The node load ratio of each node at 9 moments before sampling time t, 8 moments before sampling time t, and sampling time t; 、 、 They represent the new power system topology. j The power angle of the generator set of each node at 9 moments before sampling time t, 8 moments before sampling time t, and sampling time t; 、 、 They represent the new power system topology. i The node frequencies of the nodes at 9 moments before sampling time t, 8 moments before sampling time t, and sampling time t; For the entire system topology, its sampling set is the node feature matrix for: Step 3 requires forming a sample set of all node observation states in the power system topology:

[0087] in,

[0088] wherein, respectively represent the node voltage amplitude set, the node load proportion set, the power angle set and the node frequency set of all nodes in the selected power system at sampling time t, and the node voltage amplitude of the previous 9, 8 time.

[0089] , , respectively represent the node voltage amplitude, the node load proportion and the node frequency of the 300th node in the selected power system at sampling time t. represent the power angle of the 56th generator set in the selected power system at sampling time t.

[0090] the 300th node and the 56th node in the selected power system, i j , i =1, 2, …, 300. j =1, 2, …, 300.

[0091] The number of layers of the graph convolutional neural network selected in the embodiment is 2.

[0092] , respectively represent the system instability reward coefficient and the steady-state reward coefficient, and the values thereof are 0.8 and 0.2 respectively; a, b, c, d are the set reward function coefficients, and the values thereof are 55, 45, 30, 20 and 10 in turn. g=1000. is a discount factor, and the value thereof is 0.5. The priority adjustment factor a is 0.6; every 100 training steps, the synthesized sample capacity is expanded at a growth rate but not more than 30% of the basic capacity; Through the above series of steps, the output value of the main network is forced to approach the target Q value. Subsequently, the action corresponding to the maximum Q value is selected, and thus the optimal low-voltage load shedding control scheme is obtained. Compared with the defects of the current reinforcement learning technology in dealing with the imbalance of power system data in emergency control, the present application can perform adaptive data expansion on the reinforcement learning experience pool, and at the same time, the experience playback mechanism is matched to speed up the training process of reinforcement learning. In addition, it can capture the correlation of the power system topology in the time and space dimensions, extract key characteristics, be suitable for various fault scenarios in the power system, have strong adaptability, and be able to give an efficient and economic emergency control low-voltage load shedding strategy of the power system.

[0093] ​Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the application embrace any and all variations of the present application that fall within the scope of the included claims. It should be understood that the application is not limited to the precise structures as set forth above and in the accompanying drawings, and that various modifications and changes can be made to the application without departing from the scope thereof. The scope of the application is limited only by the claims that follow.

Claims

1. A power system control method based on data imbalance, characterized in that: include: Set the data imbalance of the power system, fault scenarios, and action strategies, constraints, and objective functions for emergency control of the power system considering the data imbalance; Initialize the system operating status and collect observation values ​​of multiple nodes of the power grid to be tested in real time. Randomly disconnect two buses to simulate relay maintenance. Randomly select the faulty bus and remove the fault after the fault persists for a certain period of time. Obtain the node feature matrix and perform preprocessing. The preprocessed node feature matrix is ​​used to extract the power system topology information through a graph convolutional neural network, and a topology feature matrix is ​​generated; The topological feature matrix is ​​input into the D3QN network as the observation state. The optimal emergency control strategy is learned through competition between the target network and the main network, and interactive learning between the value stream and the advantage stream network. During the training process, the unbalanced data in the experience pool is processed by the adaptive generative adversarial network enhancement algorithm, supplemented by the experience replay mechanism, to form a power system control method.

2. The power system control method based on data imbalance according to claim 1, characterized in that: The action strategy of the power system emergency control is low voltage load shedding; The constraints include: Dynamic behavior constraints of the generator and its controller, Network coupling constraint relationships between generators, transmission branches and loads, Inequality constraints on node voltage, load ratio, frequency, and power angle, Where, x t are the algebraic variables in the operation of the power grid, including node voltage amplitude, frequency, and power angle; y t Represents the power grid system parameters, including system topology and given node power; d t Indicates disturbances in the power grid; a t Provides low voltage load shedding action for the power system; The objective function includes: Where, is the total cost function of low voltage load shedding control, including the load shedding cost function , voltage cost function , other cost functions ; 、 、 They are load reduction factor, voltage factor and other factors respectively; Corresponding node i The apparent power, For nodes i The bus voltage, is the line impedance, L is the total load demand; Busbar node i The minimum voltage value, 1 pu It is the per-unit reference value.

3. The power system control method based on data imbalance according to claim 1, characterized in that: The observation values ​​of multiple nodes of the power grid to be tested at each moment are: If and only if the node is configured with a generator set, the power angle information of the node is collected; to Corresponding to the selected system topology i Node, node voltage amplitude from time n-1 before sampling time t to sampling time t; to Corresponding to the selected system topology i Nodes, the node load ratio from time n-1 before sampling time t to sampling time t; to Corresponding to the selected system topology j Generator node, power angle from time n-1 before sampling time t to sampling time t; to Corresponding to the selected system topology i The node frequency from n-1 moments before sampling time t to sampling time t.

4. The power system control method based on data imbalance according to claim 3, characterized in that: The node feature matrix is ​​formed by the observation values ​​of multiple nodes of the power grid to be tested at each moment: in, Where, They represent the node voltage amplitude set, node load proportion set, power angle set and node frequency set of all nodes in the selected power system at sampling time t respectively; 、 、 They represent the node voltage amplitude, node load proportion, and node frequency of the mth node in the selected power system at sampling time t respectively; Indicates the first x The power angle of each generator set at sampling time t.

5. The power system control method based on data imbalance according to claim 4, characterized in that: The node feature matrix is ​​obtained and preprocessed, including: The power system topology relationship is constructed according to the node adjacency matrix A of the power system topology structure; in the node feature matrix, v i 、 v j Respectively represent the selected power system i nodes and j nodes, i =1, 2, ..., n, j =1, 2, ..., n; node adjacency matrix A Elements A ij Used to indicate v i 、 v j The connection status between nodes i and nodes j If there is a connection relationship, A ij =1, otherwise, A ij =0; Perform Z-Score normalization on the node feature matrix to obtain the normalized node feature matrix ,in is the normalized node feature matrix The element at row i and column j in .

6. The power system control method based on data imbalance according to claim 5, characterized in that: The layer-by-layer propagation rule of the graph convolutional neural network is: Initially, ; Where, is the normalized Laplace matrix, and the calculation process is: Where, A is the node adjacency matrix; D is the degree matrix, which represents the number of edges connected to the node; I is the identity matrix; Representative l The feature matrix of the layer neural network; It is l The weight matrix of the layer neural network; is the activation function.

7. The power system control method based on data imbalance according to claim 6, characterized in that: The topological feature matrix is ​​input into the D3QN network as the observation state, and the target network and the main network compete, and the value stream and advantage stream networks interact and learn to learn the optimal emergency control strategy, including: According to the constraints and objective function, the reward function is set as: Where, is the reward function of the low voltage load reduction control process, including the steady-state reward function and unstable reward function ; 、 Represent the system instability and steady-state reward coefficients respectively; a, b, c, d, e are the set reward function coefficients; V ref is the node voltage reference value; if the system low voltage load reduction control fails, a maximum penalty -g is given; is the deviation between the voltage of node i and the target voltage at time t, is the voltage amplitude of node i at time t, is the maximum deviation between the node voltage and the target voltage; is the maximum deviation between the node frequency and the reference frequency; Reduce the load on the node; is the node frequency deviation at time t, is the frequency value of node i at time t; f nom is the reference frequency; is the node power angle deviation at time t; For each time step, according to the current state s Use the greedy strategy to select the power system low-voltage load reduction action a, and obtain the reward r and the next state after executing the action. s’ ; Put these experiences (s,a,r, s’ ) is stored in the experience pool for subsequent training; the D3QN network estimates the Q value through the target network and the main network; during its training, a small batch of experience is sampled from the experience pool and the main network is used to estimate the Q value. θ Select the optimal action and use the target network θ' To evaluate the value of the selected low-voltage load reduction action, the objective function calculation formula is: Afterwards, the value stream V ( s ) and advantage flow A ( s , a ) The dual network decomposes the Q value, and its expression is: The loss function used by the D3QN network is: The Q value is updated through the loss function, and the target network is periodically synchronized through soft updates.

8. A power system control device based on data imbalance, characterized in that: include: A setting unit, used to set data imbalance conditions, fault scenarios, and action strategies, constraints, and objective functions for emergency control of the power system considering the data imbalance conditions; The preprocessing unit is used to initialize the system operating status and collect observation values ​​of multiple nodes of the power grid to be tested at each moment in real time; randomly disconnect two buses to simulate relay maintenance, randomly select the faulty bus, and remove the fault after the fault persists for a certain period of time; obtain the node feature matrix and perform preprocessing; An extraction unit is used to extract power system topology information from the preprocessed node feature matrix through a graph convolutional neural network and generate a topology feature matrix; The output unit is used to input the topological feature matrix as the observation state into the D3QN network, learn the optimal emergency control strategy through competition between the target network and the main network, and interactive learning between the value stream and the advantage stream network, and process the unbalanced data in the experience pool through the adaptive generative adversarial network enhancement algorithm during the training process, supplemented by the experience replay mechanism, to form a power system control method.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the power system control method based on data imbalance according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the power system control method based on data imbalance according to any one of claims 1 to 7 is implemented.

Images