A method for coordinated control of transmission and distribution frequency resources based on a federal reinforcement learning algorithm

Abstract

A kind of frequency resource coordination control method of transmission and distribution based on federal reinforcement learning algorithm, comprising: regional power grid is divided into one main grid area and multiple distribution grid areas;In the dispatching center of each area, an agent is set, and a corresponding DQN neural network model is established for each agent;Each agent uses the data of the corresponding area to locally train the corresponding DQN neural network model, encrypts the information of the locally trained DQN neural network model, and uploads the encrypted information to the aggregation center;The aggregation center carries out gradient average processing to the encrypted information and sends it back to each agent, and each agent uses the information processed by gradient average to carry out subsequent training on the locally trained DQN neural network model to obtain a trained DQN neural network model, and obtains the frequency regulation instruction of each unit accepting dispatching in regional power grid through the trained DQN neural network model.The design can ensure the safe and efficient operation of regional power grid and the privacy safety of frequency regulation users.

Description

Technical Field

[0001] This invention belongs to the field of automatic generation control in power systems, specifically a collaborative control method for transmission, distribution, and frequency regulation resources based on federated reinforcement learning algorithms. Background Technology

[0002] Automatic power control (APC) extends the traditional generator set technology to the adjustable load side, encompassing the original generator set AGC technology and affirming the frequency regulation capabilities of flexible resources. Most of these flexible resources are connected through the distribution network. With the development of technologies such as communication, the distribution network has gradually transformed from a unidirectional receiving network into a local grid with self-balancing capabilities. The relationship between the main and distribution networks has also changed from a master-slave dependency to a mutually supportive bidirectional interactive relationship. Traditional frequency regulation resources, such as thermal and hydropower, are mostly connected to the main grid, whose topology is simpler than the distribution network. Distributed power sources, however, are mostly connected to the distribution network. Their power generation or reduction after being dispatched has a significant impact on the distribution network's operation, as the primary function of the distribution network is always to provide reliable power to users. Against this backdrop, how to integrate these resources with different characteristics and environments to participate in APC is a challenging issue in exploring the development of new power systems.

[0003] The current regional power grid APC closed-loop control process mainly consists of two steps: 1) collecting the grid frequency deviation and tie-line power deviation, calculating the real-time regional control deviation ACE, and obtaining a total generation power command through a PI controller; 2) then distributing this command to each APC unit using relevant power allocation methods. Currently, the total regulation power command is mainly allocated based on the adjustable capacity of the units, but this strategy cannot meet the optimal control requirements of the system. Meanwhile, traditional centralized control suffers from high computational load, centralized communication, poor reliability, and inability to adapt to the flexible and ever-changing structure of active distribution networks. Therefore, the system control mode is gradually shifting from centralized control to distributed control. However, due to the large dispersion of distributed power sources, it is difficult to achieve overall optimization of the autonomous region by installing intelligent agents for distributed control on each frequency regulation unit. Furthermore, the increasing number of distributed power sources exacerbates the trend of diversified entities, threatening the privacy of multiple entities. Based on the existing problems of distributed control, a flexible optimal power allocation strategy is needed to ensure the safe and efficient operation of the regional power grid and the privacy security of frequency regulation users. Summary of the Invention

[0004] The purpose of this invention is to overcome the problems that existing control methods in the prior art are unable to meet the optimal control requirements of the system, cannot adapt to the flexible and ever-changing active distribution network structure, and pose a threat to the privacy and security of frequency regulation users. This invention provides a collaborative control method for transmission, distribution and frequency regulation resources based on federated reinforcement learning algorithm that can meet the optimal control requirements of the system, adapt to the flexible and ever-changing active distribution network structure, ensure the safe and efficient operation of the regional power grid and the privacy and security of frequency regulation users.

[0005] To achieve the above objectives, the technical solution of the present invention is:

[0006] A collaborative control method for transmission and distribution frequency modulation resources based on a federated reinforcement learning algorithm, the control method comprising the following steps:

[0007] S1. Divide the regional power grid into one main grid area and multiple distribution grid areas;

[0008] S2. Set up an agent in the dispatch center of each region and build a corresponding DQN neural network model for each agent;

[0009] S3. Each agent uses local data from its respective region to train the corresponding DQN neural network model locally, performs homomorphic encryption on the information of the locally trained DQN neural network model, and uploads the encrypted information to the aggregation center.

[0010] S4. The aggregation center performs gradient averaging on all encrypted information and sends the gradient-averaged information to each agent. Each agent receives the gradient-averaged information and performs subsequent training on the corresponding locally trained DQN neural network model based on the gradient-averaged information to obtain the trained DQN neural network model. The trained DQN neural network model is then used to obtain the frequency regulation instructions for each unit to be scheduled.

[0011] In step S3, when each agent uses local data from the corresponding region to train the DQN neural network model locally, the state space, action space and reward function of each agent are set according to the Markov decision process.

[0012] Setting the state space of agent z specifically includes:

[0013] The magnitude of the total frequency adjustment command that determines the total deviation of the frequency response during the frequency allocation process is used as the state space of agent z.

[0014] Then the state of agent z at time t is ;

[0015] Setting the action space for agent z specifically includes:

[0016] Define the action space that agent z can make decisions on. All control behaviors of agent z originate from the action space. Selected from;

[0017] Control behavior of agent z at time t It can be represented as:

[0018] (1);

[0019] In formula (1): The active power output of the oth thermal power unit controlled by the z-th intelligent agent at time t; The active power output of the m-th energy storage device controlled by the z-th intelligent agent at time t; The active power output of the nth wind turbine controlled by agent z at time t; The active power output of the j-th electric vehicle group controlled by agent z at time t;

[0020] Setting the reward function for agent z specifically includes:

[0021] To determine the reward for the control behavior of agent z, the reward function for agent z is constructed with the objective of minimizing the deviation between the power command value and the power response value.

[0022] (2);

[0023] (3);

[0024] In equations (2)-(3), Let be the reward function of agent z at time t; Q be the number of control periods; q be the number of APC units in the area corresponding to agent z; i be the i-th APC unit in the area corresponding to agent z; t be the t-th discrete control cycle. The input regulation power command value is the i-th APC unit within the area corresponding to agent z. Let be the power response value of the i-th APC unit within the area corresponding to agent z;

[0025] The objective function is obtained by summing the cumulative discounts. Value function:

[0026] (4);

[0027] In equation (4), To control behavior The average of all accumulated rewards generated is taken. ∈[0,1], This is the discount factor; This is the accumulation of the reward function corresponding to multiple consecutive actions of agent z.

[0028] In step S3, agent z uses local data from the corresponding region to locally train the corresponding DQN neural network model, specifically including:

[0029] S31, agent z initializes the current network parameters of the corresponding DQN neural network model and copies a target network with the same structure;

[0030] Agents S32 and Z train the DQN neural network model using state data from 96 time periods within the corresponding area during the day, and update the parameters of the target network.

[0031] In step S32, the training of the DQN neural network model by agent z using state data from 96 time periods within the corresponding area during the day includes:

[0032] S321. Select the state data of one time period from the state data of 96 time periods within the day as the current state of agent z. ;

[0033] S322, Based on the current state of agent z An ε-greedy strategy is used for trial and error, that is, a random strategy is used with probability ε to select the control behavior. Select the current optimal control behavior with probability 1-ε. , , ∈ ;

[0034] in: (5);

[0035] S323. Based on the selected control action a, the current network in the DQN neural network model calculates the size of the reward function after executing the control action a. The Q value is calculated using the following function:

[0036] (6);

[0037] In equations (5) and (6), This is the current Q value; The target Q value; The learning rate; This is the reward decay coefficient;

[0038] S324. Based on the selected control action a, obtain the next state returned by the environment after agent z executes the selected control action a. , to obtain empirical samples ( ,a, , ), and the experience sample ( ,a, , Stored in the experience replay pool;

[0039] S325. Update the current state of agent z to the next state returned by the environment, and repeat steps S322-S324 until the experience replay pool is full.

[0040] S326. After the experience replay pool is full, Ω experience samples are extracted from the experience replay pool for calculation, and the loss function is updated:

[0041] (7);

[0042] In equation (7): The loss function; Let z be the reward function for agent z. This represents the Q-value of the current network. The target Q value is the one corresponding to the empirical sample.

[0043] In step S3, performing additive homomorphic encryption on the information of the locally trained DQN neural network model and uploading the encrypted information to the aggregation center specifically includes:

[0044] S34. Each agent uses the Paillier fully homomorphic encryption public key K to encrypt the loss function in the corresponding locally trained DQN neural network model to obtain the encrypted loss function.

[0045] S35. Each agent transmits the encrypted loss function to the aggregation center.

[0046] In step S4, the aggregation center performs gradient averaging on all encrypted information and sends the gradient-averaged information to each agent. Each agent receives the gradient-averaged information and performs subsequent training on the corresponding locally trained DQN neural network model based on the gradient-averaged information. Specifically, this includes:

[0047] S41. The aggregation center calculates the comprehensive loss function based on the encrypted loss function sent by each agent. :

[0048] (8);

[0049] In equation (8): This indicates summing multiple encrypted loss functions. Let z be the reward function for agent z. Let z be the Q-value of the target network corresponding to agent z. This represents the current Q-value corresponding to agent z. The learning rate; y represents the reward decay coefficient; y represents the total number of agents.

[0050] S42, The aggregation center will integrate the loss function. The data is sent to each agent, and each agent uses the current network and the comprehensive loss function in its locally trained DQN neural network model. Calculate gradient information;

[0051] S43. Each agent adds a safety mask to the gradient information and transmits the gradient information with the added safety mask to the aggregation center.

[0052] S44. After receiving the gradient information with the added security mask, the aggregation center decrypts the fully homomorphic encryption and returns the result of decryption to the corresponding agent.

[0053] S45. Each agent receives the result after decryption of the fully homomorphic encryption, removes the subnet mask from the result, and obtains the unencrypted gradient information. Each agent updates the parameters of the current network in the corresponding locally trained DQN neural network model using the unencrypted gradient information. .

[0054] In step S45, the parameters of the current network in the corresponding locally trained DQN neural network model are updated. The formula is:

[0055] (9);

[0056] In equation (9): F is the loss function, The updated network parameters for agent z. These are the current network parameters before agent z was updated.

[0057] The control behavior in the action space conforms to the power supply characteristic constraints and the system balance constraints.

[0058] The power supply characteristic constraints specifically include:

[0059] Thermal power unit operating constraints:

[0060] (10);

[0061] In equation (10), , These are the upper and lower limits of the output of the i-th thermal power unit, respectively; Let be the gradeability of the i-th thermal power unit; Let be the output of the i-th thermal power unit at time t; Let be the output of the i-th thermal power unit at time t-1;

[0062] Energy storage device operating constraints:

[0063] (11);

[0064] In equation (11), , Let m be the capacity constraint range for the m-th energy storage device; Let be the capacity of the m-th energy storage device at time t; Let be the charging power of the m-th energy storage device at time t; , The charging power constraint range for the m-th energy storage device; Let be the discharge power of the m-th energy storage device at time t; and Let m be the discharge power constraint range of the m-th energy storage device; Let be the battery capacity of the m-th energy storage device at time t+1; Let be the self-discharge efficiency of the m-th energy storage device; The charging efficiency of the m-th energy storage device; Let be the discharge efficiency of the m-th energy storage device;

[0065] Distributed wind farm unit operation constraints:

[0066] (12);

[0067] In equation (12), This is the lower limit of the output power of the nth wind turbine; This represents the upper limit of the output power of the nth wind turbine. Let t be the output power of the nth wind turbine.

[0068] Electric vehicle swarm state constraints:

[0069] (13);

[0070] (14);

[0071] (15);

[0072] (16);

[0073] In equations (13)-(16), and Let SOC be the range of the j-th electric vehicle; Let the SOC of the j-th electric vehicle be; , The constraint range for the output power increment of the Mth electric vehicle charging station; For the incremental output power of the Mth electric vehicle charging station; , [This refers to the time period during which a single electric vehicle connects to a charging station;] Let be the upper limit of the charging and discharging power of the j-th electric vehicle at time t; Let be the lower limit of the charging and discharging power of the j-th electric vehicle at time t; and It is affected by factors such as the number of vehicles j in the charging station, the SOC capacity of a single electric vehicle, and its charging / discharging state. Let be the SOC of the j-th electric vehicle at time t; The rated charging power for the j-th electric vehicle; Let be the rated discharge power of the j-th electric vehicle; Let be the output power increment of the Mth electric vehicle charging pile at time t; Let M be the upper limit of the incremental output power of the Mth electric vehicle charging pile at time t. Let t be the lower limit of the output power increment of the Mth electric vehicle charging pile at time t; Let be the power of the j-th electric vehicle at time t.

[0074] The system equilibrium constraints specifically include:

[0075] (17);

[0076] (18);

[0077] (19);

[0078] In equations (17)-(19), , , , Let represent the active power output of the i-th thermal power unit, the m-th energy storage device, the n-th wind turbine, and the j-th electric vehicle at time t, respectively. , , , These represent the quantities of thermal power units, energy storage devices, wind turbine units, and electric vehicles, respectively. This represents the load disturbance at time t; Let b be the set of branches whose first node is b. Let be the set of branches whose final node is node b; and Branch roads Active power and reactive power at time t; and Branch roads Resistance and reactance, Let be the voltage amplitude at the relaxed node at time t; Let be the voltage amplitude at node b at time t; The active power of the generator set connected to node b; The reactive power of the generator set connected to node b; The active power of the load connected to node b; The reactive power of the load connected to node b; and Branch roads Upper and lower limits of active power; and Branch roads Upper and lower limits of reactive power; and These are the upper and lower limits of the voltage at node b, respectively.

[0079] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0080] 1. This invention, a collaborative control method for transmission and distribution frequency regulation resources based on a federated reinforcement learning algorithm, divides the regional power grid into one main grid area and multiple distribution grid areas. A regional intelligent agent is set up in the dispatch center of each area. The intelligent agent controls the corresponding area, achieving joint frequency regulation of various types of generators, including traditional units on the main grid side and distributed power sources on the distribution grid side. Information interaction between regional intelligent agents is used to allocate frequency regulation commands to the dispatching units, achieving overall optimization of the autonomous region. Therefore, this design fully utilizes the main and distribution grid structure of the regional power grid to divide areas and set up intelligent agents, using information interaction between intelligent agents to allocate frequency regulation commands to the dispatching units, achieving overall optimization of the autonomous region.

[0081] 2. This invention, a method for coordinated control of transmission and distribution frequency regulation resources based on federated reinforcement learning, utilizes a distributed algorithm of federated reinforcement learning to solve the coordination problem among multiple agents. When training each DQN neural network, offline training is performed using the daily states of 96 points in the corresponding distribution network. Offline training effectively shortens the online decision-making time and further improves the real-time performance of instruction execution. Therefore, this design effectively shortens the online decision-making time and improves the real-time performance of instruction execution by utilizing offline training.

[0082] 3. In the collaborative control method for frequency modulation resources based on federated reinforcement learning algorithm of the present invention, when the information of the local model of the DQN neural network is subjected to additive homomorphic encryption and the encrypted information is uploaded to the aggregation center, each agent uses the Paillier fully homomorphic encryption public key to encrypt the corresponding loss function to obtain the encrypted loss function, and transmits the encrypted loss function to the aggregation center. The aggregation center calculates the comprehensive loss function based on the encrypted loss function sent by each agent and sends the comprehensive loss function to each agent. Each agent calculates the current network in the corresponding local model of the DQN neural network relative to the comprehensive loss function. The gradient information of the loss function is obtained, and then each agent adds a security mask to the gradient information and transmits the masked gradient information to the aggregation center. The aggregation center receives the masked gradient information, decrypts it, and returns the decrypted result to the corresponding agent. Each agent updates the current network parameters in its local DQN neural network model using the decrypted result. In this method, only the model parameters are transmitted and processed, and the model parameters are encrypted using a fully homomorphic encryption public key, effectively improving the data privacy and security of information interaction between agents and avoiding the risk of data leakage during transmission and storage. Therefore, this design only transmits and processes the model parameters, and the model parameters are encrypted using a fully homomorphic encryption public key to avoid the risk of data leakage during transmission and storage and ensure the privacy and security of FM users.

[0083] 4. In the transmission and distribution frequency regulation resource coordinated control method based on federated reinforcement learning algorithm of this invention, the neural network training framework of federated reinforcement learning adopts the DQN training model. The optimal strategy is obtained by optimizing an iteratively calculated state-action pair-valued function matrix Q(s,a), maximizing the sum of expected discounted rewards. The DQN training model can be well integrated into the multi-source coordinated frequency control framework of the main and distribution networks and is more suitable for solving the decentralized optimization problem of APC power dynamic allocation. Therefore, the DQN training model is adopted in this design, which is more suitable for solving the decentralized optimization problem of APC power dynamic allocation. Attached Figure Description

[0084] Figure 1This is a framework diagram of a frequency modulation resource collaborative control system based on federated reinforcement learning algorithms.

[0085] Figure 2 This is a schematic diagram of a federated reinforcement learning framework based on DQN-trained networks. Detailed Implementation

[0086] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0087] See Figures 1 to 2 A collaborative control method for frequency modulation resources in power transmission and distribution based on a federated reinforcement learning algorithm, the control method comprising the following steps:

[0088] S1. Divide the regional power grid into one main grid area and multiple distribution grid areas;

[0089] S2. Set up an agent in the dispatch center of each region and build a corresponding DQN neural network model for each agent;

[0090] Each region's dispatch center is equipped with an intelligent agent and a DQN neural network model. The intelligent agents located in the same region's dispatch center correspond to the DQN neural network model. The intelligent agents only train, control, and operate their corresponding DQN neural network models.

[0091] S3. Each agent uses the local data of its respective region to train the corresponding DQN neural network model locally, performs additive homomorphic encryption on the information of the locally trained DQN neural network, and uploads the encrypted information to the aggregation center.

[0092] After obtaining the overall frequency regulation command from the aggregation center, the training task is issued—the frequency regulation command is allocated to each unit. After the training task is issued by the aggregation center, all agents begin to execute the same task, that is, each agent uses the local data of its own area to simultaneously train the DQN neural network model locally.

[0093] S4. The aggregation center performs gradient averaging on all encrypted information and sends the gradient-averaged information to each agent. Each agent receives the gradient-averaged information and further trains its locally trained DQN neural network based on the gradient-averaged information to obtain the corresponding trained DQN neural network model. The agent then obtains the frequency modulation instructions for each unit in the corresponding area that is being scheduled through the corresponding trained DQN neural network model.

[0094] In step S3, when each agent uses local data from the corresponding region to train the DQN neural network model locally, the state space, action space and reward function of each agent are set according to the Markov decision process.

[0095] The state space of the defined intelligent agent specifically includes:

[0096] Setting the state space of agent z specifically includes:

[0097] The magnitude of the total frequency adjustment command that determines the total deviation of the frequency response during the frequency allocation process is used as the state space of agent z.

[0098] The amplitude of the frequency change is divided into eight intervals:

[0099] {[∞,-0.2),[-0.2,-0.15),[-0.15,-0.10),[-0.10,0.003),[0.03,0.10),[0.10,0.15),[0.15,0.2),[0.2,+∞)};

[0100] Then the state of agent z at time t is , ={S1,S2,S3,S4,S5,S6,S 7， S8}, where S1 and S8 represent the states corresponding to the minimum and maximum values ​​of the total frequency adjustment command of the system under a certain type of disturbance, respectively;

[0101] The action space of the defined intelligent agent specifically includes:

[0102] Define the action space that agent z can make decisions on. All control behaviors of agent z originate from the action space. Selected from;

[0103] Control behavior of agent z at time t It can be represented as:

[0104] (1);

[0105] In formula (1): The active power output of the oth thermal power unit controlled by the z-th intelligent agent at time t; The active power output of the m-th energy storage device controlled by the z-th intelligent agent at time t; The active power output of the nth wind turbine controlled by agent z at time t; The active power output of the j-th electric vehicle group controlled by agent z at time t;

[0106] The control behavior of an intelligent agent must comply with the constraints of the action space. The constraints of the action space include the following two categories: power characteristic constraints considering the dynamic response transmission process of the generating units; and system balance constraints considering the overall stable operation of the power system. Among them, the system balance constraints mainly consider the difference between the joint frequency regulation of the main and distribution networks and the previous multi-source coordinated frequency regulation.

[0107] The power supply characteristic constraints specifically include:

[0108] Thermal power unit operating constraints:

[0109] (10);

[0110] In equation (10), , These are the upper and lower limits of the output of the i-th thermal power unit, respectively; Let be the gradeability of the i-th thermal power unit; Let be the output of the i-th thermal power unit at time t; Let be the output of the i-th thermal power unit at time t-1;

[0111] Energy storage device operating constraints:

[0112] (11);

[0113] In equation (11), Let m be the minimum capacity of the m-th energy storage device; Let m be the maximum capacity of the m-th energy storage device; Let be the battery capacity of the m-th energy storage device at time t; Let be the charging power of the m-th energy storage device at time t; , The charging power constraint range for the m-th energy storage device; Let be the discharge power of the m-th energy storage device at time t; and Let m be the discharge power constraint range of the m-th energy storage device; Let be the battery capacity of the m-th energy storage device at time t+1; Let be the self-discharge efficiency of the m-th energy storage device; The charging efficiency of the m-th energy storage device; Let be the discharge efficiency of the m-th energy storage device;

[0114] Distributed wind farm unit operation constraints:

[0115] (12);

[0116] In equation (12), This is the lower limit of the output power of the nth wind turbine; This represents the upper limit of the output power of the nth wind turbine. Let t be the output power of the nth wind turbine.

[0117] Electric vehicle swarm state constraints:

[0118] (13);

[0119] (14);

[0120] (15);

[0121] (16);

[0122] In equations (13)-(16), and Let SOC be the range of the j-th electric vehicle; Let the SOC of the j-th electric vehicle be; , The constraint range for the output power increment of the Mth electric vehicle charging station; For the incremental output power of the Mth electric vehicle charging station; , [This refers to the time period during which a single electric vehicle connects to a charging station;] Let be the upper limit of the charging and discharging power of the j-th electric vehicle at time t; Let be the lower limit of the charging and discharging power of the j-th electric vehicle at time t; and It is affected by factors such as the number of vehicles j in the charging station, the SOC capacity of a single electric vehicle, and its charging / discharging state. Let be the SOC of the j-th electric vehicle at time t; The rated charging power for the j-th electric vehicle; Let be the rated discharge power of the j-th electric vehicle; Let be the output power increment of the Mth electric vehicle charging pile at time t; Let M be the upper limit of the incremental output power of the Mth electric vehicle charging pile at time t. Let t be the lower limit of the output power increment of the Mth electric vehicle charging pile at time t; Let be the power of the j-th electric vehicle at time t.

[0123] The system equilibrium constraints specifically include:

[0124] (17);

[0125] (18);

[0126] (19);

[0127] Among them, equation (17) is the system power balance constraint, and equations (18) and (19) are the distribution network operation related constraints;

[0128] In equations (17)-(19), , , , Let represent the active power output of the i-th thermal power unit, the m-th energy storage device, the n-th wind turbine, and the j-th electric vehicle at time t, respectively. , , , These represent the quantities of thermal power units, energy storage devices, wind turbine units, and electric vehicles, respectively. This represents the load disturbance at time t; Let b be the set of branches whose first node is b. Let be the set of branches whose final node is node b; and Branch roads Active power and reactive power at time t; and Branch roads Resistance and reactance, Let be the voltage amplitude at the relaxed node at time t; Let be the voltage amplitude at node b at time t; Let be the voltage amplitude of node b at time t+1; The active power of the generator set connected to node b; The reactive power of the generator set connected to node b; The active power of the load connected to node b; The reactive power of the load connected to node b; and Branch roads Upper and lower limits of active power; and Branch roads Upper and lower limits of reactive power; and These are the upper and lower limits of the voltage at node b, respectively.

[0129] Setting the reward function for agent z specifically includes:

[0130] The environment is configured to reward the control behavior of agent z. The objective function of agent z is constructed by modifying its control behavior to minimize the deviation between the power command value and the power response value. and reward function :

[0131] (2);

[0132] The reward function is then:

[0133] (3);

[0134] In equations (2)-(3), Let be the reward function of agent z at time t; Q be the number of control periods; q be the number of APC units in the area corresponding to agent z; i be the i-th APC unit in the area corresponding to agent z; t be the t-th discrete control cycle. The input regulation power command value is the i-th APC unit within the area corresponding to agent z. Let be the power response value of the i-th APC unit within the area corresponding to agent z;

[0135] The objective function is obtained by summing the cumulative discounts. Value function:

[0136] (4);

[0137] In equation (4), For agent z in controlling behavior The function that rewards the controlled behavior accordingly; To control behavior The average of all accumulated rewards generated is taken. ∈[0,1], This is the discount factor; This is the accumulation of the reward function corresponding to multiple consecutive actions of agent z.

[0138] In step S3, agent z uses local data from the corresponding region to locally train the DQN neural network model, specifically including:

[0139] S31. Initialize the current network parameters of the corresponding DQN neural network model for the agent. , ... And replicate a target network with the same structure as the current network. , ... ;

[0140] Agents S32 and Z train the DQN neural network model using state data from 96 time periods within the corresponding area during the day, and update the parameters of the target network.

[0141] The 24-hour period is divided into 15-minute intervals, resulting in 96 intervals. State data for the region corresponding to agent z within each of these 96 intervals is obtained, representing the state data for the 96 daily intervals. This data is then used to train the corresponding DQN neural network model. The agent trains the DQN neural network model multiple times. After each training iteration, the agent copies the current network parameters to the target network, updating the parameters of the target network.

[0142] In step S32, the training of the DQN neural network model by agent z using state data from 96 time periods within the corresponding area during the day includes:

[0143] S321. Agent Z obtains the state data of 96 time periods within the day for the corresponding power grid area, and selects the state data of one time period from the 96 state periods as the current state s of Agent Z. t ;

[0144] S322, Based on the current state s of agent z t An ε-greedy strategy is used for trial and error, that is, a random strategy is used with probability ε to select the control behavior. Select the current optimal control behavior with probability 1-ε. , , ∈ ;

[0145] in: (5);

[0146] Equation (5) indicates that the optimal Q value is selected as the current Q value;

[0147] S323. Based on the selected control action a, the current network in the DQN neural network model calculates the size of the reward function after executing the control action a. The corresponding Q value is calculated using the following function:

[0148] (6);

[0149] In equations (5) and (6), This represents the Q-value of the current network. The target network's Q-value; The learning rate; This is the reward decay coefficient; ∈{r 1,t ,r m,t ,...,r n,t}, {r 1,t ,r m,t ,...,r n,t} represents the set of reward functions for agent z;

[0150] S324. Based on the selected control action a, obtain the next state returned by the environment after agent z executes the selected control action a. , to obtain empirical samples ( ,a, , ), and the experience sample ( ,a, , Stored in the experience replay pool;

[0151] S325. Update the current state of agent z to the next state returned by the environment, and repeat steps S322-S324 until the experience replay pool is full.

[0152] S326. After the experience replay pool is full, Ω experience samples are extracted from the experience replay pool for calculation, and the loss function is updated:

[0153] (7);

[0154] In equation (7): The loss function; For the reward function; The Q-value of the current network corresponding to the empirical sample; The Q-value of the target network corresponding to the empirical sample; This indicates that both the current action state and the target network action state belong to the state space set of agent z; This indicates that both the current action and the target network action belong to the action space set of agent z.

[0155] In step S3, performing additive homomorphic encryption on the information of the locally trained DQN neural network and uploading the encrypted information to the aggregation center specifically includes:

[0156] S34. Each agent uses the Paillier fully homomorphic encryption public key K to encrypt the loss function in its corresponding locally trained DQN neural network model, obtaining the encrypted loss function. , This represents the result after fully homomorphic encryption;

[0157] S35. Each agent will use the encrypted loss function. Transmitted to the aggregation center.

[0158] In step S4, the aggregation center performs gradient averaging on all encrypted information and sends the gradient-averaged information to each agent, specifically including:

[0159] The aggregation center calculates the comprehensive loss function based on the encrypted loss function sent by each agent. and the comprehensive loss function Send to each agent:

[0160] (8);

[0161] In equation (8): This indicates summing multiple encrypted loss functions. Let z be the reward function for agent z. Let z be the Q-value of the target network corresponding to agent z. This represents the current network Q-value corresponding to agent z. The learning rate; y represents the reward decay coefficient; y represents the total number of agents.

[0162] Each agent receives the information after gradient averaging and uses this information to further train its locally trained DQN neural network. Specifically, this includes:

[0163] Loss function of each agent And calculate the current network in the corresponding locally trained DQN neural network model for the comprehensive loss function. gradient information;

[0164] Each agent adds a safety mask to the gradient information and transmits the gradient information with the safety mask added to the aggregation center;

[0165] After receiving the gradient information with the added security mask, the aggregation center decrypts the fully homomorphic encryption and returns the decrypted result to the corresponding agent.

[0166] Each agent receives the result after decryption of the fully homomorphic encryption, removes the subnet mask from the result, and obtains the unencrypted gradient information. Each agent then updates the parameters of the current network in its locally trained DQN neural network model using the unencrypted gradient information. .

[0167] Update the parameters of the current network in the corresponding locally trained DQN neural network model. The formula is:

[0168] (9).

[0169] In equation (9): F is the loss function, The updated network parameters for agent z. These are the current network parameters before agent z was updated.

[0170] Federated reinforcement learning follows a Markov decision process (MDP). Using DQN as the training network neural model, the MDP followed by each agent can be represented as a tuple (z, s). t a t ,r,s t+1 ), where z is the agent number; s t Let a be the state of the agent at time t; t r is the control action executed by the agent at time t; r is the control action performed by the agent in state s. t Perform action a t The reward obtained later; s t+1 For an agent in state s t Execute control behavior a t The state transitioned to in the next time step. In federated reinforcement learning, such as... Figure 2 As shown, an initial state is first randomly selected, and then a control action is selected based on this initial state. After the control action selection is completed, the agent will execute this control action in the environment, and then the environment will return to the next time-series state s. t+1 And the reward r obtained, then the quadruple (z, s) t a t ,r,s t+1 Store it in the experience pool. Next, store the state s for the next moment. t+1 Consider the current state s t Repeat the above steps until the experience pool is full. Then, use the aggregation center to perform gradient averaging on the error gradient function of the Q-network training for each agent. The aggregation center then returns the gradient-averaged information to each agent for subsequent training guidance. In this way, multiple agents train on the same task and exchange information.

[0171] The principle of this invention is explained as follows:

[0172] The intelligent agents include main network agents and distribution network agents. The area agents corresponding to the main network areas are main network agents, and the area agents corresponding to the distribution network areas are distribution network agents. The main network agents also function as the aggregation center in federated reinforcement learning. Figure 1The diagram shows the framework of a collaborative control system for transmission and distribution frequency regulation resources based on federated reinforcement learning algorithms. The regional power grid is divided into a main grid and multiple distribution networks, with regional agents set up in the dispatch centers of each main grid and distribution network. This design optimizes the collaborative control problem of transmission and distribution frequency regulation resources based on the existing main and distribution network structure, its basic structure, main functional orientation, and connected power source characteristics. In the transition from centralized control to distributed control, considering that the computing and information interaction platform should shift from the original individual entities and the main grid dispatch center to the regional network side represented by the distribution network, this not only reduces the communication and computing pressure on the main grid as a high-level dispatch center during dispatch but also fully leverages the active control capabilities of the distribution network dispatch center under the characteristics of future active distribution networks with localization and activation. Secondly, since most of the distributed power generation units participating in frequency regulation are from enterprises and users, this group is particularly sensitive to privacy and security issues. Using federated reinforcement learning algorithms to solve the collaboration problem among multiple agents can shorten online decision-making time through offline training while ensuring user privacy and security, meeting the real-time decision-making requirements of distributed execution.

[0173] The APC unit refers to a generator unit that automatically tracks power dispatch instructions and adjusts its power generation / consumption in real time according to a certain adjustment rate within a specified power output adjustment range in order to meet the requirements of power system active power balance, frequency stability and tie-line power control.

[0174] Example 1:

[0175] A collaborative control method for transmission and distribution frequency modulation resources based on a federated reinforcement learning algorithm, the control method comprising the following steps:

[0176] S1. Divide the regional power grid into one main grid area and multiple distribution grid areas;

[0177] S2. Set up an agent in the dispatch center of each region and build a corresponding DQN neural network model for each agent;

[0178] S3. Each agent uses the local data of its respective region to train the corresponding DQN neural network model locally, performs additive homomorphic encryption on the information of the locally trained DQN neural network model, and uploads the encrypted information to the aggregation center.

[0179] S4. The aggregation center performs gradient averaging on all encrypted information and sends the gradient-averaged information to each agent. Each agent receives the gradient-averaged information and uses it to train its local DQN neural network model to obtain a trained DQN neural network model. The trained DQN neural network model is then used to obtain the frequency modulation instructions for each unit in the corresponding area that is being scheduled.

[0180] In step S3, when each agent uses local data from the corresponding region to train the DQN neural network model locally, the state space, action space and reward function of each agent are set according to the Markov decision process.

[0181] Setting the state space of agent z specifically includes:

[0182] The magnitude of the total frequency adjustment command that determines the total deviation of the frequency response during the frequency allocation process is used as the state space of agent z.

[0183] Then the state of agent z at time t is ;

[0184] Setting the action space for agent z specifically includes:

[0185] Define the action space that agent z can make decisions on. All control behaviors of agent z originate from the action space. Selected from;

[0186] Control behavior of agent z at time t It can be represented as:

[0187] (1);

[0188] In formula (1): The active power output of the oth thermal power unit controlled by the z-th intelligent agent at time t; The active power output of the m-th energy storage device controlled by the z-th intelligent agent at time t; The active power output of the nth wind turbine controlled by agent z at time t; The active power output of the j-th electric vehicle group controlled by agent z at time t;

[0189] Setting the reward function for agent z specifically includes:

[0190] To determine the reward for the control behavior of agent z, the reward function for agent z is constructed with the objective of minimizing the deviation between the power command value and the power response value.

[0191] (2);

[0192] (3);

[0193] In equations (2)-(3), Let be the reward function of agent z at time t; Q be the number of control periods; q be the number of APC units in the area corresponding to agent z; i be the i-th APC unit in the area corresponding to agent z; t be the t-th discrete control cycle. The input regulation power command value is the i-th APC unit within the area corresponding to agent z. Let be the power response value of the i-th APC unit within the area corresponding to agent z;

[0194] The objective function is obtained by summing the cumulative discounts. Value function:

[0195] (4);

[0196] In equation (4), To control behavior The average of all accumulated rewards generated is taken. ∈[0,1], This is the discount factor; This is the accumulation of the reward function corresponding to multiple consecutive actions of agent z.

[0197] In step S3, agent z uses local data from the corresponding region to locally train the DQN neural network model, specifically including:

[0198] S31, agent z initializes the current network parameters of the corresponding DQN neural network model and copies a target network with the same structure;

[0199] Agents S32 and Z train the DQN neural network model using state data from 96 time periods within the corresponding area during the day, and update the parameters of the target network.

[0200] In step S32, the training of the DQN neural network model by agent z using state data from 96 time periods within the corresponding area during the day includes:

[0201] S321. Select the state data of one time period from the state data of 96 time periods within the day as the current state of agent z. ;

[0202] S322, Based on the current state of agent z An ε-greedy strategy is used for trial and error, that is, a random strategy is used with probability ε to select the control behavior. Select the current optimal control behavior with probability 1-ε. , , ∈ ;

[0203] in: (5);

[0204] S323. Based on the selected control action a, the current network in the DQN neural network model calculates the size of the reward function after executing the control action a. The Q value is calculated using the following function:

[0205] (6);

[0206] In equations (5) and (6), This is the current Q value; The target Q value; The learning rate; This is the reward decay coefficient;

[0207] S324. Based on the selected control action a, obtain the next state returned by the environment after agent z executes the selected control action a. , to obtain empirical samples ( ,a, , ), and the experience sample ( ,a, , Stored in the experience replay pool;

[0208] S325. Update the current state of agent z to the next state returned by the environment, and repeat steps S322-S324 until the experience replay pool is full.

[0209] S326. After the experience replay pool is full, Ω experience samples are extracted from the experience replay pool for calculation, and the loss function is updated:

[0210] (7);

[0211] In equation (7): The loss function; Let z be the reward function for agent z. This represents the Q-value of the current network. The target Q value is the one corresponding to the empirical sample.

[0212] In step S3, performing additive homomorphic encryption on the information of the locally trained DQN neural network model and uploading the encrypted information to the aggregation center specifically includes:

[0213] S34. Each agent uses the Paillier fully homomorphic encryption public key K to encrypt the loss function in the corresponding locally trained DQN neural network model to obtain the encrypted loss function.

[0214] S35. Each agent transmits the encrypted loss function to the aggregation center.

[0215] In step S4, the aggregation center performs gradient averaging on all encrypted information and sends the gradient-averaged information to each agent. Each agent receives the gradient-averaged information and performs subsequent training on the corresponding locally trained DQN neural network model based on the gradient-averaged information. Specifically, this includes:

[0216] S41. The aggregation center calculates the comprehensive loss function based on the encrypted loss function sent by each agent. :

[0217] (8);

[0218] In equation (8): This indicates summing multiple encrypted loss functions. Let z be the reward function for agent z. Let z be the Q-value of the target network corresponding to agent z. This represents the current Q-value corresponding to agent z. The learning rate; y represents the reward decay coefficient; y represents the total number of agents.

[0219] S42, The aggregation center will integrate the loss function. The data is sent to each agent, and each agent calculates the current network's performance relative to the comprehensive loss function in its locally trained DQN neural network model. gradient information;

[0220] S43. Each agent adds a safety mask to the gradient information and transmits the gradient information with the added safety mask to the aggregation center.

[0221] S44. After receiving the gradient information with the added security mask, the aggregation center decrypts the fully homomorphic encryption and returns the result of decryption to the corresponding agent.

[0222] S45. Each agent receives the result after decryption of the fully homomorphic encryption, removes the subnet mask from the result, and obtains the unencrypted gradient information. Each agent updates the parameters of the current network in the corresponding locally trained DQN neural network model using the unencrypted gradient information. .

[0223] In step S45, the parameters of the current network in the corresponding locally trained DQN neural network model are updated. The formula is:

[0224] (9);

[0225] In equation (9): F is the loss function, The updated network parameters for agent z. These are the current network parameters before agent z was updated.

[0226] Example 2:

[0227] Example 2 is basically the same as Example 1, except that:

[0228] The control behavior in the action space conforms to the power supply characteristic constraints and the system balance constraints.

[0229] The power supply characteristic constraints specifically include:

[0230] Thermal power unit operating constraints:

[0231] (10);

[0232] In equation (10), , These are the upper and lower limits of the output of the i-th thermal power unit, respectively; Let be the gradeability of the i-th thermal power unit; Let be the output of the i-th thermal power unit at time t; Let be the output of the i-th thermal power unit at time t-1;

[0233] Energy storage device operating constraints:

[0234] (11);

[0235] In equation (11), , Let m be the capacity constraint range for the m-th energy storage device; Let be the capacity of the m-th energy storage device at time t; Let be the charging power of the m-th energy storage device at time t; , The charging power constraint range for the m-th energy storage device; Let be the discharge power of the m-th energy storage device at time t; and Let m be the discharge power constraint range of the m-th energy storage device; Let be the battery capacity of the m-th energy storage device at time t+1; Let be the self-discharge efficiency of the m-th energy storage device; The charging efficiency of the m-th energy storage device; Let be the discharge efficiency of the m-th energy storage device;

[0236] Distributed wind farm unit operation constraints:

[0237] (12);

[0238] In equation (12), This is the lower limit of the output power of the nth wind turbine; This represents the upper limit of the output power of the nth wind turbine. Let t be the output power of the nth wind turbine.

[0239] Electric vehicle swarm state constraints:

[0240] (13);

[0241] (14);

[0242] (15);

[0243] (16);

[0244] In equations (13)-(16), and Let SOC be the range of the j-th electric vehicle; Let the SOC of the j-th electric vehicle be; , The constraint range for the output power increment of the Mth electric vehicle charging station; For the incremental output power of the Mth electric vehicle charging station; , [This refers to the time period during which a single electric vehicle connects to a charging station;] Let be the upper limit of the charging and discharging power of the j-th electric vehicle at time t; Let be the lower limit of the charging and discharging power of the j-th electric vehicle at time t; and It is affected by factors such as the number of vehicles j in the charging station, the SOC capacity of a single electric vehicle, and its charging / discharging state. Let be the SOC of the j-th electric vehicle at time t; The rated charging power for the j-th electric vehicle; Let be the rated discharge power of the j-th electric vehicle; Let be the output power increment of the Mth electric vehicle charging pile at time t; Let M be the upper limit of the incremental output power of the Mth electric vehicle charging pile at time t. Let t be the lower limit of the output power increment of the Mth electric vehicle charging pile at time t; Let be the power of the j-th electric vehicle at time t.

[0245] The system equilibrium constraints specifically include:

[0246] (17);

[0247] (18);

[0248] (19);

[0249] In equations (17)-(19), , , , Let represent the active power output of the i-th thermal power unit, the m-th energy storage device, the n-th wind turbine, and the j-th electric vehicle at time t, respectively. , , , These represent the quantities of thermal power units, energy storage devices, wind turbine units, and electric vehicles, respectively. This represents the load disturbance at time t; Let b be the set of branches whose first node is b. Let be the set of branches whose final node is node b; and Branch roads Active power and reactive power at time t; and Branch roads Resistance and reactance, Let be the voltage amplitude at the relaxed node at time t; Let be the voltage amplitude at node b at time t; The active power of the generator set connected to node b; The reactive power of the generator set connected to node b; The active power of the load connected to node b; The reactive power of the load connected to node b; and Branch roads Upper and lower limits of active power; and Branch roads Upper and lower limits of reactive power; and These are the upper and lower limits of the voltage at node b, respectively.

[0250] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A method for collaborative control of transmission and distribution frequency regulation resources based on a federated reinforcement learning algorithm, comprising the following steps: S1, dividing a regional power grid into a main grid area and a plurality of distribution grid areas; S2, setting an agent in a dispatch center of each area, and establishing a corresponding DQN neural network model for each agent; S3, each agent uses local data of the corresponding area to locally train the corresponding DQN neural network model, homomorphically encrypts information of the locally trained DQN neural network model, and uploads the encrypted information to an aggregation center; S4, the aggregation center performs gradient averaging processing on all encrypted information, and sends the gradient averaging processed information to each agent, each agent receives the gradient averaging processed information, and performs subsequent training on the corresponding locally trained DQN neural network model according to the gradient averaging processed information to obtain a trained DQN neural network model, and obtains frequency regulation instructions for each unit accepting dispatching through the trained DQN neural network model; In the step S3, when each agent uses local data of the corresponding area to locally train the DQN neural network model, the state space, action space and reward function of each agent are set according to a Markov decision process; The state space of the zth agent includes: The size of the total frequency regulation instruction that determines the total frequency response deviation in the frequency allocation process is set as the state space of the zth agent; The action space of the zth agent includes: The state of the zth agent at time t is then ; The reward function of the zth agent includes: Setting the action space of the z-th agent that can be decided All control behaviors of the z-th agent are selected from the action space ​ the control behavior of the zth agent at time t may be represented as: （1）； In formula (1): Pz,o,t is the active power output of the oth thermal power unit controlled by the zth agent at time t; Pz,m,t is the active power output of the mth energy storage device controlled by the zth agent at time t; Pz,n,t is the active power output of the nth wind power unit controlled by the zth agent at time t; Pz,j,t is the active power output of the jth electric vehicle group controlled by the zth agent at time t; The reward of the control behavior of the environment to the zth agent is set, and the reward function of the zth agent is constructed to minimize the deviation between the regulation power instruction value and the power response value; 2.The method according to claim 1, wherein in the step S3, the zth agent uses local data of the corresponding area to locally train the corresponding DQN neural network model, including: （2）； （3）； In formula (2)-(3), is the reward function of the z-th agent at time t; Q is the number of control periods; q is the number of APC units in the area corresponding to the z-th agent; i is the i-th APC unit in the area corresponding to the z-th agent; t is the t-th discrete control period; is the input adjustment power instruction value of the i-th APC unit in the area corresponding to the z-th agent; is the power response value of the i-th APC unit in the area corresponding to the z-th agent; The value function for the objective function is obtained by discounting the cumulative sum : （4）； In formula (4), for the control behavior all cumulative rewards generated are averaged; ∈[0, 1], is a discount factor; is the cumulative of the reward function corresponding to a plurality of consecutive behaviors of the agent numbered z. S31, the zth agent initializes the current network parameters of the corresponding DQN neural network model, and copies a target network with the same network structure as the current network; S32, the zth agent trains the DQN neural network model with state data of 96 time periods in a day of the corresponding area, and updates the parameters of the target network. 3.The method according to claim 2, wherein in the step S32, the zth agent trains the DQN neural network model with state data of 96 time periods in a day of the corresponding area, including: S325, updating the current state of the zth agent to the next state returned by the environment, and repeating steps S322-S324 until the experience replay pool is full; S326, after the experience replay pool is full, extracting Ω experience samples from the experience replay pool for calculation, and updating the loss function: ​ S321, select the state data of a time period from the state data of 96 time periods in a day as the current state of the zth agent ; S322, based on the current state of the z-number agent , using an epsilon-greedy policy to try and error, i.e. with probability epsilon using a random policy to select a control action , with probability 1-epsilon selecting the currently optimal control action , , ∈ ; wherein: (5); S323、According to the selected control action a, the reward function size after executing the control action a is calculated and the Q value is calculated by the following function: （6）； In formula (5), (6), is the current network Q value; is the target network Q value; is the learning rate; is the reward decay coefficient; S324, according to the selected control action a, obtaining the next state returned by the environment after the intelligent agent with the number z executes the selected control action a , obtaining an experience sample ( , a, , ), and storing the experience sample ( , a, , ) into an experience replay pool; ​ ​ （7）； In formula (7): is a loss function; is a reward function for the zth agent; is a Q-value of the current network; is a Q-value of the target network.

4. The method of claim 1, wherein the method further comprises: performing homomorphic encryption on information of the locally trained DQN neural network model, and uploading the encrypted information to the aggregation center. S34, each agent encrypts a loss function in the locally trained DQN neural network model using a Paillier full homomorphic encryption public key K to obtain an encrypted loss function. S35, each agent transmits the encrypted loss function to the aggregation center.

5. The method of claim 4, wherein the method further comprises: performing gradient average processing on all encrypted information at the aggregation center, and sending the gradient average processed information to each agent. S43, each agent adds a security mask to the gradient information, and transmits the gradient information with the security mask to the aggregation center. S44, the aggregation center receives the gradient information with the security mask, and performs full homomorphic encryption removal on the gradient information to return the result to the corresponding agent. S41, the aggregation center calculates a comprehensive loss function according to the encrypted loss functions sent by the intelligent agents : （8）； In formula (8): denotes summing a plurality of encrypted loss functions, is a reward function of the z-th agent, is a target network Q value corresponding to the z-th agent; is a current network Q value corresponding to the z-th agent; is a learning rate; is a reward decay coefficient; y is the total number of agents; S42, the aggregation center sends the comprehensive loss function to each agent, and each agent calculates gradient information according to the current network in the corresponding locally trained DQN neural network model and the comprehensive loss function S42, the aggregation center sends the comprehensive loss function to each agent, and each agent calculates gradient information according to the current network in the corresponding locally trained DQN neural network model and the comprehensive loss function S42, the aggregation center sends the comprehensive loss function to each agent, and each agent calculates gradient information according to the current network in the 6. The method of claim 5, wherein the method further comprises:

7. The method of claim 1, wherein the control behavior in the action space meets power source characteristic constraint conditions and system balance constraint conditions. S45, each agent receives the result after the full homomorphic encryption is removed, and removes the subnet mask in the result to obtain gradient information without encryption, and each agent updates the parameters of the current network in the corresponding local trained DQN neural network model through the gradient information without encryption .

8. The method of claim 7, wherein the power source characteristic constraint conditions comprise: In the step S45, the parameters of the current network in the corresponding local trained DQN neural network model are updated The formula is: （9）； In formula (9), F is a loss function, is the updated current network parameter of the zth agent, is the current network parameter of the zth agent before updating. thermal power unit operation constraints; energy storage device operation constraints; distributed wind power plant unit operation constraints; electric vehicle group state constraints.

9. The method of claim 7, wherein the system balance constraint conditions comprise: （10）； In formula (10), , are the upper and lower limits of the output of the i th thermal power unit, respectively; is the ramp rate of the i th thermal power unit; is the output of the i th thermal power unit at time t; is the output of the i th thermal power unit at time t-1; ​ （11）； In formula (11), , is a capacity constraint range of the mth energy storage device; is a capacity of the mth energy storage device at time t; is a charging power of the mth energy storage device at time t; , is a charging power constraint range of the mth energy storage device; is a discharging power of the mth energy storage device at time t; and is a discharging power constraint range of the mth energy storage device; is a battery capacity of the mth energy storage device at time t+1; is a self-discharge efficiency of the mth energy storage device; is a charging efficiency of the mth energy storage device; is a discharging efficiency of the mth energy storage device; ​ （12）； In formula (12), Pn,low is a lower limit of output power of the n-th wind power generator; Pn,high is an upper limit of output power of the n-th wind power generator; Pn(t) is output power of the n-th wind power generator at time t. ​ （13）； （14）； （15）； （16）； In formula (13)-(16), and is the SOC constraint range of the jth electric vehicle; is the SOC of the jth electric vehicle; , is the output power increment constraint range of the Mth electric vehicle charging station; is the output power increment of the Mth electric vehicle charging station; , is the time period when a single electric vehicle accesses the charging station; is the upper limit of the charging and discharging power of the jth electric vehicle at time t; is the lower limit of the charging and discharging power of the jth electric vehicle at time t; and is affected by the number of vehicles j in the charging station, the SOC capacity of a single electric vehicle, and the charging and discharging state; is the SOC of the jth electric vehicle at time t; is the rated charging power of the jth electric vehicle; is the rated discharging power of the jth electric vehicle; is the output power increment of the Mth electric vehicle charging pile at time t; is the upper limit of the output power increment of the Mth electric vehicle charging pile at time t; is the lower limit of the output power increment of the Mth electric vehicle charging pile at time t; is the power of the jth electric vehicle at time t. ​ ​ （17）； （18）； （19）； In formula (17)-(19), respectively, represents the active power of the i-th thermal power unit, the m-th energy storage device, the n-th wind power unit, and the j-th electric vehicle at time t; 、 、 、 、 、 、 represents the load disturbance at time t; represents the branch set with the head node being node b; represents the branch set with the end node being node b; and respectively represent the active power and the reactive power of the branch at time t; and respectively represent the resistance and the reactance of the branch , represents the voltage amplitude of the slack node at time t; represents the voltage amplitude of node b at time t; represents the active power of the generator set connected to node b; represents the reactive power of the generator set connected to node b; represents the active power of the load connected to node b; represents the reactive power of the load connected to node b; and respectively represent the upper limit and the lower limit of the active power of the branch ; and respectively represent the upper limit and the lower limit of the reactive power of the branch ; and respectively represent the upper limit and the lower limit of the voltage of node b.​​

Images