Large-scale power grid multi-agent reinforcement learning rapid scheduling method

Abstract

The invention belongs to the technical field of power systems, and more specifically relates to a large-scale power grid multi-agent reinforcement learning rapid scheduling method, and the method comprises the steps: obtaining power grid data in real time based on a data collection system, and improving the change trend of a prediction load through a load prediction model; a heuristic algorithm and a parallel computing architecture are adopted, so that the generation and execution process of a scheduling scheme can be accelerated, and the problems of efficiency and expandability of a scheduling algorithm are solved; therefore, the problem of limitation in the aspect of real-time data processing in a large-scale power grid multi-agent reinforcement learning rapid scheduling method is effectively solved, and the power grid scheduling efficiency and response speed are improved.

Description

Technical Field

[0001] The present application belongs to the technical field of power systems, and more specifically, to a fast dispatching method for multi-agent reinforcement learning in large-scale power grids. Background Art

[0002] With the development of society and the growing demand for energy, the safe, stable and efficient operation of the power system, as an indispensable infrastructure of modern society, has become particularly important. In large-scale power grids, it has become a challenging technical task to achieve effective dispatching and distribution of electricity to cope with dynamic changes in loads and emergencies. Traditional power system dispatching methods often face problems such as high computational complexity and slow response speed, especially in dealing with the complexity and real-time nature of large-scale power grids.

[0003] In order to solve this problem, intelligent dispatching technology has gradually become a hot topic in research and application in recent years. Among them, the intelligent dispatching method based on reinforcement learning has gradually attracted widespread attention due to its advantages such as strong adaptability and high learning ability. In this method, the power grid is regarded as an intelligent system, and each part is abstracted as an individual with certain intelligence and decision-making ability. They realize intelligent decision-making on power grid dispatching through learning and optimization.

[0004] Although intelligent dispatching methods have made some progress in improving the efficiency and response speed of power grid dispatching, they still face some challenges. The current dispatching methods still have certain limitations in processing real-time data. Summary of the invention

[0005] The present invention provides a large-scale power grid multi-agent reinforcement learning fast scheduling method, which is intended to solve the problem that current scheduling methods have certain limitations in processing real-time data.

[0006] A large-scale power grid multi-agent reinforcement learning fast dispatching method is characterized by comprising the following steps:

[0007] Step 1: Build a data acquisition system to obtain real-time power grid data and pre-process the real-time data stream;

[0008] Step 2: Use the preprocessed real-time data stream and the constructed power grid load prediction model to predict the load change trend of the power grid;

[0009] Step 3: Based on the load change trend of the power grid and combined with the heuristic algorithm, a preliminary dispatching plan is quickly generated;

[0010] Step 4: Treat each part of the power grid as an intelligent agent and adopt a distributed computing architecture to allow each intelligent agent to calculate the scheduling plan in parallel;

[0011] Step 5: Based on the scheduling plan obtained in step 4, each agent executes the scheduling plan in parallel.

[0012] The present invention acquires power grid data in real time based on a data acquisition system, and improves the prediction of load change trends through a load prediction model; and adopts a heuristic algorithm and a parallel computing architecture to accelerate the generation and execution of scheduling plans, solving the efficiency and scalability problems of scheduling algorithms; therefore, the present invention effectively solves the limitations of real-time data processing in large-scale power grid multi-agent reinforcement learning fast scheduling methods, and improves the efficiency and response speed of power grid scheduling.

[0013] Preferably, the power grid load prediction model adopts an LSTM model based on an attention mechanism; the specific structure is as follows:

[0014] Input layer: receives the feature data of time series as input;

[0015] Attention layer: dynamically calculates attention weights based on the features in the input sequence to determine the degree of attention the model pays to different time steps in the input sequence; the attention weights are obtained by weighted averaging;

[0016] LSTM layer: used to process time series data and learn the time series features in the sequence; the LSTM layer includes multiple LSTM units, each of which controls the transmission and forgetting of information through a gating mechanism;

[0017] Output layer: receives the output of the LSTM layer and maps it to the target value of load prediction;

[0018] The loss function of the power grid load prediction model adopts the mean square error loss function; the optimization algorithm adopts the stochastic gradient descent algorithm.

[0019] Preferably, step 3 comprises the following steps:

[0020] Step 3.1: Use the grid load forecast results as input to generate a set of preliminary dispatching schemes, and then optimize each dispatching scheme based on the genetic algorithm;

[0021] Step 3.2: Evaluate the optimized scheduling plan and select the best scheduling plan as the final scheduling plan based on the preset evaluation indicators.

[0022] Preferably, the step 3.1 comprises the following steps:

[0023] Based on the grid load prediction results, a load balancing algorithm is used to reasonably distribute the grid load to each node or part of the grid;

[0024] Based on the load distribution results and the grid operation status, a resource optimization scheduling algorithm is used to schedule the resources of each node or part of the grid to obtain multiple preliminary scheduling plans;

[0025] Genetic algorithm is used to optimize each preliminary plan and obtain multiple optimized scheduling plans.

[0026] Preferably, the steps of obtaining the load distribution result are as follows:

[0027] Obtaining a load prediction value for each electromechanical device based on the grid load prediction result;

[0028] Add the load prediction values ​​of each node to get the total load of the power grid;

[0029] According to the load prediction value of the node and the total load of the power grid, the load distribution amount of each node is calculated, and finally the load distribution amount of each node is obtained; the calculation formula of the load distribution amount of each node is as follows:

[0030]

[0031] Where: R i Represents node N i Load distribution, N i represents the i-th node; L total Indicates the total load of the power grid; L i Represents node N i Load prediction value; L j Represents node N j The load forecast value.

[0032] Preferably, the resource optimization scheduling algorithm adopts an integer programming scheduling algorithm, wherein the integer programming scheduling algorithm is established as follows:

[0033] Define the decision variables and objective function:

[0034] Define x i represents the power generation of node i or the corresponding decision variable;

[0035] Objective function:

[0036]

[0037] Where: c i represents the power generation cost or cost coefficient of node i;

[0038] Set up constraints:

[0039] The total load of the power grid is equal to the sum of the load distribution of each node:

[0040]

[0041] Where: L total Indicates the total load of the power grid;

[0042] Node power generation upper limit constraint:

[0043] x i ≤capacity i ;

[0044] Where: capacity i represents the capacity of the generator or other resources at node i;

[0045] An integer programming scheduling algorithm is obtained based on the defined decision variables, objective function and constraints, and then the integer programming scheduling algorithm established by the integer programming solver is solved to obtain multiple preliminary scheduling solutions.

[0046] Preferably, the specific steps of optimizing each initial scheduling scheme using a genetic algorithm are as follows:

[0047] A. Generate an initial population. Each individual in the initial population represents a preliminary scheduling plan, which consists of the power generation of each node;

[0048] B. Evaluate the fitness of each individual, calculate the performance index of the individual in the current environment, and evaluate the fitness based on the objective function of the cost size;

[0049] C. According to the fitness value of each individual, use roulette wheel to select some individuals as parents;

[0050] D. Perform crossover operation on the selected parent individuals to generate new individuals;

[0051] E. Perform mutation operation on the generated new individuals to obtain the mutated individuals;

[0052] F. Merge the parent individuals, the new individuals generated by crossover, and the mutated individuals to form a new population;

[0053] G. Repeat steps B to F until the maximum number of iterations is reached or a satisfactory solution is found. When the termination condition is reached, the optimal individual is output as the optimized scheduling solution.

[0054] The beneficial effects of the present invention include:

[0055] The present invention acquires power grid data in real time based on a data acquisition system, and improves the prediction of load change trends through a load prediction model; and adopts a heuristic algorithm and a parallel computing architecture to accelerate the generation and execution of scheduling plans, solving the efficiency and scalability problems of scheduling algorithms; therefore, the present invention effectively solves the limitations of real-time data processing in large-scale power grid multi-agent reinforcement learning fast scheduling methods, and improves the efficiency and response speed of power grid scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0057] Figure 1 A flowchart of the steps provided in the embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0060] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0061] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0062] See also Figure 1 As shown, the best embodiment of the present invention is further described;

[0063] A large-scale power grid multi-agent reinforcement learning fast dispatching method is characterized by comprising the following steps:

[0064] Step 1: Build a data acquisition system to obtain real-time power grid data and pre-process the real-time data stream;

[0065] Specifically: Design the network architecture: Build a distributed data acquisition system, including multiple data acquisition nodes (such as solar power stations, wind power stations, substations, etc.), a data transmission network (using Internet of Things technology or wired network) and a data processing center.

[0066] Select sensor equipment: Install appropriate sensor equipment at each key node, such as photovoltaic current sensors at solar power stations and current sensors at substations, to collect relevant data in real time.

[0067] Develop data collection protocols: Set a unified data collection protocol, such as MQTT or Modbus protocol, to ensure that sensor devices can accurately transmit data to the data processing center.

[0068] Deploy data acquisition system: deploy data acquisition nodes and establish a data transmission network to ensure that the data acquisition system can run stably and in real time.

[0069] Real-time data preprocessing: In the data processing center, the collected real-time data is preprocessed, such as removing outliers and normalizing data, to ensure the accuracy and availability of the data.

[0070] Through the above steps, we have established a data acquisition system that can obtain various data of the power grid in real time and provide a reliable data basis for subsequent data analysis and scheduling.

[0071] Step 2: Use the preprocessed real-time data stream and the constructed power grid load prediction model to predict the load change trend of the power grid;

[0072] Data preparation: Collect historical grid data, including power generation, load, weather and other information, i.e. the data preprocessed in step 1. These data will be used to build a load forecasting model.

[0073] Build a load prediction model: Design an LSTM model based on the attention mechanism. The specific structure includes input layer, attention layer, LSTM layer and output layer.

[0074] Input layer: receives time series feature data as input, such as power generation, load, temperature, humidity, etc. in the past few hours or days.

[0075] Attention layer: Dynamically calculates attention weights based on the features in the input sequence to determine how much the model pays attention to different time steps. This helps the model pay more attention to time points that have an important impact on the current prediction when making predictions.

[0076] LSTM layer: used to process time series data and learn the temporal features in the sequence. The LSTM layer controls the transmission and forgetting of information through a gating mechanism and is suitable for processing data with long-term dependencies.

[0077] Output layer: receives the output of the LSTM layer and maps it to the target value of load forecasting, that is, the predicted value of the power grid load at a certain point in the future.

[0078] Training model: Use historical power grid data to train the constructed load forecasting model. Divide the historical data into a training set and a validation set, use the mean square error loss function as the loss function of the model, and use the stochastic gradient descent algorithm to optimize the model parameters.

[0079] Model evaluation: Use the validation set to evaluate the performance of the model. By comparing the error between the load values ​​predicted by the model and the actual observed values, the accuracy and generalization ability of the model can be evaluated.

[0080] Load forecasting: Apply the trained load forecasting model to the real-time data stream, that is, the grid data in the future time period. The model will generate a predicted grid load change trend, providing an important reference for grid dispatching. Step 3: Based on the load change trend of the grid and combined with the heuristic algorithm, a preliminary dispatching plan is quickly generated;

[0081] The step 3 comprises the following steps:

[0082] Step 3.1: Use the grid load forecast results as input to generate a set of preliminary dispatching schemes, and then optimize each dispatching scheme based on the genetic algorithm;

[0083] Step 3.2: Evaluate the optimized scheduling plan and select the best scheduling plan as the final scheduling plan based on the preset evaluation indicators.

[0084] The step 3.1 comprises the following steps:

[0085] Based on the grid load prediction results, a load balancing algorithm is used to reasonably distribute the grid load to each node or part of the grid;

[0086] Based on the load distribution results and the grid operation status, a resource optimization scheduling algorithm is used to schedule the resources of each node or part of the grid to obtain multiple preliminary scheduling plans;

[0087] Genetic algorithm is used to optimize each preliminary plan and obtain multiple optimized scheduling plans.

[0088] Preferably, the steps of obtaining the load distribution result are as follows:

[0089] Obtaining a load prediction value for each electromechanical device based on the grid load prediction result;

[0090] Add the load prediction values ​​of each node to get the total load of the power grid;

[0091] According to the load prediction value of the node and the total load of the power grid, the load distribution amount of each node is calculated, and finally the load distribution amount of each node is obtained; the calculation formula of the load distribution amount of each node is as follows:

[0092]

[0093] Where: R i Represents node N i Load distribution, N i represents the i-th node; L total Indicates the total load of the power grid; L i Represents node N i Load forecast value; L j Represents node N j The load forecast value.

[0094] Preferably, the resource optimization scheduling algorithm adopts an integer programming scheduling algorithm, wherein the integer programming scheduling algorithm is established as follows:

[0095] Define the decision variables and objective function:

[0096] Define x i represents the power generation of node i or the corresponding decision variable;

[0097] Objective function:

[0098]

[0099] Where: c i represents the power generation cost or cost coefficient of node i;

[0100] Set up constraints:

[0101] The total load of the power grid is equal to the sum of the load distribution of each node:

[0102]

[0103] Where: L total Indicates the total load of the power grid;

[0104] Node power generation upper limit constraint:

[0105] x i ≤capacity i ;

[0106] Where: capacity i represents the capacity of the generator or other resources at node i;

[0107] An integer programming scheduling algorithm is obtained based on the defined decision variables, objective function and constraints, and then the integer programming scheduling algorithm established by the integer programming solver is solved to obtain multiple preliminary scheduling solutions.

[0108] Preferably, the specific steps of optimizing each initial scheduling scheme using a genetic algorithm are as follows:

[0109] A. Generate an initial population. Each individual in the initial population represents a preliminary scheduling plan, which consists of the power generation of each node;

[0110] B. Evaluate the fitness of each individual, calculate the performance index of the individual in the current environment, and evaluate the fitness based on the objective function of the cost size;

[0111] C. According to the fitness value of each individual, use roulette wheel to select some individuals as parents;

[0112] D. Perform crossover operation on the selected parent individuals to generate new individuals;

[0113] E. Perform mutation operation on the generated new individuals to obtain the mutated individuals;

[0114] F. Merge the parent individuals, the new individuals generated by crossover, and the mutated individuals to form a new population;

[0115] G. Repeat steps B to F until the maximum number of iterations is reached or a satisfactory solution is found. When the termination condition is reached, the optimal individual is output as the optimized scheduling solution.

[0116] Step 4: Treat each part of the power grid as an intelligent agent and adopt a distributed computing architecture to allow each intelligent agent to calculate the scheduling plan in parallel;

[0117] Divide Agents: First, we divide the power grid into multiple agents, each of which represents a region or part of the power grid. For example, a city can be divided into several geographical regions, and each region is an agent responsible for managing the power resources in that region.

[0118] Parallel computing architecture: Using a distributed computing architecture, each intelligent agent is deployed on different computing nodes to achieve parallel computing and scheduling.

[0119] Scheduling plan generation: Each agent independently generates a preliminary scheduling plan based on factors such as load demand, generator capacity, and energy storage equipment in its own area. These plans may include start and stop control of generators, charge and discharge control of energy storage equipment, etc.

[0120] Coordination and optimization: After each agent generates a preliminary dispatch plan, it communicates and coordinates its own dispatch plan with neighboring agents through distributed communication and collaboration. This can help each agent better understand the operation of the entire power grid and coordinate dispatch plans to solve the problem of cross-regional power supply and demand balance.

[0121] The details are as follows:

[0122] Information exchange and sharing: One agent can send a proposal to another agent, proposing a certain resource allocation or scheduling plan. The two agents share each other's real-time data, load forecast information, generator status, energy storage device status, etc., in order to more accurately assess the current situation and formulate plans.

[0123] Proposal formulation: Each agent formulates its own proposal based on its own data and algorithms. Each agent evaluates the pros and cons of its own proposal, taking into account local interests and the overall interests of the grid.

[0124] Negotiation and discussion: Agents exchange proposals, discuss and evaluate them. Based on the feedback and opinions of the other party, the agent can modify its proposal to better meet the needs of both parties.

[0125] Reaching consensus: During the discussion and negotiation process, the agents can gradually adjust the proposed solution to reach a consensus acceptable to both parties. Both parties should take the overall interests of the power grid as a common goal during the negotiation process, not just local interests.

[0126] Reaching an agreement: After many rounds of negotiation and discussion, the two parties can form a final consensus plan. Once the two parties reach an agreement, they can take this consensus plan as the final agreement and start to implement it.

[0127] Parallel execution of scheduling schemes: After coordinated optimization, each agent executes the optimized scheduling scheme in parallel. Each agent controls local generators, energy storage devices and other equipment according to its own scheduling scheme to achieve a balance between power supply and demand and stable operation of the power grid.

[0128] Step 5: Based on the scheduling plan obtained in step 4, each agent executes the scheduling plan in parallel.

[0129] The present invention acquires power grid data in real time based on a data acquisition system, and improves the prediction of load change trends through a load prediction model; and adopts a heuristic algorithm and a parallel computing architecture to accelerate the generation and execution of scheduling plans, solving the efficiency and scalability problems of scheduling algorithms; therefore, the present invention effectively solves the limitations of real-time data processing in large-scale power grid multi-agent reinforcement learning fast scheduling methods, and improves the efficiency and response speed of power grid scheduling.

[0130] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. Rapid dispatching method of multi-agent reinforcement learning for large-scale power grids, It is characterized in that The following steps are involved: Step 1: Build a data acquisition system to obtain real-time power grid data and pre-process the real-time data stream; Step 2: Use the preprocessed real-time data stream and the constructed power grid load prediction model to predict the load change trend of the power grid; Step 3: Based on the load change trend of the power grid and combined with the heuristic algorithm, a preliminary dispatching plan is quickly generated; Step 4: Treat each part of the power grid as an intelligent agent and adopt a distributed computing architecture to allow each intelligent agent to calculate the scheduling plan in parallel; Step 5: Based on the scheduling plan obtained in step 4, each agent executes the scheduling plan in parallel.

2. The large-scale power grid multi-agent reinforcement learning fast dispatching method according to claim 1, It is characterized in that The power grid load prediction model adopts an LSTM model based on the attention mechanism; the specific structure is as follows: Input layer: receives the feature data of time series as input; Attention layer: dynamically calculates attention weights based on the features in the input sequence to determine the degree of attention the model pays to different time steps in the input sequence; the attention weights are obtained by weighted averaging; LSTM layer: used to process time series data and learn the temporal features in the sequence; The LSTM layer includes multiple LSTM units, each of which controls the transmission and forgetting of information through a gating mechanism; Output layer: receives the output of the LSTM layer and maps it to the target value of load prediction; The loss function of the power grid load prediction model adopts the mean square error loss function; the optimization algorithm adopts the stochastic gradient descent algorithm.

3. The large-scale power grid multi-agent reinforcement learning fast dispatching method according to claim 1, It is characterized in that The step 3 comprises the following steps: Step 3.1: Use the grid load forecast results as input to generate a set of preliminary dispatching schemes, and then optimize each dispatching scheme based on the genetic algorithm; Step 3.2: Evaluate the optimized scheduling plan and select the best scheduling plan as the final scheduling plan based on the preset evaluation indicators.

4. The large-scale power grid multi-agent reinforcement learning fast dispatching method according to claim 3, It is characterized in that The step 3.1 comprises the following steps: Based on the grid load prediction results, a load balancing algorithm is used to reasonably distribute the grid load to each node or part of the grid; Based on the load distribution results and the grid operation status, a resource optimization scheduling algorithm is used to schedule the resources of each node or part of the grid to obtain multiple preliminary scheduling plans; Genetic algorithm is used to optimize each preliminary plan and obtain multiple optimized scheduling plans.

5. The large-scale power grid multi-agent reinforcement learning fast dispatching method according to claim 4, It is characterized in that The steps for obtaining the load distribution result are as follows: Obtaining a load prediction value for each electromechanical device based on the grid load prediction result; Add the load prediction values ​​of each node to get the total load of the power grid; According to the load prediction value of the node and the total load of the power grid, the load distribution amount of each node is calculated, and finally the load distribution amount of each node is obtained; the calculation formula of the load distribution amount of each node is as follows: Where: R i Represents node N i Load distribution, N i represents the i-th node; L total Indicates the total load of the power grid; L i Represents node N i Load forecast value; L j Represents node N j The load forecast value.

6. The large-scale power grid multi-agent reinforcement learning fast dispatching method according to claim 4, It is characterized in that The resource optimization scheduling algorithm adopts an integer programming scheduling algorithm, wherein the integer programming scheduling algorithm is established as follows: Define the decision variables and objective function: Define x i represents the power generation of node i or the corresponding decision variable; Objective function: Where: c i represents the power generation cost or cost coefficient of node i; Set up constraints: The total load of the power grid is equal to the sum of the load distribution of each node: Where: L total Indicates the total load of the power grid; Node power generation upper limit constraint: x i ≤capacity i ; Where: capacity i represents the capacity of the generator or other resources at node i; An integer programming scheduling algorithm is obtained based on the defined decision variables, objective function and constraints, and then the integer programming scheduling algorithm established is solved by an integer programming solver to obtain multiple preliminary scheduling solutions.

7. The large-scale power grid multi-agent reinforcement learning fast dispatching method according to claim 4, It is characterized in that The specific steps of using genetic algorithm to optimize each initial scheduling scheme are as follows: A. Generate an initial population. Each individual in the initial population represents a preliminary scheduling plan, which consists of the power generation of each node; B. Evaluate the fitness of each individual, calculate the performance index of the individual in the current environment, and evaluate the fitness based on the objective function of the cost size; C. According to the fitness value of each individual, use roulette wheel to select some individuals as parents; D. Perform crossover operation on the selected parent individuals to generate new individuals; E. Perform mutation operation on the generated new individuals to obtain the mutated individuals; F. Merge the parent individuals, the new individuals generated by crossover, and the mutated individuals to form a new population; G. Repeat steps B to F until the maximum number of iterations is reached or a satisfactory solution is found. When the termination condition is reached, the optimal individual is output as the optimized scheduling solution.

Images