Virtual power plant intelligent control method and system based on multiple agents

Abstract

The invention discloses a multi-agent-based virtual power plant intelligent control method and system, and the method comprises the steps: dividing a virtual power plant into a plurality of sub-virtual power plants, deploying an agent in each sub-virtual power plant, collecting a local resource state through each agent, and predicting a load demand and the output of a distributed power supply, a hierarchical control unit is adopted to carry out collaborative optimization among the sub-virtual power plants according to a prediction result, and an upper-layer optimization control module constructs a linear programming model according to the prediction result and solves the linear programming model to obtain an initial scheduling scheme; and the lower-layer reinforcement learning control module performs local adjustment on the preliminary scheduling scheme according to a multi-agent depth deterministic strategy gradient algorithm to obtain a decision scheme. Based on a distributed control strategy of a multi-agent architecture, the fault-tolerant capability and reliability of the system are improved, a hierarchical control architecture is adopted, global optimization and local adjustment are organically combined, and efficient coordination and real-time adjustment capability of global resources are realized.

Description

Technical Field

[0001] The present invention relates to the field of virtual power plant control, and in particular to a multi-agent-based intelligent control method and system for a virtual power plant. Background Art

[0002] In recent years, with the rapid development of renewable energy and the widespread application of distributed energy around the world, virtual power plants (VPPs) are a new type of power dispatching model. Through intelligent control and dispatching technologies, they integrate and uniformly manage multiple power resources such as distributed energy, energy storage equipment, and demand response, thereby realizing functions such as power market participation, optimized dispatching, peak shaving and valley filling, and balancing supply and demand, greatly improving the flexibility and reliability of the power system.

[0003] However, virtual power plant management models also present challenges during scheduling. The distributed resources that comprise a virtual power plant typically belong to different operating entities, each with its own operational objectives, which diverge from the overall optimized operational strategy of the virtual power plant. Existing virtual power plant management and control systems are burdened by large, complex system tasks and face computational bottlenecks. Consequently, global optimized scheduling of virtual power plants presents difficulties, and there is a lack of targeted technical solutions that can effectively address the complexities of actual grid operations. Summary of the Invention

[0004] Purpose of the invention: The purpose of the present invention is to provide an efficient and stable virtual power plant intelligent control method and system, which can achieve optimal configuration of virtual power plant resources and balance supply and demand through collaboration between multiple intelligent agents.

[0005] Technical solution: The multi-agent-based virtual power plant intelligent control method of the present invention includes the following steps:

[0006] (1) Divide the virtual power plant into several sub-virtual power plants, deploy intelligent agents in each sub-virtual power plant, and establish a communication network between the intelligent agents;

[0007] (2) Each agent collects the local resource status of the sub-virtual power plant;

[0008] (3) Each intelligent agent builds a prediction model to predict load demand and distributed power output;

[0009] (4) Using hierarchical control units to perform collaborative optimization among sub-virtual power plants, including:

[0010] The upper optimization control module constructs a linear programming model based on the prediction results of load demand and distributed power output and solves it to obtain the initial scheduling plan;

[0011] The lower-level reinforcement learning control module makes local adjustments to the preliminary scheduling plan based on the multi-agent deep deterministic policy gradient algorithm to obtain a decision plan;

[0012] (5) Exchange decision plans between intelligent agents and execute them.

[0013] Furthermore, in step (4), the upper-level optimization control module establishes an objective function with the goal of minimizing the total cost of the virtual power plant, solves the linear programming problem through a linear programming solver, and obtains a preliminary scheduling plan; the total cost of the virtual power plant is the sum of fuel cost, energy storage charging and discharging cost, load scheduling cost and emission cost; the constraints of the linear programming problem include power balance constraint, energy storage device charge state constraint, device physical constraint, charging and discharging rate constraint and frequency and voltage constraint.

[0014] Furthermore, in step (4), the lower-layer reinforcement learning control module establishes a reward function for controlling the actions of each agent, and the reward function is:

[0015] r t =-α·C total (t)-β·(f(t)-f ref ) 2 -γ·(V(t)-V ref ) 2 -δ·∑|P i -P neighbor |-η·|ΔP|;

[0016] Among them, α, β, γ, δ, η are weight coefficients, C total (t) is the total cost of the virtual power plant, f(t) and V(t) represent the frequency and voltage of the virtual power plant at the current time t, respectively. ref and V ref is the reference frequency and voltage value, P i and P neighbor are the powers of agent i and its neighboring agents, |ΔP| is the penalty term, and δ is the penalty term coefficient;

[0017] The actions of each intelligent agent include the scheduling adjustments that the sub-virtual power plant can make at the current moment, including adjusting the output power of the local distributed power supply, adjusting the charging and discharging power of the local energy storage, adjusting the controllable load, and adjusting the output power according to the status of the neighboring intelligent agents.

[0018] Furthermore, the action of adjusting the output power of the local distributed power supply is a1=[ΔP DG (t)], ΔP DG (t)∈[P DG,min , P DG,max ], P DG,min and P DG,maxare the minimum output and maximum output of the local distributed power source respectively;

[0019] The action of adjusting the charging and discharging power of the local energy storage a2=[ΔP ES (t)], ΔP ES (t)∈[P ES,min , P ES,max ], P ES,min and P ES,max The minimum and maximum power for charging and discharging local energy storage;

[0020] Adjust the action of controllable load a3=[ΔP CL (t)], ΔP CL (t)∈[P CL,min , P CL,max ], P CL,min and P CL,max The minimum load adjustment range and the maximum load adjustment range of the controllable load;

[0021] The action a4 of adjusting the output power according to the state of the neighboring agent is [ΔP DG (t)-P neighbor (t)],P neighbor (t) is the output power of the local distributed power supply of the neighboring intelligent agent.

[0022] Furthermore, in step (3), the prediction model includes an input layer, an embedding layer, a prediction layer, a fully connected layer and an output layer, and takes historical load data, meteorological data and distributed power output data as input, and predicts the load demand and distributed power output of future time periods through the prediction model; wherein the prediction layer is composed of a stack of at least one long short-term memory network, or a stack of at least one gated recurrent unit.

[0023] Furthermore, step (3) also includes using Kalman filtering to correct the prediction results of load demand and distributed power output.

[0024] The multi-agent-based virtual power plant intelligent control system of the present invention includes:

[0025] Virtual power plant, which consists of several sub-virtual power plants. An intelligent agent is deployed for each sub-virtual power plant, and a communication network is established between the intelligent agents.

[0026] The status monitoring module is used to collect the local resource status of the sub-virtual power plant through each intelligent agent;

[0027] The prediction module is used to predict load demand and distributed power output by building prediction models through various intelligent agents;

[0028] Hierarchical control unit, used for collaborative optimization among sub-virtual power plants, including:

[0029] The upper optimization control module constructs a linear programming model based on the prediction results of load demand and distributed power output and solves it to obtain the initial scheduling plan;

[0030] The lower-level reinforcement learning control module makes local adjustments to the preliminary scheduling plan based on the multi-agent deep deterministic policy gradient algorithm to obtain a decision plan;

[0031] The execution module is used to exchange decision plans between intelligent agents and execute the decision plans.

[0032] Furthermore, in the hierarchical control unit, the upper-level optimization control module establishes an objective function with the goal of minimizing the total cost of the virtual power plant, solves the linear programming problem through a linear programming solver, and obtains a preliminary scheduling plan; the total cost of the virtual power plant is the sum of fuel cost, energy storage charging and discharging cost, load scheduling cost and emission cost; the constraints of the linear programming problem include power balance constraint, energy storage equipment charge state constraint, equipment physical constraint, charging and discharging rate constraint and frequency and voltage constraint.

[0033] Furthermore, in the hierarchical control unit, the lower-layer reinforcement learning control module establishes a reward function for controlling the actions of each agent. The reward function is:

[0034] r t =-α·C total (t)-β·(f(t)-f ref ) 2 -γ·(V(t)-V ref ) 2 -δ·∑|P i -P neighbor |-η·|ΔP|;

[0035] Among them, α, β, γ, δ, η are weight coefficients, C total (t) is the total cost of the virtual power plant, f(t) and V(t) represent the frequency and voltage of the virtual power plant at the current time t, respectively. ref and V ref is the reference frequency and voltage value, P i and P neighbor are the powers of agent i and its neighboring agents, |ΔP| is the penalty term, and δ is the penalty term coefficient;

[0036] The actions of each intelligent agent include the scheduling adjustments that the sub-virtual power plant can make at the current moment, including adjusting the output power of the local distributed power supply, adjusting the charging and discharging power of the local energy storage, adjusting the controllable load, and adjusting the output power according to the status of the neighboring intelligent agents.

[0037] Furthermore, the action of adjusting the output power of the local distributed power supply is a1=[ΔP DG (t)], ΔP DG (t)∈[P DG,min , P DG,max ], P DG,min and P DG,max are the minimum output and maximum output of the local distributed power source respectively;

[0038] The action of adjusting the charging and discharging power of the local energy storage a2=[ΔP ES (t)], ΔP ES (t)∈[P ES,min , P ES,max ], P ES,min and P ES,max The minimum and maximum power for charging and discharging local energy storage;

[0039] Adjust the action of controllable load a3=[ΔP CL (t)], ΔP CL (t)∈[P CL,min , P CL,max ], P CL,min and P CL,max The minimum load adjustment range and the maximum load adjustment range of the controllable load;

[0040] The action a4 of adjusting the output power according to the state of the neighboring agent is [ΔP DG (t)-P neighbor (t)],P neighbor (t) is the output power of the local distributed power supply of the neighboring intelligent agent.

[0041] Furthermore, in the prediction module, the prediction model includes an input layer, an embedding layer, a prediction layer, a fully connected layer and an output layer, and takes historical load data, meteorological data and distributed power output data as input, and predicts the load demand and distributed power output of future time periods through the prediction model; wherein the prediction layer is composed of a stack of at least one long short-term memory network, or a stack of at least one gated recurrent unit.

[0042] Furthermore, the prediction module also includes using Kalman filtering to correct the prediction results of load demand and distributed power output.

[0043] The electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is loaded into the processor, it implements the multi-agent-based virtual power plant intelligent control method.

[0044] The computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the multi-agent-based virtual power plant intelligent control method.

[0045] Beneficial effects: Compared with the prior art, the advantages of the present invention are:

[0046] (1) The present invention adopts a multi-agent distributed architecture to reduce the computational burden, improve the system response speed, and avoid the risk of single point failure; the collaborative mechanism between agents ensures the stable operation of the system and improves the control accuracy;

[0047] (2) The distributed control strategy based on the multi-agent architecture improves the fault tolerance and reliability of the system; the continuous optimization of the control strategy is achieved through reinforcement learning, and the system operation efficiency continues to improve over time; the modular design ensures good scalability of the system and facilitates the integration of new equipment;

[0048] (3) The present invention adopts a hierarchical control architecture that organically combines global optimization and local adjustment. The upper-level optimization control module adopts the model predictive control method to achieve efficient coordination of global resources. The lower-level reinforcement learning control module can quickly respond to local changes and provide real-time adjustment capabilities.

[0049] (4) The present invention significantly improves the prediction accuracy of load demand and distributed power output by adopting a deep learning model (LSTM / GRU) combined with online rolling correction technology; introduces external data sources and performs feature engineering to fully utilize multi-source information such as weather forecasts and historical data; the prediction model has adaptive capabilities and can dynamically adjust the prediction results based on real-time data;

[0050] (5) The present invention reduces operating costs through precise prediction and optimized scheduling; the optimized management of energy storage equipment extends the equipment life; the refined management of demand-side response reduces electricity costs; the efficient use of renewable energy reduces carbon emissions; multi-objective optimization ensures a balance between economic and environmental benefits; and the coordinated operation of distributed energy improves energy utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Schematic diagram of the distributed architecture of the present invention.

[0052] Figure 2 Schematic diagram of the intelligent agent functional module of the present invention.

[0053] Figure 3 This is the prediction model control block diagram of the present invention. DETAILED DESCRIPTION

[0054] This invention aims to improve the forecasting accuracy and decision-making efficiency of virtual power plants (VPPs) in uncertain environments, optimize the coordinated scheduling of distributed energy resources, and achieve the goals of low emissions, low electricity costs, and low volatility. Based on a multi-agent system (MAS), it integrates online rolling correction forecasting technology, optimization techniques (linear programming), and reinforcement learning to ensure efficient, stable, and safe operation of the system. The technical solution of this invention is further described below with reference to the accompanying figures.

[0055] like Figure 1 As shown, the present invention adopts a distributed architecture consisting of multiple sub-virtual power plants (Sub-VPPs). Each Sub-VPP is managed by an independent agent (Agent) to schedule local resources. The agents collaborate to optimize the operation of the sub-VPP. The VPP master control center obtains data from each agent and performs comprehensive control. Sub-VPPs include distributed generation (DG) such as photovoltaic, wind power, and micro gas turbines; energy storage (ES) such as batteries and supercapacitors; and controllable loads (CL) such as industrial and commercial loads.

[0056] The virtual power plant includes the following functional points.

[0057] (1) Division and communication of sub-virtual power plants.

[0058] Division method: Sub-VPPs are divided based on factors such as geographical location, grid access point, resource type and scale, for example, by distribution transformer or region.

[0059] Communication Protocol: Lightweight, highly reliable protocols such as MQTT and DDS are used for information transmission to ensure real-time and reliable data exchange. Transmission content includes power, voltage, current, energy storage status, prediction results, scheduling plans, alarm information, etc.

[0060] Topology: Adopts adaptive Mesh network and combines virtual local area network (VLAN) technology for logical isolation to enhance security and manageability.

[0061] (2) Sub-virtual power plant modeling.

[0062] Distributed Generation (DG): Photovoltaic: A dynamic prediction model based on weather (light intensity, temperature) and historical data uses methods such as Kalman filtering or particle filtering for online rolling correction; Wind power: Output is dispatched based on wind speed, wind direction, air density and wind turbine characteristic models, combined with numerical weather forecasts; Micro gas turbines: Output is adjusted according to factors such as load demand, operating status and fuel cost, taking into account constraints such as ramp rate and minimum operating power.

[0063] Energy storage equipment (ES): Dynamic scheduling based on charge and discharge efficiency, capacity, cycle life and state of charge (SOC), taking into account charge and discharge rate limits, SOC upper and lower limits and temperature effects.

[0064] Controllable load (CL): Dynamically adjusts load based on real-time electricity prices, demand response strategies, user preferences, and contractual agreements to optimize load distribution and management, taking into account load response time and maximum adjustable capacity.

[0065] (3) Communication topology and association matrix.

[0066] The initial topology is constructed based on factors such as the physical location of the Sub-VPPs, the quality and importance of their communication links, and other factors. Real-time adjustments are made based on network congestion, link failures, and node failures. A distributed routing algorithm is used, and an association matrix is used to describe the topology and guide information transmission and collaborative control.

[0067] like Figure 2 As shown in the figure, the intelligent agent includes: a status monitoring module for monitoring the real-time status of local resources; a prediction module, responsible for load demand and distributed power output prediction; a decision module, used to generate local scheduling strategies; a communication module: responsible for information exchange between intelligent agents; a hierarchical control unit, in which the upper-level optimization control module uses an optimization algorithm for global optimization, and the lower-level reinforcement learning control module uses a reinforcement learning algorithm for local adjustment and adaptation.

[0068] Specifically, the upper-level optimization control module is responsible for global, medium- to long-term optimization and scheduling, calculating a preliminary optimal scheduling plan based on forecast data. The lower-level reinforcement learning control module is responsible for local, short-term adjustments, fine-tuning the upper-level plan based on actual conditions to address forecast errors and unexpected events. The steps for controlling a virtual power plant based on hierarchical control units are as follows.

[0069] 1. Data collection and preprocessing.

[0070] Collect grid parameters, distributed power generation parameters, energy storage equipment parameters, load data and weather data from power grid companies, equipment manufacturers, smart meters, meteorological bureaus and other channels.

[0071] Specific data includes: historical load data: the system's historical load curve, usually in hours or minutes; meteorological data: temperature, humidity, wind speed, sunshine time, cloud cover, etc., and future meteorological forecast data can be obtained through the meteorological forecast API; distributed power output data: such as actual output data of photovoltaic and wind power; external data: including holidays, working days, seasonal information, etc., used to assist in predicting load and power output.

[0072] Perform preprocessing operations such as cleaning, missing value processing, outlier detection and normalization on the original data to improve the model training effect.

[0073] Feature engineering: Time feature extraction: convert date and time information into periodic features (such as hours, weeks, months, seasons, etc.); meteorological data preprocessing: standardize meteorological data and convert it into a suitable input format; lag feature construction: construct lag features of historical load data and power output, such as load and power output data for the previous hour, 24 hours, and 7 days; external factor features: encode factors such as holidays and special events as independent features.

[0074] 2. Short-term load and distributed power output forecast.

[0075] like Figure 3 As shown in the figure, deep learning models such as long short-term memory (LSTM) or gated recurrent unit (GRU) are used to combine historical data, weather data, external data, etc. to perform short-term (e.g., the next 15 minutes or 1 hour) load and distributed power output forecasts.

[0076] Model Architecture: Input Layer: Accepts multiple features, including historical load data, meteorological data, distributed power generation data, and external data sources; Embedding Layer: Embeds and transforms time information (such as days of the week and holidays) to help the model understand periodic features; LSTM (or GRU) Layer: Processes time series data to capture long-term and short-term dependencies. Multiple LSTM (or GRU) layers can be stacked to enhance the model's expressiveness; Dense Layer: Converts the LSTM (or GRU) output into specific predictions; Output Layer: Outputs predicted load demand and distributed power generation output for future time periods.

[0077] Model training and optimization: Use historical load data, meteorological data, and distributed power output data as input, and future load demand and power output as target values for supervised learning. Use mean square error (MSE) as the loss function and the Adam optimizer for model training. Cross-validation is used in the training process to evaluate the model's performance based on different training and validation sets to avoid overfitting. A rolling forecast method is used, that is, the model is retrained and forecasted using the latest data at regular intervals (for example, 5 minutes) to improve forecast accuracy.

[0078] Prediction step: Calculate the state at the next moment based on the load demand and power output forecast values for the future period output by the prediction model;

[0079] Update step: By comparing with actual observations (such as real-time load data and real-time power output data), the prediction results are adjusted to make the model more accurate;

[0080] Rolling Correction: Whenever new data arrives, the Kalman filter is used to correct previous predictions, thereby updating model parameters and improving prediction accuracy. The Kalman filter is a recursive filtering algorithm used to estimate the state of a dynamic system. When applied to this solution, the Kalman filter can correct the predicted value at each moment based on the model's predictions.

[0081] 3. Generate the initial scheduling plan through the upper-level optimization module.

[0082] The load and distributed power output forecast values obtained by using short-term load and distributed power output forecasts are used as the prediction inputs of model predictive control. The optimization goal is to minimize the operating costs in the future, including fuel costs, energy storage charging and discharging costs, load scheduling costs, and emission costs. The objective function is:

[0083]

[0084] Among them, a·P DG,i +b represents the fuel cost of the i-th distributed power source. It is assumed that the fuel cost is a linear function of the output power. If a more accurate model is needed, a nonlinear function (such as ) to better reflect the fuel consumption characteristics of actual power plants. c·|P ES,j | represents the charging and discharging cost of the jth energy storage device. The cost of charging and discharging power can be the absolute value of the power size, because the cost of energy storage equipment is usually related to its charging and discharging rate. CL,k | represents the dispatch cost of the kth controllable load. Load dispatch cost is typically related to the degree of load regulation, so this cost should be linked to the absolute value of the power. e·Emission represents the emission cost. Emission cost is typically related to the energy consumed, so it should be associated with the type and output of the distributed generation (DG). Emission cost is proportional to the product of emissions per unit of energy and power generation. T is the prediction horizon.

[0085] Constraints: Each cost in the objective function can be associated with a specific constraint. Here’s how to justify these costs in the constraints.

[0086] (1) Power balance constraints

[0087]

[0088] (2) Physical constraints of equipment:

[0089]

[0090] (3) Energy storage cost and SOC limitation

[0091] SOC constraint: SOCmin ≤SOC j (t)≤SOC max ;

[0092] Charge and discharge rate limits of energy storage devices:

[0093] The upper and lower limits of SOC of energy storage equipment (usually 0 and 1): 0≤SOC j (t)≤1, where SOC j (t) represents the state of charge of energy storage device j at time t.

[0094] (4) Load scheduling constraints

[0095] The cost of load dispatch should be modeled based on the load adjustment. For each controllable load, the power of dispatch adjustment should be limited, such as:

[0096] At the same time, the cost of load dispatch may also need to consider the losses caused to users due to load reduction, or the economic compensation that may be generated by load adjustment.

[0097] (5) Environmental costs and emission restrictions

[0098] Emission costs should be related to the output level of distributed generation. Emissions can be calculated based on the type of generation, fuel characteristics and generation volume, such as: Among them, f cmission (P DG,i ) represents the emissions of the i-th distributed generation, which may be a function linearly or nonlinearly related to the power output. For clean energy such as wind power and photovoltaic power, the emissions are zero and can be ignored.

[0099] (6) System stability constraints

[0100] The frequency and voltage of the system must be maintained within a certain range. These constraints will affect the optimization results of the scheduling scheme. Assume that the constraints of grid frequency and voltage control are: min ≤f(t)≤f max , V min ≤V(t)≤V max ,These constraints need to be maintained by adjusting the power output of each device through scheduling.

[0101] Solution method: Use an efficient linear programming solver (such as CPLEX, Gurobi, etc.) to quickly calculate the optimal output power of each device.

[0102] 4. Generate decision plans through the lower-level reinforcement learning control module.

[0103] Objective: To dynamically adjust scheduling strategies and achieve local optimization in complex and uncertain environments through reinforcement learning algorithms.

[0104] (1) State space design

[0105] In a distributed system, each agent can only observe local information, so the state space should be mainly based on local information while taking into account the global information obtained through the communication module.

[0106] The state space design includes the following local information: Local distributed power output: P DG (t); local energy storage state: SOC(t); local controllable load: P CL (t); local voltage: V(t); local frequency: f(t); local load forecast: P load (t); Local distributed power forecast: P pred (t); local energy storage charging and discharging status: P ES (t).

[0107] The state space design includes the following global information: through communication between agents, the power (P ncighbor ), voltage (V ncighbor ), frequency (f neighbor ) and other information.

[0108] In summary, the state space of the agent can be expressed as:

[0109] S t =(P DG (t), SOC(t), P CL (t), V(t), f(t), P load (t), P pred (t), P ES (t), {P neighbor (t), V neighbor (t), f neighbor (t)});

[0110] (2) Action space design: The action of each agent is the scheduling adjustment that it can make at the current moment. It is assumed that each agent has the following action space, such as adjusting the output of distributed power supply, energy storage charging and discharging, load management, etc.

[0111] Distributed power supply power regulation: Each intelligent agent adjusts the output power of the distributed power supply within a certain range:

[0112] a1=[ΔP DG (t)], ΔP DG (t)∈[P DG,min , P DG,max ];

[0113] Among them, P DG,min and P DG,max are the minimum and maximum outputs of the local distributed power source respectively.

[0114] Energy storage control: Each agent regulates the charging and discharging power of local energy storage:

[0115] a2=[ΔP ES (t)], ΔP ES (t)∈[P ES,min , P ES,max ];

[0116] Among them, P ES,min and P ES,max The minimum and maximum power of energy storage charging and discharging.

[0117] Load regulation: The agent can adjust the controllable load according to the load demand and system status:

[0118] a3=[ΔP CL (t)], ΔP CL (t)∈[P CL,min , P CL,max ];

[0119] Among them, P CL,min and P CL,max The minimum and maximum load adjustment range for the controllable load.

[0120] In a multi-agent system, each agent must not only regulate local resources but also consider coordination with neighboring agents. To achieve consistency, the action space can consider coordinated adjustments with the power output or scheduling strategies of neighboring agents. Therefore, the action space should also include some coordinated adjustments, namely power adjustments based on the state of neighboring agents, and neighbor coordinated adjustments: responding to changes in the state of neighboring agents by adjusting actions (such as power adjustments).

[0121] a4=[ΔP DG (t)-P neighbor (t)];

[0122] Action space overview:

[0123] A t =[a1, a2, a3, a4];

[0124] Among them, a1 to a3 represent local scheduling actions, and a4 represents the coordination behavior with neighbors.

[0125] (3) Reward function design

[0126] The reward function aims to guide the agents to make efficient scheduling decisions, taking into account cost, system stability (frequency, voltage), and consistency among multiple agents.

[0127] The design of the reward function should include the following important goals:

[0128] Economic efficiency (fuel cost, energy storage cost, load dispatch cost): By reducing the overall cost of the system, the intelligent agent is rewarded for adopting more energy-efficient and economical dispatching solutions.

[0129] Frequency stability: The frequency deviation should be as small as possible, so the penalty term should be proportional to the square of the frequency deviation.

[0130] Voltage stability: Voltage deviation should also be minimized. Similar to frequency, the penalty term is proportional to the square of the voltage deviation.

[0131] Consistency: Strengthen the consistency of power distribution between agents and penalize the absolute value of power difference between adjacent agents. This goal is the key to the coordination of multi-agent systems. Reducing local inconsistencies can effectively improve the overall efficiency of the system. t =-α·C total (t)-β·(f(t)-f ref ) 2 -γ·(V(t)-V ref ) 2 -δ·∑|P i -P neighbor |-η·|ΔP|;

[0132] Among them, α, β, γ, δ, η are weight coefficients, |ΔP| is used to punish excessive adjustments and improve system response speed; C total (t) is the total cost of the system, which combines fuel cost, energy storage cost, load dispatch cost and emission cost; f(t) and V(t) represent the current system frequency and voltage, respectively, f ref and V ref are the reference frequency and voltage values, and the goal is to maintain the system frequency and voltage near these reference values; P i and P neighbor are the powers of agent i and its neighboring agents, respectively. δ is the coefficient of the consistency penalty, which penalizes the sum of the absolute values of the power differences between agents, and encourages the agent to reach a consistent scheduling target with its neighbors.

[0133] This reward function comprehensively considers economy (such as electricity charges and fuel costs), stability (such as frequency and voltage constraints) and consistency (such as coordination between multiple agents), and aims to optimize the scheduling strategy through reinforcement learning to respond to emergencies such as load fluctuations and weather changes.

[0134] 5. Exchange decision plans and real-time data between intelligent agents through the communication module and execute the final decision plan.

[0135] The multi-agent-based virtual power plant intelligent control system of the present invention includes:

[0136] Virtual power plant, which consists of several sub-virtual power plants. An intelligent agent is deployed for each sub-virtual power plant, and a communication network is established between the intelligent agents.

[0137] The status monitoring module is used to collect the local resource status of the sub-virtual power plant through each intelligent agent;

[0138] The prediction module is used to predict load demand and distributed power output by building prediction models through various intelligent agents;

[0139] Hierarchical control unit, used for collaborative optimization among sub-virtual power plants, including:

[0140] The upper optimization control module constructs a linear programming model based on the prediction results of load demand and distributed power output and solves it to obtain the initial scheduling plan;

[0141] The lower-level reinforcement learning control module makes local adjustments to the preliminary scheduling plan based on the multi-agent deep deterministic policy gradient algorithm to obtain a decision plan;

[0142] The execution module is used to exchange decision plans between intelligent agents and execute the decision plans.

[0143] The electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is loaded into the processor, it implements the multi-agent-based virtual power plant intelligent control method.

[0144] The computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the multi-agent-based virtual power plant intelligent control method.

[0145] The computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store program code in the form of instructions or data structures and that can be accessed by a computer.

[0146] The processor is configured to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment.

Claims

1. A multi-agent based virtual power plant intelligent control method, characterized in that: The steps include: (1) Divide the virtual power plant into several sub-virtual power plants, deploy intelligent agents in each sub-virtual power plant, and establish a communication network between the intelligent agents; (2) Each agent collects the local resource status of the sub-virtual power plant; (3) Each intelligent agent builds a prediction model to predict load demand and distributed power output; (4) Using hierarchical control units to perform collaborative optimization among sub-virtual power plants, including: The upper optimization control module constructs a linear programming model based on the prediction results of load demand and distributed power output and solves it to obtain the initial scheduling plan; The lower-level reinforcement learning control module makes local adjustments to the preliminary scheduling plan based on the multi-agent deep deterministic policy gradient algorithm to obtain a decision plan; (5) Exchange decision plans between intelligent agents and execute them.

2. The multi-agent-based virtual power plant intelligent control method according to claim 1 is characterized in that: In step (4), the upper optimization control module establishes an objective function with the goal of minimizing the total cost of the virtual power plant, solves the linear programming problem through a linear programming solver, and obtains a preliminary scheduling plan; the total cost of the virtual power plant is the sum of fuel cost, energy storage charging and discharging cost, load scheduling cost and emission cost; The constraints of the linear programming problem include power balance constraints, energy storage device charge state constraints, device physical constraints, charge and discharge rate constraints, and frequency and voltage constraints.

3. The multi-agent-based virtual power plant intelligent control method according to claim 1 is characterized in that: In step (4), the lower-level reinforcement learning control module establishes a reward function to control the actions of each agent. The reward function is: r t =-α·C total (t)-β·(f(t)-f ref ) 2 -γ·(V(t)-V ref ) 2 -δ·∑|P i -P neighbor |-η·|ΔP|; Among them, α, β, γ, δ, η are weight coefficients, C total (t) is the total cost of the virtual power plant, f(t) and V(t) represent the frequency and voltage of the virtual power plant at the current time t, respectively. ref and V ref is the reference frequency and voltage value, P i and P neighbor are the powers of agent i and its neighboring agents, |ΔP| is the penalty term, and δ is the penalty term coefficient; The actions of each intelligent agent include the scheduling adjustments that the sub-virtual power plant can make at the current moment, including adjusting the output power of the local distributed power supply, adjusting the charging and discharging power of the local energy storage, adjusting the controllable load, and adjusting the output power according to the status of the neighboring intelligent agents.

4. The multi-agent-based virtual power plant intelligent control method according to claim 3 is characterized in that: The action of adjusting the output power of the local distributed power supply a1=[ΔP DG (t)], ΔP DG (t)∈[P DG,min , P DG,max ], P DG,min and P DG,max are the minimum output and maximum output of the local distributed power source respectively; The action of adjusting the charging and discharging power of the local energy storage a2=[ΔP ES (t)], ΔP ES (t)∈[P ES,min , P ES,max ], P ES,min and P ES,max The minimum and maximum power for charging and discharging local energy storage; Adjust the action of controllable load a3=[ΔP CL (t)], ΔP CL (t)∈[P CL,min , P CL,max ], P CL,min and P CL,max The minimum load adjustment range and the maximum load adjustment range of the controllable load; The action a4 of adjusting the output power according to the state of the neighboring agent is [ΔP DG (t)-P neighbor (t)],P neighbor (t) is the output power of the local distributed power supply of the neighboring intelligent agent.

5. The multi-agent-based virtual power plant intelligent control method according to claim 1 is characterized in that: In step (3), the prediction model includes an input layer, an embedding layer, a prediction layer, a fully connected layer, and an output layer, and takes historical load data, meteorological data, and distributed power output data as input, and predicts the load demand and distributed power output of future time periods through the prediction model; The prediction layer is composed of a stack of at least one long short-term memory network, or a stack of at least one gated recurrent unit.

6. The multi-agent-based virtual power plant intelligent control method according to claim 1 is characterized in that: Step (3) also includes using Kalman filtering to correct the prediction results of load demand and distributed power output.

7. A multi-agent based virtual power plant intelligent control system based on the method of claim 1, characterized in that: include: Virtual power plant, which consists of several sub-virtual power plants. An intelligent agent is deployed for each sub-virtual power plant, and a communication network is established between the intelligent agents. The status monitoring module is used to collect the local resource status of the sub-virtual power plant through each intelligent agent; The prediction module is used to predict load demand and distributed power output by building prediction models through various intelligent agents; Hierarchical control unit, used for collaborative optimization among sub-virtual power plants, including: The upper optimization control module constructs a linear programming model based on the prediction results of load demand and distributed power output and solves it to obtain the initial scheduling plan; The lower-level reinforcement learning control module makes local adjustments to the preliminary scheduling plan based on the multi-agent deep deterministic policy gradient algorithm to obtain a decision plan; The execution module is used to exchange decision plans between intelligent agents and execute the decision plans.

8. The multi-agent-based virtual power plant intelligent control system according to claim 7 is characterized in that: In the hierarchical control unit, the upper optimization control module establishes an objective function with the goal of minimizing the total cost of the virtual power plant, solves the linear programming problem using a linear programming solver, and obtains a preliminary scheduling plan; the total cost of the virtual power plant is the sum of fuel cost, energy storage charging and discharging cost, load scheduling cost, and emission cost; The constraints of the linear programming problem include power balance constraints, energy storage device charge state constraints, device physical constraints, charge and discharge rate constraints, and frequency and voltage constraints.

9. The multi-agent-based virtual power plant intelligent control system according to claim 7 is characterized in that: In the hierarchical control unit, the lower-layer reinforcement learning control module establishes a reward function to control the actions of each agent. The reward function is: r t =-α·C total (t)-β·(f(t)-f ref ) 2 -γ·(V(t)-V ref ) 2 -δ·∑|P i -P neighbor |-η·|ΔP|; Among them, α, β, γ, δ, η are weight coefficients, C total (t) is the total cost of the virtual power plant, f(t) and V(t) represent the frequency and voltage of the virtual power plant at the current time t, respectively. ref and V ref are the reference frequency and voltage values, P and P neighbor are the powers of agent i and its neighboring agents, |ΔP| is the penalty term, and δ is the penalty term coefficient; The actions of each intelligent agent include the scheduling adjustments that the sub-virtual power plant can make at the current moment, including adjusting the output power of the local distributed power supply, adjusting the charging and discharging power of the local energy storage, adjusting the controllable load, and adjusting the output power according to the status of the neighboring intelligent agents.

10. The multi-agent based virtual power plant intelligent control system according to claim 9, characterized in that: The action of adjusting the output power of the local distributed power supply a1=[ΔP DG (t)], ΔP DG (t)∈[P DG,min , P DG,max ], P DG,min and p DG,max are the minimum output and maximum output of the local distributed power source respectively; The action of adjusting the charging and discharging power of the local energy storage a2=[ΔP ES (t)], ΔP ES (t)∈[P ES,min , P ES,max ], p ES,min and P ES,max The minimum and maximum power for charging and discharging local energy storage; Adjust the action of controllable load a3=[ΔP CL (t)], ΔP CL (t)∈[P CL,min , P CL,max ], P CL,min and P CL,max The minimum load adjustment range and the maximum load adjustment range of the controllable load; The action a4 of adjusting the output power according to the state of the neighboring agent is [ΔP DG (t)-p neighbor (t)],p neighbor (t) is the output power of the local distributed power supply of the neighboring intelligent agent.

11. The multi-agent-based virtual power plant intelligent control system according to claim 7, characterized in that: In the prediction module, the prediction model includes an input layer, an embedding layer, a prediction layer, a fully connected layer, and an output layer. It takes historical load data, meteorological data, and distributed power output data as input, and uses the prediction model to predict the load demand and distributed power output of future time periods. The prediction layer is composed of a stack of at least one long short-term memory network, or a stack of at least one gated recurrent unit.

12. The multi-agent-based virtual power plant intelligent control system according to claim 7, characterized in that: The prediction module also includes using Kalman filtering to correct the prediction results of load demand and distributed power output.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is loaded into the processor, the multi-agent-based virtual power plant intelligent control method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multi-agent-based virtual power plant intelligent control method according to any one of claims 1 to 6 is implemented.

Images