A multi-level coordinated control method and system for a virtual power plant

Abstract

The present invention discloses a multi-level coordinated control method and system for a virtual power plant. The adjustable space of various distributed devices in the virtual power plant is quantitatively described through a unified modeling method, and evaluation models of real-time output and response capability are established respectively. In view of the online coordinated scheduling problem between the virtual power plant and the distribution network, a centralized-distributed hierarchical scheduling structure is adopted, and the scheduling efficiency of the system and the response capability of the equipment are effectively improved through unified modeling and hierarchical optimization. The present invention can effectively reduce the total operating cost of the virtual power plant, avoid the comprehensive information sharing problem required for traditional centralized scheduling through privacy protection design, realize efficient interaction and information security between the virtual power plant and the distribution network, meet the needs of real-time economic scheduling in complex scenarios, and have strong scalability and adaptability. At the same time, it conforms to the current development direction of optimized scheduling of distribution networks, and plays a vital role in promoting the transformation of energy structure and accelerating the consumption of renewable energy.

Description

Technical Field

[0001] The present invention belongs to the technical field of distribution network operation optimization, and specifically relates to a multi-level coordinated control method and system for a virtual power plant. Background Art

[0002] In recent years, modern power systems containing a large number of renewable energy sources (RES) have developed rapidly, and distributed energy resources (DER) have been widely connected to modern power systems due to their flexibility and economy. However, RES have the characteristics of low capacity, wide geographical distribution and high volatility, and their large-scale access has brought severe challenges to the power grid. Virtual Power Plant (VPP) uses advanced metering, sensing and control technologies to manage the aggregation of DER and participate in the dispatching and operation of the distribution network (DN) as a whole, which can fully tap the potential of different types of flexible resources and provide flexible regulation capabilities for the distribution network.

[0003] In terms of optimizing VPP operation, scholars' research mainly focuses on two aspects: the interactive operation between VPP and the power grid and the optimal scheduling of distributed equipment (DE) within VPP. Traditional VPP control modes are divided into centralized and distributed. In the centralized control mode, the control center must obtain the cost and technical information of all DEs. However, privacy issues between VPPs and between VPPs and DEs make the centralized scheduling method no longer applicable. In addition, as the scale of the system continues to expand, the centralized control mode needs to process a large amount of device-level and system-level data. Therefore, some scholars have proposed a distributed optimal scheduling algorithm for VPP, which achieves adaptive adjustment by iteratively adjusting the excitation signal. However, the optimal scheduling solution obtained by the distributed control mode does not take into account the operating status of the system and requires a large number of iterations, resulting in slow convergence.

[0004] In addition, the above studies mainly focus on traditional day-ahead dispatch, which requires strict adherence to the dispatch plan on the dispatch day, but due to the uncertainty within the VPP, the day-ahead dispatch plan will inevitably deviate when applied to real-time dispatch. In addition, in actual operation scenarios, the regulation requirements sent by the distribution network to the VPP will change frequently, and traditional optimization methods are difficult to adapt to and meet the rapidly changing environment and regulation requirements. Based on the above problems, in order to maximize the potential of DER and respond to real-time regulation requirements quickly, accurately and economically, it is urgent to design an efficient dispatch method to achieve online optimal dispatch of different DEs.

[0005] Deep reinforcement learning (DRL) is suitable for scenarios where the environment model and data distribution are unknown. In the offline phase, DRL agents trained based on a large amount of data can make millisecond-level decisions in the online scheduling phase, meeting the needs of fast and accurate adjustment of VPP. However, in general, there are few studies on using DRL methods to achieve multi-level coordinated adjustment and privacy protection of hierarchical VPP.

[0006] In addition, since VPP contains a large number of DEs, the action space of traditional DRL methods is usually defined within the response range of all DEs, which significantly reduces learning efficiency and generates constraint-violating policies. Moreover, due to the privacy protection requirements, it is difficult for VPP to collect accurate models and parameters. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a multi-level coordinated control method and system for a virtual power plant in response to the deficiencies in the above-mentioned prior art, which is used to solve the technical problem of coordinated control between the scheduling of distributed equipment in the virtual power plant and the overall economic scheduling of the distribution network, and to achieve hierarchical control of the upper and lower layers of the system and real-time optimized response of the equipment in the virtual power plant.

[0008] The present invention adopts the following technical solutions:

[0009] A multi-level coordinated control method for a virtual power plant comprises the following steps: based on the active power boundary, power generation boundary and ramp rate boundary of distributed equipment, an adjustable space six-tuple modeling method is used to obtain the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment;

[0010] Based on the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed device, combined with the network security constraints of the distribution network, a hierarchical double-delay deep deterministic policy gradient method is constructed to obtain the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL, which are used to realize the economic dispatch of the distribution network to the virtual power plant and the real-time response of the distributed devices in the virtual power plant respectively.

[0011] The hierarchical DRL method under the centralized-decentralized scheduling framework is adopted to train the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL. In the offline stage, historical data is used for strategy learning, and real-time scheduling is performed through soft updates in the online stage to achieve coordinated regulation of the upper and lower layers of the system.

[0012] Preferably, the aggregated external characteristics and adjustable spatial model of the virtual power plant are as follows:

[0013] Using sextuples Describing Virtual Power Plants Distributed equipment At the moment The power generation characteristics of the six-element Determine distributed equipment The distributed devices in the virtual power plant respond to the decomposed dispatch instructions, and after each response cycle, the remaining capacity status of the distributed devices is updated, where: Virtual Power Plant Distributed equipment At the moment The upper limit of power generation, Virtual Power Plant Distributed equipment At the moment The lower limit of power generation, Virtual Power Plant Distributed equipment At the moment The upper limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The lower limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The upper limit of the climbing rate, Virtual Power Plant Distributed equipment At the moment The lower limit of the climbing rate.

[0014] Preferably, the active power boundary, power generation boundary and ramp rate boundary of the distributed equipment are as follows:

[0015]

[0016]

[0017]

[0018] in, Virtual Power Plant Distributed equipment At the moment The power generation capacity, Virtual Power Plant Distributed equipment At the moment The power generation capacity, represents the scheduling interval, It is a virtual power plant Distributed equipment The initial power.

[0019] Preferably, the distributed device The adjustable capacity is as follows:

[0020]

[0021]

[0022]

[0023] in, and Virtual Power Plant Distributed equipment At the moment The upper and lower limits of the adjustable space when responding to the virtual power plant regulation instructions, Virtual Power Plant Distributed equipment At the moment The response adjustment, is the energy conversion efficiency of the virtual power plant, is the time step.

[0024] Preferably, the adjustable spatial model of the distributed equipment includes:

[0025] Adjustable spatial model of photovoltaic units:

[0026]

[0027]

[0028] in, Virtual Power Plant Photovoltaic unit At the moment The lower limit of power generation, Virtual Power Plant Photovoltaic unit At the moment The power at the maximum power point of the photovoltaic Virtual Power Plant Photovoltaic unit At the moment The upper limit of the photovoltaic power generation capacity, Virtual Power Plant Photovoltaic unit At the moment The upper limit of the adjustable space of the photovoltaic unit, Virtual Power Plant Photovoltaic unit At the moment The lower limit of the adjustable space of the photovoltaic unit, Virtual Power Plant Photovoltaic unit At the moment of photovoltaic power generation;

[0029] Micro gas turbine unit adjustable space model:

[0030]

[0031]

[0032] in, Virtual Power Plant Micro gas turbine unit At the moment The real-time power, and Virtual Power Plant Micro gas turbine unit At the moment The power generation efficiency and heat loss coefficient, Virtual Power Plant Micro gas turbine unit At the moment The adjustable upper limit, Virtual Power Plant Micro gas turbine unit At the moment The upper limit of power generation, Virtual Power Plant Micro gas turbine unit At the moment The upper limit of the climbing rate, Virtual Power Plant Micro gas turbine unit At the moment The upper limit of gas consumption, Virtual Power Plant Micro gas turbine unit At the moment The current amount of gas consumed, Virtual Power Plant Micro gas turbine unit At the moment The power generation efficiency, represents the time step, Virtual Power Plant Micro gas turbine unit At the moment The adjustable lower limit of Virtual Power Plant Micro gas turbine unit At the moment The lower limit of power generation, Virtual Power Plant Micro gas turbine unit At the moment The lower limit of the climbing rate, Virtual Power Plant Micro gas turbine unit At the moment The lower limit of gas consumption;

[0033] Adjustable space model of energy storage system:

[0034]

[0035]

[0036] in, Energy storage system The battery capacity, Virtual Power Plant Energy Storage System At the moment Real-time state of charge, and Virtual Power Plant Energy Storage System At the moment Real-time charge / discharge power, and Represents energy storage system The discharge / charge efficiency, Virtual Power Plant Energy Storage System At the moment The energy storage adjustable power limit, Virtual Power Plant Energy Storage System At the moment The upper limit of energy storage power generation, Virtual Power Plant Energy Storage System At the moment The lower limit of the state of charge, Virtual Power Plant Energy Storage System At the moment The lower limit of the adjustable power of energy storage, Virtual Power Plant Energy Storage System At the moment The upper limit of discharge power, Virtual Power Plant Energy Storage System At the moment The upper limit of the state of charge;

[0037] Building air conditioning adjustable space model:

[0038]

[0039]

[0040] in, and Virtual Power Plant Medium air conditioning At the moment Maximum and minimum cooling power limited by room temperature, and Respectively represent air conditioning The heat capacity and thermal conductivity of the room, and Respectively represent air conditioning The maximum and minimum indoor temperature of the room, and Virtual Power Plant Medium air conditioning At the moment Real-time indoor and outdoor temperatures;

[0041] Output power of air conditioner:

[0042]

[0043]

[0044] in, Virtual Power Plant Medium air conditioning At the moment The real-time power, and Respectively represent air conditioning The maximum and minimum frequencies, Indicates the upper limit of the adjustable power of the air conditioner. Indicates the lower limit of the adjustable power of the air conditioner. is the coefficient of a linear function;

[0045] Interruptible load adjustable space model:

[0046]

[0047] in, Virtual Power Plant Medium interruptible load At the moment The real-time power, Virtual Power Plant Medium interruptible load At the moment The upper limit of the interruptible load regulation, Virtual Power Plant Medium interruptible load At the moment The lower limit of the interruptible load regulation.

[0048] Preferably, the upper distribution network centralized online dispatching model of the single-agent DRL is specifically:

[0049] Taking the distribution network as an intelligent entity, introducing strategies represents the conditional probability distribution of each action when the current state is known, where Initial state The probability distribution of To indicate the status Take action The conditional probability of To indicate the status Next action After that, the system moves to the next state The conditional probability of Get the reward value when it is known Guide the training of the agent and use the gradient ascent method to update the strategy to maximize the reward value; define the state value function Evaluate the state performance of the parameterized neural network; during the training process, the goal of the agent is to select the best strategy to maximize the state value function; use the judgment network to simulate the real state value function, and use the policy network to obtain a deterministic strategy; finally, use a soft update strategy to update the network parameters;

[0050] The decentralized online dispatch model of the lower virtual power plant of multi-agent DRL is as follows:

[0051] The virtual power plant aggregates data information from distributed devices and trains device agents centrally. The evaluation network of each device agent estimates the strategies of other agents to obtain more accurate evaluation information. The model trained based on offline data is deployed on distributed devices, and each device agent independently adjusts the scheduling strategy according to its real-time status. The strategy network obtains a greater expected cumulative reward by continuously updating parameters. The evaluation network updates parameters by minimizing the time difference error. Finally, all target networks use soft updates to regularly update the network parameters of the evaluation network.

[0052] Preferably, the action space of the centralized online dispatching model of the upper distribution network of the single-agent DRL is , state space , environment and reward function They are:

[0053] Action Space ,in, For the distribution network at time Virtual Power Plant Active power dispatch instructions, Virtual Power Plant At the moment The scheduling output decision of is the set of virtual power plants in the region;

[0054] State Space , the real-time actual power of each virtual power plant and its adjustable space, Virtual Power Plant At the moment The state space of

[0055] During offline training, the policy network is based on After making a decision, the power flow model of the distribution network obtains the injected power of each virtual power plant node, and obtains the power purchase cost and network loss cost as the reward value. The judgment network will evaluate the action and guide the agent to the next state, and then the agent will adjust the strategy until the end of the training phase;

[0056] The reward function obtained by the distribution network during the training process for:

[0057]

[0058] in, is the total operating cost of the virtual power plant, is the network loss cost, The cost of purchasing electricity, is the total number of virtual power plants.

[0059] Preferably, the action space of the decentralized online scheduling model of the underlying virtual power plant of the multi-agent DRL is , state space , environment and reward function They are:

[0060] State Space ,in, is the total number of virtual power plants, is the total number of types of distributed resources in all virtual power plants;

[0061] State space of any distribution network in a virtual power plant The definition is as follows:

[0062]

[0063] in, Indicates at time Send to Active power dispatch instructions for virtual power plants, Virtual Power Plant A device with a certain type of distributed resource At the moment The power, Virtual Power Plant A device with a certain type of distributed resource At the moment In addition to power, other required parameters, such as energy storage charge state, indoor temperature where the air conditioner is located, etc. Virtual Power Plant A device with a certain type of distributed resource At the moment The adjustable space limit, Virtual Power Plant A device with a certain type of distributed resource At the moment The lower limit of the adjustable space, Virtual Power Plant Energy Storage System At the moment The state of charge, Virtual Power Plant Medium air conditioning equipment At the moment The indoor temperature Virtual Power Plant Photovoltaic equipment At the moment The power generation capacity, Virtual Power Plant Photovoltaic equipment At the moment The adjustable space limit, Virtual Power Plant Photovoltaic equipment At the moment The lower limit of the adjustable space, Virtual Power Plant Energy Storage System At the moment The power generation capacity, Virtual Power Plant Energy Storage System At the moment The adjustable space limit, Virtual Power Plant Energy Storage System At the moment The lower limit of the adjustable space, Virtual Power Plant Micro gas turbine At the moment The power generation capacity, Virtual Power Plant Micro gas turbine At the moment The adjustable space limit, Virtual Power Plant Micro gas turbine At the moment The lower limit of the adjustable space, Virtual Power Plant Medium air conditioning equipment At the moment The power consumption, Virtual Power Plant Medium air conditioning equipment At the moment The adjustable space limit, Virtual Power Plant Medium air conditioning At the moment The lower limit of the adjustable space, Virtual Power Plant Medium interruptible load At the moment The power, Virtual Power Plant Medium interruptible load At the moment The upper limit of adjustment, Virtual Power Plant Medium interruptible load At the moment The lower limit of adjustment, Virtual Power Plant China Photovoltaic At the moment The state space of Virtual Power Plant Energy Storage System At the moment The state space of Virtual Power Plant Micro gas turbine At the moment The state space of Virtual Power Plant Air conditioning equipment At the moment The state space of Virtual Power Plant Medium interruptible load At the moment The state space of

[0064] The relationship between the adjustment rate of various distributed resources and the scheduling instructions is described as follows:

[0065]

[0066] Among them, DER refers to a certain type of distributed resources. Virtual Power Plant Distributed equipment At the moment Adjustment instructions, Virtual Power Plant Distributed equipment At the moment The lower limit of the adjustable space, Virtual Power Plant Distributed equipment At the moment The adjustment rate, Virtual Power Plant Distributed equipment At the moment The upper limit of the adjustable space, the distributed devices represented in the formula refer to the devices included in this type of resources;

[0067] The policy network will take actions based on the current state to determine the output of the distributed equipment. Then, the virtual power plant operating environment model will feedback the cost of each equipment agent and then move to the next state until the training is completed.

[0068] Reward Function as follows:

[0069]

[0070] in, Virtual Power Plant Distributed resources in the device At the moment The running cost, Virtual Power Plant At the moment The power deviation cost from the distribution network dispatch instructions.

[0071] Preferably, the hierarchical DRL method under the centralized-decentralized scheduling framework is used to train the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL. Specifically:

[0072] During the offline training phase, the virtual power plant in the distribution network collects and aggregates the adjustable spatial information of distributed resources. , Virtual Power Plant A device with a certain type of distributed resource At the moment The adjustable space limit, Virtual Power Plant A device with a certain type of distributed resource At the moment The adjustable space lower limit forms the external features , Virtual Power Plant At the moment The external characteristics of the adjustable space upper limit, Virtual Power Plant At the moment The lower limit of the adjustable space of the external characteristics;

[0073] External features The environmental status is transmitted to the upper distribution network, which determines the dispatch output of each virtual power plant based on the current operating status. , and transmitted to each virtual power plant in the lower layer, Virtual Power Plant At the moment The scheduling output decision of For the upper power grid according to the time The operating state is a virtual power plant The scheduling output of the decision;

[0074] The distributed equipment agents in the virtual power plant respond to the dispatching instructions sent by the upper layer and the operating status of the distributed equipment. , for the actual output Make decisions and obtain the operating cost of training; the total operating cost of the virtual power plant, network loss cost and power purchase cost will form the upper-level reward value and be fed back to the distribution network for training until the end of the training phase. Virtual power plant for offline training phase Distributed equipment At the moment The output decision Indicates at time Send to Active power dispatch instructions for virtual power plants, Virtual Power Plant A device with a certain type of distributed resource At the moment The power, Virtual Power Plant A device with a certain type of distributed resource At the moment In addition to power, other required parameters, such as energy storage charge state, indoor temperature where the air conditioner is located, etc. Virtual Power Plant A device with a certain type of distributed resource At the moment The adjustable space limit, Virtual Power Plant A device with a certain type of distributed resource At the moment The lower limit of adjustable space;

[0075] In the online application stage, the trained upper-layer distribution network agent is based on the aggregated external characteristics. Publish real-time dispatch plans for each virtual power plant , the lower-level distribution network intelligent agent responds to the dispatching instructions sent by the upper layer and the real-time environmental status collected Determine real-time output , to achieve coordinated regulation between upper and lower levels, Virtual Power Plant At the moment The state space of is the set of virtual power plants in the region, Virtual Power Plant Distributed equipment At the moment The state space of is a distributed set of devices in a virtual power plant. Virtual Power Plant for Online Application Distributed equipment At the moment Real-time output decision making.

[0076] In a second aspect, an embodiment of the present invention provides a multi-level coordinated control system for a virtual power plant, including:

[0077] The construction module uses the adjustable space six-tuple modeling method to obtain the aggregated external characteristics and adjustable space model of the virtual power plant and the adjustable space model of each distributed device based on the active power boundary, power generation boundary and ramp rate boundary of the distributed devices;

[0078] Design module, based on the aggregate external characteristics and adjustable space model of the constructed virtual power plant, as well as the adjustable space model of each distributed device, combined with the network security constraints of the distribution network, construct a hierarchical double-delay deep deterministic policy gradient method, and obtain the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL, which are used to realize the economic dispatch of the distribution network to the virtual power plant and the real-time response of the distributed devices in the virtual power plant respectively;

[0079] The control module adopts the hierarchical DRL method under the centralized-decentralized scheduling framework to train the upper-layer distribution network centralized online scheduling model of the single-agent DRL and the lower-layer virtual power plant decentralized online scheduling model of the multi-agent DRL. In the offline stage, historical data is used for strategy learning, and real-time scheduling is performed through soft updates in the online stage to achieve coordinated control of the upper and lower layers of the system.

[0080] In a third aspect, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the multi-level coordinated control method of a virtual power plant when executing the computer program.

[0081] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned virtual power plant multi-level coordinated control method.

[0082] In a fifth aspect, a chip comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the multi-level coordinated control method of a virtual power plant when executing the computer program.

[0083] In a sixth aspect, an embodiment of the present invention provides an electronic device, comprising a computer program, which, when executed by the electronic device, implements the steps of the above-mentioned virtual power plant multi-level coordinated control method.

[0084] Compared with the prior art, the present invention has at least the following beneficial effects:

[0085] A multi-level coordinated control method for virtual power plants uses a unified modeling method to quantitatively describe the adjustable space of various distributed equipment in the virtual power plant, and establishes evaluation models for their real-time output and response capabilities respectively; for the online coordinated scheduling problem between the virtual power plant and the distribution network, a centralized-distributed hierarchical scheduling structure is adopted. The upper layer is based on the TD3 method, and optimizes the overall economic scheduling strategy of the virtual power plant on the basis of considering network security constraints; the lower layer adopts the Multi-agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) method to perform real-time scheduling of the internal equipment of the virtual power plant based on the upper-layer scheduling instructions to ensure the response speed and privacy protection of the equipment. The method of the present invention effectively improves the system's dispatching efficiency and equipment's responsiveness through unified modeling and hierarchical optimization; compared with traditional centralized and distributed dispatching algorithms, the method of the present invention can effectively reduce the total operating cost of the virtual power plant; in addition, the method of the present invention avoids the problem of comprehensive information sharing required for traditional centralized dispatching through privacy protection design, realizes efficient interaction and information security between the virtual power plant and the distribution network, can meet the needs of real-time economic dispatching in complex scenarios, and has strong scalability and adaptability.

[0086] Furthermore, the aggregated external characteristics and adjustable space model of the virtual power plant describe the power generation characteristics of distributed devices through six-tuples, so as to quantify the adjustable capacity of each device and provide consistent control of multiple devices within the virtual power plant. This setting can effectively optimize the scheduling strategy, improve the flexibility and adaptability of scheduling, and ensure the rational allocation of resources. In addition, by clarifying the adjustable capacity, it is helpful to evaluate the device response capability and the remaining dispatchable space in real time to cope with complex scheduling scenarios.

[0087] Furthermore, active power boundaries, power generation boundaries, and ramp rate boundaries are used to determine the operating range of distributed devices at different times. The setting of these boundaries ensures that the equipment operates within a safe and reasonable range, preventing the stability of the system from being affected by overload or unstable power output. At the same time, the introduction of the ramp rate boundary can control the speed of power change, thereby reducing the impact of sudden load changes on the stability of the equipment and the power grid, and optimizing the dynamic response performance.

[0088] Furthermore, adjustable capacity is used to represent the ability of distributed equipment to respond to adjustment instructions within a specific time step. Setting the upper and lower limits of adjustable capacity helps to accurately quantify the flexibility of the equipment during the adjustment process, thereby providing basic data support for hierarchical optimization. In addition, clear capacity limits can ensure that while meeting scheduling needs, the service life of the equipment is extended and equipment performance is prevented from being degraded due to excessively frequent adjustments.

[0089] Furthermore, the adjustable space model of each distributed device can achieve a comprehensive evaluation of the equipment's operating status by accurately describing the adjustable power range of different types of equipment (such as photovoltaic units, energy storage systems, air conditioners, etc.). This setting helps optimize the economic benefits and resource utilization efficiency of the equipment and improve the overall scheduling flexibility of the virtual power plant. At the same time, by quantitatively describing the adjustable space, the equipment's adjustment potential can be maximized while ensuring the safety of equipment operation.

[0090] Furthermore, the upper-layer centralized dispatch model based on the single-agent DRL method is responsible for the overall economic dispatch of the virtual power plant, while the lower-layer decentralized dispatch model based on the multi-agent DRL method performs real-time adjustments to each distributed device. Through the two-layer structure combining centralization and decentralization, it is possible to take into account the overall optimization of the system and the real-time response of each device, ensuring the unity of economy and adjustment flexibility. At the same time, the multi-agent architecture ensures the information privacy between devices and avoids the security risks brought about by comprehensive information sharing.

[0091] Furthermore, the design of the action space, state space, environment and reward function in the upper-level dispatch model aims to enable the distribution network agent to take into account both safety and cost objectives in the process of optimizing economic dispatch. The reward function combines the total operating cost of the virtual power plant, network loss and electricity purchase cost, so that the agent can continuously adjust its strategy during the training process to maximize the overall economic benefits. This design enables the model to achieve better dispatch results while meeting network security constraints.

[0092] Furthermore, the lower-level scheduling model is based on a multi-agent architecture, and the settings of its action space, state space, environment, and reward function enable each device agent to make independent decisions. Through feedback on the real-time status, each agent continuously adjusts its strategy to obtain the maximum expected reward. This design not only improves the response speed and flexibility of each device, but also realizes the overall economic and efficient operation of the virtual power plant.

[0093] Furthermore, the training process of the hierarchical DRL method includes an offline training phase and an online training phase. In the offline training phase, the agent uses historical data to learn strategies, while in the online training phase, real-time scheduling is achieved through soft updates. This training method can effectively reduce the complexity of online calculations and ensure the gradual optimization of strategies to adapt to changing scheduling requirements, thereby achieving coordinated regulation and continuous optimization of the system.

[0094] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0095] In summary, the present invention effectively solves the problem of online real-time scheduling of internal equipment in virtual power plants through a hierarchical deep reinforcement learning method, significantly reduces the operating cost of virtual power plants, and improves the utilization rate of new energy. While ensuring scheduling efficiency, it can ensure information privacy between devices and adapt to complex scheduling environments. It provides a feasible solution for realizing flexible scheduling of large-scale distributed energy, which is of great significance to promoting the transformation of energy structure and promoting the development of smart grids.

[0096] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 is a flow chart of the method of the present invention;

[0098] Figure 2 A training graph for a deep reinforcement learning algorithm;

[0099] Figure 3 Comparison chart of scenarios for optimizing scheduling for data center clusters;

[0100] Figure 4 A schematic diagram of a computer device provided by an embodiment of the present invention;

[0101] Figure 5 The present invention is a block diagram of an electronic device provided according to an embodiment. DETAILED DESCRIPTION

[0102] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0103] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0104] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0105] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0106] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0107] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0108] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0109] The present invention provides a multi-level coordinated control method for a virtual power plant. First, the adjustable space of various distributed devices in the virtual power plant is quantitatively described by a unified modeling method, and evaluation models of their real-time output and response capabilities are established respectively. In view of the online coordinated scheduling problem between the virtual power plant and the distribution network, a centralized-distributed hierarchical scheduling structure is adopted, wherein the upper layer is based on the TD3 method, and the overall economic scheduling strategy of the virtual power plant is optimized on the basis of considering network security constraints; the lower layer adopts the multi-agent MATD3 method, and the internal equipment of the virtual power plant is scheduled in real time based on the upper layer scheduling instructions to ensure the response speed and privacy protection of the equipment; the method of the present invention effectively improves the scheduling efficiency of the system and the response capability of the equipment through unified modeling and hierarchical optimization. Compared with traditional centralized and distributed scheduling algorithms, the method of the present invention can effectively reduce the total operating cost of the virtual power plant. In addition, the method of the present invention also avoids the comprehensive information sharing problem required for traditional centralized scheduling through privacy protection design, realizes efficient interaction and information security between the virtual power plant and the distribution network, can meet the needs of real-time economic scheduling in complex scenarios, and has strong scalability and adaptability.

[0110] See also Figure 1 The present invention provides a multi-level coordinated control method for a virtual power plant, comprising the following steps:

[0111] S1. Based on the active power boundary, power generation boundary and ramp rate boundary of distributed equipment, a unified adjustable space six-tuple modeling method is constructed to quantify the aggregated external characteristics and adjustable space of the virtual power plant;

[0112] 1. The aggregated external characteristics and adjustable space model of the virtual power plant are:

[0113] VPP middle DE At the moment The power generation characteristics can be expressed as a sextet General description: The six-tuple parameters represent the active power boundary, power generation boundary and ramp rate boundary of DE, as follows:

[0114]

[0115]

[0116]

[0117] in, Virtual Power Plant Distributed equipment At the moment The upper limit of power generation, Virtual Power Plant Distributed equipment At the moment The lower limit of power generation, Virtual Power Plant Distributed equipment At the moment The upper limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The lower limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The upper limit of the climbing rate, Virtual Power Plant Distributed equipment At the moment The lower limit of the climbing rate, It is a virtual power plant Distributed equipment The initial power, Virtual Power Plant Distributed equipment At the moment The power generation capacity, Virtual Power Plant Distributed equipment At the moment The cumulative power, Indicates the scheduling interval.

[0118] In the real-time scheduling of the system, internal distributed equipment needs to provide the virtual power plant with its adjustable capacity in advance.

[0119] Distributed equipment The adjustable capacity of is determined by the above six-tuple, as follows:

[0120]

[0121]

[0122]

[0123] in, and Virtual Power Plant Distributed equipment At the moment The upper and lower limits of the adjustable space when responding to the virtual power plant regulation instructions, Virtual Power Plant Distributed equipment At the moment Response adjustment.

[0124] The distributed devices in the virtual power plant respond to the decomposed dispatch instructions, and the dynamic response characteristics are as follows:

[0125]

[0126] in, It is a virtual power plant Distributed equipment At the moment Real-time power acquisition.

[0127] After each response cycle, the remaining capacity status of the distributed devices should be updated as follows:

[0128]

[0129] in, and represent the conversion efficiency of energy input and output respectively, Represents the time step.

[0130] 2. Adjustable spatial model of distributed equipment:

[0131] 1) Photovoltaic (PV)

[0132] By coordinating and managing the output of internal photovoltaic equipment, VPP can reduce its impact on the distribution network. The photovoltaic units connected to the VPP are equipped with maximum power point tracking technology, and the power collected in real time is regarded as the maximum power. , the power is continuously adjusted from 0 to , so the adjustable space of the photovoltaic unit is as follows:

[0133]

[0134]

[0135] in, Virtual Power Plant Photovoltaic unit At the moment The lower limit of power generation, Virtual Power Plant Photovoltaic unit At the moment The power at the maximum power point of the photovoltaic Virtual Power Plant Photovoltaic unit At the moment The upper limit of the photovoltaic power generation capacity, Virtual Power Plant Photovoltaic unit At the moment The upper limit of the adjustable space of the photovoltaic unit, Virtual Power Plant Photovoltaic unit At the moment The lower limit of the adjustable space of the photovoltaic unit, Virtual Power Plant Photovoltaic unit At the moment of photovoltaic power generation;

[0136] 2) Microturbine unit (MT):

[0137] The micro gas turbine includes a turbine, a high-speed generator, a rectifier and an inverter; the traditional MT mainly needs to consider its output power constraint, ramp rate constraint and energy constraint corresponding to gas consumption. Its adjustable space is expressed as:

[0138]

[0139]

[0140] in, Virtual Power Plant Micro gas turbine unit At the moment The real-time power, and Virtual Power Plant Micro gas turbine unit At the moment The power generation efficiency and heat loss coefficient, Virtual Power Plant Micro gas turbine unit At the moment The adjustable upper limit of Virtual Power Plant Micro gas turbine unit At the moment The upper limit of power generation, Virtual Power Plant Micro gas turbine unit At the moment The upper limit of the climbing rate, Virtual Power Plant Micro gas turbine unit At the moment The upper limit of gas consumption, Virtual Power Plant Micro gas turbine unit At the moment The current amount of gas consumed, Virtual Power Plant Micro gas turbine unit At the moment The power generation efficiency, represents the time step, Virtual Power Plant Micro gas turbine unit At the moment The adjustable lower limit of Virtual Power Plant Micro gas turbine unit At the moment The lower limit of power generation, Virtual Power Plant Micro gas turbine unit At the moment The lower limit of the climbing rate, Virtual Power Plant Micro gas turbine unit At the moment The lower limit of gas consumption.

[0141] 3) Energy Storage System (ESS):

[0142] The energy storage system needs to consider its power ramping constraints and energy state constraints during scheduling operations. The upper and lower limits of its adjustable space are:

[0143]

[0144]

[0145] in, Energy storage system The battery capacity, Virtual Power Plant Energy Storage System At the moment Real-time charging status, and Virtual Power Plant Energy Storage System At the moment Real-time charge / discharge power, and Represents energy storage system The discharge / charge efficiency, Virtual Power Plant Energy Storage System At the moment The energy storage adjustable power limit, Virtual Power Plant Energy Storage System At the moment The upper limit of energy storage power generation, Virtual Power Plant Energy Storage System At the moment The lower limit of the state of charge, Virtual Power Plant Energy Storage System At the moment The lower limit of the adjustable power of energy storage, Virtual Power Plant Energy Storage System At the moment The upper limit of discharge power, Virtual Power Plant Energy Storage System At the moment upper limit of state of charge.

[0146] 4) Building Air Conditioning (AC):

[0147] Air conditioning is a flexible resource with huge regulation potential. Its adjustable potential comes from the thermal storage characteristics of the building and the comfortable temperature range of the human body. Referring to the equivalent thermal model of air conditioning, the relationship between air conditioning cooling power and room parameters and temperature is established:

[0148]

[0149]

[0150] in, and Virtual Power Plant Medium air conditioning At the moment Maximum and minimum cooling power limited by room temperature, and denote the heat capacity and thermal conductivity of the room, respectively. and Respectively represent the highest and lowest indoor temperatures of the room, and Virtual Power Plant Medium air conditioning At the moment Real-time indoor and outdoor temperatures.

[0151] Since the scheduling of variable frequency air conditioners is based on the compressor frequency, the output power of the air conditioner is affected by the compressor frequency:

[0152]

[0153]

[0154] in, Virtual Power Plant Medium air conditioning At the moment The real-time power, and Respectively represent air conditioning The maximum and minimum frequencies, Indicates the upper limit of the adjustable power of the air conditioner. Indicates the lower limit of the adjustable power of the air conditioner. is the coefficient of a linear function.

[0155] 5) Interruptible load (IL):

[0156] The industrial load on the user side is regarded as a flexible resource in the economic dispatch of VPP, which can reduce the economic cost of VPP by reducing the controllable load. and non-adjustable load , the controllable load is mainly considered in the economic dispatch of VPP, as follows:

[0157]

[0158] in, Virtual Power Plant Medium interruptible load At the moment The real-time power, Virtual Power Plant Medium interruptible load At the moment The upper limit of the interruptible load regulation, Virtual Power Plant Medium interruptible load At the moment The lower limit of the interruptible load regulation.

[0159] S2. Based on the adjustable space model of the virtual power plant constructed in step S1 and combined with the network security constraints of the distribution network, a hierarchical twin delayed deep deterministic policy gradient method (Hierarchical-TD3) is designed. The upper layer uses the single-agent TD3 method to realize the economic dispatch of the distribution network to the virtual power plant, and the lower layer uses the multi-agent MATD3 method to realize the real-time response of distributed equipment in the virtual power plant.

[0160] S201. The upper-layer DN centralized online scheduling model based on the single-agent DRL method is:

[0161] In the implementation of the TD3 method, DN is used as the intelligent agent. In order to find the matching relationship between state-action pairs at any time, a strategy is introduced. represents the conditional probability distribution of each action when the current state is known. When known, the reward value can be obtained To guide the training of the agent, and use the gradient ascent method to update the strategy to maximize the reward value:

[0162]

[0163] Therefore, define a state value function To evaluate the state performance of the parameterized neural network, specifically:

[0164]

[0165] During the training process, the goal of the agent is to choose the best strategy to maximize the state value function, specifically:

[0166]

[0167] It consists of two neural networks, using the judgment network Simulate the real state value function and use the policy network Get the deterministic strategy; the update strategy of these two neural networks is expressed as:

[0168]

[0169]

[0170] in, Represents the current actual state value function, is the partial derivative of the state value function with respect to the network parameters, is the expectation of the state-value function deviation.

[0171] Finally, a soft update strategy will be used to update the network parameters, namely:

[0172]

[0173]

[0174] in, is the soft update parameter, is the updated parameter of the evaluation network, is the parameter of the evaluation network before updating, are the parameters of the updated policy network, are the parameters of the policy network before updating.

[0175] The action space, state space, environment and reward function of the single-agent TD3 method are defined as:

[0176] (1) Action Space :

[0177] Power adjustment of each VPP, where For DN at time Issued to VPP Active power dispatch instructions.

[0178] (2) State space :

[0179] The real-time actual power and adjustable range of each VPP are as follows:

[0180] ;

[0181] in, Virtual Power Plant i At the moment t The actual power, Virtual Power Plant i At the moment t The adjustable space limit, Virtual Power Plant i At the moment t The lower limit of the adjustable space.

[0182] (3) Environment:

[0183] During offline training, the policy network will be based on To make decisions , Virtual Power Plant i At the moment t After the DN power flow model obtains the injected power of each VPP node, it obtains the power purchase cost and network loss cost as the reward value. The judgment network will evaluate the action and guide the agent to the next state, and then the agent will adjust the strategy until the end of the training phase.

[0184] (4) Reward value :

[0185] The reward value obtained by DN during training Including network loss costs, electricity purchase costs and total VPP operating costs:

[0186]

[0187] in, is the total operating cost of the virtual power plant, is the network loss cost, The cost of purchasing electricity, is the total number of virtual power plants.

[0188] S202. The lower-layer VPP decentralized online scheduling model based on the multi-agent DRL method is constructed as follows:

[0189] During the offline training phase, VPP will aggregate the data information of DE and train the device agents centrally. The evaluation network of each agent will estimate the strategies of other agents to obtain more accurate evaluation information. The model trained based on offline data can be deployed on DE. Each device agent only needs to independently adjust the scheduling strategy according to its own real-time status, without passing data to VPP, which can effectively reduce the burden of data communication, and is therefore more suitable for actual scheduling scenarios.

[0190] The system includes agents, then the state space It is the overall state information. The policy network obtains a larger expected cumulative reward by continuously updating its parameters, which is defined as:

[0191]

[0192] in, is the state value function in the centralized mode, including the joint actions of all agents, but each agent only needs to consider its own learning process and update itself. It is the experience pool that stores all training experiences.

[0193] The critic network updates its parameters by minimizing the temporal difference error, defined as:

[0194]

[0195]

[0196] in, is the expectation of the state-value function deviation, is the current actual state value function, are known self-set parameters, -y is the deviation of the state value function.

[0197] Finally, all target networks use soft updates to periodically update the network parameters of the evaluation network. , defined as:

[0198]

[0199] in, is a soft update parameter and .

[0200] The action space, state space, environment and reward function of the multi-agent TD3 method are defined as:

[0201] (1) State space :

[0202] It consists of the real-time output, energy state, up and down adjustment space and DN scheduling instructions of different DEs aggregated in VPP. The state space of any DE in VPP The definition is as follows:

[0203]

[0204] in, Indicates at time Send to Active power dispatch instructions for virtual power plants, Virtual Power Plant A device with a certain type of distributed resource At the moment The power, Virtual Power Plant A device with a certain type of distributed resource At the moment In addition to power, other required parameters, such as energy storage charge state, indoor temperature where the air conditioner is located, etc. Virtual Power Plant A device with a certain type of distributed resource At the moment The adjustable space limit, Virtual Power Plant A device with a certain type of distributed resource At the moment The lower limit of the adjustable space, Virtual Power Plant Energy Storage System At the moment The state of charge, Virtual Power Plant Air conditioning equipment At the moment Indoor temperature, Virtual Power Plant Photovoltaic equipment At the moment The power generation capacity, Virtual Power Plant Photovoltaic equipment At the moment The adjustable space limit, Virtual Power Plant Photovoltaic equipment At the moment The lower limit of the adjustable space, Virtual Power Plant Energy Storage System At the moment The power generation capacity, Virtual Power Plant Energy Storage System At the moment The adjustable space limit, Virtual Power Plant Energy Storage System At the moment The lower limit of the adjustable space, Virtual Power Plant Micro gas turbine At the moment The power generation capacity, Virtual Power Plant Micro gas turbine At the moment The adjustable space limit, Virtual Power Plant Micro gas turbine At the moment The lower limit of the adjustable space, Virtual Power Plant Air conditioning equipment At the moment The power consumption, Virtual Power Plant Air conditioning equipment At the moment The adjustable space limit, Virtual Power Plant Medium air conditioning At the moment The lower limit of the adjustable space, Virtual Power Plant Medium interruptible load At the moment The power, Virtual Power Plant Medium interruptible load At the moment The upper limit of adjustment, Virtual Power Plant Medium interruptible load At the moment The lower limit of adjustment, Virtual Power Plant China Photovoltaic At the moment The state space of Virtual Power Plant Energy Storage System At the moment The state space of Virtual Power Plant Micro gas turbine At the moment The state space of Virtual Power Plant Air conditioning equipment At the moment The state space of Virtual Power Plant Medium interruptible load At the moment The state space of;.

[0205] (2) Action space:

[0206] The action space of DE in the virtual power plant corresponds to the decision variables in the optimization scheduling model, that is, the adjustment output of different DE. The relationship between the adjustment rate of various distributed resources and the scheduling instructions is described as follows:

[0207]

[0208] Among them, DER refers to a certain type of distributed resources. Virtual Power Plant Distributed equipment At the moment Adjustment instructions, Virtual Power Plant Distributed equipment At the moment The lower limit of the adjustable space, Virtual Power Plant Distributed equipment At the moment The adjustment rate, Virtual Power Plant Distributed equipment At the moment The upper limit of the adjustable space, the distributed devices represented in the formula refer to the devices included in this type of resources;

[0209] (3) Environment:

[0210] During the training process of the multi-agent DRL method, the policy network will take actions based on the current state, which is equivalent to determining the output of DE. Then, the VPP operating environment model will feedback the cost of each device agent and then move to the next state until the training is completed.

[0211] (4) Reward value :

[0212] The reward value will guide the agent to choose actions. In the MATD3 method, the reward value of each agent includes the running cost of each DE and the power deviation cost of the DN scheduling instruction.

[0213]

[0214] in, Virtual Power Plant A device with a certain type of distributed resource At the moment The running cost, Virtual Power Plant At the moment The power deviation cost from the distribution network dispatch instructions.

[0215] S3. Based on the hierarchical double-delay deep deterministic policy gradient method proposed in step S2, the training process of the hierarchical DRL method under the centralized-decentralized scheduling framework is designed. In the offline stage, the intelligent agent uses historical data for strategy learning, and in the online stage, real-time scheduling is performed through soft updates, realizing coordinated regulation of the upper and lower layers of the system, meeting the needs of real-time response and privacy protection.

[0216] The training process of the hierarchical DRL method under the centralized-decentralized scheduling framework is:

[0217] During the offline training phase, the VPP in the DN will collect the adjustable spatial information of the aggregated DE , forming its external features ;

[0218] Then, the external features are passed to the upper-layer DN as the environment state, and the DN will determine the scheduling output of each VPP based on the current operating state. , and transmit it to each VPP in the lower layer;

[0219] The DE agent in the VPP responds to the scheduling instructions sent by the upper layer and the running status of the DE. , for the actual output Make a decision and obtain the operating cost of training, where Virtual power plant for offline training phase Distributed equipment At the moment The output decision Indicates at time Send to Active power dispatch instructions for virtual power plants, Virtual Power Plant A device with a certain type of distributed resource At the moment The power, Virtual Power Plant A device with a certain type of distributed resource At the moment In addition to power, other required parameters, such as energy storage charge state, indoor temperature where the air conditioner is located, etc. Virtual Power Plant A device with a certain type of distributed resource At the moment The adjustable space limit, Virtual Power Plant A device with a certain type of distributed resource At the moment At the same time, the total operating cost of the VPP, as well as the network loss cost and the electricity purchase cost will form the upper-layer reward value and be fed back to the DN for training until the end of the training phase.

[0220] In the online application phase, the trained upper-layer DN agent is trained based on its aggregated external features. Publish real-time scheduling plans for each VPP , and the lower-level DE agent follows the scheduling instructions sent by the upper layer and the real-time environmental status it collects Determines its real-time output This achieves coordinated regulation between the upper and lower levels.

[0221] The present invention also provides a virtual power plant multi-level coordination and control system, which can be used to implement the above-mentioned virtual power plant multi-level coordination and control method. Specifically, the virtual power plant multi-level coordination and control system includes a construction module, a design module and a control module.

[0222] Among them, the construction module, based on the active power boundary, power generation boundary and ramp rate boundary of distributed equipment, adopts the adjustable space six-tuple modeling method to obtain the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment;

[0223] Design module, based on the aggregate external characteristics and adjustable space model of the constructed virtual power plant, as well as the adjustable space model of each distributed device, combined with the network security constraints of the distribution network, construct a hierarchical double-delay deep deterministic policy gradient method, and obtain the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL, which are used to realize the economic dispatch of the distribution network to the virtual power plant and the real-time response of the distributed devices in the virtual power plant respectively;

[0224] The control module adopts the hierarchical DRL method under the centralized-decentralized scheduling framework to train the obtained single-agent DRL upper-layer distribution network centralized online scheduling model and the multi-agent DRL lower-layer virtual power plant decentralized online scheduling model. In the offline stage, historical data is used for strategy learning, and real-time scheduling is performed through soft updates in the online stage to achieve coordinated control of the upper and lower layers of the system.

[0225] The present invention also provides a terminal device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the multi-level coordinated control method of a virtual power plant, including:

[0226] Based on the active power boundary, power generation boundary and ramp rate boundary of distributed equipment, the adjustable space six-tuple modeling method is used to obtain the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment; based on the constructed aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment, combined with the network security constraints of the distribution network, a hierarchical double-delay deep deterministic policy gradient method is constructed to obtain the upper-layer distribution network centralized online dispatching model of the single-agent DRL and the lower-layer virtual power plant decentralized online dispatching model of the multi-agent DRL, which are used to realize the economic dispatch of the distribution network to the virtual power plant and the real-time response of the distributed equipment in the virtual power plant respectively; the hierarchical DRL method under the centralized-decentralized dispatching framework is used to train the obtained single-agent DRL upper-layer distribution network centralized online dispatching model and the multi-agent DRL lower-layer virtual power plant decentralized online dispatching model, and historical data are used for strategy learning in the offline stage. In the online stage, real-time dispatching is performed through soft updates to realize coordinated regulation of the upper and lower layers of the system.

[0227] See also Figure 4, the terminal device is a computer device. The computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the method for calculating the fluid composition in the reservoir transformation wellbore in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the multi-level coordinated control system of the virtual power plant in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0228] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 4 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0229] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0230] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0231] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0232] See also Figure 5 The terminal device 600 is an electronic device, and the electronic device is in the form of a general computing device. The components of the electronic device may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0233] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0234] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0235] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0236] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0237] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0238] The present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory.

[0239] One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the corresponding steps of the multi-level coordinated control method of a virtual power plant in the above embodiment; one or more instructions in the computer-readable storage medium may be loaded and executed by a processor as follows:

[0240] Based on the active power boundary, power generation boundary and ramp rate boundary of distributed equipment, the adjustable space six-tuple modeling method is used to obtain the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment; based on the constructed aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment, combined with the network security constraints of the distribution network, a hierarchical double-delay deep deterministic policy gradient method is constructed to obtain the upper-layer distribution network centralized online dispatching model of the single-agent DRL and the lower-layer virtual power plant decentralized online dispatching model of the multi-agent DRL, which are used to realize the economic dispatch of the distribution network to the virtual power plant and the real-time response of the distributed equipment in the virtual power plant respectively; the hierarchical DRL method under the centralized-decentralized dispatching framework is used to train the obtained single-agent DRL upper-layer distribution network centralized online dispatching model and the multi-agent DRL lower-layer virtual power plant decentralized online dispatching model, and historical data are used for strategy learning in the offline stage. In the online stage, real-time dispatching is performed through soft updates to realize coordinated regulation of the upper and lower layers of the system.

[0241] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0242] The multi-level coordinated control method of virtual power plants based on hierarchical security deep reinforcement learning has been applied in actual operation scenarios. In order to highlight the compatibility and scalability of the proposed method, a case study is designed. The test case is a modified IEEE-33 node system, which contains two virtual power plants connected to nodes 14 and 30 respectively. Each virtual power plant aggregates photovoltaic units, energy storage, micro gas turbines, air conditioners and interruptible loads.

[0243] See also Figure 2, the figure shows the changes in round rewards of different algorithms during the training process of the virtual power plant dispatching system. As the number of rounds increases, the virtual power plant agent based on the hierarchical TD3 method gradually converges after about 50 rounds, and the total reward value remains stable after convergence, indicating that the agent has learned a near-optimal scheduling strategy. In contrast, the centralized TD3 and distributed MATD3 methods converge slowly during the training process, and the total reward value fluctuates greatly, especially the centralized TD3 method in the early random exploration stage, the reward value fluctuates violently. This shows that the centralized scheduling method is less efficient when facing large-scale complex environments, while the hierarchical TD3 method achieves a faster and more stable convergence effect through hierarchical optimization scheduling, further verifying the advantages of this algorithm in multi-level coordinated control of virtual power plants.

[0244] In order to verify the effectiveness of the hierarchical TD3 method in reducing the operating costs of virtual power plants and optimizing scheduling efficiency, the present invention designs three different scenarios to evaluate the impact of different scheduling methods on costs.

[0245] Scenario 1: Using the hierarchical TD3 method for overall scheduling of virtual power plants.

[0246] Scenario 2: Use the decentralized MATD3 method for independent device scheduling.

[0247] Scenario 3: Use the centralized TD3 method for global unified scheduling.

[0248] Figure 3 The test results of the three scenarios are compared. Compared with scenario 1, the total operating costs of scenarios 2 and 3 increased by 1.77% and 4.51%, respectively, indicating that the hierarchical TD3 method is effective in reducing the total operating costs of virtual power plants. In addition, the hierarchical TD3 method achieves the economy of overall scheduling and ensures the real-time response of each distributed device through hierarchical optimization, greatly improving the scheduling efficiency. In contrast, the decentralized MATD3 method in scenario 2 lacks global unified scheduling, resulting in poor scheduling coordination of the system, thereby increasing operating costs.

[0249] In scenario 3, since the centralized TD3 method needs to process the scheduling data of a large number of distributed devices in the virtual power plant, its computational complexity is high and the convergence speed is slow, resulting in a significant increase in the total operating cost. In addition, the scheduling accuracy and efficiency of the centralized TD3 method are also affected, and it is unable to quickly adapt to changes in the system, further pushing up the cost of purchasing electricity and network loss costs. Overall, the scheduling performance of the centralized scheduling method in large-scale systems is obviously inferior to the hierarchical TD3 method proposed in this invention.

[0250] In summary, the multi-level coordinated control method and system of a virtual power plant of the present invention significantly reduces the operating cost of the virtual power plant and improves the dispatching efficiency through hierarchical optimization and real-time response of distributed equipment; the method of the present invention can not only converge quickly in a complex system environment, but also realize efficient coordinated dispatching between the virtual power plant and the distribution network, which is of great significance for promoting the optimization of the energy management system and reducing the operating costs.

[0251] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0252] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0253] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0254] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0255] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0256] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0257] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0258] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0259] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0260] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0261] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A multi-level coordinated control method for a virtual power plant, characterized in that: The following steps are included Based on the active power boundary, power generation boundary and ramp rate boundary of distributed equipment, the adjustable space six-tuple modeling method is adopted to obtain the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment. The aggregated external characteristics and adjustable space model of the virtual power plant are as follows: Using sextuples Describing Virtual Power Plants Distributed equipment At the moment The power generation characteristics of the six-element Determine distributed equipment The distributed devices in the virtual power plant respond to the decomposed dispatch instructions, and after each response cycle, the remaining capacity status of the distributed devices is updated, where: Virtual Power Plant Distributed equipment At the moment The upper limit of power generation, Virtual Power Plant Distributed equipment At the moment The lower limit of power generation, Virtual Power Plant Distributed equipment At the moment The upper limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The lower limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The upper limit of the climbing rate, Virtual Power Plant Distributed equipment At the moment The lower limit of the climbing rate; Based on the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed device, combined with the network security constraints of the distribution network, a hierarchical double-delay deep deterministic policy gradient method is constructed to obtain the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL, which are used to realize the economic dispatch of the distribution network to the virtual power plant and the real-time response of the distributed devices in the virtual power plant respectively. The hierarchical DRL method under the centralized-decentralized scheduling framework is adopted to train the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL. In the offline stage, historical data is used for strategy learning, and real-time scheduling is performed through soft updates in the online stage to achieve coordinated regulation of the upper and lower layers of the system.

2. The multi-level coordinated control method of a virtual power plant according to claim 1 is characterized in that: The active power boundary, power generation boundary and ramp rate boundary of distributed equipment are as follows: in, Virtual Power Plant Distributed equipment At the moment The power generation capacity, Virtual Power Plant Distributed equipment At the moment The power generation capacity, represents the scheduling interval, It is a virtual power plant Distributed equipment The initial power.

3. The multi-level coordinated control method of a virtual power plant according to claim 1 is characterized in that: Distributed equipment The adjustable capacity is as follows: in, Virtual Power Plant Distributed equipment At the moment The power generation capacity, and Virtual Power Plant Distributed equipment At the moment The upper and lower limits of the adjustable space when responding to the virtual power plant regulation instructions, Virtual Power Plant Distributed equipment At the moment The response adjustment, is the energy conversion efficiency of the virtual power plant, is the time step.

4. The multi-level coordinated control method of a virtual power plant according to claim 1, characterized in that: Adjustable spatial models for distributed equipment include: Adjustable spatial model of photovoltaic units: in, Virtual Power Plant Photovoltaic unit At the moment The lower limit of power generation, Virtual Power Plant Photovoltaic unit At the moment The power at the maximum power point of the photovoltaic Virtual Power Plant Photovoltaic unit At the moment The upper limit of the photovoltaic power generation capacity, Virtual Power Plant Photovoltaic unit At the moment The upper limit of the adjustable space of the photovoltaic unit, Virtual Power Plant Photovoltaic unit At the moment The lower limit of the adjustable space of the photovoltaic unit, Virtual Power Plant Photovoltaic unit At the moment of photovoltaic power generation; Micro gas turbine unit adjustable space model: in, Virtual Power Plant Micro gas turbine unit At the moment The real-time power, and Virtual Power Plant Micro gas turbine unit At the moment The power generation efficiency and heat loss coefficient, Virtual Power Plant Micro gas turbine unit At the moment The adjustable upper limit of Virtual Power Plant Micro gas turbine unit At the moment The upper limit of power generation, Virtual Power Plant Micro gas turbine unit At the moment The upper limit of the climbing rate, Virtual Power Plant Micro gas turbine unit At the moment The upper limit of gas consumption, Virtual Power Plant Micro gas turbine unit At the moment The current amount of gas consumed, Virtual Power Plant Micro gas turbine unit At the moment The power generation efficiency, represents the time step, Virtual Power Plant Micro gas turbine unit At the moment The adjustable lower limit of Virtual Power Plant Micro gas turbine unit At the moment The lower limit of power generation, Virtual Power Plant Micro gas turbine unit At the moment The lower limit of the climbing rate, Virtual Power Plant Micro gas turbine unit At the moment The lower limit of gas consumption; Adjustable space model of energy storage system: in, Energy storage system The battery capacity, Virtual Power Plant Energy Storage System At the moment Real-time state of charge, and Virtual Power Plant Energy Storage System At the moment Real-time charge / discharge power, and Represents energy storage system The discharge / charge efficiency, Virtual Power Plant Energy Storage System At the moment The energy storage adjustable power limit, Virtual Power Plant Energy Storage System At the moment The upper limit of energy storage power generation, Virtual Power Plant Energy Storage System At the moment The lower limit of the state of charge, Virtual Power Plant Energy Storage System At the moment The lower limit of the adjustable power of energy storage, Virtual Power Plant Energy Storage System At the moment The upper limit of discharge power, Virtual Power Plant Energy Storage System At the moment The upper limit of the state of charge; Building air conditioning adjustable space model: in, and Virtual Power Plant Medium air conditioning At the moment Maximum and minimum cooling power limited by room temperature, and Respectively represent air conditioning The heat capacity and thermal conductivity of the room, and Respectively represent air conditioning The maximum and minimum indoor temperature of the room, and Virtual Power Plant Medium air conditioning At the moment Real-time indoor and outdoor temperatures; Output power of air conditioner: in, Virtual Power Plant Medium air conditioning At the moment The real-time power, and Respectively represent air conditioning The maximum and minimum frequencies, Indicates the upper limit of the adjustable power of the air conditioner. Indicates the lower limit of the adjustable power of the air conditioner. is the coefficient of a linear function; Interruptible load adjustable space model: in, Virtual Power Plant Medium interruptible load At the moment The real-time power, Virtual Power Plant Medium interruptible load At the moment The upper limit of the interruptible load regulation, Virtual Power Plant Medium interruptible load At the moment The lower limit of the interruptible load regulation.

5. The multi-level coordinated control method of a virtual power plant according to claim 1, characterized in that: The upper distribution network centralized online dispatching model of single-agent DRL is as follows: Taking the distribution network as an intelligent entity, introducing strategies represents the conditional probability distribution of each action when the current state is known, where Initial state The probability distribution of To indicate the status Take action The conditional probability of To indicate the status Next action After that, the system moves to the next state The conditional probability of Get the reward value when it is known Guide the training of the agent and use the gradient ascent method to update the strategy to maximize the reward value; define the state value function Evaluate the state performance of the parameterized neural network; during the training process, the goal of the agent is to select the best strategy to maximize the state value function; use the judgment network to simulate the real state value function, and use the policy network to obtain a deterministic strategy; finally, use a soft update strategy to update the network parameters; The decentralized online dispatch model of the lower virtual power plant of multi-agent DRL is as follows: The virtual power plant aggregates data information from distributed devices and trains device agents centrally. The evaluation network of each device agent estimates the strategies of other agents to obtain more accurate evaluation information. The model trained based on offline data is deployed on distributed devices, and each device agent independently adjusts the scheduling strategy according to its real-time status. The strategy network obtains a greater expected cumulative reward by continuously updating parameters. The evaluation network updates parameters by minimizing the time difference error. Finally, all target networks use soft updates to regularly update the network parameters of the evaluation network.

6. The multi-level coordinated control method of a virtual power plant according to claim 5 is characterized in that: Action space of the centralized online dispatching model of the upper distribution network based on single-agent DRL , state space , environment and reward function They are: Action Space ,in, For the distribution network at time Virtual Power Plant Active power dispatch instructions, Virtual Power Plant At the moment The scheduling output decision of is the set of virtual power plants in the region; State Space , the real-time actual power of each virtual power plant and its adjustable space, Virtual Power Plant At the moment The state space of During offline training, the policy network is based on After making a decision, the power flow model of the distribution network obtains the injected power of each virtual power plant node, and obtains the power purchase cost and network loss cost as the reward value. The judgment network will evaluate the action and guide the agent to the next state, and then the agent will adjust the strategy until the end of the training phase; The reward function obtained by the distribution network during the training process for: in, is the total operating cost of the virtual power plant, is the network loss cost, The cost of purchasing electricity, is the total number of virtual power plants.

7. The multi-level coordinated control method of a virtual power plant according to claim 5, characterized in that: Action space of decentralized online scheduling model of virtual power plant under multi-agent DRL , state space , environment and reward function They are: State Space ,in, is the total number of virtual power plants, is the total number of types of distributed resources in all virtual power plants; State space of any distribution network in a virtual power plant The definition is as follows: in, Indicates at time Send to Active power dispatch instructions for virtual power plants, Virtual Power Plant Distributed resources in the device At the moment The power, Virtual Power Plant Distributed resources in the device At the moment In addition to power, other required parameters, Virtual Power Plant Distributed equipment At the moment The adjustable space limit, Virtual Power Plant Distributed equipment At the moment The lower limit of the adjustable space, Virtual Power Plant Energy Storage System At the moment The state of charge, Virtual Power Plant Air conditioning equipment At the moment The indoor temperature Virtual Power Plant Photovoltaic equipment At the moment The power generation capacity, Virtual Power Plant Photovoltaic equipment At the moment The adjustable space limit, Virtual Power Plant Photovoltaic equipment At the moment The lower limit of the adjustable space, Virtual Power Plant Energy Storage System At the moment The power generation capacity, Virtual Power Plant Energy Storage System At the moment The adjustable space limit, Virtual Power Plant Energy Storage System At the moment The lower limit of the adjustable space, Virtual Power Plant Micro gas turbine At the moment The power generation capacity, Virtual Power Plant Micro gas turbine At the moment The adjustable space limit, Virtual Power Plant Micro gas turbine At the moment The lower limit of the adjustable space, Virtual Power Plant Air conditioning equipment At the moment The power consumption, Virtual Power Plant Air conditioning equipment At the moment The adjustable space limit, Virtual Power Plant Medium air conditioning At the moment The lower limit of the adjustable space, Virtual Power Plant Medium interruptible load At the moment The power, Virtual Power Plant Medium interruptible load At the moment The upper limit of adjustment, Virtual Power Plant Medium interruptible load At the moment The lower limit of adjustment, Virtual Power Plant China Photovoltaic At the moment The state space of Virtual Power Plant Energy Storage System At the moment The state space of Virtual Power Plant Micro gas turbine At the moment The state space of Virtual Power Plant Air conditioning equipment At the moment The state space of Virtual Power Plant Medium interruptible load At the moment The state space of The relationship between the adjustment rate of various distributed resources and the scheduling instructions is described as follows: Among them, DER refers to distributed equipment, Virtual Power Plant Distributed equipment At the moment Adjustment instructions, Virtual Power Plant Distributed equipment At the moment The lower limit of the adjustable space, Virtual Power Plant Distributed equipment At the moment The adjustment rate, Virtual Power Plant Distributed equipment At the moment The adjustable space limit; The policy network will take actions based on the current state to determine the output of the distributed equipment. Then, the virtual power plant operating environment model will feedback the cost of each equipment agent and then move to the next state until the training is completed. Reward Function as follows: in, Virtual Power Plant Distributed resources in the device At the moment The running cost, Virtual Power Plant At the moment The power deviation cost from the distribution network dispatch instructions.

8. The multi-level coordinated control method of a virtual power plant according to claim 1, characterized in that: The hierarchical DRL method under the centralized-decentralized scheduling framework is used to train the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL. Specifically: During the offline training phase, the virtual power plant in the distribution network collects and aggregates the adjustable spatial information of distributed resources. , Virtual Power Plant Distributed equipment At the moment The adjustable space limit, Virtual Power Plant Distributed equipment At the moment The adjustable space lower limit forms the external features , Virtual Power Plant At the moment The upper limit of the adjustable space of the external characteristics, Virtual Power Plant At the moment The lower limit of the adjustable space of the external characteristics; External features The environmental status is transmitted to the upper distribution network, which determines the dispatch output of each virtual power plant based on the current operating status. , and transmitted to each virtual power plant in the lower layer, Virtual Power Plant At the moment The scheduling output decision of For the upper power grid according to the time The operating state is a virtual power plant The scheduling output of the decision; The distributed equipment agents in the virtual power plant respond to the dispatching instructions sent by the upper layer and the operating status of the distributed equipment. , for the actual output Make decisions and obtain the operating cost of training; the total operating cost of the virtual power plant, network loss cost and power purchase cost will form the upper-level reward value and be fed back to the distribution network for training until the end of the training phase. Virtual power plant for offline training phase Distributed equipment At the moment The output decision Indicates at time Send to Active power dispatch instructions for virtual power plants, Virtual Power Plant Distributed resources in the device At the moment The power, Virtual Power Plant Equipment with certain distributed resources At the moment In addition to power, other required parameters, Virtual Power Plant Distributed resources in the device At the moment The adjustable space limit, Virtual Power Plant Distributed resources in the device At the moment The lower limit of adjustable space; In the online application stage, the trained upper-layer distribution network agent is used to aggregate external characteristics. Publish real-time dispatch plans for each virtual power plant , the lower-level distribution network intelligent agent responds to the dispatching instructions sent by the upper layer and the real-time environmental status collected Determine real-time output , to achieve coordinated regulation between upper and lower levels, Virtual Power Plant At the moment The state space of is the set of virtual power plants in the region, Virtual Power Plant Distributed equipment At the moment The state space of is a distributed set of devices in a virtual power plant. Virtual Power Plant for Online Application Distributed equipment At the moment Real-time output decision making.

9. A multi-level coordinated control system for a virtual power plant, characterized in that: include: The construction module uses the adjustable space six-tuple modeling method based on the active power boundary, power generation boundary and ramp rate boundary of distributed equipment to obtain the aggregated external characteristics and adjustable space model of the virtual power plant, as well as the adjustable space model of each distributed equipment. The aggregated external characteristics and adjustable space model of the virtual power plant are as follows: Using sextuples Describing Virtual Power Plants Distributed equipment At the moment The power generation characteristics of the six-element Determine distributed equipment The distributed devices in the virtual power plant respond to the decomposed dispatch instructions, and after each response cycle, the remaining capacity status of the distributed devices is updated, where: Virtual Power Plant Distributed equipment At the moment The upper limit of power generation, Virtual Power Plant Distributed equipment At the moment The lower limit of power generation, Virtual Power Plant Distributed equipment At the moment The upper limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The lower limit of the cumulative power, Virtual Power Plant Distributed equipment At the moment The upper limit of the climbing rate, Virtual Power Plant Distributed equipment At the moment The lower limit of the climbing rate; Design module, based on the aggregate external characteristics and adjustable space model of the constructed virtual power plant, as well as the adjustable space model of each distributed device, combined with the network security constraints of the distribution network, construct a hierarchical double-delay deep deterministic policy gradient method, and obtain the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL, which are used to realize the economic dispatch of the distribution network to the virtual power plant and the real-time response of the distributed devices in the virtual power plant respectively; The control module adopts the hierarchical DRL method under the centralized-decentralized scheduling framework to train the upper distribution network centralized online scheduling model of the single-agent DRL and the lower virtual power plant decentralized online scheduling model of the multi-agent DRL. In the offline stage, historical data is used for strategy learning, and real-time scheduling is performed through soft updates in the online stage to achieve coordinated control of the upper and lower layers of the system.

Images