Multi-agent reinforcement learning driven park power guarantee self-adaptive cooperation method and multi-agent reinforcement learning driven park power guarantee self-adaptive cooperation system

Abstract

The invention discloses a multi-agent reinforcement learning driven park power protection self-adaptive cooperation method and system. The method comprises the following steps: S1, establishing a multi-agent reinforcement learning simulation environment model of a park power system; s2, on the basis of the simulation environment model, a reinforcement learning algorithm is adopted to carry out parallel cooperative training on multiple agents, and a multi-agent strategy model is obtained; s3, deploying the multi-agent strategy model in a park edge control device in a distributed manner, independently making a decision by each agent according to system state information sensed in real time, and self-organizing to emerge a global adaptive power guarantee control strategy through implicit collaboration among the agents; and S4, issuing an execution power guarantee control strategy, monitoring a system operation state in real time to collect feedback data, and performing evaluation and optimization based on the feedback data to form a closed-loop adaptive process. According to the invention, interaction between the reinforcement learning algorithm and multiple agents is utilized to automatically generate the power protection strategy adaptive to dynamic changes.

Description

Technical Field

[0001] This invention relates to the fields of power system operation control and artificial intelligence, specifically to a multi-agent reinforcement learning-driven adaptive collaborative method and system for power supply protection in industrial parks. Background Technology

[0002] During major sporting events, cultural activities, or political summits, the power systems of venues / parks face severe challenges, including numerous temporary access devices, rapid topology changes, and large and uncertain load fluctuations. National energy authorities and power grid companies require the safe and stable operation of the power grid during these events, ensuring no power outages for key users and rapid fault handling, placing stringent demands on the intelligence and timeliness of power supply protection strategies. However, current traditional power supply protection methods for parks mainly rely on manual inspections and pre-established fixed plans, which suffer from problems such as slow response and insufficient coordination in practical applications.

[0003] Specifically, existing power supply protection strategies have two major flaws:

[0004] (1) High dependence on human intervention: When faced with rapid disturbances such as temporary equipment access and sudden load increases, the fixed contingency plan library is difficult to adapt and adjust in real time. It still relies heavily on human experience for decision-making and intervention, resulting in a slow response speed and failing to meet the real-time requirements of highly dynamic scenarios.

[0005] (2) Insufficient cross-device / cross-regional coordination: In the existing strategy, each control unit (such as distributed power supply, energy storage, load switch, etc.) often operates independently and lacks a global coordination mechanism, making it difficult to achieve optimized coordination across devices and regions. Especially when communication bandwidth and computing power are limited, the centralized control scheme cannot make real-time collaborative decisions for a large number of devices.

[0006] The aforementioned problems make traditional power supply strategies ill-suited for real-time collaborative control and adaptive adjustment requirements in high-uncertainty scenarios. To address this, this invention proposes an intelligent emergence method based on multi-agent model reinforcement learning. This method achieves decentralized, self-organizing collaborative control by constructing a high-fidelity environment model, parallel training of multiple agents, distributed policy reasoning, and closed-loop feedback optimization. Summary of the Invention

[0007] The technical problem this invention aims to solve is that existing power supply strategies for industrial parks struggle to achieve near real-time collaborative control and adaptive adjustment when dealing with highly uncertain scenarios such as numerous temporary equipment connections, rapid topology changes, and large load fluctuations. The goal is to provide a multi-agent reinforcement learning-driven adaptive collaborative power supply method for industrial parks, which solves the technical problem of existing solutions being unable to achieve near real-time collaboration and adaptation.

[0008] This invention is achieved through the following technical solution:

[0009] Firstly, this application provides a multi-agent reinforcement learning-driven adaptive collaborative method for power supply protection in a campus, comprising the following steps:

[0010] A multi-agent reinforcement learning simulation environment model for the park's power system is established. The simulation environment model includes a state space defined based on the physical topology and operating characteristics of the park's power grid, an action space defined based on the control permissions and capabilities of each controllable unit within the park, and a multi-objective reward function for quantifying business objectives.

[0011] Based on the simulation environment model, a reinforcement learning algorithm is used to perform parallel collaborative training on multiple agents to obtain a multi-agent policy model.

[0012] The multi-agent strategy model is deployed in a distributed manner in the edge control equipment of the park. Each agent makes independent decisions based on the real-time perceived system status information. Through implicit collaboration among agents, a globally adaptive power supply control strategy emerges in a self-organized manner.

[0013] The power supply control strategy is issued and executed, and the system operation status is monitored in real time to collect feedback data. The effect of the strategy is evaluated and optimized based on the feedback data to form a closed-loop adaptive process.

[0014] A further optimization scheme is to define the state space S of the reinforcement learning agent, based on the physical topology and operating characteristics of the park's power grid, and the state variables... As shown in the following formula:

[0015] ;

[0016] In the formula, Represents the voltage magnitude vector of each key bus; This represents the voltage phase angle vector of each key busbar; Indicates the system frequency; These represent the active and reactive power output vectors of each distributed power source, respectively. Indicates the state of charge of the energy storage system; This represents the active power vector of each load; The state of each switch or circuit breaker is represented by a binary vector. This represents a vector of environmental parameters.

[0017] A further optimized solution is to define a joint action space A for the intelligent agent based on the control permissions and capabilities of each controllable unit within the park. The executable control instructions can be represented as:

[0018] ;

[0019] In the formula, For the actions of photovoltaic intelligent agents; For the actions of wind power intelligent agents; For the actions of the generator intelligent agent; For the actions of energy storage intelligent agents; To control the actions of the intelligent agent under load; To control the actions of the intelligent agent.

[0020] A further optimization scheme is as follows: the multi-objective reward function for:

[0021] ;

[0022] in, These are the weighting coefficients. As a reward for power supply reliability, As a reward for safe operation, As an economic reward, To constrain the use of punishment and rewards.

[0023] A further optimized solution is proposed, with the calculation formulas for the power supply reliability reward, safe operation reward, economic reward, and constraint penalty reward as shown in the following equations:

[0024] ;

[0025] In the formula, For the set of load nodes; For the set of busbars; A collection of transmission lines; It is a collection of diesel generators and mobile generator vehicles; This is the weighting coefficient for the load; the higher the priority, the larger the value. These are the voltage and upper limit of the j-th bus, respectively; These are the current and upper limit of the k-th line, respectively; This refers to the amount of new energy consumed; This is the energy storage operation cost coefficient; The charging and discharging power for energy storage; This is the fuel cost coefficient; For the active power output of the nth diesel generator or mobile generator vehicle; This is the penalty coefficient; , These represent the upper and lower limits of the state of charge of the energy storage system.

[0026] A further optimized solution involves establishing a multi-agent reinforcement learning simulation environment model for the park's power system. This simulation environment model includes a state space defined based on the park's power grid physical topology and operational characteristics, an action space defined based on the control permissions and capabilities of each controllable unit within the park, and a multi-objective reward function for quantifying business objectives. The solution also includes the following steps:

[0027] Noise that simulates dynamic uncertainty is injected into the simulation environment model. The noise includes: adding random disturbances to the load power based on a statistical model of historical data that conform to the actual fluctuation characteristics; and randomly setting faults of key equipment components with a preset probability to simulate typical fault scenarios in which a single component in the power system randomly goes out of operation.

[0028] A further optimized solution involves using a reinforcement learning algorithm to perform parallel collaborative training on multiple agents based on the simulation environment model to obtain a multi-agent policy model. This specifically includes the following steps:

[0029] Each agent is configured with an actor network and a critic network to construct a multi-agent reinforcement learning network based on the actor-critic framework;

[0030] Based on the multi-agent reinforcement learning network, a centralized training paradigm is adopted during the training phase, enabling each agent's commentator network to obtain global state information and the action information of other agents. Each agent selects actions according to the current policy through its actor network. The joint actions of all agents work together to trigger state transitions and generate training data containing the next generation of states and immediate rewards.

[0031] Based on the generated training data, the parameters of the actor network and the critic network of all agents are updated synchronously using the gradient descent algorithm to obtain the parameter-updated actor network, which is then used as the multi-agent policy model.

[0032] A further optimized solution involves issuing and executing the power supply control strategy, monitoring the system's operating status in real time to collect feedback data, evaluating and optimizing the strategy's effectiveness based on the feedback data, and forming a closed-loop adaptive process. This specifically includes the following steps:

[0033] The power supply control command is sent to the corresponding execution unit via edge computing nodes or industrial IoT communication networks;

[0034] After the control command is executed, the system operation status data is collected in real time through the sensor network deployed in the park, thereby obtaining feedback status data;

[0035] The effectiveness of the strategy is evaluated based on the obtained feedback status data. If the system status does not return to normal or new anomalies occur, a new round of intelligent decision-making process is triggered to regenerate and deploy the power supply control strategy.

[0036] Secondly, this application provides an intelligent generation system for campus power supply protection strategies based on multi-agent reinforcement learning, comprising:

[0037] The status awareness module is used to collect operating parameters of power equipment and environmental status data through sensors deployed in the park's power network;

[0038] The environment model module, connected to the state perception module, is used to receive and, based on the state data, construct and run the multi-agent reinforcement learning simulation environment model, which defines the state space, action space, reward function, and state transition logic.

[0039] A multi-agent decision-making module, connected to the environment model module, is used to run reinforcement learning algorithms in the simulation environment model, perform parallel collaborative training of multiple agents to obtain a policy model, and perform policy reasoning based on real-time state information to generate an optimized power-saving control policy.

[0040] The strategy execution module, connected to the multi-agent decision-making module, is used to send the generated control strategy commands to the corresponding distributed power sources, energy storage systems, load switches and circuit breakers in the park through edge computing nodes or industrial IoT communication networks.

[0041] The feedback optimization module is connected to both the state perception module and the multi-agent decision-making module. It receives system feedback data collected by the state perception module after the strategy is executed, evaluates the effect of the strategy, and feeds back the evaluation result to the multi-agent decision-making module to trigger the optimization or regeneration of the strategy.

[0042] Thirdly, this application provides a computer-readable storage medium storing a multi-agent reinforcement learning-driven adaptive collaborative program for power supply protection in a campus, wherein when the multi-agent reinforcement learning-driven adaptive collaborative program for power supply protection in a campus is executed by a processor, it implements the steps of the multi-agent reinforcement learning-driven adaptive collaborative method for power supply protection in a campus as described above.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] This technical solution systematically realizes a fundamental transformation of the park's power supply strategy from static manual pre-setting to dynamic autonomous intelligent emergence by constructing four major processes: "high-fidelity environment modeling to lay the foundation for digital twins, distributed collaborative training to learn global optimization strategies, decentralized strategy emergence to achieve real-time intelligent decision-making, and online closed-loop optimization to form continuous adaptive capabilities." This enhances the system's rapid response and collaborative control capabilities in dynamic and uncertain scenarios such as load fluctuations and equipment failures. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0046] Figure 1 The implementation steps of the intelligent emergence method for power supply strategy in a park based on multi-agent model reinforcement learning provided in the embodiments of this application;

[0047] Figure 2 This application provides a flowchart of the strategy push closed-loop feedback process in an embodiment.

[0048] Figure 3 The diagram shows the structure of the intelligent generation system for power supply strategy in a park based on multi-agent model reinforcement learning, which is provided in the embodiments of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0050] Firstly, such as Figure 1 As shown, this application provides a multi-agent reinforcement learning-driven adaptive collaborative method for power supply protection in a campus, including the following steps:

[0051] Step S1: Establish a multi-agent reinforcement learning simulation environment model for the park's power system. The simulation environment model includes a state space defined based on the physical topology and operating characteristics of the park's power grid, an action space defined based on the control permissions and capabilities of each controllable unit within the park, and a multi-objective reward function for quantifying business objectives.

[0052] Step S2: Based on the simulation environment model, use reinforcement learning algorithm to perform parallel collaborative training on multiple agents to obtain a multi-agent policy model;

[0053] Step S3: Distribute the multi-agent strategy model in the park edge control equipment. Each agent makes independent decisions based on the real-time perceived system status information. Through implicit collaboration between agents, a globally adaptive power supply control strategy emerges in a self-organized manner.

[0054] Step S4: Issue and execute the power supply control strategy, and monitor the system operation status in real time to collect feedback data. Based on the feedback data, evaluate and optimize the effect of the strategy to form a closed-loop adaptive process.

[0055] This embodiment systematically realizes the fundamental transformation of the park's power supply strategy from static manual preset to dynamic autonomous intelligent emergence by constructing four major processes: "high-fidelity environment modeling to lay the foundation for digital twins, distributed collaborative training to learn global optimization strategies, decentralized strategy emergence to achieve real-time intelligent decision-making, and online closed-loop optimization to form continuous adaptive capabilities." This enhances the system's rapid response and collaborative control capabilities in dynamic and uncertain scenarios such as load fluctuations and equipment failures.

[0056] In one embodiment, based on the physical topology and operational characteristics of the campus power grid, the state space S of the reinforcement learning agent is defined, and the state variables are... As shown in the following formula:

[0057] ;

[0058] In the formula, Represents the voltage magnitude vector of each key bus; This represents the voltage phase angle vector of each key busbar; Indicates the system frequency; These represent the active and reactive power output vectors of each distributed power source (photovoltaic, wind power, diesel generator, mobile generator vehicle); Indicates the state of charge of the energy storage system; This represents the active power vector of each load; The state of each switch or circuit breaker is represented by a binary vector. This represents a vector of environmental parameters such as light intensity, wind speed, and temperature.

[0059] In one embodiment, a joint action space A for the intelligent agent is defined based on the control permissions and capabilities of each controllable unit within the park. The executable control instructions can be represented as:

[0060] ;

[0061] In the formula, The actions of a photovoltaic intelligent agent can be defined as follows: ,in These represent the active and reactive power output values ​​of photovoltaic systems, respectively. The actions of a wind power intelligent agent can be defined as follows: ,in These represent the active and reactive power output values ​​of wind power, respectively. The actions of a generator agent can be defined as follows: ,in This indicates the start-up and shutdown actions of the diesel generator. Indicates the on / off status of the mobile generator access point; The actions of an energy storage intelligent agent can be defined as follows: ,in These represent the power of energy storage charging and discharging, respectively; The actions of a load control agent can be defined as offline actions. Where 0 represents load shedding and 1 represents load commissioning; The action of the line control agent can be defined as a binary vector of length N, representing the state of N controllable switches / circuit breakers.

[0062] In one embodiment, a reward function is designed with the core objective of ensuring power supply reliability and system safety and stability. This quantifies business objectives into calculable reward signals. The reward function consists of a weighted average of multiple objectives, as shown in the following equation:

[0063] ;

[0064] In the formula, These are weighting coefficients used to balance the importance of different optimization objectives; As a reward for power supply reliability, a positive reward is given for continuous power supply to important loads, and a high negative reward is given for power outage events. To reward safe operation, penalties are imposed on unsafe conditions such as voltage or current exceeding limits, frequency deviation, etc., with the severity of the penalty related to the degree of exceeding the limit; As an economic incentive, appropriate rewards will be given for consuming renewable energy, while taking into account the certain operating costs of distributed power generation. To constrain rewards and punishments, penalties are imposed on actions that violate physical or operational constraints.

[0065] In one embodiment, the power supply reliability reward Safe operation reward Economic rewards and constraints, punishments, and rewards The calculation formula is shown below:

[0066] ;

[0067] In the formula, For the set of load nodes; For the set of busbars; A collection of transmission lines; It is a collection of diesel generators and mobile generator vehicles; This is the weighting coefficient for the load; the higher the priority, the larger the value. These are the voltage and upper limit of the j-th bus, respectively; These are the current and upper limit of the k-th line, respectively; This refers to the amount of new energy consumed; This is the energy storage operation cost coefficient; The charging and discharging power for energy storage; This is the fuel cost coefficient; For the active power output of the nth diesel generator or mobile generator vehicle; This is the penalty coefficient; , These represent the upper and lower limits of the state of charge of the energy storage system.

[0068] In one embodiment, the establishment of a multi-agent reinforcement learning simulation environment model for the park's power system includes a state space defined based on the physical topology and operational characteristics of the park's power grid, an action space defined based on the control permissions and capabilities of each controllable unit within the park, and a multi-objective reward function for quantifying business objectives. The model also includes the following steps:

[0069] Noise that simulates dynamic uncertainty is injected into the simulation environment model. The noise includes: adding random disturbances to the load power based on a statistical model of historical data that conform to the actual fluctuation characteristics; and randomly setting faults of key equipment components with a preset probability to simulate typical fault scenarios in which a single component in the power system randomly goes out of operation.

[0070] This embodiment introduces two types of noise interference: random load fluctuations and occasional equipment failures. This effectively enhances the resilience learning agent's adaptability to various uncertainties and unexpected events in real-world environments, thereby training a more robust and generalizable power supply control strategy. This noise injection mechanism simulates the natural fluctuations in load and the characteristics of occasional equipment failures in a real power grid, providing a more realistic and complex environment for strategy training.

[0071] An environmental model of the park's power grid is constructed based on power system simulation software. Considering the physical constraints of the power system and typical disturbance scenarios, a digital environment simulating the dynamic behavior of the park's power grid is established. The simulation environment ultimately needs to be implemented based on its dynamic interaction logic, which is determined by the core state transition function. The internal logic of this state transition function is as follows: First, it receives the set of actions of the agent. Then, actions are applied to the environment model to modify the corresponding device actions or states; the new system operating state is calculated. Calculate instant rewards Check termination conditions; finally return to running status. and rewards .

[0072] During the environmental modeling process, since power systems do not allow for repeated online trial and error, the above environmental model is mainly constructed through offline training. Utilizing historical data and a small amount of online experimental data, it learns the state transition patterns of the environment and the impact of faults / events on the system, forming a reinforcement learning environmental model. Through these steps, a high-fidelity simulation environment that reflects both the physical laws of the power grid in the industrial park and includes necessary uncertainties is constructed, providing a safe, efficient, and reliable environmental model for subsequent multi-agent collaborative training.

[0073] In one embodiment, after the scene modeling step is completed, a reinforcement learning algorithm is initiated to perform parallel collaborative training and optimization on all agents. This implementation preferably employs a deep deterministic policy gradient algorithm based on an actor-critic framework. Its core lies in adopting a collaborative paradigm of "centralized training and decentralized execution": during the training phase, the critic network can acquire global state information and the action information of other agents, thereby stabilizing the training process and effectively capturing the collaborative and competitive relationships between agents; while in the actual deployment and execution phase, each agent relies only on local observation information and its own actor network to make independent decisions, thus meeting the real-time and reliability requirements of the park's power supply system for distributed control; Step S2: Based on the simulation environment model, a reinforcement learning algorithm is used to perform parallel collaborative training on multiple agents to obtain a multi-agent policy model, specifically including the following steps:

[0074] Step S21: Configure an actor network and a critic network for each agent to construct a multi-agent reinforcement learning network based on the actor-critic framework; the actor network serves as a policy network, receiving environmental state information and outputting corresponding action instructions, and the critic network serves as an evaluation network, used to evaluate the state-action value function under a given system state and the joint actions of all agents.

[0075] Step S22: Based on the multi-agent reinforcement learning network, the above-described centralized training paradigm is adopted during the training phase. Each agent selects an action according to its current policy through its actor network. The joint actions of all agents act on the environment model to trigger state transitions and generate training data containing the next generation of states and immediate rewards.

[0076] Step S23: Based on the generated training data, the parameters of the actor network and critic network of all agents are synchronously updated using the gradient descent algorithm. The actor network after parameter update constitutes the multi-agent policy model, used to achieve decentralized independent decision-making in actual deployment.

[0077] Through iterative training, the agent's strategy is continuously improved, the system's cumulative reward gradually increases, and ultimately the power supply control strategy tends to the global optimum.

[0078] In one embodiment, step S3 specifically involves distributing a well-trained and stable multi-agent policy model across various edge control devices within the park, enabling each agent to independently generate control commands based on locally perceived real-time system status information. This decentralized deployment architecture effectively avoids communication bottlenecks and single-point-of-failure risks that centralized control may bring, laying a physical foundation for the system to achieve rapid response and high reliability.

[0079] Because the agents have fully learned the behavioral patterns of other agents and their impact on the system state through global information interaction during the early training process, their strategies essentially contain the equilibrium characteristics of multi-agent game. Therefore, in actual operation, although each agent makes independent decisions based only on local observations, they can achieve implicit cross-regional cooperation through the inherent correlation between strategies, thereby spontaneously maintaining the consistency of the overall system goal without the need for central coordination.

[0080] From a game theory perspective, this operating mechanism enables each agent to achieve a dynamic balance between pursuing its own optimal action and the system's global reward. Its strategy combinations can be viewed as efficient solutions that approximate Nash equilibrium under a given reward mechanism. Through continuous environmental interaction and strategy adjustment, the system exhibits significant self-organizing characteristics; that is, it can generate globally adaptive intelligent collaborative behavior based solely on local rules and interactions, enabling the agents collectively to demonstrate system efficiency that surpasses the simple sum of their individual actions.

[0081] When the system encounters unknown disturbances or sudden events not covered in the training, each agent can perform rapid online inference based on the learned strategy, and promptly generate adaptive control measures such as adjusting power output and shedding loads. To further ensure the long-term reliability of the emergent strategy, this embodiment can also introduce a game evaluation mechanism. By placing the current strategy in a high-fidelity simulation environment for multi-agent inference, its effectiveness can be dynamically verified. Once a strategy performance degradation or obvious defects are identified, a retraining process can be triggered for online optimization and updates.

[0082] In one embodiment, step S4: issuing and executing the power supply control strategy, and monitoring the system operating status in real time to collect feedback data, evaluating and optimizing the strategy effect based on the feedback data to form a closed-loop adaptive process, specifically includes the following steps:

[0083] Step S41: The power supply control command is sent to the corresponding execution unit via edge computing nodes or industrial IoT communication networks. A differentiated command transmission mechanism is adopted for different types of controlled objects: for devices with automatic execution capabilities, the control command is directly sent to the device's local controller for execution; for commands requiring manual intervention, the operation requirements are pushed to the maintenance personnel's mobile terminal App. The strategy issuance process considers communication reliability and priority management: control commands related to critical equipment and important loads are transmitted first, ensuring that core strategies can still be delivered to the execution end in a timely manner even under network bandwidth constraints or signal interference.

[0084] Step S42: After the control command is executed, the system operation status data is collected in real time by sensors deployed at each node of the park. The monitoring content includes, but is not limited to, the output power of the distributed power supply, the line opening status, the voltage and frequency of key nodes, fault indication information and the load power supply recovery status, etc., so as to obtain feedback status data. The above status data is uploaded to the edge computing platform in real time, which constitutes the basis for evaluating the strategy execution effect.

[0085] Step S43: Evaluate the effectiveness of the strategy based on the acquired feedback status data. If the system status returns to normal, the system returns to normal monitoring mode. If the system status does not return to normal or a new anomaly occurs, a new round of intelligent decision-making process is triggered. Based on the current actual status, multi-agent collaborative reasoning and strategy generation are carried out again to achieve online optimization and adaptive improvement of strategy performance.

[0086] Through the closed-loop design of data-decision-execution-feedback described above, the system continuously accumulates operational experience and feedback data, forming an intelligent power supply protection system with continuous learning capabilities.

[0087] In summary, as Figure 2 As shown, this invention achieves a fundamental shift in power supply strategy generation mode from static manual pre-setting to dynamic autonomous evolution through a complete technology chain encompassing distributed deployment, implicit collaboration, self-organizing emergence, and closed-loop optimization. The generated strategy possesses strong environmental adaptability and can effectively meet the stringent requirements for power supply assurance in complex campus environments.

[0088] Secondly, such as Figure 3As shown, this application provides an intelligent generation system for power supply strategy in a park based on multi-agent reinforcement learning, which is used to implement the multi-agent reinforcement learning-driven adaptive collaborative method for power supply in a park as described above; the system includes a state perception module 100, an environment model module 200, a multi-agent decision-making module 300, a strategy execution module 400, and a feedback optimization module 500;

[0089] The status awareness module 100 is used to collect operating parameters of power equipment and environmental status data through sensors deployed in the park's power network;

[0090] The environment model module 200 is connected to the state perception module 100 and is used to receive and, based on the state data, construct and run the multi-agent reinforcement learning simulation environment model. The model defines the state space, action space, reward function and state transition logic.

[0091] The multi-agent decision-making module 300 is connected to the environment model module 200 and is used to run reinforcement learning algorithms in the simulation environment model, perform parallel collaborative training of multiple agents to obtain a policy model, and perform policy reasoning based on real-time state information to generate an optimized power-saving control policy.

[0092] The strategy execution module 400 is connected to the multi-agent decision module 300 and is used to send the generated control strategy commands to the corresponding distributed power sources, energy storage systems, load switches and line circuit breakers in the park through edge computing nodes or industrial Internet of Things communication networks.

[0093] The feedback optimization module 500 is connected to the state perception module 100 and the multi-agent decision-making module 300 respectively. It is used to receive system feedback data after the strategy execution collected by the state perception module 100, evaluate the strategy effect, and feed back the evaluation result to the multi-agent decision-making module 300 to trigger strategy optimization or regeneration.

[0094] The state perception module, environment model module, multi-agent decision-making module, policy execution module, and feedback optimization module, through the above-mentioned connections, constitute a continuously iterative "data-decision-execution-feedback" closed-loop optimization system.

[0095] Thirdly, this application provides a multi-agent reinforcement learning-driven adaptive collaborative power supply device for a campus. The multi-agent reinforcement learning-driven adaptive collaborative power supply device for a campus can be a personal computer (PC), a laptop, a server, or other device with data processing capabilities.

[0096] In this embodiment, the multi-agent reinforcement learning-driven adaptive collaborative power supply device for the park may include a processor, a memory, a communication interface, and a communication bus.

[0097] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0098] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces enable interconnection of devices within the multi-agent reinforcement learning-driven adaptive collaborative power supply system for campuses, as well as interfaces for interconnection between the multi-agent reinforcement learning-driven adaptive collaborative power supply system for campuses and other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0099] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0100] The processor can be a general-purpose processor, which can call the multi-agent reinforcement learning-driven adaptive power supply coordination program stored in the memory and execute the multi-agent reinforcement learning-driven adaptive power supply coordination method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the multi-agent reinforcement learning-driven adaptive power supply coordination program is called can be referred to the various embodiments of the multi-agent reinforcement learning-driven adaptive power supply coordination method of this application, and will not be repeated here.

[0101] Fourthly, embodiments of this application also provide a readable storage medium.

[0102] This application stores a multi-agent reinforcement learning-driven adaptive collaborative program for campus power supply protection on a readable storage medium, wherein when the multi-agent reinforcement learning-driven adaptive collaborative program for campus power supply protection is executed by a processor, it implements the steps of the multi-agent reinforcement learning-driven adaptive collaborative method for campus power supply protection as described above.

[0103] The method implemented when the multi-agent reinforcement learning-driven adaptive collaborative program for power supply protection in the park is executed can be referred to in the various embodiments of the multi-agent reinforcement learning-driven adaptive collaborative program for power supply protection in the park of this application, and will not be repeated here.

[0104] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-agent reinforcement learning-driven adaptive collaborative method for power supply protection in a park, characterized in that, Includes the following steps: A multi-agent reinforcement learning simulation environment model for the park's power system is established. The simulation environment model includes a state space defined based on the physical topology and operating characteristics of the park's power grid, an action space defined based on the control permissions and capabilities of each controllable unit within the park, and a multi-objective reward function for quantifying business objectives. Based on the simulation environment model, a reinforcement learning algorithm is used to perform parallel collaborative training on multiple agents to obtain a multi-agent policy model. The multi-agent strategy model is deployed in a distributed manner in the edge control equipment of the park. Each agent makes independent decisions based on the real-time perceived system status information. Through implicit collaboration among agents, a globally adaptive power supply control strategy emerges in a self-organized manner. The power supply control strategy is issued and executed, and the system operation status is monitored in real time to collect feedback data. The effect of the strategy is evaluated and optimized based on the feedback data to form a closed-loop adaptive process.

2. The multi-agent reinforcement learning-driven adaptive collaborative power supply method for industrial parks according to claim 1, characterized in that, Based on the physical topology and operational characteristics of the campus power grid, the state space S of the reinforcement learning agent is defined, and the state variables are... As shown in the following formula: ； In the formula, Represents the voltage magnitude vector of each key bus; This represents the voltage phase angle vector of each key bus. Indicates the system frequency; These represent the active and reactive power output vectors of each distributed power source, respectively. Indicates the state of charge of the energy storage system; This represents the active power vector of each load; The state of each switch or circuit breaker is represented by a binary vector. This represents a vector of environmental parameters.

3. The multi-agent reinforcement learning-driven adaptive collaborative power supply method for industrial parks according to claim 1, characterized in that, Based on the control permissions and capabilities of each controllable unit within the park, a joint action space A for the intelligent agent is defined, wherein the actions... The executable control instructions can be represented as: ； In the formula, For the actions of photovoltaic intelligent agents; For the actions of wind power intelligent agents; For the actions of the generator intelligent agent; For the actions of energy storage intelligent agents; To control the actions of the intelligent agent under load; To control the actions of the intelligent agent.

4. The multi-agent reinforcement learning-driven adaptive collaborative power supply method for industrial parks according to claim 1, characterized in that, The multi-objective reward function for: ； in, These are the weighting coefficients. As a reward for power supply reliability, As a reward for safe operation, As an economic reward, To constrain punishment and reward.

5. The multi-agent reinforcement learning-driven adaptive collaborative power supply method for industrial parks according to claim 4, characterized in that, The calculation formulas for the power supply reliability reward, safe operation reward, economic reward, and constraint penalty reward are as follows: ； In the formula, For the set of load nodes; For the set of busbars; A collection of transmission lines; It is a collection of diesel generators and mobile generator vehicles; This is the weighting coefficient for the load; the higher the priority, the larger the value. These are the voltage and upper limit of the j-th bus, respectively; These are the current and upper limit of the k-th line, respectively; This refers to the amount of new energy consumed; This is the energy storage operation cost coefficient; The charging and discharging power for energy storage; This is the fuel cost coefficient; For the active power output of the nth diesel generator or mobile generator vehicle; This is the penalty coefficient; , These represent the upper and lower limits of the state of charge of the energy storage system.

6. The multi-agent reinforcement learning-driven adaptive collaborative power supply method for industrial parks according to claim 1, characterized in that, The establishment of a multi-agent reinforcement learning simulation environment model for the park's power system includes a state space defined based on the park's power grid physical topology and operating characteristics, an action space defined based on the control permissions and capabilities of each controllable unit within the park, and a multi-objective reward function for quantifying business objectives. The model also includes the following steps: Noise that simulates dynamic uncertainty is injected into the simulation environment model. The noise includes: adding random disturbances to the load power based on a statistical model of historical data that conform to the actual fluctuation characteristics; and randomly setting faults of key equipment components with a preset probability to simulate typical fault scenarios in which a single component in the power system randomly goes out of operation.

7. The method according to claim 1, characterized in that, Based on the simulation environment model, the multi-agent policy model is obtained by using reinforcement learning algorithms to perform parallel collaborative training on multiple agents, specifically including the following steps: Each agent is configured with an actor network and a critic network to construct a multi-agent reinforcement learning network based on the actor-critic framework; Based on the multi-agent reinforcement learning network, a centralized training paradigm is adopted during the training phase, enabling each agent's commentator network to obtain global state information and the action information of other agents. Each agent selects actions according to the current policy through its actor network. The joint actions of all agents work together to trigger state transitions and generate training data containing the next generation of states and immediate rewards. Based on the generated training data, the parameters of the actor network and the critic network of all agents are updated synchronously using the gradient descent algorithm to obtain the parameter-updated actor network, which is then used as the multi-agent policy model.

8. The multi-agent reinforcement learning-driven adaptive collaborative method for power supply protection in a park according to claim 1, characterized in that, The process of issuing and executing the power supply control strategy, monitoring the system's operating status in real time to collect feedback data, evaluating and optimizing the strategy's effectiveness based on the feedback data, and forming a closed-loop adaptive process specifically includes the following steps: The power supply control command is sent to the corresponding execution unit via edge computing nodes or industrial IoT communication networks; After the control command is executed, the system operation status data is collected in real time through the sensor network deployed in the park, thereby obtaining feedback status data; The effectiveness of the strategy is evaluated based on the obtained feedback status data. If the system status does not return to normal or new anomalies occur, a new round of intelligent decision-making process is triggered to regenerate and deploy the power supply control strategy.

9. A smart generation system for power supply strategy in a park based on multi-agent reinforcement learning, characterized in that, A system for implementing the multi-agent reinforcement learning-driven adaptive collaborative power supply method for industrial parks as described in any one of claims 1-8; the system comprises: The status awareness module is used to collect operating parameters of power equipment and environmental status data through sensors deployed in the park's power network; The environment model module, connected to the state perception module, is used to receive and, based on the state data, construct and run the multi-agent reinforcement learning simulation environment model, which defines the state space, action space, reward function, and state transition logic. A multi-agent decision-making module, connected to the environment model module, is used to run reinforcement learning algorithms in the simulation environment model, perform parallel collaborative training of multiple agents to obtain a policy model, and perform policy reasoning based on real-time state information to generate an optimized power-saving control policy. The strategy execution module, connected to the multi-agent decision-making module, is used to send the generated control strategy commands to the corresponding distributed power sources, energy storage systems, load switches and circuit breakers in the park through edge computing nodes or industrial IoT communication networks. The feedback optimization module is connected to both the state perception module and the multi-agent decision-making module. It receives system feedback data collected by the state perception module after the strategy is executed, evaluates the effect of the strategy, and feeds back the evaluation result to the multi-agent decision-making module to trigger the optimization or regeneration of the strategy.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a multi-agent reinforcement learning-driven adaptive collaborative program for power supply protection in a campus, wherein when the multi-agent reinforcement learning-driven adaptive collaborative program for power supply protection in a campus is executed by a processor, it implements the steps of the multi-agent reinforcement learning-driven adaptive collaborative method for power supply protection in a campus as described in any one of claims 1 to 8.

Images