Multi-agent scheduling method and system for wind-solar-hydrogen microgrid

Abstract

The application provides a kind of multi-agent scheduling method and system for wind light hydrogen storage microgrid, it is related to microgrid energy storage scheduling technical field, the method comprises: collecting original state data;The original state data is mapped, and time sequence state vector sequence is generated;Based on the time sequence state vector sequence, generate the first global state vector;The first global state vector is input into the multiple heterogeneous agents constructed in advance respectively, and multiple action instructions output by the multiple heterogeneous agents are obtained;The heterogeneous agent is one-to-one corresponding with the action instruction;The action instruction is executed by the heterogeneous agent, and the wind light hydrogen storage microgrid is scheduled.The multi-agent scheduling method and system for wind light hydrogen storage microgrid provided by the application, through the collaborative execution of action instruction by multiple heterogeneous agents, the wind light hydrogen storage microgrid is scheduled, and real-time, accurate and automatic optimization control of the wind light hydrogen storage microgrid is realized.

Description

Technical Field

[0001] This invention relates to the field of microgrid energy storage scheduling technology, and in particular to a multi-agent scheduling method and system for wind, solar, and hydrogen storage microgrids. Background Technology

[0002] Wind-solar-storage-hydrogen microgrids (or microgrids for short) are key carriers for integrating distributed energy and realizing flexible energy regulation. Their dispatching capabilities directly determine the operating efficiency, power supply reliability, and absorption capacity of the new power system for fluctuating renewable energy.

[0003] Currently, the mainstream scheduling methods for wind, solar, and hydrogen storage microgrids mainly fall into three categories: rule-based methods, mathematical programming-based methods, and heuristic algorithms. Rule-based methods suffer from poor adaptability in their scheduling strategies, making it difficult to handle complex dynamic scenarios within microgrids. While mathematical programming-based methods can find theoretically optimal solutions, they heavily rely on accurate predictions of future wind and solar power output and load demand, making them prone to model mismatch and decreased microgrid performance under the influence of strong uncertainties in renewable energy. Heuristic algorithms, on the other hand, suffer from low solution efficiency and decision-making lag, making it difficult to meet the real-time scheduling requirements of microgrids. Summary of the Invention

[0004] This invention provides a multi-agent scheduling method and system for wind, solar, and hydrogen storage microgrids, to solve the technical problems of poor real-time performance and low accuracy in the scheduling of wind, solar, and hydrogen storage microgrids in the prior art.

[0005] This invention provides a multi-agent scheduling method for wind, solar, and hydrogen storage microgrids, comprising the following steps: Collect raw state data; the raw state data includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid; The original state data is mapped to generate a time-series state vector sequence; Based on the temporal state vector sequence, a first global state vector is generated; The first global state vector is input into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands. The heterogeneous intelligent agent executes the action commands to schedule the wind, solar, and hydrogen storage microgrid.

[0006] According to the present invention, a multi-agent scheduling method for wind, solar, and hydrogen storage microgrids is provided, wherein generating a first global state vector based on the time-series state vector sequence includes: The temporal state vector sequence is subjected to time-step feature extraction using a multi-layer long short-term memory network to obtain multiple hidden state sequences; the multiple hidden state sequences include the current hidden state at the current time step and the historical hidden states at multiple historical time steps; Based on the current hidden state, a query vector is generated; Based on the historical hidden state, a historical key vector and a historical value vector are generated; Calculate the similarity score between the query vector and the historical key vector; The similarity scores are normalized to obtain attention weights; Based on the attention weights, the historical value vector is weighted and summed to generate the first global state vector.

[0007] According to the present invention, a multi-agent scheduling method for wind, solar, and hydrogen storage microgrids is provided, wherein the heterogeneous agents include a policy network; the policy network is specifically used for: Collect the local observation vector of the heterogeneous intelligent agent at the current moment; Based on the first global state vector and the local observation vector, the current action strategy is generated; The current action strategy is sampled to generate the action instruction.

[0008] According to the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids provided by the present invention, the heterogeneous agents further include an update module; the update module is specifically used for: Collect the second global state vector of all heterogeneous agents after executing the action command; Based on the second global state vector and the preset multi-objective reward function, the immediate reward corresponding to the action command is calculated; Experience tuple data is constructed using the first global state vector, the action instruction, the immediate reward, and the second global state vector; The action instructions are updated based on the experience tuple data.

[0009] According to the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids provided by the present invention, the step of updating the action instructions based on the empirical tuple data includes: Based on the aforementioned empirical tuple data, the individual advantage function of each heterogeneous agent is calculated using the counterfactual baseline method. Based on the individual advantage function, the contribution of each agent to the global value estimation after executing the action instruction is quantified; The action command is updated based on the contribution level.

[0010] According to the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids provided by the present invention, the heterogeneous agents further include a value network; the value network is specifically used for: The global value estimate is generated based on the second global state vector.

[0011] This invention also provides a multi-agent scheduling system for wind, solar, and hydrogen storage microgrids, comprising the following modules: The data acquisition module is used to collect raw state data, which includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid. The mapping module is used to map the original state data to generate a time-series state vector sequence; The generation module is used to generate a first global state vector based on the time-series state vector sequence; The decision module is used to input the first global state vector into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands; The scheduling module is used to execute the action commands through the heterogeneous intelligent agent to schedule the wind, solar and hydrogen storage microgrid.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids as described above.

[0015] This invention provides a multi-agent scheduling method and system for wind-solar-hydrogen storage microgrids. It collects raw state data and maps this data to generate a time-series state vector sequence, thus standardizing and organizing multi-source, heterogeneous raw data into structured time-series data. This eliminates dimensional differences and provides a reliable input foundation for subsequent data-driven intelligent decision-making that does not rely on precise mathematical models. Based on the time-series state vector sequence, a first global state vector is generated, utilizing time-series dependencies to avoid short-sighted decision-making that could lead to local optima. The first global state vector is then input into multiple pre-constructed heterogeneous agents to obtain multiple action commands output by these agents. This enables distributed collaborative decision-making by multiple agents under a unified global situational awareness, improving adaptability to the dynamically changing environment of the wind-solar-hydrogen storage microgrid and thus increasing scheduling accuracy. By executing action commands through the heterogeneous agents, the wind-solar-hydrogen storage microgrid is scheduled, enabling collaborative work among the agents and achieving real-time, precise, and automatic optimization control. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids provided by the present invention.

[0018] Figure 2 This is a schematic diagram of the structure of the multi-agent scheduling device for wind, solar, and hydrogen storage microgrids provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] Existing wind-solar-hydrogen storage microgrid dispatching methods still have certain shortcomings when dealing with complex dynamic environments, particularly in capturing time-series dynamic characteristics, designing multi-agent collaborative mechanisms, and balancing multiple objectives. For example, they neglect the inherent strong time-series dependence of the power system (such as the diurnal periodicity of load, the continuous decline trend of wind power output, or the daily variation of photovoltaic power), leading to short-sighted decision-making and failing to plan energy storage charging and discharging or hydrogen energy conversion in advance. The lack of effective information interaction and impact assessment mechanisms in the design of multi-agent collaborative mechanisms may lead to a "credit allocation dilemma," where it is impossible to accurately determine whether global performance changes are caused by its own behavior or the behavior of others, resulting in policy learning oscillations, slow convergence, or even getting trapped in local optima.

[0022] To this end, the present invention provides a multi-agent scheduling method and system for wind, solar, and hydrogen storage microgrids. It aims to accurately capture the temporal dynamic characteristics of the microgrid system by introducing an attention mechanism, construct an efficient multi-agent collaborative communication mechanism, and support dynamic trade-offs of multiple objectives, thereby improving the intelligent operation level of the microgrid and meeting the economic and environmental benefits brought about by the coupling of high proportions of renewable energy and hydrogen energy.

[0023] The following is combined Figures 1 to 3 This invention describes a multi-agent scheduling method and system for wind, solar, and hydrogen storage microgrids.

[0024] Figure 1 This is a flowchart illustrating the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps: Step 101: Collect raw state data; the raw state data includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid. Specifically, by deploying sensors and measurement devices in each unit of the wind-solar-hydrogen storage microgrid, wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data are periodically collected to form raw status data.

[0025] For example, in one embodiment, wind power generation data is acquired via an anemometer and power transmitter; photovoltaic power generation data is acquired via an irradiance sensor and inverter; the state of charge (SOC) data of the energy storage is acquired via the battery management system (BMS) in the wind-solar-hydrogen storage microgrid; hydrogen storage tank pressure data is acquired via a pressure transmitter; electrolyzer operating status data (such as start-up / shutdown and efficiency) and fuel cell operating status data are read via the Controller Area Network (CAN) bus; and load power data is acquired via a smart meter. This data is uploaded to the scheduling system via a communication interface unit (supporting the Modbus TCP / IP protocol).

[0026] The embodiments of the present invention realize comprehensive and real-time perception of the operating status of the electricity-hydrogen-storage multi-element coupled system, laying a solid foundation for data-driven intelligent scheduling, abandoning the dependence on precise mathematical models, and enhancing the system's adaptability to uncertainty.

[0027] Step 102: Map the original state data to generate a time-series state vector sequence; Specifically, missing values ​​are filled and outliers are removed for each data item in the original state dataset. The maximum-minimum normalization method is used to linearly map the data of each dimension to the interval [0, 1] to form a standardized time-series state vector sequence.

[0028] For example, in one embodiment, the raw value (e.g., 0.5) of the energy storage state of charge (SOC) data is mapped to a normalized value (e.g., 0.375, assuming the allowable range of SOC data is [0.1, 0.9]). All the normalized data at each scheduling time (e.g., 15 minutes) constitute a state vector, which, when arranged in chronological order, forms a time-series state vector sequence.

[0029] This invention transforms raw data into standard, clean time-series data, eliminating the influence of dimensions and forming a structured data sequence with a time dimension. This provides a reliable data foundation for subsequent data-driven intelligent decision-making that does not rely on precise mathematical models.

[0030] Step 103: Generate a first global state vector based on the time-series state vector sequence; Further, generating the first global state vector based on the temporal state vector sequence includes: The temporal state vector sequence is subjected to time-step feature extraction using a multi-layer long short-term memory network to obtain multiple hidden state sequences; the multiple hidden state sequences include the current hidden state at the current time step and the historical hidden states at multiple historical time steps; Based on the current hidden state, a query vector is generated; Based on the historical hidden state, a historical key vector and a historical value vector are generated; Calculate the similarity score between the query vector and the historical key vector; The similarity scores are normalized to obtain attention weights; Based on the attention weights, the historical value vector is weighted and summed to generate the first global state vector.

[0031] Specifically, the temporal state vector sequence is input into a multi-layer Long Short-Term Memory (LSTM) network to extract its hidden state sequence. Then, based on the current hidden state corresponding to the current time step, a query vector is generated, and based on the multiple historical hidden states corresponding to multiple historical time steps, historical key vectors and historical value vectors are generated. Then, attention weights are calculated through an attention mechanism. Finally, based on the attention weights, the historical value vectors are weighted and summed to generate a first global state vector that integrates key historical information (i.e., contextual information).

[0032] The current hidden state of the Long Short-Term Memory network at the current time step t Update using the following expression: In the formula, Let be the standardized state vector at time t. and These represent the hidden state and cell state of the previous time step, respectively.

[0033] Based on the query vector at the current time step (Depend on (Obtained through linear transformation) and the historical key vectors of all historical time steps (Depend on (Obtained through linear transformation) Calculate attention weights : In the formula, , is the dimension of the historical key vector, and T is the length of the historical time window.

[0034] The first global state vector obtained by weighted fusion The expression is: In the formula, To hide from history The corresponding historical value vector.

[0035] For example, in one embodiment, a two-layer LSTM network is first constructed. After inputting the temporal state vector sequence into the LSTM network, the LSTM network outputs the hidden state sequence for each time step. For the current time step t, its corresponding hidden state is... A trainable weight matrix Wq is linearly transformed into a query vector qt. The historical hidden states corresponding to each historical time step are then used. The historical key vectors are transformed using trainable weight matrices Wk and Wv, respectively. and historical value vector The similarity score is obtained by calculating the dot product of qt with all ki and scaling it, and then normalizing it using Softmax to get the attention weights. Finally, the attention weights are applied to all historical value vectors. We perform a weighted summation to obtain the first global state vector.

[0036] The embodiments of the present invention realize dynamic focusing of time-series characteristics, enabling microgrid systems to adaptively focus on historical events most relevant to the current decision, overcoming the short-sighted decision-making problem caused by neglecting time-series dependencies in traditional methods, and improving the foresight and robustness of scheduling strategies.

[0037] Step 104: Input the first global state vector into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands; Furthermore, the heterogeneous intelligent agent includes a policy network; the policy network is specifically used for: Collect the local observation vector of the heterogeneous intelligent agent at the current moment; Based on the first global state vector and the local observation vector, the current action strategy is generated; The current action strategy is sampled to generate the action instruction.

[0038] Specifically, based on actual needs, multiple heterogeneous intelligent agents with different structures are pre-constructed. For example, for multiple physical units in a microgrid system, such as wind turbines, photovoltaics, energy storage, electrolyzers, fuel cells, and loads, wind power and photovoltaic intelligent agents, electrochemical energy storage intelligent agents, electrolyzer intelligent agents, fuel cell intelligent agents, and load intelligent agents can be pre-constructed. The strategy network of each heterogeneous intelligent agent adopts an Actor network structure.

[0039] Each heterogeneous intelligent agent, distributed across various physical units of the wind-solar-hydrogen storage microgrid, generates a current action strategy or a probability distribution of all possible actions based on shared global situation awareness (i.e., the first global state vector) and its own local situation (i.e., the local observation vector). Finally, it samples the current action strategy, selects an action, and outputs the corresponding action command. These action commands include one or more of the following: power generation unit output adjustment commands, energy storage charging and discharging commands, electrolyzer / fuel cell start / stop and power commands, and load management commands.

[0040] In some embodiments, for different heterogeneous intelligent agents, their local observation vectors The composition is as follows: For wind and solar power intelligent agents: The local observation vector includes the current actual wind power generation, the current actual photovoltaic power generation, the current ambient wind speed (if equipped with a wind speed sensor), the solar irradiance (if equipped with an irradiance sensor), and the predicted power generation value for the next moment. This local observation vector reflects the natural fluctuation characteristics of renewable energy.

[0041] For electrochemical energy storage agents: The local observation vector includes the current state of charge (SOC) of the energy storage system, the current charge / discharge power of the energy storage system, the highest battery temperature (if equipped with a temperature sensor), and the state of health (SOH). This local observation vector primarily characterizes the energy storage capacity and operational health of the energy storage device.

[0042] For the intelligent agent in the electrolytic cell: The local observation vector includes the current operating status of the electrolyzer (such as start / stop status and operating level), the actual hydrogen production power of the electrolyzer, the cell temperature, and the current hydrogen production rate. This local observation vector reflects the real-time operating conditions of the hydrogen production process.

[0043] For fuel cell intelligent agents: The local observation vector includes the current operating status of the fuel cell (e.g., start-up / shutdown status), the actual power output of the fuel cell, the fuel cell stack temperature, and the current hydrogen consumption rate. This local observation vector reflects the real-time response capability of the hydrogen power generation process.

[0044] For the load agent: The local observation vector includes the total load power at the current moment, the interruptible load power, the critical load power, and the load forecast for the next moment. This local observation vector reflects the electricity demand and adjustability potential on the user side.

[0045] For example, in one embodiment, a first global state vector is input to each heterogeneous agent. Simultaneously, each heterogeneous agent receives its own local observation vector (e.g., an energy storage agent also receives its own State of Charge (SOC)). The first global state vector and the local observation vector are concatenated and input into the policy network of the heterogeneous agent. The policy network outputs the current action policy or action command corresponding to the physical unit, such as the energy storage agent outputting the action command "-50kW" (charging). The first global state vector is broadcast via the microgrid communication bus or distributed by edge computing nodes. When communication is limited, each heterogeneous agent propagates its local observation vector forward based on its own policy network, independently outputting action commands using implicit cooperative features learned during training, thereby achieving decentralized fault-tolerant scheduling.

[0046] It should be noted that, in this embodiment of the invention, the action space of each heterogeneous intelligent agent is defined according to the characteristics of its corresponding physical unit. For example, the action space of wind and photovoltaic intelligent agents can be an adjustment coefficient or a curtailment rate for planned output. This action does not directly control physical equipment, but is used to correct short-term forecast values ​​to form planned values ​​for participating in microgrid power balance. Actual output still follows natural conditions. When the planned value is lower than the actual output, the excess is considered curtailed.

[0047] The action space of an electrochemical energy storage intelligent agent can be a charge / discharge power command; The action space of the electrolyzer's intelligent agent can be the hydrogen production power command; The action space of a fuel cell intelligent agent can be a power generation command.

[0048] The action space of the load agent can be the reduction ratio of interruptible loads.

[0049] When making decisions, the policy network of each heterogeneous agent satisfies the following constraints: (1) Power balance constraint. The power generated within the microgrid, the power of the load, and the power exchanged with the grid must be balanced in real time, that is: In the formula, the left side represents the total power supply. Specifically, Indicates the power output of wind power. Indicates the photovoltaic power supply capacity. This indicates the energy storage discharge power (where a positive value represents discharge and a negative value represents charging). Indicates the power output of the fuel cell. The right side represents the power supplied by the power grid; the right side represents the total power consumption. Specifically, Indicates the total load. This indicates a reduction in load. This indicates the load for hydrogen production via electrolysis. It is understandable that... Indicates the purchase of electricity. This indicates electricity sales (i.e., meeting the electricity sales constraints on the microgrid side).

[0050] (2) Equipment operation boundary constraints.

[0051] Wind / solar power output constraints: , ( , (Rated power of wind power equipment and photovoltaic equipment, respectively). Energy storage charge and discharge constraints: ;in, This indicates the maximum charging power of the energy storage device. This indicates the maximum discharge power of the energy storage device.

[0052] SOC constraint ; Charge and discharge power change rate constraint (To avoid power surges that could damage the battery); among which, This represents the actual charging and discharging power of the energy storage system at time t-1 (i.e., the previous scheduling period). This indicates the maximum allowable power change (ramp rate limit) between two adjacent scheduling periods of the energy storage system.

[0053] Electrolytic cell constraints: ;in, This indicates the rated maximum power of the hydrogen electrolyzer.

[0054] Start-stop constraint: The continuous running time after each start is ≥30 minutes (starting from the start time), to avoid frequent start-stop (achieved through action penalty); Fuel cell constraints: ;in, This indicates the rated maximum power of the fuel cell.

[0055] Hydrogen consumption constraints: ;in, This represents the actual hydrogen consumption rate of the fuel cell at time t. This indicates the maximum hydrogen supply rate of the hydrogen storage tank.

[0056] (3) Constraints of hydrogen storage tanks.

[0057] Pressure constraints: (To avoid damage to the equipment due to overpressure or underpressure); among which, This indicates the lower limit of the hydrogen storage tank pressure. Indicates in The pressure of the hydrogen storage tank at any given time. This indicates the upper pressure limit of the hydrogen storage tank.

[0058] Extreme scenario handling: When the pressure in the hydrogen storage tank is lower than... In such cases, the fuel cell is forced to stop generating electricity to prioritize the safety of the hydrogen storage tank. Hydrogen storage constraints: ;in, This indicates the minimum safe hydrogen storage capacity allowed by the hydrogen storage tank (i.e., the amount of remaining hydrogen at the corresponding lower pressure limit). This indicates the actual physical amount of hydrogen stored in the hydrogen storage tank at time t (standard cubic meters or mass). This indicates the maximum hydrogen storage capacity of the hydrogen storage tank (the maximum amount of hydrogen that can be stored at the corresponding pressure limit).

[0059] Hydrogen storage change rate constraint (Matching hydrogen production and consumption rates); where, This indicates the actual amount of hydrogen stored in the hydrogen storage tank at time t-1 (i.e., the previous scheduling period). This indicates the maximum allowable change in hydrogen storage capacity within two adjacent scheduling periods.

[0060] (4) Power grid interaction constraints.

[0061] Interactive power constraints: ( For maximum power purchase capacity, (The maximum power sales capacity is set by the power grid dispatching department). Electricity sales timing constraints: Electricity sales are prohibited during certain periods (such as peak electricity consumption periods) and must be matched with time-of-use pricing policies.

[0062] (5) Load constraints.

[0063] Maximum interruptible load reduction: This is used to ensure the reliability of power supply to core loads. Among them, This represents the total power of interruptible loads participating in demand response within the microgrid system at time t.

[0064] This invention utilizes multiple distributed heterogeneous intelligent agents to make collaborative decisions based on shared global situational awareness and their own local circumstances. This ensures that each agent's decision considers both the overall system optimization and its own unique state and constraints, achieving efficient and rational distributed decision-making. Furthermore, adding new energy units only requires adding corresponding heterogeneous intelligent agents to integrate them into the system, enabling system collaboration and flexible expansion.

[0065] Step 105: The heterogeneous intelligent agent executes the action command to schedule the wind-solar-hydrogen storage microgrid.

[0066] Specifically, each heterogeneous intelligent agent executes corresponding action instructions, which are applied to the microgrid system to complete the scheduling action.

[0067] For example, in one embodiment, each heterogeneous intelligent agent executes a corresponding action command, which is then converted into control commands executable by the underlying programmable logic controller (PLC) or energy management system (EMS) of each physical unit via a communication interface unit (such as the IEC 61850 protocol) and issued. The devices in each physical unit of the microgrid execute these commands to achieve power redistribution and optimized scheduling.

[0068] The embodiments of the present invention form a complete "perception-decision-execution" closed loop by having multiple heterogeneous intelligent agents collaboratively execute action commands, thereby realizing real-time, accurate, and automatic optimized scheduling of wind, solar, and hydrogen storage microgrids.

[0069] This invention provides a multi-agent scheduling method for wind-solar-hydrogen storage microgrids. It collects raw state data and maps this data to generate a time-series state vector sequence, thus standardizing and organizing multi-source, heterogeneous raw data into structured time-series data. This eliminates dimensional differences and provides a reliable input foundation for subsequent data-driven intelligent decision-making that does not rely on precise mathematical models. Based on the time-series state vector sequence, a first global state vector is generated, utilizing time-series dependencies to avoid short-sighted decision-making that could lead to local optima. The first global state vector is then input into multiple pre-constructed heterogeneous agents to obtain multiple action commands output by these agents. This enables distributed collaborative decision-making by multiple agents under a unified global situational awareness, improving adaptability to the dynamically changing environment of the wind-solar-hydrogen storage microgrid and thus increasing scheduling accuracy. By executing action commands through the heterogeneous agents, the wind-solar-hydrogen storage microgrid is scheduled, enabling collaborative work among the agents and achieving real-time, automatic optimization control of the microgrid.

[0070] Furthermore, the heterogeneous intelligent agent also includes an update module; the update module is specifically used for: Collect the second global state vector of all heterogeneous agents after executing the action command; Based on the second global state vector and the preset multi-objective reward function, the immediate reward corresponding to the action command is calculated; Experience tuple data is constructed using the first global state vector, the action instruction, the immediate reward, and the second global state vector; The action instructions are updated based on the experience tuple data.

[0071] Specifically, after each heterogeneous intelligent agent executes the action command, it re-collects the wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid, obtaining new state data. Steps 102 and 103 are then performed on the new state data to obtain the corresponding second global state vector (i.e., the new state). ).

[0072] Based on a multi-objective reward function, the immediate reward obtained by each heterogeneous agent for executing the action instruction is calculated. The state before execution (i.e., the first global state vector) is then considered. Execution of action instructions Instant rewards received New status after execution Constituting empirical tuple data This experience tuple data is stored in the experience replay pool for subsequent parameter updates to the policy network, which in turn updates the action instructions using the updated policy network.

[0073] Among them, multi-objective reward function The design integrates economic efficiency, stability, and environmental protection goals, specifically as follows: In the formula, the electricity purchase cost item Cost of purchasing electricity from the upper-level power grid during time period t; power imbalance penalty term. The absolute value of the system's net power imbalance, used to penalize power instability; equivalent carbon emission cost item. Equivalent carbon emission cost; Renewable energy integration rate incentives The immediate absorption rate of renewable energy; This indicates the penalty for violating the constraint; Let be the weight coefficients of each objective term, satisfying... The specific definitions and explanations of the above parameters are as follows: (1) Electricity purchase cost (Yuan).

[0074] In the formula, the purchased power is... , Hourly (15-minute dispatch cycle); electricity sales revenue can offset electricity purchase costs, i.e. ( (This refers to the electricity sales price).

[0075] (2) Power imbalance penalty (kW).

[0076] The larger the absolute value, the heavier the penalty, ensuring system stability.

[0077] (3) Equivalent carbon emission cost item (Yuan).

[0078] In the formula, Carbon emission factor for electricity purchase (kgCO2 / kWh). The cost per unit of carbon emissions (RMB / kgCO2) is used to encourage a reduction in reliance on fossil fuels.

[0079] (4) Renewable energy consumption rate incentive items (Dimensionless).

[0080] In the formula, Let be the curtailment power of wind and solar power (kW); Boundary conditions: when At that time, the default (No consumption is required when there is no wind or light output, and no penalty is incurred). The higher the consumption rate, the greater the reward, with a maximum value of 1.

[0081] (5) Constraints on penalties for violations (Yuan).

[0082] When an agent's actions violate equipment operating constraints (such as exceeding SOC limits, abnormal hydrogen storage pressure, or sudden power fluctuations), a penalty is triggered: In the formula, The penalty coefficient (set according to the importance of the constraint, such as a safety constraint) The value is calculated based on the cost of equipment failure (RMB / kW). To constrain the degree of violation (e.g., a difference of less than 20% in SOC).

[0083] This invention establishes a data link from environmental feedback to policy optimization through an update module, enabling heterogeneous agents to continuously improve scheduling strategies directly from real-time interactions with the environment without relying on prediction models, through a "trial and error-learning" approach.

[0084] Furthermore, updating the action instruction based on the experience tuple data includes: Based on the aforementioned empirical tuple data, the individual advantage function of each heterogeneous agent is calculated using the counterfactual baseline method. Based on the individual advantage function, the contribution of each agent to the global value estimation after executing the action instruction is quantified; The action command is updated based on the contribution level.

[0085] Furthermore, the heterogeneous intelligent agent also includes a value network; the value network is specifically used for: The global value estimate is generated based on the second global state vector.

[0086] Specifically, the experience tuple data is stored in the experience replay pool. Once a certain number has been accumulated (the threshold can be set according to actual needs), a batch of experience tuple data corresponding to historical time steps is randomly sampled from the experience replay pool and used as training data.

[0087] For empirical tuple data , Indicates the execution of the action instruction. Previous, in the current time step The first global state vector, Indicates the execution of the action instruction. The next step The second global state vector, This represents the execution action instructions calculated through a multi-objective optimization function. The immediate reward that follows.

[0088] The second global state vector Input the value network to generate the corresponding global value estimate. The value network employs a Critic network structure.

[0089] The individual advantage function of each heterogeneous agent is calculated separately using the counterfactual baseline method. For any heterogeneous agent... Its individual advantage function Defined as: In the formula, This is the global state-action value function, i.e., global value estimation. Including heterogeneous intelligent agents All other heterogeneous intelligent agents execute the joint action corresponding to the action instruction. heterogeneous intelligent agents The current action strategy, Its movement trajectory.

[0090] The value of the individual advantage function can be used to quantify any heterogeneous intelligent agent. Execute action instructions Compared to the additional value gain that each agent can bring by averaging its performance according to its own strategy, the contribution of each agent to the global value estimate after executing action instructions is obtained (i.e., (Specific value).

[0091] The gradient of the policy network parameters is updated based on the contribution. The expression is: In the formula, Indicates the first The policy objective function of an agent Its parameters The gradient is used to update the policy network. This represents the state obtained by sampling according to the joint policy π. ,action The distribution takes the expectation. Indicates the first The advantage function (contribution) of each agent is used to measure actions. Relative to state Value gain of the average action.

[0092] The parameters of the policy network can be directly updated based on the updated gradient. Simultaneously, the parameters of the value network are updated using the temporal difference error. The immediate reward from the empirical tuple is then used. Second global state vector Calculate the temporal difference objective, i.e., the Q-value estimate under joint actions. Then, pair the temporal difference objective with the value network. The estimated values ​​are compared, the loss is calculated, and the parameters of the value network are updated. In the next time step, each heterogeneous agent generates new action instructions based on the updated policy network and value network, thus updating the action instructions.

[0093] This invention addresses the common credit allocation dilemma in multi-agent reinforcement learning by collaboratively training and updating the policy network and value network of each heterogeneous agent. This accurately evaluates the marginal contribution of each agent's actions to global multi-objective performance (economic, stable, and environmentally friendly), thereby guiding agents to learn more efficient and collaborative distributed scheduling strategies, accelerating strategy convergence and improving final performance. Through real-time interaction with the environment and continuous learning, the strategy is dynamically adjusted, achieving real-time, precise, and automatic optimization control of wind-solar-hydrogen storage microgrids. This significantly enhances the adaptive optimization scheduling capability and overall operational efficiency of wind-solar-hydrogen storage microgrids under conditions of high renewable energy access and strong randomness.

[0094] The following describes the multi-agent scheduling device for wind, solar, and hydrogen storage microgrids provided by the present invention. The multi-agent scheduling device for wind, solar, and hydrogen storage microgrids described below can be referred to in correspondence with the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids described above.

[0095] Figure 2 This is a schematic diagram of the structure of the multi-agent scheduling device for wind, solar, and hydrogen storage microgrids provided by the present invention, as shown below. Figure 2 As shown. This invention provides a multi-agent scheduling device for wind, solar, and hydrogen storage microgrids, comprising a data acquisition module 201, a mapping module 202, a generation module 203, a decision-making module 204, and a scheduling module 205, wherein: The data acquisition module 201 is used to acquire raw state data, including wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid. The mapping module 202 is used to map the raw state data to generate a time-series state vector sequence. The generation module 203 is used to generate a first global state vector based on the time-series state vector sequence. The decision module 204 is used to input the first global state vector into multiple pre-constructed heterogeneous intelligent agents to obtain multiple action commands output by the multiple heterogeneous intelligent agents. The heterogeneous intelligent agents correspond one-to-one with the action commands. The scheduling module 205 is used to schedule the wind-solar-hydrogen storage microgrid by executing the action commands through the heterogeneous intelligent agents.

[0096] This invention provides a multi-agent scheduling system for wind-solar-hydrogen storage microgrids. It collects raw state data and maps this data to generate a time-series state vector sequence, thus standardizing and organizing multi-source, heterogeneous raw data into structured time-series data. This eliminates dimensional differences and provides a reliable input foundation for subsequent data-driven intelligent decision-making that does not rely on precise mathematical models. Based on the time-series state vector sequence, a first global state vector is generated, utilizing time-series dependencies to avoid short-sighted decision-making that could lead to local optima. The first global state vector is then input into multiple pre-constructed heterogeneous agents to obtain multiple action commands output by these agents. This enables distributed collaborative decision-making by multiple agents under a unified global situational awareness, improving adaptability to the dynamically changing environment of the wind-solar-hydrogen storage microgrid and thus increasing scheduling accuracy. By executing action commands through the heterogeneous agents, the system schedules the wind-solar-hydrogen storage microgrid, enabling collaborative work among the agents and achieving real-time, automatic optimization control of the microgrid.

[0097] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a multi-agent scheduling method for wind-solar-hydrogen storage microgrids, which includes: Collect raw state data; the raw state data includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid; The original state data is mapped to generate a time-series state vector sequence; Based on the temporal state vector sequence, a first global state vector is generated; The first global state vector is input into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands. The heterogeneous intelligent agent executes the action commands to schedule the wind, solar, and hydrogen storage microgrid.

[0098] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids provided by the above methods, the method including: Collect raw state data; the raw state data includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid; The original state data is mapped to generate a time-series state vector sequence; Based on the temporal state vector sequence, a first global state vector is generated; The first global state vector is input into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands. The heterogeneous intelligent agent executes the action commands to schedule the wind, solar, and hydrogen storage microgrid.

[0100] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids provided by the above methods, the method comprising: Collect raw state data; the raw state data includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid; The original state data is mapped to generate a time-series state vector sequence; Based on the temporal state vector sequence, a first global state vector is generated; The first global state vector is input into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands. The heterogeneous intelligent agent executes the action commands to schedule the wind, solar, and hydrogen storage microgrid.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0104] In this application's embodiments, "determine B based on A" means that factor A must be considered when determining B. It is not limited to "B can be determined based solely on A," but should also include: "determine B based on A and C," "determine B based on A, C, and E," "determine C based on A, and further determine B based on C," etc. Additionally, it can include using A as a condition for determining B, for example, "when A meets the first condition, determine B using the first method"; another example, "when A meets the second condition, determine B," etc.; another example, "when A meets the third condition, determine B based on the first parameter," etc. Of course, it can also be a condition where A is a factor in determining B, for example, "when A meets the first condition, determine C using the first method, and further determine B based on C," etc.

[0105] It should also be noted that the terms "target," "first," and "second" in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more.

[0106] In this invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-agent scheduling method for wind-solar-hydrogen storage microgrids, characterized in that, include: Collect raw state data; the raw state data includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid; The original state data is mapped to generate a time-series state vector sequence; Based on the temporal state vector sequence, a first global state vector is generated; The first global state vector is input into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands. The heterogeneous intelligent agent executes the action commands to schedule the wind, solar, and hydrogen storage microgrid.

2. The multi-agent scheduling method for wind-solar-hydrogen storage microgrids according to claim 1, characterized in that, The step of generating the first global state vector based on the temporal state vector sequence includes: The temporal state vector sequence is subjected to time-step feature extraction using a multi-layer long short-term memory network to obtain multiple hidden state sequences; the multiple hidden state sequences include the current hidden state at the current time step and the historical hidden states at multiple historical time steps; Based on the current hidden state, a query vector is generated; Based on the historical hidden state, a historical key vector and a historical value vector are generated; Calculate the similarity score between the query vector and the historical key vector; The similarity scores are normalized to obtain attention weights; Based on the attention weights, the historical value vector is weighted and summed to generate the first global state vector.

3. The multi-agent scheduling method for wind-solar-hydrogen storage microgrids according to claim 1, characterized in that, The heterogeneous intelligent agent includes a policy network; the policy network is specifically used for: Collect the local observation vector of the heterogeneous intelligent agent at the current moment; Based on the first global state vector and the local observation vector, the current action strategy is generated; The current action strategy is sampled to generate the action instruction.

4. The multi-agent scheduling method for wind-solar-hydrogen storage microgrids according to claim 1, characterized in that, The heterogeneous intelligent agent further includes an update module; the update module is specifically used for: Collect the second global state vector of all heterogeneous agents after executing the action command; Based on the second global state vector and the preset multi-objective reward function, the immediate reward corresponding to the action command is calculated; Experience tuple data is constructed using the first global state vector, the action instruction, the immediate reward, and the second global state vector; The action instructions are updated based on the experience tuple data.

5. The multi-agent scheduling method for wind-solar-hydrogen storage microgrids according to claim 4, characterized in that, The step of updating the action instruction based on the experience tuple data includes: Based on the aforementioned empirical tuple data, the individual advantage function of each heterogeneous agent is calculated using the counterfactual baseline method. Based on the individual advantage function, the contribution of each agent to the global value estimation after executing the action instruction is quantified; The action command is updated based on the contribution level.

6. The multi-agent scheduling method for wind-solar-hydrogen storage microgrids according to claim 5, characterized in that, The heterogeneous intelligent agent also includes a value network; the value network is specifically used for: The global value estimate is generated based on the second global state vector.

7. A multi-agent scheduling system for wind-solar-hydrogen storage microgrids, characterized in that, include: The data acquisition module is used to collect raw state data, which includes wind power generation data, photovoltaic power generation data, energy storage state of charge data, hydrogen storage tank pressure data, electrolyzer operating status data, fuel cell operating status data, and load power data of the wind-solar-hydrogen storage microgrid. The mapping module is used to map the original state data to generate a time-series state vector sequence; The generation module is used to generate a first global state vector based on the time-series state vector sequence; The decision module is used to input the first global state vector into a plurality of pre-constructed heterogeneous agents to obtain a plurality of action commands output by the plurality of heterogeneous agents; the heterogeneous agents correspond one-to-one with the action commands; The scheduling module is used to execute the action commands through the heterogeneous intelligent agent to schedule the wind, solar and hydrogen storage microgrid.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-agent scheduling method for wind, solar, and hydrogen storage microgrids as described in any one of claims 1 to 6.

Images