Microgrid scheduling method, device and equipment based on multi-agent reinforcement learning
By employing a multi-agent reinforcement learning approach, the problems of multi-objective optimization and insufficient robustness in microgrid scheduling are addressed, achieving a balance between economy, reliability, and equipment lifespan, thereby improving the operational stability and equipment lifespan of the microgrid.
Patent Information
- Application Number
- CN202511693596.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-18
- Publication Date
- 2026-02-17
AI Technical Summary
Existing microgrid scheduling methods are difficult to handle multi-objective optimization, lack security guarantees, ignore equipment lifespan, and have insufficient robustness, making them unable to effectively cope with complex and ever-changing operating conditions and system uncertainties.
A microgrid scheduling method based on multi-agent reinforcement learning is adopted. By determining the input state vector, an initial scheduling scheme is generated using a preset multi-agent model. The predicted state is verified using a security check function. Based on the state check results, the target scheduling scheme is determined, and control commands are generated to schedule the operation of the microgrid.
It achieves multi-objective synergistic optimization, significantly reduces operating costs, extends equipment lifespan, enhances system robustness, provides security assurance, and adapts to complex operating environments.
Smart Images
Figure CN121546550A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of microgrid control technology, and in particular to a microgrid scheduling method, apparatus, and device based on multi-agent reinforcement learning. Background Technology
[0002] As the global energy structure transitions towards a low-carbon model, the penetration rate of renewable energy sources, such as photovoltaics and wind power, in the power system is increasing daily. However, the inherent intermittency and uncertainty of renewable energy pose significant challenges to the stable operation of the power grid. Microgrids, as an effective form of integrating distributed power sources, energy storage devices, controllable loads, and monitoring and protection devices, enable the flexible and efficient application of distributed energy, improve power supply reliability and power quality, and are an important component of the future energy internet. Composite energy storage systems, especially those combining fast-response, high-energy-density battery energy storage with long-cycle, large-capacity hydrogen energy storage, can fully utilize the advantages of different energy storage technologies to achieve energy complementarity and coordination across multiple time scales, making them a key technology for improving the flexibility and economy of microgrids. However, optimizing the scheduling of such complex composite energy storage systems to achieve a balance between multiple objectives such as economy, reliability, renewable energy absorption rate, and equipment lifespan remains a highly challenging problem.
[0003] Traditional microgrid energy management methods mainly include rule-based methods and model-optimized methods. While rule-based methods are simple, they struggle to adapt to complex and changing operating conditions, neglect equipment lifespan, and are susceptible to environmental influences. Model-optimized methods, such as linear programming, mixed-integer linear programming, and dynamic programming, can find theoretically optimal solutions, but they heavily rely on accurate modeling of the system and environment, resulting in high computational complexity and difficulty in handling the system's high nonlinearity and uncertainty. Summary of the Invention
[0004] This disclosure provides a microgrid scheduling method, apparatus, and device based on multi-agent reinforcement learning, which solves the problems in the prior art where single agents have difficulty handling multi-objective optimization, lack of security guarantees, neglect of equipment lifespan, and insufficient robustness.
[0005] Firstly, a microgrid scheduling method based on multi-agent reinforcement learning is provided, including:
[0006] The input state vector is determined based on the microgrid's operational data; the input state vector includes a first state vector, a second state vector, and a third state vector; the operational data includes at least renewable energy data, local load data, energy storage status data, interactive data, market electricity price and environmental data;
[0007] The first state vector, the second state vector, and the third state vector are respectively input into the corresponding intelligent agent sub-models of the preset multi-agent model to determine the initial scheduling scheme; the preset multi-agent model includes an economic optimization intelligent agent sub-model, a lifetime management intelligent agent sub-model, a renewable energy consumption intelligent agent model, and a decision-maker;
[0008] The predicted state of the microgrid after executing the scheduling scheme is determined using a security verification function. The security verification function is constructed based on the set of security constraints of the microgrid, which includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints.
[0009] Based on the state verification results, a target scheduling scheme is determined; the state verification results are determined based on the predicted state and the current state of the microgrid.
[0010] Control commands are generated according to the target scheduling scheme to schedule the operation of the microgrid.
[0011] Secondly, a microgrid scheduling device based on multi-agent reinforcement learning is provided, comprising:
[0012] An input state vector determination module is used to determine input state vectors based on the microgrid's operational data; the input state vectors include a first state vector, a second state vector, and a third state vector; the operational data includes at least renewable energy data, local load data, energy storage status data, interactive data, market electricity price and environmental data;
[0013] The initial scheduling scheme determination module is used to input the first state vector, the second state vector, and the third state vector into the corresponding intelligent agent sub-models of the preset multi-agent model to determine the initial scheduling scheme; the preset multi-agent model includes an economic optimization intelligent agent sub-model, a lifetime management intelligent agent sub-model, a renewable energy consumption intelligent agent model, and a decision-maker;
[0014] The predicted state determination module is used to determine the predicted state of the microgrid after executing the scheduling scheme using a security verification function; the security verification function is constructed based on the set of security constraints of the microgrid, and the set of security constraints includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints.
[0015] The scheduling scheme determination module is used to determine the target scheduling scheme based on the state verification result; the state verification result is determined based on the predicted state and the current state of the microgrid.
[0016] The scheduling module is used to generate control commands according to the target scheduling scheme, so as to schedule the operation of the microgrid through the control commands.
[0017] Thirdly, an electronic device is provided, comprising:
[0018] At least one processor; and,
[0019] A memory communicatively connected to the at least one processor; wherein,
[0020] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the microgrid scheduling method based on multi-agent reinforcement learning as described in the first aspect above.
[0021] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the microgrid scheduling method based on multi-agent reinforcement learning as described in the first aspect above.
[0022] Fifthly, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the microgrid scheduling method based on multi-agent reinforcement learning as described in the first aspect above.
[0023] This disclosure presents a microgrid scheduling method, apparatus, and device based on multi-agent reinforcement learning. The method includes: determining an input state vector based on microgrid operating data; the input state vector includes a first state vector, a second state vector, and a third state vector; the operating data includes at least renewable energy data, local load data, energy storage status data, interaction data, market electricity price, and environmental data; inputting the first state vector, the second state vector, and the third state vector into corresponding agent sub-models of a preset multi-agent model to determine an initial scheduling scheme; the preset multi-agent model includes an economic optimization agent sub-model and a lifetime management agent sub-model. This technical solution involves three sub-models: an intelligent agent model, a renewable energy consumption intelligent agent model, and a decision-maker. A security verification function is used to determine the predicted state of the microgrid after executing the scheduling scheme. This security verification function is constructed based on a set of security constraints for the microgrid, including at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints. A target scheduling scheme is determined based on the state verification result, which is determined based on the predicted state and the current state of the microgrid. Control commands are generated according to the target scheduling scheme to schedule the operation of the microgrid. This solution extracts multi-dimensional state vectors (renewable energy, load, energy storage, electricity price, etc.) from real-time microgrid operation data and feeds them into three intelligent agent sub-models. The decision-maker quickly generates an initial scheduling scheme. Subsequently, a security verification function is called to determine the predicted state after the initial scheduling scheme is executed, and the result is compared with the current state to provide a verification result. If the verification passes, the scheme becomes the target scheduling scheme, and control commands are generated according to the target scheduling scheme to schedule the operation of the microgrid. This solves the problems in the existing technology where a single agent is unable to handle multi-objective optimization, lacks security guarantees, ignores equipment lifespan, and has insufficient robustness. It realizes multi-objective collaborative optimization, significantly reduces the operating cost of the microgrid, extends the service life of the microgrid equipment, and enhances the robustness of the microgrid system.
[0024] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this disclosure, nor is it intended to limit the scope of the embodiments of this disclosure. Other features of the embodiments of this disclosure will become readily apparent from the following description. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is a flowchart of a microgrid scheduling method based on multi-agent reinforcement learning provided in Embodiment 1 of this disclosure;
[0027] Figure 2 This is a schematic diagram of the structure of a microgrid scheduling device based on multi-agent reinforcement learning provided in Embodiment 2 of this disclosure;
[0028] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this disclosure. Detailed Implementation
[0029] To enable those skilled in the art to better understand the solutions of the embodiments of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the protection scope of the embodiments of this disclosure.
[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0031] Example 1
[0032] Figure 1 This is a flowchart of a microgrid scheduling method based on multi-agent reinforcement learning provided in Embodiment 1 of this disclosure. This embodiment is applicable to the scheduling of microgrids. The method can be executed by a microgrid scheduling device based on multi-agent reinforcement learning. This device can be implemented in hardware and / or software and can be configured in an electronic device, including but not limited to computers, PCs, electronic devices, and servers, etc., which have data processing capabilities. Figure 1 As shown, the method includes:
[0033] S110. Determine the input state vector based on the microgrid's operating data; the input state vector includes a first state vector, a second state vector, and a third state vector; the operating data includes at least renewable energy data, local load data, energy storage status data, interactive data, market electricity price and environmental data.
[0034] In this embodiment, a microgrid can be a small-scale autonomous power generation and distribution system composed of distributed power sources (such as photovoltaics, wind turbines, and diesel generators), energy storage devices, energy conversion devices, loads, and monitoring and protection devices. A microgrid may include photovoltaic (PV) power generation units, wind power generation units (optional), composite energy storage units (lithium-ion battery energy storage system (BESS) and hydrogen energy storage system (HESS) consisting of an electrolyzer-hydrogen storage tank-fuel cell), local loads, and an interface with the main grid (Point of Common Coupling, PCC).
[0035] As described above, the operating data of a microgrid may include: renewable energy data (real-time output power of photovoltaic arrays and / or real-time output power of wind turbines), local load data (real-time power of total local load), energy storage status data (state of charge, health status, internal temperature, and current charging and discharging power of battery energy storage systems; hydrogen storage tank pressure, electrolyzer operating power, and / or fuel cell operating power of hydrogen energy storage systems), interaction data (interaction power with the main grid (positive for purchasing electricity, negative for selling electricity)), and market electricity price and environmental data (time-of-use electricity price, outdoor temperature, and / or solar irradiance, etc.).
[0036] Specifically, an input state vector can be determined based on operational data. This input state vector can include a first state vector, a second state vector, and a third state vector. The first state vector is an economic optimization vector, which can include economically relevant state information such as time-of-use pricing of the power grid, load forecasting, photovoltaic output forecasting, and battery energy storage state (State of Charge, SOC). The second state vector is a lifespan management vector, which can include information related to the energy storage device's state, such as battery state of health (SOH), remaining useful life (RUL), cycle count, charge / discharge rate, depth of discharge, and temperature. The third state vector is a renewable energy consumption vector, which can include renewable energy-related state information such as real-time photovoltaic output, predicted photovoltaic output, load, and hydrogen energy storage status.
[0037] S120. Input the first state vector, the second state vector, and the third state vector into the corresponding intelligent agent sub-models of the preset multi-agent model to determine the initial scheduling scheme; the preset multi-agent model includes an economic optimization intelligent agent sub-model, a life management intelligent agent sub-model, a renewable energy consumption intelligent agent sub-model, and a decision maker.
[0038] In this embodiment, after obtaining the first state vector, the second state vector, and the third state vector, they can be input into the corresponding agent sub-models of a preset multi-agent model, respectively. The initial scheduling scheme can then be determined through the preset multi-agent model. The preset multi-agent model can be a pre-trained multi-agent model, including an economic optimization agent sub-model, a lifetime management agent sub-model, a renewable energy consumption agent sub-model, and a decision-maker.
[0039] It should be explained that the pre-defined multi-agent model can selectively extract information from the global state vector according to its specific optimization objective, form its own local observations, and make decisions based on these observations to obtain the initial scheduling scheme.
[0040] S130. Use a security verification function to determine the predicted state of the microgrid after the execution of the initial scheduling scheme. The security verification function is constructed based on the set of security constraints of the microgrid. The set of security constraints includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints.
[0041] Specifically, a security verification function can be constructed using the set of security constraints of the microgrid. For example, the security verification function can be a Lyapunov function, which can be a non-negative, continuous function that monotonically does not increase with time; as long as the function value always decreases or remains constant, the equilibrium point of the microgrid can be considered stable. The Lyapunov function can be expressed as... This function can quantify the current state. The "distance" from the boundary of the safe zone can typically be constructed based on the degree of constraint violation, for example: ,in Indicates the first A constraint is violated if and only if the state... When within a safe zone, .
[0042] Based on the above description, all inviolable physical constraints and microgrid operating rules can be formally defined as a set of safety constraints. This set of safety constraints includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints.
[0043] For example, the battery energy storage state constraint can be: (like: ),in, This could be the battery's energy storage state. This can be the battery's minimum energy storage state. This represents the battery's maximum energy storage state. The hydrogen storage capacity constraint can be: ,in, It could be hydrogen storage capacity. This can be the minimum capacity for hydrogen energy storage. This could be the maximum capacity of hydrogen storage. Equipment power constraints could mean that the charging / discharging / operating power of each device (battery, electrolyzer, fuel cell) must not exceed its rated maximum power. Grid interconnection power constraints: ,in, It can represent the power exchange between power grids. This can represent the minimum value of the power exchange between power grids. This can represent the maximum value of power exchanged between power grids. Hard constraints on power balance: ,in, It can represent the sum of all generated power in the microgrid at time t. It can represent the total power consumption at time t.
[0044] As described above, the constructed security verification function can be used to determine the predicted state of the microgrid after the initial scheduling scheme is executed.
[0045] S140. Determine the target scheduling scheme based on the state verification results; the state verification results are determined based on the predicted state and the current state of the microgrid.
[0046] It is known that by using the constructed safety verification function to determine the predicted state of the microgrid after executing the initial scheduling scheme, the difference between the predicted state and the current state of the microgrid can be used to determine the state verification result. After the state verification result is determined, it can be used to judge whether the initial scheduling scheme is safe and feasible. If the difference is within the allowable range, the initial scheduling scheme is set as the target scheduling scheme to be executed; otherwise, it is rolled back for correction or recalculation.
[0047] S150. Generate control commands based on the target scheduling scheme to schedule the operation of the microgrid through the control commands.
[0048] Specifically, after determining the target scheduling scheme, control commands can be generated based on the scheme. These commands can be control signals that the microgrid can recognize. The generated control commands can be sent to the corresponding controllable devices in the microgrid to schedule their operation. These controllable devices can include battery power conversion systems, controllers for electrolyzers and fuel cells, and controllers at the connection points with the main grid.
[0049] This embodiment provides a microgrid scheduling method based on multi-agent reinforcement learning, including: determining an input state vector based on microgrid operating data; the input state vector includes a first state vector, a second state vector, and a third state vector; the operating data includes at least renewable energy data, local load data, energy storage status data, interaction data, market electricity price, and environmental data; inputting the first state vector, the second state vector, and the third state vector into corresponding agent sub-models of a preset multi-agent model to determine an initial scheduling scheme; the preset multi-agent model includes an economic optimization agent sub-model and a lifetime management agent sub-model. This invention comprises a renewable energy consumption intelligent agent sub-model and a decision-maker; a security verification function is used to determine the predicted state of the microgrid after executing the scheduling scheme; the security verification function is constructed based on the set of security constraints of the microgrid, which includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints; a target scheduling scheme is determined based on the state verification result; the state verification result is determined based on the predicted state and the current state of the microgrid; control commands are generated according to the target scheduling scheme to schedule the operation of the microgrid. This technical solution solves the problems of existing technologies, such as the difficulty of single intelligent agents in handling multi-objective optimization, lack of security guarantees, neglect of equipment lifespan, and insufficient robustness. It achieves multi-objective collaborative optimization, significantly reduces the operating cost of microgrids, extends the service life of microgrid equipment, and enhances the robustness of microgrid systems.
[0050] As an optional implementation of this embodiment, the microgrid scheduling method based on multi-agent reinforcement learning provided in this embodiment further includes, before determining the input state vector based on the microgrid's operating data:
[0051] 1) Obtain the initial operating data of the microgrid.
[0052] Specifically, initial operating data of the microgrid can be collected in real time through sensors and metering devices deployed throughout the microgrid. This initial operating data includes, but is not limited to, photovoltaic output data, load power data, grid electricity price data, and energy storage status data.
[0053] 2) Preprocess the initial running data to obtain the running data.
[0054] Specifically, after obtaining the initial running data, the initial running data can be preprocessed to obtain the processed running data. The preprocessing can include operations such as outlier removal and missing value imputation on the collected initial running data to ensure data quality.
[0055] As an optional implementation of this embodiment, determining the input state vector based on the microgrid's operating data includes:
[0056] 1) Based on the operational data and weather information, a set of prediction sequences within a preset time period is determined using a preset prediction model. The set of prediction sequences includes a photovoltaic power output prediction sequence, a load prediction sequence, a microgrid electricity price prediction sequence, and the standard deviation corresponding to each prediction sequence.
[0057] Specifically, weather information can be obtained, such as weather forecasts. Based on operational data and weather information, a set of prediction sequences within a preset time period is determined using a pre-set prediction model. This pre-trained prediction model can be a long short-term memory network or a gated recurrent unit (GRU). The set of prediction sequences can include photovoltaic power output prediction sequences, load prediction sequences, microgrid electricity price prediction sequences, and the standard deviations corresponding to each prediction sequence.
[0058] For example, short-term forecasting models (such as Long Short-Term Memory networks (LSTM) or Gated Recurrent Units (GRUs) generate photovoltaic output forecast sequences for the next N time steps (e.g., N=24 hours) based on historical data and weather forecast information. Load forecast sequence and electricity price forecast series Simultaneously, the uncertainty of the prediction results is quantified to obtain the standard deviation of the prediction error. and .
[0059] 2) Based on the operational data, the predicted sequence set, and the energy storage device status, construct the input state vector; the energy storage device status includes at least the health status of the energy storage device and the remaining usage time of the energy storage device.
[0060] Specifically, the input state vector can be constructed from the operating data, the predicted sequence set, and the energy storage device status. For example, the operating data, the predicted sequence set, uncertainty data (such as the standard deviation of the prediction error), and state information fed back from other modules (such as SOH, Lyapunov function values, etc.) can be combined into a unified, high-dimensional input state vector S(t). This input state vector contains all the information describing the current complete state of the microgrid and has undergone normalization processing. This input state vector will serve as the data basis for each agent in the subsequent multi-agent cooperative scheduling module to generate its local observations, rather than being directly used as input to each agent.
[0061] It should be noted that the status of energy storage devices includes at least their health status and remaining service life. Key factors affecting the lifespan of energy storage devices can be obtained, such as cycle count, depth of discharge (DoD), and charge / discharge rate. And battery temperature. By monitoring these parameters in real time, the current SOH (State of Health) of the battery (i.e., the health status of the energy storage device) can be dynamically assessed. For example, an empirical model can be used: ,in This is the increment of the equivalent loop count. Where T can be the battery temperature and C can be the charge / discharge rate. This could be "state of health degradation," which indicates the extent to which the state of health (SOH) decreases after one equivalent cycle. It is the battery's health status value (percentage) at the current time t. It was the previous moment The module measures the battery's health status value. Based on the current SOH value and recent average degradation rate, it can predict when the battery will reach end-of-life conditions (e.g., SOH drops to a certain level). How much longer or how many cycles are needed, i.e., the remaining service life (RUL). ,in, It could be the battery health status measured at the current time t. It can indicate the battery's end-of-life health status as specified by the battery manufacturer. It can represent the average rate of decline in health status per unit of time.
[0062] As an optional implementation of this embodiment, determining the target scheduling scheme based on the state verification result includes:
[0063] 1) Determine whether the status verification result meets the preset judgment conditions.
[0064] In this embodiment, the preset judgment condition can be a pre-set condition for judging the state verification result. For example, the preset judgment condition can be that the state verification result is less than a preset threshold, and the preset threshold can be 0.
[0065] 2) If the conditions are met, the initial scheduling scheme shall be determined as the target scheduling scheme.
[0066] Specifically, if the status verification result meets the preset judgment conditions, the initial scheduling scheme can be determined as the target scheduling scheme.
[0067] 3) If the conditions are not met, the initial scheduling scheme is modified using a preset security correction strategy, and the modified scheme is determined as the target scheduling scheme.
[0068] As described above, if the status verification result does not meet the preset judgment conditions, the initial scheduling scheme can be modified using a preset security correction strategy, and the modified scheme can be determined as the target scheduling scheme. The preset security correction strategy can be a strategy pre-set to modify the initial scheduling scheme.
[0069] For example, the state verification result can be represented as ,in, This can represent the predicted state of a microgrid after the initial scheduling scheme is executed, determined using a security check function. This can represent the current state of the microgrid. If... This indicates that the initial scheduling scheme is safe, and the microgrid's system state will remain or move further away from the unsafe boundary. In this case, the initial scheduling scheme can be executed.
[0070] Following the above description, if This indicates that the initial scheduling scheme is dangerous and may cause the microgrid system state to exceed the safety boundary. In this case, the initial scheduling scheme can be rejected, and a built-in safety correction strategy can be initiated. The goal of this strategy is to find a new scheduling scheme that minimizes the increment of the Lyapunov function, thereby determining the new scheduling scheme and setting the corrected new scheduling scheme as the target scheduling scheme.
[0071] As an optional implementation of this embodiment, the step of inputting the first state vector, the second state vector, and the third state vector into the corresponding agent sub-models of the preset multi-agent model to determine the initial scheduling scheme includes:
[0072] 1) Input the first state vector into the economic optimization intelligent agent sub-model to obtain the microgrid power dispatch behavior.
[0073] In this embodiment, the economic optimization agent sub-model can be a pre-trained agent model. The main objective of the economic optimization agent sub-model is to minimize the short-term operating cost of the microgrid. Its reward function can be defined as the negative value of the operating cost, and the reward function can be expressed as: Then we have:
[0074]
[0075] in This can cover the cost of losses during the energy storage conversion process. This can be the power purchased from the main grid at time t. The electricity purchase price at time t can be used. This can be the power supplied to the main grid at time t. This can be the electricity price at time t. The agent's local observations mainly consist of economically relevant state information, such as the grid time-of-use price, load forecast, photovoltaic output forecast, and battery state of charge (SOC) extracted from the global state vector. Based on this input, the agent primarily makes decisions regarding strategies related to the grid's electricity purchase and sale behavior.
[0076] Specifically, the first state vector can be input into the economic optimization intelligent agent sub-model to obtain the microgrid power dispatch behavior, such as power purchase and sale plans.
[0077] 2) Input the second state vector into the lifetime management agent sub-model to determine the microgrid charge and discharge limiting strategy.
[0078] In this embodiment, the lifespan management agent sub-model focuses on protecting energy storage devices and extending their lifespan, thereby minimizing the long-term, implicit costs of the system. Its reward function can be defined as the negative of the device degradation cost. The agent's local observations mainly consist of information related to the health status of the energy storage device, such as the state of battery health (SOH), remaining useful life (RUL), cycle count, charge / discharge rate, depth of discharge, and temperature extracted from the global state vector. Based on this input, the agent mitigates device aging by suggesting adjustments to charge / discharge power and limiting the depth of discharge. It should be noted that the abstract concept of "health" can also be transformed into a concrete economic cost. For example, calculating the equivalent degradation cost per cycle: ,in It's the battery replacement cost. This is its total design cycle life. This cost value will be used as a penalty signal input into the reward function of the lifespan management agent sub-model.
[0079] As described above, the second state vector is input into the lifetime management agent sub-model to determine the microgrid charging and discharging limitation strategy, such as: depth of discharge limitation and / or power adjustment amount, etc.
[0080] 3) Input the third state vector into the renewable energy consumption intelligent agent sub-model to determine the microgrid power adjustment strategy.
[0081] In this embodiment, the renewable energy consumption intelligent agent sub-model aims to maximize the utilization rate of local renewable energy. Its reward function... Defined as a negative value for the curtailment penalty for light / wind curtailment: ,in, This can represent the penalty weight for curtailing solar / wind power, used to quantify the penalty cost or mismatch consequence of reducing electricity output per unit. This can reduce the power output of renewable energy sources, i.e., "curtailed solar / wind power". The agent's local observations mainly consist of state information related to renewable energy consumption, such as real-time photovoltaic output, predicted photovoltaic output, current load, and hydrogen storage status extracted from the input state vector. Based on this input, during periods of high photovoltaic power generation, the agent tends to decide to increase the power of the electrolyzer to produce hydrogen, converting excess electrical energy into hydrogen for storage.
[0082] Specifically, the third state vector can be input into the renewable energy consumption intelligent agent sub-model, thereby determining the microgrid power adjustment strategy, such as the electrolyzer start-up and shutdown plan and / or power allocation scheme.
[0083] 4) Input the microgrid power dispatch behavior, the microgrid charging and discharging limitation strategy, and the microgrid power adjustment strategy into the decision-maker to determine the initial dispatch scheme through the decision-maker.
[0084] In this embodiment, the decision-maker can be a Pareto optimization decision-maker, which can be a multi-objective conflict coordination mechanism. It first optimizes multiple objectives such as economy, energy storage lifetime, and renewable energy consumption simultaneously to obtain a set of Pareto optimal solutions that "cannot be improved by each other". Then, it selects a compromise optimal solution as the final scheduling scheme through preset weights, preferences or interaction rules to achieve the best balance of the overall performance of the microgrid.
[0085] Specifically, the microgrid power dispatch behavior, the microgrid charging and discharging limitation strategy, and the microgrid power adjustment strategy can be input into the decision-maker, which can then determine the initial dispatch scheme.
[0086] It should be noted that the pre-defined multi-agent model can adopt a "centralized training, decentralized execution" model. During offline training, all agents share one or more centralized critic networks, which evaluate the value of the global state and joint actions, effectively guiding each agent to learn how to collaborate with other agents. During online execution, each agent can quickly generate action suggestions based solely on its local observations and the pre-trained policy network, ensuring real-time decision-making. Simultaneously, the reward function for each sub-agent model is an adaptive reward function, replacing traditional fixed weights or reference value tracking methods. Specifically, a smooth reward function based on the SOC boundary is adopted, using the tanh function to provide a smooth penalty signal when the SOC approaches its upper and lower limits, giving agents greater optimization freedom within a safe range. Furthermore, an adaptive weight mechanism is introduced, dynamically adjusting the weight of each sub-agent in the total reward based on the completion of its sub-objectives, achieving dynamic balance among the sub-objectives. Robust optimization techniques are also incorporated into the training process of the pre-defined multi-agent model to address uncertainty. For example, adversarial training can be used to train an additional agent as an adversary to simulate worst-case scenarios of renewable energy or load fluctuations, forcing the pre-defined multi-agent model to learn strategies that still perform well under these worst-case conditions.
[0087] Meanwhile, the pre-set multi-agent model can also perform online learning and policy fine-tuning. It can continuously collect real-world operational data and store it in an experience replay pool. This new data can be used to periodically fine-tune the pre-set multi-agent model, enabling the scheduling strategy to adapt to changing power grid environments, load patterns, or equipment aging, achieving true continuous evolution and lifelong learning.
[0088] As an optional implementation of this embodiment, the step of inputting the microgrid power dispatch behavior, the microgrid charge / discharge limitation strategy, and the microgrid power adjustment strategy into the decision-maker to determine the initial dispatch scheme through the decision-maker includes:
[0089] 1) Based on the microgrid power dispatching behavior, the microgrid charging and discharging limitation strategy, and the microgrid power adjustment strategy, the decision-maker determines the set of dispatching schemes.
[0090] Specifically, the power dispatching behavior, charging and discharging limitation strategies, and power adjustment strategies of the microgrid can be used by a decision-maker to determine a set of dispatching schemes. This set of schemes can be composed of non-dominated solutions. A non-dominated solution can be understood as a solution for which no other solution is better in all objectives, and at least one objective is stronger. In other words, it is optimal in at least one objective and not worse than other solutions in others, thus not being completely overwhelmed by other solutions and belonging to the candidate solutions on the Pareto front.
[0091] 2) Determine the initial scheduling scheme from the set of scheduling schemes according to the preset decision algorithm.
[0092] In this embodiment, the preset decision algorithm can be a pre-defined algorithm, which may be a weighted approach preset by the operator (e.g., prioritizing economic efficiency during peak electricity price periods and battery maintenance at night) or a multi-criteria decision-making method. The initial scheduling scheme can be determined from the set of scheduling schemes based on the preset decision algorithm.
[0093] For example, the decision-maker first collects candidate actions or policy suggestions from the three agents mentioned above. Since the agents' objectives may conflict (e.g., increasing battery deep discharge to reduce costs, thus impairing battery life), the decision-maker handles this conflict through Pareto optimization. It first identifies the non-dominated solutions among all candidate actions, forming a Pareto optimal set (i.e., the set of scheduling schemes). Then, based on the operator's preset weights (e.g., prioritizing economy during peak electricity price periods and battery maintenance at night) or using a multi-criteria decision-making method (such as TOPSIS), it selects the final action A(t) that best balances current demand from the Pareto optimal set; that is, the initial scheduling scheme.
[0094] The technical effects that the above technical solution can achieve include:
[0095] 1) Achieved multi-objective collaborative optimization: Through an innovative multi-agent framework and Pareto optimization decision-making mechanism, the complex single optimization problem is decomposed into sub-problems handled by multiple agents, effectively solving the balance and coordination among multiple conflicting objectives such as economy, equipment life, safety, and absorption rate, and improving the overall performance of the system.
[0096] 2) Provides provable security assurance: For the first time, security reinforcement learning based on Lyapunov theory is introduced into the field of composite energy storage scheduling. Through an independent, physically constrained security verification algorithm, the security of the microgrid system is guaranteed throughout the training and operation process, avoiding dangerous operations and greatly enhancing the reliability and deployability of the technology in actual physical systems.
[0097] 3) Significantly extended the lifespan of energy storage devices: By establishing a dedicated lifespan management sub-agent and combining the health status and remaining usage time of energy storage devices, the total lifespan cost of the devices is incorporated into the optimization objective. The learned scheduling strategy can effectively reduce battery wear and tear, extend battery lifespan, and significantly reduce the long-term operating costs of the system.
[0098] 4) Enhanced system robustness to uncertainty: By introducing robust optimization techniques such as adversarial training, the trained scheduling strategy can maintain high performance and stability even when there are large errors in renewable energy and load forecasting, making it more suitable for the complex operating environment of microgrids in the real world.
[0099] Example 2
[0100] Figure 2 This is a schematic diagram of a microgrid scheduling device based on multi-agent reinforcement learning, provided in Embodiment 2 of this disclosure; as shown... Figure 2 As shown, the device includes: an input state vector determination module 210, an initial scheduling scheme determination module 220, a predicted state determination module 230, a scheduling scheme determination module 240, and a scheduling module 250.
[0101] The input state vector determination module 210 is used to determine the input state vector based on the microgrid's operating data; the input state vector includes a first state vector, a second state vector, and a third state vector; the operating data includes at least renewable energy data, local load data, energy storage status data, interactive data, market electricity price and environmental data;
[0102] The initial scheduling scheme determination module 220 is used to input the first state vector, the second state vector, and the third state vector into the corresponding intelligent agent sub-models of the preset multi-agent model to determine the initial scheduling scheme; the preset multi-agent model includes an economic optimization intelligent agent sub-model, a lifetime management intelligent agent sub-model, a renewable energy consumption intelligent agent model, and a decision-maker;
[0103] The prediction state determination module 230 is used to determine the predicted state of the microgrid after executing the scheduling scheme using a security verification function; the security verification function is constructed based on the set of security constraints of the microgrid, and the set of security constraints includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints.
[0104] The scheduling scheme determination module 240 is used to determine the target scheduling scheme based on the state verification result; the state verification result is determined based on the predicted state and the current state of the microgrid.
[0105] The scheduling module 250 is used to generate control commands according to the target scheduling scheme, so as to schedule the operation of the microgrid through the control commands.
[0106] Embodiment 2 of this disclosure provides a microgrid scheduling device based on multi-agent reinforcement learning, which solves the problems in the prior art where a single agent is unable to handle multi-objective optimization, lacks security guarantees, ignores equipment lifespan, and has insufficient robustness. It realizes multi-objective collaborative optimization, significantly reduces the operating cost of microgrids, extends the service life of microgrid equipment, and enhances the robustness of microgrid systems.
[0107] Furthermore, the initial scheduling scheme determination module 220 also includes:
[0108] The first determining unit is used to input the first state vector into the economic optimization intelligent agent sub-model to obtain the microgrid power dispatch behavior;
[0109] The second determining unit is used to input the second state vector into the lifetime management intelligent agent sub-model to determine the microgrid charging and discharging limitation strategy;
[0110] The third determining unit is used to input the third state vector into the renewable energy consumption intelligent agent sub-model to determine the microgrid power adjustment strategy;
[0111] The decision unit is used to input the microgrid power dispatch behavior, the microgrid charging and discharging limitation strategy, and the microgrid power adjustment strategy into the decision generator, so as to determine the initial dispatch scheme through the decision generator.
[0112] Furthermore, the decision-making unit is also used for:
[0113] Based on the microgrid power dispatching behavior, the microgrid charging and discharging limitation strategy, and the microgrid power adjustment strategy, the decision-maker determines a set of dispatching schemes.
[0114] The initial scheduling scheme is determined from the set of scheduling schemes according to a preset decision algorithm.
[0115] Furthermore, the input state vector determination module 210 is used for:
[0116] Based on the operational data and weather information, a set of prediction sequences within a preset time period is determined using a preset prediction model. The set of prediction sequences includes a photovoltaic power output prediction sequence, a load prediction sequence, a microgrid electricity price prediction sequence, and the standard deviation of each prediction sequence.
[0117] Based on the operational data, the predicted sequence set, and the energy storage device status, the input state vector is constructed; the energy storage device status includes at least the health status of the energy storage device and the remaining usage time of the energy storage device.
[0118] Furthermore, the scheduling scheme determination module 240 is also used for:
[0119] Determine whether the status verification result meets the preset judgment conditions;
[0120] If the conditions are met, the initial scheduling scheme will be determined as the target scheduling scheme.
[0121] If the conditions are not met, the initial scheduling scheme is modified using a preset security correction strategy, and the modified scheme is determined as the target scheduling scheme.
[0122] Furthermore, the device also includes:
[0123] The data acquisition module is used to acquire the initial operating data of the microgrid;
[0124] The preprocessing module is used to preprocess the initial running data to obtain the running data.
[0125] The microgrid scheduling device based on multi-agent reinforcement learning provided in this disclosure can execute the microgrid scheduling method based on multi-agent reinforcement learning provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.
[0126] Example 3
[0127] Figure 3 A schematic diagram of the structure of an electronic device 10 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the embodiments of the present disclosure described and / or claimed herein.
[0128] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0129] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0130] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microprocessor, etc. Processor 11 performs the various methods and processes described above, such as a microgrid scheduling method based on multi-agent reinforcement learning.
[0131] In some embodiments, the microgrid scheduling method based on multi-agent reinforcement learning can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the microgrid scheduling method based on multi-agent reinforcement learning described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the microgrid scheduling method based on multi-agent reinforcement learning by any other suitable means (e.g., by means of firmware).
[0132] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0133] Computer programs for implementing the methods of embodiments of this disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0134] In the context of embodiments of this disclosure, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0136] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0137] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0138] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the embodiments of this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of the embodiments of this disclosure can be achieved, and this document does not impose any limitations.
[0139] The specific embodiments described above do not constitute a limitation on the scope of protection of the embodiments disclosed herein. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the embodiments disclosed herein should be included within the scope of protection of the embodiments disclosed herein.
[0140] This disclosure also provides a computer program product, including a computer program and / or instructions, which, when executed by a processor, implements the microgrid scheduling method based on multi-agent reinforcement learning as provided in any embodiment of this application.
[0141] In implementing a computer program product, computer program code for performing the operations of the embodiments of this disclosure can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0142] Note that the above are merely preferred embodiments and the technical principles applied in this disclosure. Those skilled in the art will understand that this disclosure is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the protection scope of this disclosure. Therefore, although the embodiments of this disclosure have been described in detail above, this disclosure is not limited to the above embodiments. More other equivalent embodiments may be included without departing from the concept of this disclosure, and the scope of this disclosure is determined by the scope of the appended claims.
Claims
1. A microgrid scheduling method based on multi-agent reinforcement learning, characterized in that, include: The input state vector is determined based on the microgrid's operational data; the input state vector includes a first state vector, a second state vector, and a third state vector; the operational data includes at least renewable energy data, local load data, energy storage status data, interactive data, market electricity price and environmental data; The first state vector, the second state vector, and the third state vector are respectively input into the corresponding intelligent agent sub-models of the preset multi-agent model to determine the initial scheduling scheme; The preset multi-agent model includes an economic optimization agent sub-model, a life management agent sub-model, a renewable energy consumption agent sub-model, and a decision-maker; The predicted state of the microgrid after executing the scheduling scheme is determined using a security verification function. The security verification function is constructed based on the set of security constraints of the microgrid, which includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints. Based on the state verification results, a target scheduling scheme is determined; the state verification results are determined based on the predicted state and the current state of the microgrid. Control commands are generated according to the target scheduling scheme to schedule the operation of the microgrid.
2. The method according to claim 1, characterized in that, The step of inputting the first state vector, the second state vector, and the third state vector into the corresponding agent sub-models of the preset multi-agent model to determine the initial scheduling scheme includes: The first state vector is input into the economic optimization intelligent agent sub-model to obtain the microgrid power dispatch behavior; The second state vector is input into the lifetime management agent sub-model to determine the microgrid charge and discharge limitation strategy; The third state vector is input into the renewable energy consumption intelligent agent sub-model to determine the microgrid power adjustment strategy; The microgrid power dispatching behavior, the microgrid charging and discharging limitation strategy, and the microgrid power adjustment strategy are input into the decision-maker to determine the initial dispatching scheme.
3. The method according to claim 2, characterized in that, The step of inputting the microgrid power dispatch behavior, the microgrid charge / discharge limitation strategy, and the microgrid power adjustment strategy into the decision-maker to determine the initial dispatch scheme includes: Based on the microgrid power dispatching behavior, the microgrid charging and discharging limitation strategy, and the microgrid power adjustment strategy, the decision-maker determines a set of dispatching schemes. The initial scheduling scheme is determined from the set of scheduling schemes according to a preset decision algorithm.
4. The method according to claim 1, characterized in that, The determination of the input state vector based on the microgrid's operational data includes: Based on the operational data and weather information, a set of prediction sequences within a preset time period is determined using a preset prediction model. The set of prediction sequences includes a photovoltaic power output prediction sequence, a load prediction sequence, a microgrid electricity price prediction sequence, and the standard deviation of each prediction sequence. Based on the operational data, the predicted sequence set, and the energy storage device status, the input state vector is constructed; the energy storage device status includes at least the health status of the energy storage device and the remaining usage time of the energy storage device.
5. The method according to claim 1, characterized in that, The step of determining the target scheduling scheme based on the status verification result includes: Determine whether the status verification result meets the preset judgment conditions; If the conditions are met, the initial scheduling scheme will be determined as the target scheduling scheme. If the conditions are not met, the initial scheduling scheme is modified using a preset security correction strategy, and the modified scheme is determined as the target scheduling scheme.
6. The method according to claim 1, characterized in that, Before determining the input state vector based on the microgrid's operational data, the method further includes: Obtain the initial operating data of the microgrid; The initial running data is preprocessed to obtain the running data.
7. A microgrid scheduling device based on multi-agent reinforcement learning, characterized in that, include: An input state vector determination module is used to determine input state vectors based on the microgrid's operational data; the input state vectors include a first state vector, a second state vector, and a third state vector; the operational data includes at least renewable energy data, local load data, energy storage status data, interactive data, market electricity price and environmental data; The initial scheduling scheme determination module is used to input the first state vector, the second state vector, and the third state vector into the corresponding intelligent agent sub-models of the preset multi-agent model, respectively, in order to determine the initial scheduling scheme. The preset multi-agent model includes an economic optimization agent sub-model, a life management agent sub-model, a renewable energy consumption agent sub-model, and a decision-maker; The predicted state determination module is used to determine the predicted state of the microgrid after executing the scheduling scheme using a security verification function; the security verification function is constructed based on the set of security constraints of the microgrid, and the set of security constraints includes at least battery energy storage state constraints, energy storage capacity constraints, equipment power constraints, grid interaction power constraints, and power balance constraints. The scheduling scheme determination module is used to determine the target scheduling scheme based on the state verification result; the state verification result is determined based on the predicted state and the current state of the microgrid. The scheduling module is used to generate control commands according to the target scheduling scheme, so as to schedule the operation of the microgrid through the control commands.
8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the microgrid scheduling method based on multi-agent reinforcement learning as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the microgrid scheduling method based on multi-agent reinforcement learning as described in any one of claims 1-6.
10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the microgrid scheduling method based on multi-agent reinforcement learning as described in any one of claims 1-6.