Artificial intelligence enabled microgrid energy management and economic operation system

Abstract

This invention discloses an AI-enabled microgrid energy management and economic operation system, relating to the field of microgrid energy management technology. It includes a data acquisition module that collects real-time operational data from distributed generation units, energy storage units, and load units, as well as external market environment data; a scenario construction module that builds a set of dynamic operating scenarios across multiple time scales based on multi-source data; a learning and decision-making module that inputs the scenarios into a multi-objective reinforcement learning decision model to generate scenario-adaptive energy dispatch strategies and market trading instructions; an execution and control module that controls generation and energy storage output and executes power interactions; and a rolling optimization module that monitors status and prices and provides feedback on optimization strategies. This invention replaces static libraries with dynamic scenario construction and combines multi-objective reinforcement learning to balance economic and reliability objectives, thereby improving the accuracy, economic efficiency, and power supply reliability of microgrid energy management.

Description

Technical Field

[0001] This invention belongs to the field of microgrid energy management technology, specifically an AI-enabled microgrid energy management and economic operation system. Background Technology

[0002] Microgrids need to coordinate the real-time output of distributed generation units, energy storage units, and load units while participating in main grid or electricity market transactions to achieve a balance between economic operation and power supply reliability. Existing technologies have built energy management and control systems around this goal, adjusting the power interaction of each component through preset strategies or simple optimization algorithms.

[0003] Existing technical solutions have shortcomings. Scenario construction often relies on static historical data to generate typical scenario libraries, failing to integrate real-time operational data and dynamic changes in the external market environment. This leads to a disconnect between scenarios and actual operating conditions, making it difficult to cope with complex situations such as fluctuations in wind and solar power output, sudden load changes, and real-time electricity price adjustments. Decision-making models are often driven by a single economic objective, neglecting the synergy of multiple objectives such as energy storage lifespan loss and power supply reliability. Furthermore, the high complexity of these models makes it difficult to generate strategies adapted to different scenarios online, resulting in a mismatch between dispatch instructions and actual needs.

[0004] To address the aforementioned problems, this invention constructs a dynamic set of operating scenarios covering multiple time scales by collecting multi-source data such as the output of distributed generation units, the status of energy storage units, load demand, and external electricity prices and market rules in real time. This replaces the static scenario library to cope with real-time changes. At the same time, a multi-objective reinforcement learning decision model is introduced, which takes the dynamic scenarios as input and outputs energy scheduling strategies and market trading instructions adapted to each scenario. This solves the problem of multi-objective imbalance under single-objective optimization and improves the system's decision-making accuracy and overall operational efficiency in a variable environment. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art; To this end, the present invention proposes an AI-enabled microgrid energy management and economic operation system, comprising: The data acquisition module is used to collect real-time operating data and external market environment data from distributed generation units, energy storage units, and load units in the microgrid system. The scenario construction module constructs a set of dynamic operating scenarios with multiple time scales based on the real-time operating data and external market environment data. The learning decision module is used to input the set of dynamic operating scenarios into a pre-trained multi-objective reinforcement learning decision model to generate energy scheduling strategies and market trading instructions adapted to each scenario. The execution control module controls the output of distributed generation units and energy storage units based on the energy dispatch strategy, and executes power interaction with the main grid or electricity market based on the market trading instructions. The rolling optimization module monitors the operating status and market clearing price of the microgrid system in real time, and feeds the monitoring data back to the multi-objective reinforcement learning decision model to initiate rolling optimization of the strategy.

[0006] Furthermore, the construction of a dynamic set of operating scenarios encompassing multiple time scales based on the real-time operating data and external market environment data specifically includes: The real-time operating data is preprocessed to extract the predicted output curve of the distributed generation unit, the predicted demand curve of the load unit, and the real-time state of charge of the energy storage unit. Based on the aforementioned external market environment data, analyze future time-of-use electricity price information, renewable energy quota indicators, and grid dispatch instructions; The predicted output curve, predicted demand curve, real-time state of charge, time-of-use electricity price information, renewable energy quota indicators and grid dispatch instructions are classified and aligned according to three time scales: ultra-short-term, short-term and medium-term. Based on historical data and weather forecasts, scenario generation technology is used to simulate the uncertainty of renewable energy output, the randomness of load fluctuations, and the possibility of market price fluctuations at different time scales in the future. By combining the various uncertain sequences generated by simulation with the basic data after classification and alignment, multiple dynamic operating scenarios with probabilistic characteristics are formed, and a set of dynamic operating scenarios is constructed.

[0007] Furthermore, the step of inputting the set of dynamic operating scenarios into a pre-trained multi-objective reinforcement learning decision model to generate energy scheduling strategies and market trading instructions adapted to each scenario specifically includes: Construct a multi-objective reinforcement learning decision model with the optimization objectives of minimizing microgrid operating costs, maximizing renewable energy consumption, and smoothing grid interaction power. The input state space of the multi-objective reinforcement learning decision model includes various data in the dynamic operation scenario set, and its action space includes the set power values ​​of each controllable power generation unit and energy storage unit, as well as the power declaration values ​​for buying and selling electricity with the external market. The multi-objective reinforcement learning decision model is trained offline using an experience replay library containing historical operational data and market transaction data. The model parameters are updated through a multi-objective policy gradient algorithm, enabling the model to learn to balance multiple optimization objectives while satisfying system constraints. During the online application phase, the dynamically constructed set of running scenarios is input into the trained multi-objective reinforcement learning decision model. For each dynamic operating scenario, the multi-objective reinforcement learning decision model outputs a set of optimal action sequences. After the action sequences are decoded by the policy interpreter, corresponding energy scheduling strategies and market trading instructions are generated.

[0008] Furthermore, the offline training of the multi-objective reinforcement learning decision model using an experience replay library containing historical operational data and market transaction data specifically includes: Extract time-series aligned generation data, load data, energy storage data, market price data, and actual transaction data from the microgrid historical database and the electricity market historical database; The extracted historical data is constructed into experience tuples that include status, action, immediate reward, next status, and goal achievement marker; The experience tuples are stored in a fixed-capacity experience replay library and indexed according to the time attributes and scene types of the data. During training, a batch of experience tuples are randomly sampled from the experience replay library to calculate the multi-objective policy gradient of the multi-objective reinforcement learning decision model. Based on the calculated multi-objective policy gradient, update the policy network and value network parameters of the multi-objective reinforcement learning decision model, and iterate this process until the model converges.

[0009] Furthermore, the step of controlling the output of distributed generation units and energy storage units based on the energy dispatch strategy, and executing power interaction with the main grid or electricity market based on the market trading instructions, specifically includes: The energy dispatch strategy is parsed into a specific, time-stamped sequence of control commands, which includes the power setpoint of each distributed generation unit, the charging and discharging power of each energy storage unit, and the operating mode. The control command sequence is sent to the corresponding distributed generation unit actuator and energy storage unit management system through the microgrid central controller or local controller to achieve precise control of the output. The market transaction instructions are parsed into power purchase curves, power sales curves, and corresponding price information submitted on the power market trading platform. Before the deadline for the electricity market's designated trading period, the buyer submits the power purchase curve, seller power curve, and price quote information through the market interface agent to complete the transaction declaration; During trading hours, based on market clearing results, power interactions involving the buying or selling of electricity are actually executed through the common connection point between the microgrid and the main grid.

[0010] Furthermore, the real-time monitoring of the microgrid system's operating status and market clearing price, and the feedback of the monitoring data to the multi-objective reinforcement learning decision model to initiate rolling optimization of the strategy, specifically includes: By deploying sensors and metering devices at each node of the microgrid, the system frequency, node voltage, line power flow, actual output of the power generation unit, actual state of charge of the energy storage unit, and actual power of the load unit are collected in real time to form an operational status monitoring dataset. By using the electricity market information release interface, the latest market clearing prices, system marginal electricity prices, and ancillary service prices can be obtained in real time to form a market price monitoring dataset; The operational status monitoring dataset and the market price monitoring dataset are merged and timestamped to form a real-time feedback dataset; The real-time feedback dataset is compared with the currently executing energy scheduling strategy and the market trading instructions to calculate the strategy execution deviation; If the policy execution deviation exceeds a preset threshold or reaches a preset rolling optimization cycle, the real-time feedback dataset is used as a new initial state to trigger the multi-objective reinforcement learning decision model to generate a new round of policy and output instructions, thereby achieving rolling optimization.

[0011] Furthermore, the simulation of the uncertainty of renewable energy output at different time scales in the future, based on historical data and weather forecasts and using scenario generation technology, specifically includes: Obtain historical wind speed and historical light intensity data for the target area, as well as corresponding weather forecast data for the time period; Establish mapping relationship models between wind speed and wind power output, and between solar irradiance and photovoltaic power output, respectively; An autoregressive moving average model was used to model the forecast error series of wind speed and light intensity to characterize their temporal correlation. Based on the meteorological forecast data, a deterministic basic forecast curve for renewable energy is generated using the mapping relationship model; On the basic prediction curve, multiple possible error sequences generated by the autoregressive moving average model are superimposed to generate multiple renewable energy output scenario curves that reflect power uncertainty.

[0012] Furthermore, the input state space of the multi-objective reinforcement learning decision model includes various data from the dynamic operating scenario set, and its action space includes the set power values ​​of each controllable power generation unit and energy storage unit, specifically including: The input state space at each decision moment includes: the real-time output and ultra-short-term predicted output of all distributed renewable energy units, the available capacity and operating cost of all controllable distributed generation units, the real-time state of charge and charging / discharging efficiency of all energy storage units, the real-time demand and short-term predicted demand of all load units, the time-of-use electricity price for the current and predicted future periods, and the upper and lower limits of grid dispatch instructions. The action space at each decision moment includes: the planned output value of each controllable distributed generation unit in a future scheduling period, and the planned charging and discharging power value of each group of energy storage units in a future scheduling period. The action space, at the moment of decision involving market transactions, also includes: the amount of electrical power planned to be purchased from the main grid and the amount of electrical power planned to be sold to the main grid in a future trading period; All action values ​​in the action space must satisfy the power balance constraints, equipment physical operation constraints, and network security constraints of the microgrid. These constraints are reflected in the reward function design during model training.

[0013] Furthermore, the extraction of historical data is constructed into experience tuples including state, action, immediate reward, next state, and goal achievement marker, specifically including: The historical data is sliced ​​at fixed time intervals, and each slice contains all the state variable values ​​defined in the input state space at that moment, which is a historical state; The sequence of control instructions and market trading instructions that are actually executed immediately following the historical state are mapped into the format defined in the action space, and this is taken as a historical action. Based on the historical state and the actual economic cost, actual renewable energy consumption, and smoothness index of power interaction with the grid after the historical actions are executed, a comprehensive multi-objective instantaneous reward value is calculated. The state variable values ​​collected in the next time interval slice after the historical action is executed will be used as the next historical state. Based on whether the scenario corresponding to the historical state smoothly transitions to the scenario corresponding to the next historical state, it is determined whether the decision is conducive to achieving the long-term goal, and a goal achievement marker is generated. The historical state, historical action, multi-objective instant reward value, next historical state, and objective achievement marker are combined to form a complete experience tuple.

[0014] Furthermore, the step of parsing the energy scheduling strategy into a specific, time-stamped sequence of control instructions specifically includes: Receive an action sequence output from the multi-objective reinforcement learning decision model, the action sequence containing planned values ​​for multiple consecutive decision moments within a future optimization cycle; Each planned value in the action sequence is mapped to the corresponding physical device control parameters to generate basic instructions that the device can execute; Each basic instruction is attached with a precise execution time stamp, which is synchronized with the microgrid system clock and the electricity market trading hours; All time-tagged basic instructions are sorted according to their execution time, and the continuity between instructions and the rationality of equipment actions are checked to form the final control instruction sequence. Before the control command sequence is sent to the actuator, a safety check is performed to ensure that the command does not exceed the safe operating range of the equipment.

[0015] Compared with the prior art, the beneficial effects of the present invention are: A dynamic set of operating scenarios across multiple time scales is constructed based on real-time operational data and external market environment data. The data acquisition module obtains real-time output curves of distributed generation units, state of charge and health status of energy storage units, and active and reactive power demand fluctuation characteristics of load units. Simultaneously, real-time electricity prices, grid interaction constraints, market demand response signals, and clearing rules are collected. A Markov chain Monte Carlo simulation considering time correlation, combined with clustering algorithms, is used to generate multi-time scale scenarios at 15-minute ultra-short-term, hourly short-term, and daily medium-term levels. The combined relationships between generation output, energy storage state transitions, load distribution, and market prices are clarified for each scenario, and the scenario set is iteratively updated in real-time with monitoring data. This ensures that the scenario library dynamically reflects the actual operating status of the microgrid, avoiding strategy failures caused by the lag of static scenarios. It provides an accurate input basis reflecting current and expected conditions for subsequent decision-making, enhancing the strategy's adaptability to different operating conditions.

[0016] A set of dynamic operating scenarios is input into a pre-trained multi-objective reinforcement learning decision model to generate scenario-adaptive energy dispatch strategies and market trading instructions. The model optimizes multiple directions, including economic and reliability objectives, and employs a deep deterministic policy gradient multi-objective extension architecture. Through a policy network, it outputs instructions corresponding to each scenario: energy dispatch strategies include output adjustments for distributed generation units and charging / discharging power curves for energy storage units; market trading instructions include power settings for interaction with the main grid and time-of-use bidding quantities and prices. This approach overcomes the limitations of single-objective optimization, achieving Pareto balance among multiple objectives. It generates customized strategies for different scenario characteristics, avoiding performance degradation of general strategies under multi-objective conflicts, and improving the overall performance of microgrid economic operation and reliable power supply. Attached Figure Description

[0017] Figure 1A timing diagram of the AI-enabled microgrid energy management and economic operation system described in this invention; Figure 2 A flowchart for generating energy scheduling strategies and market trading instructions for a multi-objective reinforcement learning decision-making model. Detailed Implementation

[0018] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] See Figure 1 The data acquisition module continuously collects real-time operational data from distributed generation units, energy storage units, and load units within the microgrid, while simultaneously acquiring external electricity market environment data. Subsequently, the scenario construction module receives this data and constructs a dynamic set of operational scenarios across multiple time scales. This set of dynamic operational scenarios is input into a pre-trained multi-objective reinforcement learning decision model in the learning and decision module. The model generates appropriate energy dispatch strategies and market trading instructions for each scenario in the set. The execution control module receives these strategies and instructions, parses them, and transforms them into specific control commands and market declaration information. This controls the output of distributed generation units and energy storage units within the microgrid and executes power interaction transactions with the main grid or electricity market. Throughout the system's operation, the rolling optimization module monitors the actual operating status of the microgrid system and the clearing price of the electricity market in real time, feeding this monitoring data back to the multi-objective reinforcement learning decision model in the learning and decision module. This triggers rolling optimization updates to the energy dispatch strategies and market trading instructions, forming a closed-loop intelligent management and control process.

[0020] In one embodiment of the present invention, the scenario construction module, based on real-time operational data provided by the data acquisition module and external market environment data, performs an operation to construct a dynamic set of operational scenarios encompassing multiple time scales. The scenario construction module receives real-time operational data from distributed generation units, energy storage units, and load units in the microgrid system from the data acquisition module, and performs preprocessing operations on the real-time operational data. The preprocessing operations extract the predicted output curves of the distributed generation units, the predicted demand curves of the load units, and the real-time state of charge of the energy storage units. Simultaneously, the scenario construction module receives external market environment data and, in conjunction with this data, parses future time-of-use electricity price information, renewable energy quota indicators, and grid dispatch instructions.

[0021] In some embodiments, the scenario construction module categorizes the extracted predicted output curves, predicted demand curves, real-time state of load, and parsed time-of-use pricing information, renewable energy quota indicators, and grid dispatch instructions into three distinct time scales: ultra-short-term, short-term, and medium-term. It also aligns data from different sources along the time axis. Based on historical data and weather forecast data, the scenario construction module employs scenario generation technology to simulate the uncertainty of renewable energy output, the randomness of load fluctuations, and the probability of market price fluctuations at different future time scales. The scenario generation technology characterizes various uncertainties by generating multiple sequences with different probabilities.

[0022] In some embodiments, when simulating the uncertainty of renewable energy output, the scenario construction module acquires historical wind speed data, historical solar irradiance data, and corresponding weather forecast data for the target area. The scenario construction module establishes mapping models between wind speed and wind power output, and between solar irradiance and photovoltaic power output, respectively. The module uses an autoregressive moving average model to model the wind speed forecast error sequence and the solar irradiance forecast error sequence to characterize the temporal correlation of the error sequences. Based on the weather forecast data, the scenario construction module uses the mapping models to generate a deterministic basic renewable energy forecast curve. On the basic renewable energy forecast curve, the scenario construction module overlays multiple possible error sequences generated by the autoregressive moving average model, thereby generating multiple renewable energy output scenario curves reflecting the uncertainty of output.

[0023] In practice, the scenario construction module combines various simulated uncertainty sequences with previously classified and aligned basic data such as predicted output curves, predicted demand curves, real-time state of load, time-of-use pricing information, renewable energy quota indicators, and grid dispatch instructions. The combination operation of the scenario construction module pairs an uncertainty sequence with basic data at a corresponding time scale, forming dynamic operating scenarios with specific probabilistic characteristics. Finally, all generated dynamic operating scenarios are aggregated and organized into a complete set of dynamic operating scenarios, which serves as input for the subsequent learning and decision-making module.

[0024] Optionally, during the construction of a dynamic set of operating scenarios, the scenario construction module can incorporate the statistical distribution characteristics of historical load data to simulate the stochasticity of load fluctuations. These statistical distribution characteristics include the mean, variance, and autocorrelation function. Based on these characteristics, the scenario construction module generates multiple load fluctuation scenario curves that conform to historical statistical features using Latin hypercube sampling or Monte Carlo simulation methods.

[0025] Optionally, when simulating the possibility of market price fluctuations, the scenario building module can use a geometric Brownian motion model or a mean regression model to describe the stochastic process of time-of-use electricity prices. It can be understood that the geometric Brownian motion model is suitable for characterizing long-term trends and fluctuations in electricity prices, while the mean regression model is suitable for describing the characteristics of electricity prices fluctuating around a long-term mean. Using the selected model and forward-looking market price data as input, the scenario building module generates multiple possible future electricity price evolution paths, forming a scenario of market price fluctuations.

[0026] See Figure 2 In one embodiment of the present invention, the learning decision module inputs the set of dynamic operating scenarios output by the scenario construction module into a pre-trained multi-objective reinforcement learning decision model. The multi-objective reinforcement learning decision model generates energy dispatch strategies and market trading instructions adapted to each dynamic operating scenario. The multi-objective reinforcement learning decision model constructed by the learning decision module takes minimizing microgrid operating costs, maximizing renewable energy consumption, and smoothing grid interaction power as optimization objectives. The input state space of the multi-objective reinforcement learning decision model contains various data in the set of dynamic operating scenarios, while the action space of the multi-objective reinforcement learning decision model contains the set power values ​​of each controllable power generation unit and energy storage unit, as well as the power declaration values ​​for buying and selling electricity with the external market.

[0027] In some embodiments, the dynamic operating scenario set data contained in the input state space of the multi-objective reinforcement learning decision model at each decision moment specifically includes: the real-time output and ultra-short-term predicted output of all distributed renewable energy units (such as photovoltaic arrays and wind turbines), the available capacity and operating costs of all controllable distributed generation units (such as gas turbines and diesel generators), the real-time state of charge and charge / discharge efficiency of all energy storage units (such as battery energy storage systems), the real-time demand and short-term predicted demand of all load units (such as industrial loads and commercial loads), the time-of-use electricity prices for current and predicted future periods, and the upper and lower limits of dispatch instructions from the upper-level power grid. The specific actions contained in the action space of the multi-objective reinforcement learning decision model at each decision moment include: the planned output value of each controllable distributed generation unit (such as a gas turbine) in a future dispatch period, and the planned charge / discharge power value of each group of energy storage units (such as battery energy storage systems) in a future dispatch period. At decision moments involving market transactions, the action space of the multi-objective reinforcement learning decision model also additionally includes the planned power purchase from the main grid and the planned power sale to the main grid in a future trading period.

[0028] In practical implementation, offline training of the multi-objective reinforcement learning decision model requires the use of an experience replay library. The data in the experience replay library comes from historical databases of microgrids and the electricity market, extracting time-aligned generation data, load data, energy storage data, market price data, and actual transaction data. The extracted historical data is constructed into experience tuples, including states, actions, immediate rewards, next states, and goal achievement markers. The states in the experience tuples are the values ​​of all state variables defined in the input state space at a given historical moment. The actions in the experience tuples are the values ​​mapped from the control command sequence and market transaction commands actually executed immediately after that historical moment. The immediate rewards in the experience tuples are a comprehensive value calculated based on the actual economic costs incurred after the execution of the state and actions, the actual consumption of renewable energy, and the smoothness index of power interaction with the grid. The next state in the experience tuples is the state variable value collected at the next moment after the action is executed. The goal achievement markers in the experience tuples are generated based on whether the scenario corresponding to the historical state smoothly transitions to the scenario corresponding to the next state. The constructed experience tuples are stored in a fixed-capacity experience replay library and indexed according to the time attributes and scene types of the data.

[0029] In practice, the offline training process of the multi-objective reinforcement learning decision model randomly samples a batch of experience tuples from the experience replay library. These sampled experience tuples are then used to calculate the multi-objective policy gradient of the multi-objective reinforcement learning decision model. The multi-objective policy gradient is used to update the policy network parameters and value network parameters of the multi-objective reinforcement learning decision model. The model parameter update process is iterative until the model converges. The model parameter update process can adopt the following form of policy gradient update formula: in: Indicates policy network parameters Update volume This represents the preset learning rate. Indicates the parameter Find the gradient. This represents a multi-objective performance function estimated based on empirical tuples from sampled batches.

[0030] In practical implementation, during the online application phase, the learning decision module inputs a dynamically constructed set of operating scenarios into the trained multi-objective reinforcement learning decision model. For each dynamic operating scenario input from the set, the multi-objective reinforcement learning decision model outputs a set of optimal action sequences. These action sequences are then decoded by the policy interpreter within the learning decision module. The policy interpreter maps the action sequences into instruction formats that can be recognized and executed by the microgrid control system and the electricity market, thereby generating corresponding energy dispatch strategies and market trading instructions.

[0031] In one embodiment of the present invention, the execution control module controls the output of internal microgrid units based on the energy dispatch strategy generated by the learning decision module, and executes power interaction with the external power grid based on market trading instructions generated by the learning decision module. The execution control module receives the energy dispatch strategy, which is parsed into a specific sequence of control instructions with precise time stamps. This sequence includes the power setpoints of each distributed generation unit, the charging and discharging power instructions of each energy storage unit, and the operating mode instructions of each energy storage unit. Through the microgrid central controller or the local controllers of the distributed generation units and energy storage units, the execution control module sends the control instruction sequence to the corresponding distributed generation unit actuators and energy storage unit battery management systems. The distributed generation unit actuators and energy storage unit battery management systems precisely control the output of the generation units and the charging and discharging of the energy storage units according to the instructions. Simultaneously, the execution control module receives market trading instructions, which are parsed into the power purchase power curve, power sale power curve, and corresponding price information required for application on the power market trading platform. Before the deadline of the trading period stipulated by the power market, the execution control module submits the parsed power purchase power curve, power sale power curve, and price information through the market interface agent, completing the transaction application with the power market. During the trading period, the microgrid system, based on the clearing results of the electricity market, actually performs power exchange by buying or selling electricity through the common connection point between the microgrid and the main grid.

[0032] In some embodiments, the rolling optimization module monitors the operating status of the microgrid system and the clearing price of the electricity market in real time. The rolling optimization module collects real-time data on system frequency, node voltage, line power flow, actual output of generating units, actual state of charge of energy storage units, and actual power of load units through sensors, measurement units, and metering devices deployed at various nodes and key branches of the microgrid, forming an operating status monitoring dataset. The rolling optimization module also obtains the latest market clearing price, system marginal price, and various ancillary service price information in real time through the electricity market information publishing interface, forming a market price monitoring dataset. Finally, the rolling optimization module merges the operating status monitoring dataset and the market price monitoring dataset, and performs timestamp alignment on data points from different sources to form a unified, time-stamped real-time feedback dataset.

[0033] In practical implementation, the rolling optimization module compares the real-time feedback dataset with the currently executing energy dispatch strategy and market trading instructions to calculate the strategy execution deviation. The strategy execution deviation reflects the degree of deviation between the actual operating state, actual market price, and the predetermined strategy and instructions. If the strategy execution deviation exceeds a preset threshold or reaches the system's preset fixed rolling optimization cycle, the rolling optimization module inputs the latest real-time feedback dataset as a new initial state into the multi-objective reinforcement learning decision model in the learning decision module. The multi-objective reinforcement learning decision model is triggered to perform a new round of strategy calculation, generating updated energy dispatch strategies and market trading instructions based on the new initial state, thereby achieving closed-loop rolling optimization of the entire microgrid's energy management and economic operation strategy. One way the rolling optimization module calculates the strategy execution deviation is through quantification using the following formula: in: A scalar value representing the deviation in strategy execution. This indicates the number of time points included in the current evaluation period. This represents the actual measured value of the microgrid's net load power at time t. This represents the planned value of the microgrid's net load power at time t. This represents the actual value of the market clearing price at time t. This represents the predicted market clearing price at time t. and These are the preset weighting coefficients for power deviation and price deviation, respectively.

[0034] In one embodiment of the present invention, the scenario construction module, based on historical data and meteorological forecast data, employs scenario generation technology to simulate the uncertainty of renewable energy output at different time scales in the future. The specific implementation process of the scenario generation technology involves modeling the forecast errors of wind speed and solar irradiance. The scenario construction module acquires historical wind speed data, historical solar irradiance data, and corresponding meteorological forecast data for the target area. The module establishes mapping models between wind speed and wind power output, and between solar irradiance and photovoltaic power output, respectively. The module uses an autoregressive moving average model to model the wind speed forecast error sequence and the solar irradiance forecast error sequence. The autoregressive moving average model can characterize the temporal correlation and fluctuation clustering characteristics in the error sequences. Based on the latest meteorological forecast data, the module generates a deterministic basic renewable energy forecast curve using the mapping models of wind speed and wind power output, and solar irradiance and photovoltaic power output. On the basic renewable energy forecast curve, the module overlays multiple possible and representative error sequences generated by the autoregressive moving average model, thereby generating multiple renewable energy output scenario curves reflecting the uncertainty of power output. The autoregressive moving average model used to generate the error series can be expressed as: in: Indicates the current moment Forecast error, This indicates the order of the autoregressive model. Indicates the first One autoregressive coefficient, Indicates the past number Error at any given moment This indicates the order of the moving average model. Indicates the first A moving average coefficient, Indicates the past number White noise perturbation at each moment This represents the white noise disturbance at the current moment, which follows a normal distribution with zero mean and fixed variance.

[0035] In some embodiments, the input state space of the multi-objective reinforcement learning decision-making model specifically includes various data elements from the set of dynamic operating scenarios. Referring to Table 1, the data elements included in the input state space of the multi-objective reinforcement learning decision-making model at each decision moment are listed.

[0036] Table 1: Composition of the input state space of a multi-objective reinforcement learning decision-making model In practical implementation, the action space of the multi-objective reinforcement learning decision-making model includes the set power values ​​of each controllable power generation unit and energy storage unit. At each decision moment, the specific actions included in the action space of the multi-objective reinforcement learning decision-making model are: the planned output value of each controllable distributed power generation unit (e.g., a micro gas turbine) in a future scheduling period; and the planned charge / discharge power value of each energy storage unit (e.g., a battery energy storage array) in a future scheduling period. A positive planned charge / discharge power value indicates discharging, and a negative value indicates charging. At decision moments involving market transactions, the action space of the multi-objective reinforcement learning decision-making model also includes the planned power purchase from the main grid and the planned power sale to the main grid in a future trading period. The purchased and sold power values ​​are typically non-negative and not simultaneously non-zero.

[0037] Optionally, the planned charge / discharge power values ​​for the energy storage unit in the action space can be normalized based on the physical characteristics of the energy storage unit. This normalization maps the charge / discharge power values ​​to the range [-1, 1], where -1 represents charging at the maximum acceptable power, 0 represents no action, and 1 represents discharging at the maximum release power. The multi-objective reinforcement learning decision model outputs the normalized action values, which are then decoded into actual power commands by the execution control module.

[0038] Optionally, all action values ​​in the action space must satisfy the physical and operational constraints of the microgrid system. These constraints can be understood to include real-time power balance constraints of the microgrid, upper and lower output limits of distributed generation units, upper and lower state of charge limits and charging / discharging power limits of energy storage units, and power interaction security constraints at the point of common coupling between the microgrid and the main grid. When training a multi-objective reinforcement learning decision model, these constraints are reflected through a designed reward function. Actions that violate these constraints will result in negative rewards for the model during training, thereby guiding the model to learn strategies that comply with the constraints.

[0039] In practical implementation, time-aligned historical data extracted from the microgrid historical database and the electricity market historical database are constructed into empirical tuples including states, actions, immediate rewards, next states, and goal achievement markers. The construction process first slices the historical data at fixed time intervals. Each time slice contains the specific values ​​of all state variables defined in the input state space of the multi-objective reinforcement learning decision model at that moment; this data set is defined as a historical state. The sequence of control commands and market transaction commands actually executed by the microgrid system immediately following this historical state are mapped and transformed according to the format defined in the action space of the multi-objective reinforcement learning decision model. The mapped and transformed set of commands is defined as a historical action. Based on the economic costs generated by the system after the actual execution of this historical state and historical action, the actual amount of renewable energy absorbed, and the smoothness of power interaction with the grid, a comprehensive multi-objective immediate reward value is calculated. The calculation of the multi-objective immediate reward value can be performed using a linear weighted formula. in: This represents the calculated multi-objective instantaneous reward value. This represents the total operating costs (including power generation costs, operation and maintenance costs, and market transaction costs) incurred within the decision interval. This indicates the amount of renewable energy electricity consumed within the decision interval. It represents the square of the power change at the point of common coupling between the microgrid and the main grid within the decision interval (used to measure the smoothness of power fluctuations). , and These are the preset weight coefficients for the corresponding items. After executing this historical action, all state variable values ​​collected in the next fixed time interval slice are defined as a next historical state. Based on whether the operating scenario corresponding to this historical state smoothly transitions to the operating scenario corresponding to this next historical state, it is determined whether the decision contributes to achieving the long-term optimization goal, and a binary goal achievement marker is generated. Finally, this historical state, this historical action, this multi-objective immediate reward value, this next historical state, and this goal achievement marker are combined to form a complete empirical tuple that can be used for training the reinforcement learning model.

[0040] In some embodiments, the execution control module parses the energy dispatch strategy issued by the learning decision module into a specific, time-stamped sequence of control instructions. The parsing process receives an action sequence output from a multi-objective reinforcement learning decision model, which contains planned values ​​for multiple consecutive decision moments within a future optimization cycle. The execution control module maps each planned value in the action sequence to the corresponding physical device control parameters in the microgrid, generating basic instructions that the physical devices can directly execute. A precise execution time stamp is attached to each basic instruction, synchronized with the microgrid system's master clock and the electricity market's trading time clock. The execution control module sorts all time-stamped basic instructions according to the order of their execution time stamps, checking the temporal continuity of the instructions and the physical rationality of the device actions to form the final, issueable sequence of control instructions. Before issuing the control instruction sequence to the distributed generation unit's actuator or the energy storage unit's management system, the execution control module performs a safety verification procedure to ensure that each instruction in the sequence does not exceed the safe operating range of the corresponding physical device.

[0041] Optionally, when calculating the multi-objective immediate reward value, the specific data sources for economic costs, renewable energy consumption, and power smoothness can be further clarified. It is understood that economic costs... The data comes from the settlement records and market transaction records of the microgrid energy management system, and the renewable energy consumption. The data comes from the electricity meter readings of renewable energy power generation units, power smoothness. The data comes from continuous sampling values ​​from the power sensor at the point of common coupling.

[0042] Optionally, when generating the target achievement marker, the criterion for determining whether the scenario transitioned smoothly can be based on whether the changes of multiple state variables exceed preset stability thresholds. These state variables can be understood to include the microgrid system frequency, critical bus voltage, and the overall state of charge of the energy storage unit. If these variables remain within their respective stability threshold ranges after the state transition, it is determined to be a smooth transition and a positive marker is generated; otherwise, a negative marker is generated. The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An AI-enabled microgrid energy management and economic operation system, characterized in that, include: The data acquisition module is used to collect real-time operating data and external market environment data from distributed generation units, energy storage units, and load units in the microgrid system. The scenario construction module constructs a set of dynamic operating scenarios with multiple time scales based on the real-time operating data and external market environment data. The learning decision module is used to input the set of dynamic operating scenarios into a pre-trained multi-objective reinforcement learning decision model to generate energy scheduling strategies and market trading instructions adapted to each scenario. The execution control module controls the output of distributed generation units and energy storage units based on the energy dispatch strategy, and executes power interaction with the main grid or electricity market based on the market trading instructions. The rolling optimization module monitors the operating status and market clearing price of the microgrid system in real time, and feeds the monitoring data back to the multi-objective reinforcement learning decision model to initiate rolling optimization of the strategy.

2. The AI-enabled microgrid energy management and economic operation system according to claim 1, characterized in that, The construction of a dynamic set of operating scenarios with multiple time scales based on the real-time operating data and external market environment data specifically includes: The real-time operating data is preprocessed to extract the predicted output curve of the distributed generation unit, the predicted demand curve of the load unit, and the real-time state of charge of the energy storage unit. Based on the aforementioned external market environment data, analyze future time-of-use electricity price information, renewable energy quota indicators, and grid dispatch instructions; The predicted output curve, predicted demand curve, real-time state of charge, time-of-use electricity price information, renewable energy quota indicators and grid dispatch instructions are classified and aligned according to three time scales: ultra-short-term, short-term and medium-term. Based on historical data and weather forecasts, scenario generation technology is used to simulate the uncertainty of renewable energy output, the randomness of load fluctuations, and the possibility of market price fluctuations at different time scales in the future. By combining the various uncertain sequences generated by simulation with the basic data after classification alignment, multiple dynamic operating scenarios with probabilistic characteristics are formed, and a set of dynamic operating scenarios is constructed.

3. The AI-enabled microgrid energy management and economic operation system according to claim 2, characterized in that, The step of inputting the set of dynamic operating scenarios into a pre-trained multi-objective reinforcement learning decision model to generate energy scheduling strategies and market trading instructions adapted to each scenario specifically includes: Construct a multi-objective reinforcement learning decision model with the optimization objectives of minimizing microgrid operating costs, maximizing renewable energy consumption, and smoothing grid interaction power. The input state space of the multi-objective reinforcement learning decision model includes various data in the dynamic operation scenario set, and its action space includes the set power values ​​of each controllable power generation unit and energy storage unit, as well as the power declaration values ​​for buying and selling electricity with the external market. The multi-objective reinforcement learning decision model is trained offline using an experience replay library containing historical operational data and market transaction data. The model parameters are updated through a multi-objective policy gradient algorithm, enabling the model to learn to balance multiple optimization objectives while satisfying system constraints. During the online application phase, the dynamically constructed set of running scenarios is input into the trained multi-objective reinforcement learning decision model. For each dynamic operating scenario, the multi-objective reinforcement learning decision model outputs a set of optimal action sequences. After the action sequences are decoded by the policy interpreter, corresponding energy scheduling strategies and market trading instructions are generated.

4. The AI-enabled microgrid energy management and economic operation system according to claim 3, characterized in that, The offline training of the multi-objective reinforcement learning decision model using an experience replay library containing historical operational data and market transaction data specifically includes: Extract time-series aligned generation data, load data, energy storage data, market price data, and actual transaction data from the microgrid historical database and the electricity market historical database; The extracted historical data is constructed into experience tuples that include status, action, immediate reward, next status, and goal achievement marker; The experience tuples are stored in a fixed-capacity experience replay library and indexed according to the time attributes and scene types of the data. During training, a batch of experience tuples are randomly sampled from the experience replay library to calculate the multi-objective policy gradient of the multi-objective reinforcement learning decision model. Based on the calculated multi-objective policy gradient, update the policy network and value network parameters of the multi-objective reinforcement learning decision model, and iterate this process until the model converges.

5. The AI-enabled microgrid energy management and economic operation system according to claim 4, characterized in that, The control of the output of distributed generation units and energy storage units based on the energy dispatch strategy, and the execution of power interaction with the main grid or electricity market based on the market trading instructions, specifically include: The energy dispatch strategy is parsed into a specific, time-stamped sequence of control commands, which includes the power setpoint of each distributed generation unit, the charging and discharging power of each energy storage unit, and the operating mode. The control command sequence is sent to the corresponding distributed generation unit actuator and energy storage unit management system through the microgrid central controller or local controller to achieve precise control of the output. The market transaction instructions are parsed into power purchase curves, power sales curves, and corresponding price information submitted on the power market trading platform. Before the deadline for the electricity market's designated trading period, the buyer submits the power purchase curve, seller power curve, and price quote information through the market interface agent to complete the transaction declaration; During trading hours, based on market clearing results, power interactions involving the buying or selling of electricity are actually executed through the common connection point between the microgrid and the main grid.

6. The AI-enabled microgrid energy management and economic operation system according to claim 5, characterized in that, The real-time monitoring of the microgrid system's operating status and market clearing price, and the feedback of the monitoring data to the multi-objective reinforcement learning decision model to initiate rolling optimization of the strategy, specifically includes: By deploying sensors and metering devices at each node of the microgrid, the system frequency, node voltage, line power flow, actual output of the power generation unit, actual state of charge of the energy storage unit, and actual power of the load unit are collected in real time to form an operational status monitoring dataset. By using the electricity market information release interface, the latest market clearing prices, system marginal electricity prices, and ancillary service prices can be obtained in real time to form a market price monitoring dataset; The operational status monitoring dataset and the market price monitoring dataset are merged and timestamped to form a real-time feedback dataset; The real-time feedback dataset is compared with the currently executing energy scheduling strategy and the market trading instructions to calculate the strategy execution deviation; If the policy execution deviation exceeds a preset threshold or reaches a preset rolling optimization cycle, the real-time feedback dataset is used as a new initial state to trigger the multi-objective reinforcement learning decision model to generate a new round of policy and output instructions, thereby achieving rolling optimization.

7. The AI-enabled microgrid energy management and economic operation system according to claim 6, characterized in that, The method, based on historical data and weather forecasts, employs scenario generation technology to simulate the uncertainty of renewable energy output at different future time scales, specifically includes: Obtain historical wind speed and historical light intensity data for the target area, as well as corresponding weather forecast data for the time period; Establish mapping relationship models between wind speed and wind power output, and between solar irradiance and photovoltaic power output, respectively; An autoregressive moving average model was used to model the forecast error series of wind speed and light intensity to characterize their temporal correlation. Based on the meteorological forecast data, a deterministic basic forecast curve for renewable energy is generated using the mapping relationship model; On the basic prediction curve, multiple possible error sequences generated by the autoregressive moving average model are superimposed to generate multiple renewable energy output scenario curves that reflect power uncertainty.

8. The AI-enabled microgrid energy management and economic operation system according to claim 7, characterized in that, The input state space of the multi-objective reinforcement learning decision model includes various data from the dynamic operating scenario set, and its action space includes the set power values ​​of each controllable power generation unit and energy storage unit, specifically including: The input state space at each decision moment includes: the real-time output and ultra-short-term predicted output of all distributed renewable energy units, the available capacity and operating cost of all controllable distributed generation units, the real-time state of charge and charging / discharging efficiency of all energy storage units, the real-time demand and short-term predicted demand of all load units, the time-of-use electricity price for the current and predicted future periods, and the upper and lower limits of grid dispatch instructions. The action space at each decision moment includes: the planned output value of each controllable distributed generation unit in a future scheduling period, and the planned charging and discharging power value of each group of energy storage units in a future scheduling period. The action space, at the moment of decision involving market transactions, also includes: the amount of electrical power planned to be purchased from the main grid and the amount of electrical power planned to be sold to the main grid in a future trading period; All action values ​​in the action space must satisfy the power balance constraints, equipment physical operation constraints, and network security constraints of the microgrid. These constraints are reflected in the reward function design during model training.

9. The AI-enabled microgrid energy management and economic operation system according to claim 8, characterized in that, The process of constructing the extracted historical data into experience tuples including state, action, immediate reward, next state, and goal achievement marker specifically includes: The historical data is sliced ​​at fixed time intervals, and each slice contains all the state variable values ​​defined in the input state space at that moment, which is a historical state; The sequence of control instructions and market trading instructions that are actually executed immediately following the historical state are mapped into the format defined in the action space, and this is taken as a historical action. Based on the historical state and the actual economic cost, actual renewable energy consumption, and smoothness index of power interaction with the grid after the historical actions are executed, a comprehensive multi-objective instantaneous reward value is calculated. The state variable values ​​collected in the next time interval slice after the historical action is executed will be used as the next historical state. Based on whether the scenario corresponding to the historical state smoothly transitions to the scenario corresponding to the next historical state, it is determined whether the decision is conducive to achieving the long-term goal, and a goal achievement marker is generated. The historical state, historical action, multi-objective instant reward value, next historical state, and objective achievement marker are combined to form a complete experience tuple.

10. The AI-enabled microgrid energy management and economic operation system according to claim 9, characterized in that, The step of parsing the energy scheduling strategy into a specific, time-stamped sequence of control instructions specifically includes: Receive an action sequence output from the multi-objective reinforcement learning decision model, the action sequence containing planned values ​​for multiple consecutive decision moments within a future optimization cycle; Each planned value in the action sequence is mapped to the corresponding physical device control parameters to generate basic instructions that the device can execute; Each basic instruction is attached with a precise execution time stamp, which is synchronized with the microgrid system clock and the electricity market trading hours; All time-tagged basic instructions are sorted according to their execution time, and the continuity between instructions and the rationality of equipment actions are checked to form the final control instruction sequence. Before the control command sequence is sent to the actuator, a safety check is performed to ensure that the command does not exceed the safe operating range of the equipment.

Images