Intelligent park electric-carbon coupling generalized energy storage optimal scheduling method based on lyapunov safety reinforcement learning

Abstract

The application discloses a kind of wisdom park electric-carbon coupling generalized energy storage optimization scheduling methods based on Lyapunov safety reinforcement learning, constructs wisdom park electric-carbon coupling energy system model;Air conditioning, battery energy storage and electric vehicle are characterized as generalized energy storage resources, and power-energy envelope and recoverability constraint are established;Carbon account book and multi-queue Lyapunov mechanism are constructed, and comfort, battery recovery, electric vehicle service and carbon budget pressure are characterized;Resource budget action is output by deep deterministic policy gradient intelligent agent, and RPCD signal is formed based on renewable energy abundance, electricity price, carbon pressure and deadline pressure, and coordinated action is obtained through marginal coordinator;Then, pre-execution safety correction is carried out by unified generalized energy storage safety evaluation and optimization module, to obtain safe execution action, and the strategy is updated using safety correction difference feedback.

Description

Technical Field

[0001] This invention belongs to the fields of smart park energy management, generalized energy storage optimization scheduling, electricity-carbon coupling control, and safe reinforcement learning. Specifically, it relates to a smart park electricity-carbon coupling generalized energy storage optimization scheduling method based on Lyapunov safe reinforcement learning. This method is designed for smart parks containing photovoltaic, wind power, base load, HVAC clusters, battery energy storage systems, and electric vehicle fleets. Under the conditions of meeting comfort, energy storage recoverability, electric vehicle off-site service, and daily carbon budget constraints, it achieves low-cost, low-carbon, and safe real-time scheduling. Background Technology

[0002] With the large-scale integration of renewable energy, flexible loads, battery storage, and electric vehicles, smart parks are gradually becoming an important carrier of demand-side flexibility in the power system. The objects of energy management in parks are no longer limited to single load peak shaving or electricity price response, but need to consider factors such as photovoltaic, wind power, base load, HVAC thermal inertia, battery storage, electric vehicle mobile energy storage, external grid power, and dynamic carbon emission intensity.

[0003] Existing methods based on deterministic optimization, stochastic optimization, robust optimization, or model predictive control can handle power balance, equipment constraints, and multi-period coupling constraints under conditions of accurate models and reliable predictions. However, they still suffer from high computational cost, strong model dependence, and insufficient adaptability to uncertain environments in real-time park scheduling. While conventional deep reinforcement learning methods have advantages such as fast online inference speed and the ability to learn nonlinear policies, they typically incorporate constraints into reward and penalty terms, making it difficult to reliably guarantee long-term constraints such as comfort, energy storage terminal recovery, electric vehicle off-site energy, and daily cumulative carbon emissions.

[0004] Furthermore, traditional safety layers often employ action pruning or projection to correct inactive actions. While this approach can alleviate single-step boundary violations, it fails to fully express the coupling relationships between cross-period constraints such as HVAC virtual energy storage, battery energy storage recoverability, electric vehicle time constraints, and carbon budgets. For smart parks, charging, discharging, load shifting, and V2G behaviors within the same time period not only affect current electricity purchase costs but also subsequent periods' comfort margins, energy storage recovery capabilities, electric vehicle service capabilities, and remaining carbon budgets. Therefore, a scheduling method is needed that integrates generalized energy storage modeling, Lyapunov long-term constraint pressure, safety action correction, and reinforcement learning strategy updates. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning. This method addresses the difficulty of guaranteeing long-term safety constraints in the multi-resource, multi-constraint, and electricity-carbon coupled scheduling of smart parks using conventional reinforcement learning. The method unifies HVAC clusters, BES energy storage systems, and EV fleets as generalized energy storage resources, constructs a queue-enhanced GES-A-CMDP model, and generates the final execution actions through an RPCD marginal coordinator and a unified GSAOM safety correction module.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning includes the following steps: S1. Construct a smart park electricity-carbon coupling system model and collect park status data: base load, photovoltaic output, wind power output, electricity price, electricity sales price, dynamic grid carbon intensity, outdoor temperature, HVAC status, BES status, EV fleet status, and cumulative carbon emission status, and establish the relationship between park power balance and carbon emission calculation. S2. Construct a generalized energy storage aggregation model for HVAC, BES, and EV: Represent the HVAC cluster as thermal virtual energy storage, the BES as recoverable physical energy storage, and the EV fleet as time-aware mobile energy storage, forming a unified power-energy envelope. S3. Construct a queue-enhanced generalized energy storage assessment constrained Markov decision process: distinguish between original actions, coordinated actions, and safe execution actions, so that the original actions generated by the reinforcement learning strategy do not directly affect the physical system; S4. Construct a Lyapunov multi-cohort long-term constraint pressure representation: Construct an HVAC comfort cohort, a BES recovery cohort, an EV service cohort, and a carbon budget cohort to describe cross-period constraint pressure; S5. Construct RPCD signals and generate coordinated actions through a generalized energy storage marginal coordinator: Utilize renewable energy sufficiency, price, carbon pressure, and EV term pressure to perform marginal coordination and resource allocation on the resource budget actions output by the Actor; S6. Perform pre-execution safety corrections for coordinated actions through a unified GSAOM: sequentially or in parallel correct EV life-term safety, BES recoverability, HVAC comfort, and carbon budget protection to ensure that the final action meets physical and recoverability constraints. S7. Execute safety actions and use safety correction feedback to update reinforcement learning strategies: Apply safety actions to the smart park environment, record safety correction amounts and constraint activation reports, and use them for Actor-Critic training updates to reduce subsequent reliance on safety corrections.

[0007] Furthermore, step S1 includes the following sub-steps: S1-1. Collect the exogenous conditions of the park, including base load, photovoltaic output, wind power output, electricity purchase price, electricity sales price, dynamic grid carbon intensity, and outdoor temperature; (1); in, Based on the load, and The outputs are solar and wind power, respectively. and These are the electricity purchase price and the electricity sales price, For dynamic grid carbon intensity, Outdoor temperature For discrete scheduling time period indexes, For exogenous state vectors, the superscript is... This represents the transpose of a vector.

[0008] The process of S1-1 is as follows: S1-1-1. Calculate the net exchange power between the park and the external power grid based on the collected operating status of the park's equipment.

[0009] (2); in, For HVAC power, and These represent the charge and discharge power of the BES, and These are the EV charging power and V2G discharging power, respectively. S1-1-2. Determine the power purchased and power sold based on the net exchange power. (3); in, The net exchange power between the park and the external power grid. and These are the power purchased and the power sold, respectively. Indicates taking the positive part; when When the park purchases electricity, Electricity is sold in the industrial park.

[0010] S1-2. Calculate the current carbon emissions based on the dynamic grid carbon intensity, and update the cumulative carbon emissions status based on the daily carbon budget; (4); in, For time period The amount of carbon emissions generated For time period Dynamic grid carbon intensity, For the power purchase capacity, The length of a single scheduling period. and The deadlines are respectively and The cumulative carbon emissions.

[0011] S1-3. Construct electricity-carbon equivalent price and renewable energy sufficiency signals for subsequent RPCD marginal coordination; (5); in, For time period The electricity-carbon equivalent price, For electricity purchase price, As carbon cost weight, This is the price conversion factor for carbon emissions. For dynamic grid carbon intensity; For the abundance of renewable energy, and The outputs are solar and wind power, respectively. Based on the load, For comfort neutral HVAC reference power, The required charging power for EVs.

[0012] Furthermore, step S2 includes the following sub-steps: S2-1. Establish a unified power-energy envelope for generalized energy storage, for any generalized energy storage resource. , where its aggregation state is defined. and signed power Where positive values ​​represent energy absorbed and negative values ​​represent energy released; where, For a generalized energy storage resource type index, , , and These represent HVAC thermal virtual energy storage, BES physical energy storage, and EV mobile energy storage, respectively. For resources During the period The energy state of aggregation, Its signed power.

[0013] (6); in, and Resources During the period The maximum energy release power and the maximum energy absorption power, and These are the upper and lower bounds of the aggregation energy state, respectively.

[0014] Establish a generalized energy storage state transition model: (7); in, This is the self-dissipation or thermal relaxation coefficient. and For charging and discharging efficiency, and For terms related to the arrival and departure of exogenous energy; and Resources The aggregated energy state in adjacent time periods This is the self-dissipation or thermal relaxation coefficient. and These are charging efficiency and discharging efficiency, respectively. For signed power, , The length of the scheduling period. and These are the arrival and departure terms of exogenous energy, respectively. For resource type indexing, This is a time-period index.

[0015] S2-2. Model the HVAC cluster as a virtual thermal energy storage system based on Equivalent State of Charge (ESOC) for buildings. Define the midpoint of the comfort temperature range. and width And according to the operating mode Construct a thermal virtual energy storage state; (8); in, For architecture During the period The HVAC equivalent state of charge, with a value ranging from 0 to 1; Indoor temperature, The midpoint of the comfortable temperature range, To allow for deviations in comfort temperature, This is the operating mode symbol; 1 for cooling mode and -1 for heating mode.

[0016] With comfort neutral HVAC power Using the reference power as a benchmark, the increase in actual HVAC power relative to the reference power is considered as thermal virtual charging, and the decrease is considered as thermal virtual discharging; (9); in, For architecture HVAC relative reference power, This represents the actual HVAC power. For comfort neutral reference power, and These are the thermal virtual charging power and the energy dissipation power, respectively. .

[0017] S2-3. Model BES as a recoverable physical energy storage system, and construct a dynamic recovery baseline based on the current energy, the maximum recoverable energy for the remaining time period, and the reference terminal energy. (10); (11); in, From the time period The maximum recoverable energy of BES at the end of the scheduling cycle. To improve BES charging efficiency, The maximum rated charging power for BES, The last period of the scheduling cycle, The duration of the time period; For time period The lower limit of dynamic recovery energy. It is the physical minimum energy. As the reference energy at the end of the cycle, To restore safety margin.

[0018] The upper limit of BES discharge is constrained by both the physical SOC lower limit and the recovery lower limit. (12); in, For BES in time period The upper limit of the allowable discharge power, The rated maximum discharge power of BES, For current energy, It is the physical minimum energy. To dynamically restore the lower limit of energy, For discharge efficiency, This represents the length of the time period.

[0019] S2-4. Model the EV fleet as a time-sensitive mobile energy storage system. EVs are classified into private cars, taxis, buses and logistics vehicles according to vehicle type, and the time gap is calculated based on arrival, departure, target SOC and rechargeable capacity. (13); (14); in, For EV vehicle type index, This represents the target energy level for this type of EV before departure. For time period The current aggregated energy, For the target energy gap, This is recoverable energy that can be replenished through subsequent charging before leaving the station. To address the urgent time gap after deducting recoverable energy, .

[0020] The process in S2-4 is as follows: S2-4-1. Determine the mandatory charging power for EVs based on the time gap and separate it from the optional flexible charging power; (15); (16); in, for Total charging power for EV-like vehicles To ensure the necessary charging power is available to meet the target within the specified timeframe, Optional flexible charging power, This is the upper limit of charging power. Due to the tight deadline, For charging efficiency, This represents the length of the time period.

[0021] S2-4-2. Set a minimum safe energy limit for EVs for the current period, so that V2G only uses energy higher than the minimum safe energy limit for the current period. (17); (18); in, for EV-like during the time period The lower limit of safe energy for the specified period. It is the physical minimum energy. For energy safety margin, For the target energy at the departure station, Energy can be restored before leaving the station. For reference energy, To allow for energy deviation; This is the upper limit of V2G discharge power. The percentage of vehicles available online. The number of vehicles of this type. Rated power for single-vehicle V2G, This refers to the discharge efficiency.

[0022] S3 includes the following sub-steps: S3-1. The smart park scheduling problem is constructed as a queue-enhanced generalized energy storage assessment-constrained Markov decision process, denoted as... ,in To enhance the state space of the queue, For the original action space, As a state transition kernel, For the reward function, For queue-driven constraint signals, For the pre-execution safety assessment and correction operator; S3-2, Constructing a queue to enhance state: (19); in, For time period The park's environmental conditions, Let be the Lyapunov queue vector. , , and These are the HVAC comfort queue, BES recovery queue, EV service queue, and carbon budget queue. Enhance the state of the concatenated queue.

[0023] S3-3, Actor network outputs six-dimensional resource budgeting action: (20); in, The six-dimensional raw resource budget action output by the Actor. and These are the virtual charging and discharging budgets for HVAC, respectively. and The charging and discharging budgets for BES are respectively. For EV flexible charging budget, For the V2G discharge budget of EV, superscript This represents the transpose of a vector.

[0024] S3-4. Establish a closed-loop interaction chain of original actions, coordinated actions, and safety actions; (twenty one); in, To enhance the current queue state, For the Actor's original actions, The actions after RPCD marginal coordination The execution action after GSAOM safety correction. The next time-period queue enhancement state obtained after performing a safety action.

[0025] S4 includes the following sub-steps: S4-1. Construct queue-driven signals corresponding to HVAC, BES, EV and carbon budget, which respectively represent insufficient comfort energy storage, insufficient BES recovery, EV life gap and current carbon emissions. (twenty two); in, For queue driving signal vector, , , and They represent time periods respectively. Insufficient HVAC comfort energy storage, insufficient BES recovery, EV service gap and carbon emission impact, etc. This represents the transpose of a vector.

[0026] S4-2, Update service class Lyapunov queues and carbon budget queues: (twenty three); (twenty four); in, For service During the period Lyapunov queues, , and For queue memory or decay coefficient, For service-type queues, To correspond to the service benchmark, For carbon budget queue, This represents the current carbon emissions. To be allocated to time periods carbon budget, .

[0027] S4-3. Construct a quadratic Lyapunov function and a queue pressure term to adjust the relationship between cost minimization and long-run constraint satisfaction; (25); (26); (27); in, It is a quadratic Lyapunov function. For the first Queues; For queue pressure terms, , and For each service category, , and To correspond to the service benchmark, This represents the current carbon emissions. For the current carbon budget; As a reward for training, For operating costs, As queue pressure weight, For safety correction and auxiliary penalty costs.

[0028] Step S5 includes the following sub-steps: S5-1. Construct the RPCD signal vector, where and These respectively represent abundant and insufficient renewable energy sources. Indicates the electricity-carbon equivalent price. This indicates pressure on the normalized carbon budget. This indicates EV term pressure; (28); S5-2. Based on RPCD signals, normalized queue pressure, resource recoverability pressure, and energy storage carbon value, calculate the marginal score for each resource direction. (29); (30); in, For resource direction indexing, For time period Resource direction Marginal rating For the corresponding scoring function, For RPCD signal vectors, To normalize queue pressure, To address pressures on resource recoverability, Carbon value for energy storage; For saturated functions The coordination coefficient after restrictions For the original normalized budget, This represents the upper limit of available power in the corresponding direction. To coordinate power.

[0029] S5-3. Reconstruct the campus scheduling actions based on the coordinated HVAC, BES, and EV power. (31); (32); (33); in, , and These are the reconfigured net power of HVAC, BES, and EV, respectively. For HVAC comfort neutral reference power, and These represent the coordinated charging and discharging power of the HVAC system. and The charging and discharging power are coordinated by BES respectively. The required charging power for EVs To coordinate flexible charging power for EVs, To coordinate V2G discharge power.

[0030] Step S6 includes the following sub-steps: S6-1. Perform time-limited safety correction on EV flexible charging and V2G to ensure that the EV charging power is not reduced and to limit V2G from exceeding the time-limited safety energy limit. (34); (35); in, Total charging power for EV safety The required charging power cannot be reduced. The flexible charging power is after safety calibration. For safe V2G discharge power, To coordinate V2G power, The upper limit of V2G power given for time-limited safety constraints.

[0031] S6-2. Perform recoverability correction on BES charging and discharging to ensure that the energy of BES at the next moment is not lower than the dynamic recovery baseline. (36); (37); in, and These are the BES charging power and discharging power after safety calibration, respectively. and These are the coordinated charging power and discharging power, respectively. This is the upper limit of physical charging power. The upper limit of recoverable discharge power is required to simultaneously meet both the physical SOC baseline and the dynamic recovery baseline.

[0032] S6-3. Perform ESOC and comfort safety correction on virtual charging and discharging of HVAC systems to maintain building temperature within a comfortable range. (38); in, The actual power of the HVAC system after safety correction. For comfort neutral reference power, and These are the thermal virtual charging power and discharging power, respectively, which satisfy the constraints of ESOC and temperature comfort range.

[0033] S6-4. Protect the carbon budget by prioritizing the reduction of optional flexible charging for EVs and non-essential BES charging when predicted carbon emissions exceed the remaining budget reference value. (39); (40); in, For time period Permissible reference carbon emissions For daily carbon budget, This represents the cumulative carbon emissions up to the previous period. For carbon budget safety margin, This is the end of the scheduling cycle; For the power that needs to be reduced, The predicted carbon emissions corresponding to safety actions. For dynamic grid carbon intensity, The duration is the length of the time period. .

[0034] S6-5, Output safety actions, safety correction values, and constraint activation reports; (41); in, For safety actions Compared to coordinated movements The correction vector, The first norm of the correction vector is used to characterize the total correction magnitude; To constrain activation reports, , , and These represent the activation status of HVAC, BES, EV, and carbon budget security constraints, respectively.

[0035] Step S7 includes the following sub-steps: S7-1. Map safety actions back to the Actor budget space to form safety feedback targets; (42); in, To map the safety feedback objective back to the Actor budget space, For resource direction Safe execution power, This represents the upper limit of power in that direction. To prevent positive numbers with a denominator of zero, A saturation function to restrict the result to the range of allowed actions of the Actor.

[0036] S7-2. Construct a safety correction feedback loss to make the subsequent output of the Actor closer to the feasible region. (43); in, To safely correct feedback loss, To track losses for safety targets, For loss of consistency in action direction, and These are the weighting coefficients for the two losses, respectively.

[0037] S7-3. Update the policy network and value network through the Actor-Critic framework to obtain a closed-loop policy for online scheduling; (44); in, For Actor policy loss, This represents the expectation for the experience replay sample. For parameters The Critic state-action value function For parameters Actor strategy, To enhance the queue's state, To safely correct the feedback loss weights, To ensure safety, feedback loss is corrected.

[0038] The beneficial effects of this invention are mainly reflected in: 1. This invention models HVAC, BES and EV fleets as a generalized energy storage resource, which can describe the cross-time flexibility of thermal virtual energy storage, physical energy storage and mobile energy storage under the same power-energy envelope.

[0039] 2. This invention uses Lyapunov multi-queue representations of comfort, BES recovery, EV service and carbon budget pressure, enabling reinforcement learning strategies to perceive long-term constraints rather than relying solely on rewards and penalties to handle constraints.

[0040] 3. This invention distinguishes between original actions, coordinated actions, and safe execution actions, avoiding the direct impact of ordinary DRL strategies' output of inactive actions on the physical system, thus improving the safety of the training and deployment process.

[0041] 4. This invention introduces renewable energy, electricity price, carbon intensity and EV term pressure signals through the RPCD marginal coordinator, so that resource budget actions have an interpretable marginal allocation direction before execution.

[0042] 5. This invention achieves pre-implementation correction of EV term safety, BES recoverability, HVAC comfort, and carbon budget protection by unifying GSAOM, which can reduce the infeasibility risk under multiple constraints.

[0043] 6. This invention utilizes safety correction quantities and constraint activation reports to guide Actor learning in reverse, enabling the strategy to gradually reduce its dependence on the final safety repair, thereby improving the feasibility and stability of the closed-loop scheduling strategy. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the smart park electricity-carbon coupling energy system structure in an embodiment of the present invention; Figure 2 This is a schematic diagram of the RPCD-LyDDPG closed-loop security reinforcement learning framework in an embodiment of the present invention; Figure 3 This is a training security advantage diagram in an embodiment of the present invention, used to illustrate the security advantages of the present invention during the training phase; Figure 4 This is a final performance and layered security diagram in an embodiment of the present invention, used to illustrate the effects of the present invention in terms of final performance and layered security mechanism; Figure 5 This is a generalization advantage diagram in the embodiments of the present invention, used to illustrate the generalization ability of the present invention in different scenario families; Figure 6 This is a robustness advantage diagram in the embodiments of the present invention, used to illustrate the robustness of the present invention under external disturbance conditions. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the described embodiments are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the spirit and essence of the present invention, those skilled in the art can make equivalent substitutions or improvements to the parameters, model forms, training algorithms and execution devices.

[0046] Reference Figures 1-6 A smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning includes the following steps: S1. Construct a smart park electricity-carbon coupling system model and collect park status data: base load, photovoltaic output, wind power output, electricity price, electricity sales price, dynamic grid carbon intensity, outdoor temperature, HVAC status, BES status, EV fleet status, and cumulative carbon emission status, and establish the relationship between park power balance and carbon emission calculation. S1 in this embodiment includes the following sub-steps: S1-1. Collect the exogenous conditions of the park, including base load, photovoltaic output, wind power output, electricity purchase price, electricity sales price, dynamic grid carbon intensity, and outdoor temperature; (1); in, Based on the load, and The outputs are solar and wind power, respectively. and These are the electricity purchase price and the electricity sales price, For dynamic grid carbon intensity, Outdoor temperature For discrete scheduling time period indexes, For exogenous state vectors, the superscript is... This represents the transpose of a vector.

[0047] The process of S1-1 is as follows: S1-1-1. Calculate the net exchange power between the park and the external power grid based on the collected operating status of the park's equipment.

[0048] (2); in, For HVAC power, and These represent the charge and discharge power of the BES, and These are the EV charging power and V2G discharging power, respectively. S1-1-2. Determine the power purchased and power sold based on the net exchange power. (3); in, The net exchange power between the park and the external power grid. and These are the power purchased and the power sold, respectively. Indicates taking the positive part; when When the park purchases electricity, Electricity is sold in the industrial park.

[0049] S1-2. Calculate the current carbon emissions based on the dynamic grid carbon intensity, and update the cumulative carbon emissions status based on the daily carbon budget; (4); in, For time period The amount of carbon emissions generated For time period Dynamic grid carbon intensity, For the power purchase capacity, The length of a single scheduling period. and The deadlines are respectively and The cumulative carbon emissions.

[0050] S1-3. Construct electricity-carbon equivalent price and renewable energy sufficiency signals for subsequent RPCD marginal coordination; (5); in, For time period The electricity-carbon equivalent price, For electricity purchase price, As carbon cost weight, This is the price conversion factor for carbon emissions. For dynamic grid carbon intensity; For the abundance of renewable energy, and The outputs are solar and wind power, respectively. Based on the load, For comfort neutral HVAC reference power, The required charging power for EVs.

[0051] S2. Construct a generalized energy storage aggregation model for HVAC, BES, and EV: Represent the HVAC cluster as thermal virtual energy storage, the BES as recoverable physical energy storage, and the EV fleet as time-aware mobile energy storage, forming a unified power-energy envelope. S2 in this embodiment includes the following sub-steps: S2-1. Establish a unified power-energy envelope for generalized energy storage, for any generalized energy storage resource. , where its aggregation state is defined. and signed power Where positive values ​​represent energy absorbed and negative values ​​represent energy released; where, This is an index for generalized energy storage resource types. , , and These represent HVAC thermal virtual energy storage, BES physical energy storage, and EV mobile energy storage, respectively. For resources During the period The aggregation energy state, Its signed power; (6); in, and Resources During the period The maximum energy release power and the maximum energy absorption power, and These are the upper and lower bounds of the aggregation energy state, respectively.

[0052] Establish a generalized energy storage state transition model: (7); in, This is the self-dissipation or thermal relaxation coefficient. and For charging and discharging efficiency, and For terms related to the arrival and departure of exogenous energy; and Resources The aggregated energy state in adjacent time periods This is the self-dissipation or thermal relaxation coefficient. and These are charging efficiency and discharging efficiency, respectively. For signed power, , The length of the scheduling period. and These are the arrival and departure terms of exogenous energy, respectively. For resource type indexing, This is a time-period index.

[0053] S2-2. Model the HVAC cluster as a virtual thermal energy storage system based on Equivalent State of Charge (ESOC) for buildings. Define the midpoint of the comfort temperature range. and width And according to the operating mode Construct a thermal virtual energy storage state; (8); in, For architecture During the period The HVAC equivalent state of charge, with a value ranging from 0 to 1; Indoor temperature, The midpoint of the comfortable temperature range, To allow for deviations in comfort temperature, This is the operating mode symbol; 1 for cooling mode and -1 for heating mode.

[0054] With comfort neutral HVAC power Using the reference power as a benchmark, the increase in actual HVAC power relative to the reference power is considered as thermal virtual charging, and the decrease is considered as thermal virtual discharging; (9); in, For architecture HVAC relative reference power, This represents the actual HVAC power. For comfort neutral reference power, and These are the thermal virtual charging power and the energy dissipation power, respectively. .

[0055] S2-3. Model BES as a recoverable physical energy storage system, and construct a dynamic recovery baseline based on the current energy, the maximum recoverable energy for the remaining time period, and the reference terminal energy. (10); (11); in, From the time period The maximum recoverable energy of BES at the end of the scheduling cycle. To improve BES charging efficiency, The maximum rated charging power for BES, The last period of the scheduling cycle, The duration of the time period; For time period The lower limit of dynamic recovery energy. It is the physical minimum energy. This is the reference energy at the end of the cycle. To restore safety margin.

[0056] The upper limit of BES discharge is constrained by both the physical SOC lower limit and the recovery lower limit. (12); in, For BES in time period The upper limit of the allowable discharge power, The rated maximum discharge power of BES, For current energy, It is the physical minimum energy. To dynamically restore the lower limit of energy, For discharge efficiency, This represents the duration of the time period.

[0057] S2-4. Model the EV fleet as a time-sensitive mobile energy storage system. EVs are classified into private cars, taxis, buses and logistics vehicles according to vehicle type, and the time gap is calculated based on arrival, departure, target SOC and rechargeable capacity. (13); (14); in, For EV vehicle type index, This represents the target energy level for this type of EV before departure. For time period The current aggregated energy, For the target energy gap, This is recoverable energy that can be replenished through subsequent charging before leaving the station. To address the urgent time gap after deducting recoverable energy, .

[0058] The process in S2-4 is as follows: S2-4-1. Determine the mandatory charging power for EVs based on the time gap and separate it from the optional flexible charging power; (15); (16); in, for Total charging power for EV-like vehicles To ensure the necessary charging power is available to meet the target within the specified timeframe, Optional flexible charging power, This is the upper limit of charging power. Due to the tight deadline, For charging efficiency, This represents the duration of the time period.

[0059] S2-4-2. Set a minimum safe energy limit for EVs to ensure that V2G only uses energy above the minimum safe energy limit. (17); (18); in, for EV-like periods The lower limit of safe energy for the specified period. It is the physical minimum energy. For energy safety margin, For the target energy at the departure station, Energy can be restored before leaving the station. For reference energy, To allow for energy deviation; This represents the upper limit of V2G discharge power. The percentage of vehicles available online. The number of vehicles of this type. Rated power for V2G on a single vehicle For discharge efficiency S3. Construct a queue-enhanced generalized energy storage assessment constrained Markov decision process: distinguish between original actions, coordinated actions, and safe execution actions, so that the original actions generated by the reinforcement learning strategy do not directly affect the physical system; S3 in this embodiment includes the following sub-steps: S3-1. The smart park scheduling problem is constructed as a queue-enhanced generalized energy storage assessment-constrained Markov decision process, denoted as... ,in To enhance the state space of the queue, For the original action space, As a state transition kernel, For the reward function, For queue-driven constraint signals, For the pre-execution safety assessment and correction operator; S3-2, Constructing a queue to enhance state: (19); in, For time period The park's environmental conditions, Let be the Lyapunov queue vector. , , and These are the HVAC comfort queue, BES recovery queue, EV service queue, and carbon budget queue. Enhance the state of the concatenated queue.

[0060] S3-3, Actor network outputs six-dimensional resource budgeting action: (20); in, The six-dimensional raw resource budget action output by the Actor. and These are the virtual charging and discharging budgets for HVAC, respectively. and The charging and discharging budgets for BES are respectively. For EV flexible charging budget, For the V2G discharge budget of EV, superscript This represents the transpose of a vector.

[0061] S3-4. Establish a closed-loop interaction chain of original actions, coordinated actions, and safety actions; (twenty one); in, To enhance the current queue state, For the Actor's original actions, The actions after RPCD marginal coordination The execution action after GSAOM safety correction. The next time-period queue enhancement state obtained after performing a safety action.

[0062] S4. Construct a Lyapunov multi-cohort long-term constraint pressure representation: Construct an HVAC comfort cohort, a BES recovery cohort, an EV service cohort, and a carbon budget cohort to describe cross-period constraint pressure; S4 in this embodiment includes the following sub-steps: S4-1. Construct queue-driven signals corresponding to HVAC, BES, EV and carbon budget, which respectively represent insufficient comfort energy storage, insufficient BES recovery, EV life gap and current carbon emissions. (twenty two); in, For queue driving signal vector, , , and They represent time periods respectively. Insufficient HVAC comfort energy storage, insufficient BES recovery, EV service gap and carbon emission impact, etc. This represents the transpose of a vector.

[0063] S4-2, Update service class Lyapunov queues and carbon budget queues: (twenty three); (twenty four); in, For service During the period Lyapunov queues, , and For queue memory or decay coefficient, For service-type queues, To correspond to the service benchmark, For carbon budget queue, This represents the current carbon emissions. To be allocated to time periods carbon budget, .

[0064] S4-3. Construct a quadratic Lyapunov function and a queue pressure term to adjust the relationship between cost minimization and long-run constraint satisfaction; (25); (26); (27); in, It is a quadratic Lyapunov function. For the first Queues; For queue pressure terms, , and For each service category, , and For the corresponding service benchmark, This represents the current carbon emissions. For the current carbon budget; As a reward for training, For operating costs, As queue pressure weight, For safety correction and auxiliary penalty costs.

[0065] S5. Construct RPCD signals and generate coordinated actions through a generalized energy storage marginal coordinator: Utilize renewable energy sufficiency, price, carbon pressure, and EV term pressure to perform marginal coordination and resource allocation on the resource budget actions output by the Actor; S5 in this embodiment includes the following sub-steps: S5-1. Construct the RPCD signal vector, where and These respectively represent abundant and insufficient renewable energy sources. Indicates the electricity-carbon equivalent price. This indicates pressure on the normalized carbon budget. This indicates EV term pressure; (28); S5-2. Based on RPCD signals, normalized queue pressure, resource recoverability pressure, and energy storage carbon value, calculate the marginal score for each resource direction. (29); (30); in, For resource direction indexing, For time period Resource direction Marginal rating For the corresponding scoring function, For RPCD signal vectors, To normalize queue pressure, To address pressures on resource recoverability, Carbon value for energy storage; For saturated functions The coordination coefficient after restrictions For the original normalized budget, This represents the upper limit of available power in the corresponding direction. To coordinate power.

[0066] S5-3. Reconstruct the campus scheduling actions based on the coordinated HVAC, BES, and EV power. . (31); (32); (33); in, , and These are the reconfigured net power of HVAC, BES, and EV, respectively. For HVAC comfort neutral reference power, and These represent the coordinated charging and discharging power of the HVAC system. and The charging and discharging power are coordinated by BES respectively. The required charging power for EVs To coordinate flexible charging power for EVs, To coordinate V2G discharge power.

[0067] S6. Perform pre-execution safety corrections for coordinated actions through a unified GSAOM: sequentially or in parallel correct EV life-term safety, BES recoverability, HVAC comfort, and carbon budget protection to ensure that the final action meets physical and recoverability constraints. S6 in this embodiment includes the following sub-steps: S6-1. Perform time-limited safety correction on EV flexible charging and V2G to ensure that the EV charging power is not reduced and to limit V2G from exceeding the time-limited safety energy limit. (34); (35); in, Total charging power for EV safety The required charging power cannot be reduced. The flexible charging power is after safety calibration. For safe V2G discharge power, To coordinate V2G power, The upper limit of V2G power given for time-limited safety constraints.

[0068] S6-2. Perform recoverability correction on BES charging and discharging to ensure that the energy of BES at the next moment is not lower than the dynamic recovery baseline. (36); (37); in, and These are the BES charging power and discharging power after safety calibration, respectively. and These are the coordinated charging power and discharging power, respectively. This is the upper limit of physical charging power. The upper limit of recoverable discharge power is required to simultaneously meet both the physical SOC baseline and the dynamic recovery baseline.

[0069] S6-3. Perform ESOC and comfort safety correction on virtual charging and discharging of HVAC systems to maintain building temperature within a comfortable range. (38); in, The actual power of the HVAC system after safety correction. For comfort neutral reference power, and These are the thermal virtual charging power and discharging power, respectively, which satisfy the constraints of ESOC and temperature comfort range.

[0070] S6-4. Protect the carbon budget by prioritizing the reduction of optional flexible charging for EVs and non-essential BES charging when predicted carbon emissions exceed the remaining budget reference value. (39); (40); in, For time period Permissible reference carbon emissions For daily carbon budget, This represents the cumulative carbon emissions up to the previous period. For carbon budget safety margin, This is the end of the scheduling cycle; The power that needs to be reduced, Predicted carbon emissions corresponding to safety actions. For dynamic grid carbon intensity, The duration is the length of the time period. .

[0071] S6-5, Output safety actions, safety correction values, and constraint activation reports; (41); in, For safety actions Compared to coordinated movements The correction vector, The first norm of the correction vector is used to characterize the total correction magnitude; To constrain activation reports, , , and These represent the activation status of HVAC, BES, EV, and carbon budget security constraints, respectively.

[0072] S7. Execute safety actions and use safety correction feedback to update reinforcement learning strategies: Apply safety actions to the smart park environment, record safety correction amounts and constraint activation reports, and use them for Actor-Critic training updates to reduce subsequent reliance on safety corrections.

[0073] S7 in this embodiment includes the following sub-steps: S7-1. Map safety actions back to the Actor budget space to form safety feedback targets; (42); in, To map the safety feedback objective back to the Actor budget space, For resource direction Safe execution power, This represents the upper limit of power in that direction. To prevent positive numbers with a denominator of zero, A saturation function to restrict the result to the range of allowed actions of the Actor.

[0074] S7-2. Construct a safety correction feedback loss to make the subsequent output of the Actor closer to the feasible region. (43); in, To safely correct feedback loss, To track losses for safety targets, For the loss of consistency of action direction, and These are the weighting coefficients for the two losses, respectively.

[0075] S7-3. Update the policy network and value network through the Actor-Critic framework to obtain a closed-loop policy for online scheduling; (44); in, For Actor policy loss, This represents the expectation for the experience replay sample. For parameters The Critic state-action value function For parameters Actor strategy, To enhance the queue's state, To safely correct the feedback loss weights, To ensure safety, feedback loss is corrected.

[0076] like Figure 1 As shown, the smart park system considered in this invention includes a renewable energy side, a load side, an energy storage side, and a grid interaction side. The renewable energy side includes photovoltaic and wind power, the load side includes base load and HVAC clusters, the energy storage side includes BES and EV fleets, and the grid interaction side includes purchased power, time-varying electricity prices, and dynamic carbon intensity.

[0077] In the modeling process, HVAC clusters are represented as virtual energy storage that can move energy forward or backward within the comfortable indoor temperature range; BES (Battery Execution System) is represented as stationary energy storage with power constraints, SOC (State of Charge) constraints, and terminal recovery constraints; and EV fleets are represented as mobile energy storage with arrival / departure information, service duration, and V2G capabilities. This unified modeling approach forms a generalized set of energy storage resources and provides a unified interface for subsequent coordination and safety correction.

[0078] like Figure 2 As shown, this invention employs the RPCD-LyDDPG framework for policy training. Its input states include electricity price, renewable energy output, load, outdoor temperature, building temperature, BES status, and Lyapunov queues reflecting long-term constraint pressures. The Actor network outputs the original scheduling actions, the Critic network evaluates the state-action value, and the Replay Buffer is used to store interaction samples.

[0079] Based on ordinary DDPG, this invention introduces Lyapunov guidance information to embed queue states and costs related to long-term constraints into the reinforcement learning process; at the same time, RPCD marginal coordination and unified safe action operators are set between the original action and the final executed action, so that the policy does not simply rely on end pruning, but gradually learns the action generation method close to the feasible region during training.

[0080] To verify the effectiveness of the method described in this invention in the generalized energy storage scheduling of smart parks, this embodiment uses... Figure 1The simulated system of the park is used for verification. The park side includes photovoltaic, wind power, base load, HVAC cluster, BES energy storage system, and EV fleet, and is connected to the external power grid, electricity price signals, and dynamic carbon intensity signals; the control side uses... Figure 2 The RPCD-LyDDPG training process shown in the figure ultimately outputs safe scheduling actions that satisfy multiple types of constraints.

[0081] For comparative analysis, this embodiment sets up several representative experimental conditions: the first is the training safety verification condition, used to observe the changes in constraint violation, queue backlog, and action correction during the policy learning process; the second is the final performance verification condition, used to compare the economy and feasibility of different scheduling methods on the same typical park operation day; the third is the scenario generalization verification condition, used to examine the adaptability when renewable energy, load, HVAC pressure, BES capacity, and EV fleet conditions change; and the fourth is the disturbance robustness verification condition, used to examine the safe operation capability under external uncertain disturbances.

[0082] In terms of comparison, ordinary deep reinforcement learning methods, model predictive control methods, rule-based control methods, and the RPCD-LyDDPG method described in this invention were selected for comparison. Ordinary deep reinforcement learning methods are mainly used to illustrate that relying solely on rewards and penalties is insufficient to stably satisfy long-term constraints; model predictive control and rule-based control methods are mainly used to illustrate the insufficient adaptability of traditional control methods under complex electric-carbon coupling constraints; the method of this invention generates execution actions through Lyapunov queues, RPCD marginal coordination, and GSAOM safety correction.

[0083] In terms of evaluation methods, this embodiment adopts the feasibility-first principle, and does not use single operating cost as the sole evaluation criterion. Instead, it comprehensively considers indicators such as comfort default, BES terminal recovery deviation, EV off-site energy gap, carbon budget excess, action safety correction amount, and net operating cost. Only when the scheduling result simultaneously meets the constraints of comfort, energy storage recovery, EV service, and carbon budget is the scheduling result considered feasible.

[0084] Based on the above experimental setup, the training security verification results are as follows: Figure 3 As shown, the final performance and layered security verification results are as follows: Figure 4 As shown, the scenario generalization verification results are as follows: Figure 5 As shown, the results of the perturbation robustness verification are as follows: Figure 6 As shown in the figure. The experimental results above demonstrate the technical effectiveness of this invention from four aspects: training process, final execution effect, scene adaptability, and stability against external disturbances.

[0085] like Figure 3As shown, the training security advantage diagram illustrates the security advantages of this invention during the training phase. The diagram presents results for the verification running cost trajectory, verification constraint default trajectory, cumulative debt of the four constraints, Lyapunov queue backlog, hierarchical unsafe step rate, and security correction amount.

[0086] from Figure 3 It can be seen that this invention can maintain the verification default at a low level in the early stage of training, and as training progresses, the growth of accumulated constraint debt is significantly slower than that of the comparative method, and the Lyapunov queue backlog remains at a lower level. At the same time, the unsafe components in the original actions are first reduced by the coordinator and then further eliminated by the safe action operator, indicating that this invention can gradually learn a safer closed-loop strategy during the training process.

[0087] like Figure 4 As shown, the final performance and hierarchical safety graph are used to compare the overall performance of different methods in the final evaluation stage. The graph compares the net cost, constraint default, feasibility rate, corrected residual default, and various safety layer outputs of each method from multiple dimensions.

[0088] Depend on Figure 4 It is evident that this invention, while maintaining low operating costs, better satisfies various constraints such as comfort, BES recovery, EV service, and carbon budget; furthermore, its execution layer default is significantly lower than that of ordinary deep reinforcement learning baselines, indicating that the hierarchical safety mechanism can effectively block the direct execution of inactive actions. These results demonstrate that this invention is not only more economical but also truly achieves feasibility-prioritized safe scheduling.

[0089] like Figure 5 As shown, the generalization advantage map is used to examine the adaptability of the present invention in different scenario families. These scenario families include changes in renewable resource levels, load changes, HVAC pressure changes, and EV fleet conditions. By comparing feasible coverage, default levels, and overall performance under different scenarios, it can be verified whether the learned strategy is effective only in a single nominal scenario.

[0090] Figure 5 This demonstrates that the present invention maintains higher feasible coverage and lower default residuals in most scenario families, indicating that the proposed generalized energy storage modeling, Lyapunov multi-queue, and hierarchical safety action mechanism can enhance the policy's adaptability to changes in operating conditions, rather than simply overfitting to a fixed training scenario.

[0091] like Figure 6As shown, the robustness map is used to examine the control stability of the present invention under external disturbances. These disturbances may include renewable power output disturbances, price disturbances, temperature disturbances, carbon factor disturbances, and combined disturbances. The robustness of the strategy can be further evaluated by comparing the default level, feasibility rate, and security layer response after the disturbances.

[0092] Figure 6 This demonstrates that, under both single and combined perturbation conditions, the present invention maintains low tail defaults and relatively stable feasibility, while conventional deep reinforcement learning baselines are more prone to significant degradation when perturbations are amplified. Therefore, the present invention, by maintaining long-term constraint pressure through Lyapunov queues and suppressing unsafe actions through safe action operators, can improve the robustness of campus scheduling in real-world uncertain environments.

[0093] This invention can also be implemented as a smart park electricity-carbon coupled generalized energy storage optimization scheduling system, including a status acquisition module, a generalized energy storage aggregation module, a Lyapunov queue update module, an RPCD signal construction module, a marginal coordination module, a unified GSAOM security correction module, a reinforcement learning training module, and an online execution module.

[0094] The system comprises the following modules: a status acquisition module for acquiring exogenous park status and equipment status; a generalized energy storage aggregation module for constructing a unified power-energy envelope for HVAC, BES, and EV; a Lyapunov queue update module for updating long-term constraint pressures; an RPCD signal construction module for calculating renewable energy, prices, carbon pressure, and EV term pressures; a marginal coordination module for generating coordinated actions; a unified GSAOM safety correction module for generating safety actions; a reinforcement learning training module for updating the Actor-Critic network based on safety correction feedback; and an online execution module for distributing safety actions to the park's energy management system.

[0095] The present invention can also be implemented as an electronic device, the electronic device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method steps described in any of the above embodiments.

[0096] The present invention can also be implemented as a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0097] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning, characterized in that, Includes the following steps: S1. Construct a smart park electricity-carbon coupling system model and collect park status data: base load, photovoltaic output, wind power output, electricity price, electricity sales price, dynamic grid carbon intensity, outdoor temperature, HVAC status, BES status, EV fleet status, and cumulative carbon emission status, and establish the relationship between park power balance and carbon emission calculation. S2. Construct a generalized energy storage aggregation model for HVAC, BES, and EV: Represent the HVAC cluster as thermal virtual energy storage, the BES as recoverable physical energy storage, and the EV fleet as time-aware mobile energy storage, forming a unified power-energy envelope. S3. Construct a queue-enhanced generalized energy storage assessment constrained Markov decision process: distinguish between original actions, coordinated actions, and safe execution actions, so that the original actions generated by the reinforcement learning strategy do not directly affect the physical system; S4. Construct a Lyapunov multi-cohort long-term constraint pressure representation: Construct an HVAC comfort cohort, a BES recovery cohort, an EV service cohort, and a carbon budget cohort to describe cross-period constraint pressure; S5. Construct RPCD signals and generate coordinated actions through a generalized energy storage marginal coordinator: Utilize renewable energy sufficiency, price, carbon pressure, and EV term pressure to perform marginal coordination and resource allocation on the resource budget actions output by the Actor; S6. Perform pre-execution safety corrections for coordinated actions through a unified GSAOM: sequentially or in parallel correct EV life-term safety, BES recoverability, HVAC comfort, and carbon budget protection to ensure that the final action meets physical and recoverability constraints. S7. Execute safety actions and use safety correction feedback to update reinforcement learning strategies: Apply safety actions to the smart park environment, record safety correction amounts and constraint activation reports, and use them for Actor-Critic training updates to reduce subsequent reliance on safety corrections.

2. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in claim 1, characterized in that, S1 includes the following sub-steps: S1-1. Collect the exogenous conditions of the park, including base load, photovoltaic output, wind power output, electricity purchase price, electricity sales price, dynamic grid carbon intensity, and outdoor temperature; S1-2. Calculate the current carbon emissions based on the dynamic grid carbon intensity, and update the cumulative carbon emissions status based on the daily carbon budget; S1-3. Construct electricity-carbon equivalent price and renewable energy sufficiency signals for subsequent RPCD marginal coordination.

3. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in claim 2, characterized in that, The process of S1-1 is as follows: S1-1-1. Calculate the net exchange power between the park and the external power grid based on the collected operating status of the park's equipment; S1-1-2. Determine the power purchased and the power sold based on the net exchange power.

4. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in any one of claims 1 to 3, characterized in that, S2 includes the following sub-steps: S2-1. Establish a unified power-energy envelope for generalized energy storage, for any generalized energy storage resource. , where its aggregation state is defined. and signed power Where positive values ​​represent energy absorbed and negative values ​​represent energy released; where, For a generalized energy storage resource type index, , , and These represent HVAC thermal virtual energy storage, BES physical energy storage, and EV mobile energy storage, respectively. For resources During the period The aggregation energy state, Its signed power; Establish a generalized energy storage state transition model; S2-2. Model the HVAC cluster as a virtual thermal energy storage system based on Equivalent State of Charge (ESOC) for buildings. Define the midpoint of the comfortable temperature range. and width And according to the operating mode Construct a thermal virtual energy storage state; With comfort neutral HVAC power Using the reference power as a benchmark, the increase in actual HVAC power relative to the reference power is considered as thermal virtual charging, and the decrease is considered as thermal virtual discharging; S2-3. Model BES as a recoverable physical energy storage system, and construct a dynamic recovery baseline based on the current energy, the maximum recoverable energy for the remaining time period, and the reference terminal energy; the upper limit of BES discharge is simultaneously constrained by the physical SOC baseline and the recovery baseline. S2-4. Model the EV fleet as a time-sensitive mobile energy storage system. EVs are classified into private cars, taxis, buses and logistics vehicles according to vehicle type, and the time gap is calculated based on arrival, departure, target SOC and rechargeable capacity.

5. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in claim 4, characterized in that, The process in S2-4 is as follows: S2-4-1. Determine the mandatory charging power for EVs based on the time gap and separate it from the optional flexible charging power; S2-4-2. Set a minimum safe energy limit for EVs to ensure that V2G only uses energy above the minimum safe energy limit.

6. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in any one of claims 1 to 3, characterized in that, S3 includes the following sub-steps: S3-1. The smart park scheduling problem is constructed as a queue-enhanced generalized energy storage assessment-constrained Markov decision process, denoted as... ,in To enhance the state space of the queue, For the original action space, As a state transition kernel, For the reward function, For queue-driven constraint signals, For the pre-execution safety assessment and correction operator; S3-2, Constructing queue enhancement states; S3-3, Actor network outputs six-dimensional resource budgeting actions; S3-4. Establish a closed-loop interaction chain of original actions, coordinated actions, and safety actions.

7. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in any one of claims 1 to 3, characterized in that, S4 includes the following sub-steps: S4-1. Construct queue-driven signals corresponding to HVAC, BES, EV and carbon budget, which respectively represent insufficient comfort energy storage, insufficient BES recovery, EV life gap and current carbon emissions. S4-2, Update service class Lyapunov queues and carbon budget queues; S4-3. Construct a quadratic Lyapunov function and a queue pressure term to adjust the relationship between cost minimization and long-term constraint satisfaction.

8. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in any one of claims 1 to 3, characterized in that, S5 includes the following sub-steps: S5-1. Construct the RPCD signal vector, where and These respectively represent abundant and insufficient renewable energy sources. Indicates the electricity-carbon equivalent price. This indicates pressure on the normalized carbon budget. This indicates EV term pressure; S5-2. Based on RPCD signals, normalized queue pressure, resource recoverability pressure, and energy storage carbon value, calculate the marginal score for each resource direction. S5-3. Reconstruct the campus scheduling actions based on the coordinated HVAC, BES, and EV power.

9. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in any one of claims 1 to 3, characterized in that, Step S6 includes the following sub-steps: S6-1. Perform time-limited safety correction on EV flexible charging and V2G to ensure that the EV charging power is not reduced and to limit V2G from exceeding the time-limited safety energy limit. S6-2. Perform recoverability correction on BES charging and discharging to ensure that the energy of BES at the next moment is not lower than the dynamic recovery baseline. S6-3. Perform ESOC and comfort safety correction on virtual charging and discharging of HVAC systems to maintain building temperature within a comfortable range. S6-4. Protect the carbon budget by prioritizing the reduction of optional flexible charging for EVs and non-essential BES charging when predicted carbon emissions exceed the remaining budget reference value. S6-5, Output safety actions, safety correction values, and constraint activation reports.

10. The smart park electricity-carbon coupled generalized energy storage optimization scheduling method based on Lyapunov secure reinforcement learning as described in any one of claims 1 to 3, characterized in that, Step S7 includes the following sub-steps: S7-1. Map safety actions back to the Actor budget space to form safety feedback targets; S7-2. Construct a safety correction feedback loss to make the subsequent output of the Actor closer to the feasible region. S7-3. Update the policy network and value network through the Actor-Critic framework to obtain a closed-loop policy for online scheduling.

Images