Energy storage management system for building energy conservation

Abstract

The invention relates to the technical field of reinforcement learning, and provides an energy storage management system for building energy saving. The system comprises a data acquisition module, a data processing module, a load prediction module, an energy storage scheduling module and an intelligent scheduling control module. In the load prediction module, the system constructs a symmetric sliding window fluctuation perception CNN model, and the timeliness and sensitivity of thermal load prediction are improved; in an energy storage scheduling module, the system combines opponent disturbance modeling, a bidirectional adversarial updating mechanism and a subgradient accumulation correction technology, an Actor-Critic model is improved, an adversarial double-track Actor-Critic model is formed, and the robustness and adaptive ability of a phase change energy storage scheduling strategy in a complex dynamic environment are enhanced; according to the system, high-precision thermal load prediction and high-reliability energy storage heat charging and discharging scheduling can be achieved under the multi-source disturbance condition, the energy efficiency level of a building heating and ventilation system is remarkably improved, and operation energy consumption and cost are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of reinforcement learning, and in particular to an energy storage management system for building energy conservation. Background Art

[0002] With the increasing global energy tension and the continuous improvement of energy conservation and emission reduction requirements, the energy consumption problem in the building heating and ventilation field has attracted more and more attention; more and more energy storage management systems are introduced to improve energy utilization efficiency and reduce operating costs through means such as environmental perception, load prediction, and intelligent scheduling; however, there are still many deficiencies in the existing energy storage management systems: First, the traditional systems are still relatively rough in load prediction and thermal management strategy design, lacking a modeling mechanism for the fine-grained time series fluctuations and complex interaction relationships between features in building energy consumption data, and failing to fully explore the fine-grained laws of heat load changes under different operating conditions, resulting in lags or deviations in the regulation of energy release and storage, and affecting the overall energy-saving effect; Second, the existing systems mainly train strategies based on static or weakly dynamic environments, which easily cause slow response of energy storage units and decline in heat storage and release efficiency, restricting the stability and economy of the system. Summary of the Invention

[0003] The present invention provides an energy storage management system for building energy conservation. Aiming at the problems of large heat load fluctuations, frequent environmental disturbances, and insufficient adaptability of intelligent prediction and scheduling systems in traditional energy storage management systems, a new system architecture based on symmetric sliding window fluctuation perception CNN load prediction and adversarial dual-rail Actor-Critic energy storage scheduling optimization is proposed. This system can achieve higher-precision heat load prediction and more robust phase change energy storage charging / discharging and heat load intelligent scheduling, significantly improving the comprehensive energy efficiency and operation stability of the building heating and ventilation energy storage management system; specifically: by introducing a time dimension symmetric padding strategy, a strip-shaped feature pooling mechanism, and a weighted fluctuation perception loss function, the ability of the CNN model in extracting fine-grained fluctuation features of the load and prediction accuracy is improved, and it can more accurately capture the change laws of heating and hot water demands in different operating regions and time periods; at the same time, through opponent perturbation modeling, bidirectional adversarial update, and subgradient cumulative correction mechanisms, the robustness and adaptive ability of the energy storage scheduling strategy in the face of complex environments such as load mutations and meteorological anomalies are enhanced, and it can intelligently optimize the charging / discharging behavior of phase change energy storage devices, flexibly respond to heat loads, and coordinate with the environment for control, realizing the overall energy efficiency improvement of the building heating and ventilation system and the effective reduction of operating costs.

[0004] The present invention provides an energy storage management system for building energy conservation. The system includes a data acquisition module, a data processing module, a load prediction module, an energy storage scheduling module, and an intelligent scheduling control module;

[0005] Data acquisition module, deploy environmental sensors and energy consumption monitoring devices in the heating heat source machine room, domestic hot water equipment area, and terminal radiator area, collect the supply and return water temperatures, flow rates, terminal room temperatures, hot water usage, and state parameters of the phase change energy storage unit, obtain HVAC state data, and through edge computing, unify the format and perform preliminary denoising to generate original energy storage time series data;

[0006] Data processing module, clean the original energy storage time series data, remove outliers, synchronize time, and standardize the format to generate an energy storage data feature matrix;

[0007] Load prediction module, establish a CNN model, optimize the CNN model through a time dimension symmetric padding strategy, strip feature pooling mechanism, and weighted fluctuation perception loss function, construct a symmetric sliding window fluctuation perception CNN model, process the energy storage data feature matrix through the symmetric sliding window fluctuation perception CNN model to generate heat load prediction data; the symmetric sliding window fluctuation perception CNN model includes multiple layers of two-dimensional convolutional networks and fully connected layers;

[0008] Energy storage scheduling module, establish an Actor-Critic model, improve the Actor-Critic model through opponent perturbation modeling, two-way adversarial update mechanism, and subgradient cumulative correction, construct an adversarial dual-rail Actor-Critic model, combine the heat load prediction data, and generate an energy storage scheduling plan through the adversarial dual-rail Actor-Critic model;

[0009] Intelligent scheduling control module, generate and execute control instructions according to the energy storage scheduling plan, coordinate the charging and discharging processes of the phase change energy storage unit, start-stop and regulation of heat source equipment, and flexible scheduling of the terminal load side, and generate an energy-saving evaluation report.

[0010] Further, the process of the load prediction module generating heat load prediction data specifically includes the following steps:

[0011] Step S1: Use a fixed-step sliding window to cut the energy storage data feature matrix to form a stable structure and generate a standard sample window tensor;

[0012] Step S2: Perform symmetric padding on the standard sample window tensor to generate a symmetrically padded sample tensor;

[0013] Step S3: Extract the joint distribution features of the time dimension and feature dimension of the symmetrically padded sample tensor through a multi-layer two-dimensional convolutional network, layer by layer, to form a high-dimensional convolutional feature map; use a 2×2 convolutional kernel for each layer, slide along the time and feature directions to perceive the correlation pattern between local heat load changes and environmental variables;

[0014] Step S4: Perform strip pooling on the high-dimensional convolutional feature map, only perform pooling compression in the feature dimension direction to generate a strip pooling feature map; this strategy avoids the destruction of the time structure by traditional max pooling and can compress the spatial redundancy between different energy consumption features while retaining the time dynamic trend;

[0015] Step S5: Process the strip pooling feature map through a fully connected layer to generate initial heat load prediction data, construct a mean square error loss function, optimize the mean square error loss function through an error direction penalty, a time series fluctuation sensitive weight, and an amplification penalty mechanism, construct a weighted fluctuation perception loss function, train the symmetric sliding window fluctuation perception CNN model through the weighted fluctuation perception loss function, optimize the model parameters to obtain a trained symmetric sliding window fluctuation perception CNN model, and use the trained symmetric sliding window fluctuation perception CNN model to optimize the initial heat load prediction data to generate heat load prediction data.

[0016] Further, step S2 specifically includes: performing symmetric padding on the standard sample window tensor in the time dimension, and continuing to model the boundary structure information of the standard sample window tensor by copying the feature vectors of the first and last time steps of the standard sample window tensor to generate a symmetric padding sample tensor.

[0017] Further, the process of the energy storage scheduling module generating an energy storage scheduling plan specifically includes the following steps:

[0018] Step B1: Initialize the user Actor network, the opponent Actor network, the Critic network, and the experience replay pool of the adversarial dual-rail Actor-Critic model; define the state space and the action space; input the heat load prediction data and the HVAC state data into the state space;

[0019] Step B2: Based on the current state of the state space, use the user action in the action space selected by the user Actor network and add Gaussian exploration noise to enhance exploration. Use the opponent Actor network to generate a perturbation action according to the current state. Interact the perturbation action and the opponent action with the environment to obtain a new state and a reward, and store the five-tuple composed of the current state, the user action, the perturbation action, the reward, and the new state into the experience replay pool;

[0020] Step B3: Sample data from the experience replay pool, generate the next action through the user Actor network, calculate the target value, optimize the Critic network with the least squares loss to make the output Q value close to the target value, record the cumulative loss, and obtain the updated Critic network; define a perturbation penalty coefficient and dynamically adjust the penalty strength for the perturbation action in combination with the cumulative loss;

[0021] Step B4: Based on the updated Critic network, update the user Actor network and the opponent Actor network through the adversarial dual-track optimization mechanism; the user Actor network minimizes the cost evaluated by the Critic along the descending gradient direction; the opponent Actor network maximizes the cost evaluated by the Critic along the ascending gradient direction, and then introduce the sub-gradient correction to cumulatively correct the update direction of the opponent Actor network to enhance the perturbation coherence and attack strength of the opponent Actor;

[0022] Step B5: Iteratively loop through Step B3 - Step B4 to conduct multiple rounds of adversarial training on the user Actor network to obtain the trained user Actor network, and generate the energy storage scheduling plan according to the trained user Actor network.

[0023] Adopting the above solution, the beneficial effects achieved by the present invention are as follows:

[0024] The present invention realizes high-precision heat load prediction in the complex heat load environment of the building heating and ventilation system by introducing the symmetric sliding window fluctuation perception CNN model; through the time dimension symmetric filling strategy, the boundary features of the heat load data sequence are effectively continued, solving the problems of boundary heat load information distortion and increased prediction error in the traditional convolution processing process; adopting the strip feature pooling mechanism, accurately retains the time dynamic trend of the heat load change, and improves the model's perception ability of fine-grained fluctuations and sudden load changes; at the same time, through the weighted fluctuation perception loss function, the learning weight for micro-fluctuations and short-term sudden heat load changes is emphasized, enabling the prediction model to keenly capture the actual demand characteristics of the heating and domestic hot water systems under different operating conditions, significantly improving the load prediction accuracy and response timeliness of the energy storage management system in the real building heating and ventilation environment, and providing a highly reliable decision-making basis for subsequent phase change energy storage scheduling;

[0025] The present invention enhances the dynamic adaptability and scheduling robustness of the phase change energy storage system in the face of multi-source perturbations (such as load mutations, meteorological fluctuations, changes in user heat consumption behavior, etc.) during the operation of the building heating and ventilation by constructing an adversarial dual-track Actor-Critic energy storage scheduling optimization model; by introducing the opponent perturbation modeling and two-way adversarial update mechanism, the energy storage management system can continuously optimize the scheduling strategy under the simulated most unfavorable operating conditions, effectively solving the problems of response lag and strategy degradation of the existing intelligent scheduling system in emergency scenarios; through the sub-gradient cumulative correction mechanism, the coherence of the adversarial perturbation actions and the interference resistance ability are further improved, ensuring that the phase change energy storage unit can still stably execute the heat storage and release decision and the heat load response regulation under extreme working conditions, comprehensively improving the stability of the heating and ventilation energy storage management system and the anti-risk ability in extreme environments;

[0026] In summary, through the systematic innovation of the heat load prediction and phase change energy storage scheduling optimization module, the present invention realizes the comprehensive performance improvement of the energy-saving management of the building heating and ventilation system in a complex dynamic environment, significantly improves the load prediction accuracy and the reliability of energy storage scheduling, and solves the problems of poor adaptability and insufficient stability of the existing heating and ventilation energy storage management system; by accurately predicting the change trend of the heat load, flexibly optimizing the heat storage and release management and the control of terminal energy-consuming equipment, the overall energy consumption and operation cost of the building heating and ventilation system are effectively reduced, the indoor thermal comfort is improved, and practical technical support is provided for energy conservation, emission reduction and intelligent management in the building field. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the modules of an energy storage management system for building energy conservation proposed by the present invention;

[0028] Figure 2 It is a graph showing the change of the weight factor proposed in step S5 in Embodiment 3. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0030] Embodiment 1. According to Figure 1 , the present invention provides an energy storage management system for building energy conservation, which includes a data acquisition module, a data processing module, a load prediction module, an energy storage scheduling module, and an intelligent scheduling control module;

[0031] The data acquisition module deploys environmental sensors and energy consumption monitoring devices in the heating heat source machine room, the domestic hot water equipment area, and the terminal radiator area, collects the supply and return water temperatures, flow rates, terminal room temperatures, hot water usage amounts, and the state parameters of the phase change energy storage unit, obtains the heating and ventilation state data, and through edge computing, unifies the format and performs preliminary denoising to generate the original energy storage time series data;

[0032] The data processing module cleans the original energy storage time series data, removes outliers, synchronizes time, and standardizes the format to generate an energy storage data feature matrix;

[0033] The load forecasting module establishes a CNN model, optimizes the CNN model through a time - dimension symmetric padding strategy, a strip - type feature pooling mechanism, and a weighted fluctuation - aware loss function, constructs a symmetric sliding window fluctuation - aware CNN model, processes the energy storage data feature matrix through the symmetric sliding window fluctuation - aware CNN model, and generates heat load forecasting data; the symmetric sliding window fluctuation - aware CNN model includes multiple layers of two - dimensional convolutional networks and fully - connected layers;

[0034] The energy storage scheduling module establishes an Actor - Critic model, improves the Actor - Critic model through adversary perturbation modeling, a two - way adversarial update mechanism, and sub - gradient cumulative correction, constructs an adversarial dual - track Actor - Critic model, combines the heat load forecasting data, and generates an energy storage scheduling plan through the adversarial dual - track Actor - Critic model;

[0035] The intelligent scheduling control module generates and executes control instructions according to the energy storage scheduling plan, coordinates the charging and discharging processes of the phase - change energy storage unit, the start - stop and regulation of heat source equipment, and the flexible scheduling of the end - load side, and generates an energy - saving evaluation report.

[0036] Example 2. According to Figure 2 , this example is based on Example 1. In this example, the process of the load forecasting module generating heat load forecasting data specifically includes the following steps:

[0037] Step S1: Use a fixed - step sliding window to cut the energy storage data feature matrix to form a stable structure and generate a standard sample window tensor;

[0038] Step S2: Perform symmetric padding on the standard sample window tensor to generate a symmetric - padded sample tensor;

[0039] Step S3: Extract the joint distribution features of the time dimension and the feature dimension of the symmetric - padded sample tensor through multiple layers of two - dimensional convolutional networks, layer by layer, to form a high - dimensional convolutional feature map; each layer uses a 2×2 convolutional kernel and slides along the time and feature directions to perceive the correlation pattern between local heat load changes and environmental variables;

[0040] Step S4: Perform strip - type pooling on the high - dimensional convolutional feature map, only perform pooling compression in the feature dimension direction, and generate a strip - pooled feature map; this strategy avoids the destruction of the time structure by traditional max - pooling and can compress the spatial redundancy between different energy - consumption features while retaining the time - dynamic trend;

[0041] Step S5: Process the strip pooling feature map through a fully connected layer to generate initial heat load prediction data, construct a mean squared error loss function, optimize the mean squared error loss function through an error direction penalty, a time series fluctuation sensitive weight, and an amplification penalty mechanism, construct a weighted fluctuation perception loss function, train the symmetric sliding window fluctuation perception CNN model using the weighted fluctuation perception loss function, optimize the model parameters, obtain the trained symmetric sliding window fluctuation perception CNN model, and use the trained symmetric sliding window fluctuation perception CNN model to optimize the initial heat load prediction data to generate heat load prediction data. The formulas used are as follows:

[0042] ;

[0043] where, represents the weighted fluctuation perception loss function, represents the length of the time series, represents the time step, represents the error direction weight factor, represents the fluctuation sensitive weight factor, represents the amplification penalty coefficient, represents the true load value, represents the model prediction value, represents the squared error.

[0044] Example 3. This example is based on Example 1. In this example, the process of the load prediction module generating heat load prediction data specifically includes the following steps:

[0045] Step T1: Use a fixed-step sliding window to crop the energy storage data feature matrix to form a stable structure and generate a standard sample window tensor;

[0046] Step T2: Perform symmetric padding on the standard sample window tensor to generate a symmetrically padded sample tensor;

[0047] Step T3: Extract the joint distribution features of the time dimension and the feature dimension of the symmetrically padded sample tensor through a multi-layer two-dimensional convolutional network, layer by layer, to form a high-dimensional convolutional feature map; a 2×2 convolutional kernel is used for each layer, sliding along the time and feature directions to perceive the association pattern between local heat load changes and environmental variables;

[0048] Step T4: Perform strip pooling on the high-dimensional convolutional feature map, only performing pooling compression in the feature dimension direction to generate a strip pooling feature map; this strategy avoids the destruction of the time structure by traditional max pooling and can compress the spatial redundancy between different energy consumption features while retaining the time dynamic trend;

[0049] Step T5: Process the strip pooling feature map through a fully connected layer to generate initial heat load prediction data, construct a mean squared error loss function, train the symmetric sliding window fluctuation perception CNN model through the mean squared error loss function, optimize the model parameters to obtain the trained symmetric sliding window fluctuation perception CNN model, and use the trained symmetric sliding window fluctuation perception CNN model to optimize the initial heat load prediction data to generate heat load prediction data.

[0050] Embodiment 4: This embodiment is based on Embodiment 2. In this embodiment, step S2 specifically includes: performing symmetric padding on the standard sample window tensor in the time dimension, and by replicating the feature vectors of the first and last time steps of the standard sample window tensor, performing continuity modeling on the boundary structure information of the standard sample window tensor to generate a symmetric padding sample tensor; this method is different from ordinary zero-padding, can effectively retain the feature boundary change information, prevent the model from misjudging the energy consumption patterns at both ends of the sample during convolution, and enable the model to have context continuity when learning the energy consumption structure changes in the edge region.

[0051] Embodiment 5: This embodiment is based on Embodiment 4. In this embodiment, the process of the energy storage scheduling module generating the energy storage scheduling plan specifically includes the following steps:

[0052] Step B1: Initialize the user Actor network, opponent Actor network, Critic network, and experience replay pool of the adversarial dual-rail Actor-Critic model; define the state space and action space; input the heat load prediction data and HVAC state data into the state space.

[0053] Step B2: Based on the current state of the state space, use the user action in the action space selected by the user Actor network and add Gaussian exploration noise to enhance exploration. Use the opponent Actor network to generate a perturbation action according to the current state, interact the perturbation action and the opponent action with the environment to obtain a new state and a reward, and store the five-tuple composed of the current state, user action, perturbation action, reward, and new state in the experience replay pool.

[0054] Step B3: Sample data from the experience replay pool, generate the next action through the user Actor network, calculate the target value, optimize the Critic network with the least squares loss to make the Q value output by it close to the target value, record the cumulative loss, and obtain the updated Critic network; define a perturbation penalty coefficient and dynamically adjust the penalty intensity for the perturbation action in combination with the cumulative loss.

[0055] Step B4: Based on the updated Critic network, update the user Actor network and the opponent Actor network through the adversarial dual-rail optimization mechanism; the user Actor network minimizes the cost evaluated by the Critic along the descending gradient direction; the opponent Actor network maximizes the cost evaluated by the Critic along the ascending gradient direction, and then introduce the sub-gradient correction to cumulatively correct the update direction of the opponent Actor network to improve the perturbation coherence and attack strength of the opponent Actor. The formulas are as follows:

[0056] Gradient descent:

[0057] ;

[0058] Wherein, represents the parameters of the user Actor network, that is, the parameters of the energy storage scheduling strategy, represents the learning rate of the user Actor network, represents the sampled data in the experience replay pool, represents the parameters of the opponent Actor network, represents the Actor adversarial loss function; represents the loss function with respect to gradient;

[0059] Gradient ascent:

[0060] ;

[0061] Wherein, represents the learning rate of the opponent Actor network, represents the loss function with respect to gradient; represents the weight coefficient of the sub-gradient correction term, represents historical cumulative change amount, which is used to correct the current update direction;

[0062] Step B5: Iteratively loop through Step B3 - Step B4 to perform multiple rounds of adversarial training on the user Actor network, obtain the trained user Actor network, and generate an energy storage scheduling plan according to the trained user Actor network.

[0063] Example 6, this example is based on Example 4. In this example, the process of the energy storage scheduling module generating an energy storage scheduling plan specifically includes the following steps:

[0064] ​​Step R1: Initialize the user Actor network, opponent Actor network, Critic network, and experience replay pool of the adversarial dual-rail Actor-Critic model; define the state space and action space; the state space includes heat load prediction data and HVAC state data;

[0065] Step R2: Based on the current state of the state space, select the user action in the action space selected by the user Actor network, and add Gaussian exploration noise to enhance exploration. Use the opponent Actor network to generate a perturbation action according to the current state. Interact the perturbation action and the opponent action with the environment to obtain a new state and a reward, and store the five-tuple composed of the current state, user action, perturbation action, reward, and new state in the experience replay pool;

[0066] Step R3: Sample data from the experience replay pool, generate the next action through the user Actor network, calculate the target value, optimize the Critic network with the least squares loss to make the Q value output by it close to the target value, record the cumulative loss, and obtain the updated Critic network; define the perturbation penalty coefficient, and dynamically adjust the penalty strength for the perturbation action in combination with the cumulative loss;

[0067] Step R4: Based on the updated Critic network, update the user Actor network and the opponent Actor network through the adversarial dual-rail optimization mechanism; the user Actor network minimizes the cost evaluated by the Critic along the descending gradient direction; the opponent Actor network maximizes the cost evaluated by the Critic along the ascending gradient direction;

[0068] Step R5: Iteratively loop through Step R3 - Step R4 to perform multiple rounds of adversarial training on the user Actor network, obtain the trained user Actor network, and generate an energy storage scheduling plan according to the trained user Actor network.

[0069] The above describes the present invention and its implementation manners. This description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto; generally speaking, if those of ordinary skill in the art are inspired by it and, without departing from the gist of the present invention creation, design similar structural modes and embodiments to this technical solution without creative efforts, they shall fall within the protection scope of the present invention.

Claims

1. An energy storage management system for building energy conservation, the system comprising a data acquisition module and a data processing module, the data acquisition module acquiring heating, ventilation and air conditioning (HVAC) status data; the data processing module processing the HVAC status data to generate an energy storage data feature matrix; characterized in that: The system also includes a load forecasting module and an energy storage scheduling module; The load forecasting module constructs a symmetric sliding window fluctuation perception CNN model, processes the energy storage data feature matrix through the symmetric sliding window fluctuation perception CNN model, and generates heat load forecasting data; The energy storage scheduling module constructs an adversarial dual-rail Actor-Critic model, combines the heat load forecasting data, and generates an energy storage scheduling plan through the adversarial dual-rail Actor-Critic model.

2. The energy storage management system for building energy conservation according to claim 1, characterized in that: The construction method of the symmetric sliding window fluctuation perception CNN model is: establish a CNN model, and optimize the CNN model through a time dimension symmetric padding strategy, a strip-shaped feature pooling mechanism, and a weighted fluctuation perception loss function for construction.

3. The energy storage management system for building energy conservation according to claim 2, characterized in that: The symmetric sliding window fluctuation perception CNN model includes a multi-layer two-dimensional convolutional network and a fully connected layer.

4. The energy storage management system for building energy conservation according to claim 3, characterized in that: The process of the load forecasting module generating heat load forecasting data specifically includes the following steps: Step S1: Use a fixed-step sliding window to crop the energy storage data feature matrix to generate a standard sample window tensor; Step S2: Perform symmetric padding on the standard sample window tensor to generate a symmetrically padded sample tensor; Step S3: Extract the joint distribution features of the symmetrically padded sample tensor through a multi-layer two-dimensional convolutional network to form a high-dimensional convolutional feature map; Step S4: Perform strip-shaped pooling on the high-dimensional convolutional feature map to perform pooling compression in the feature dimension direction to generate a strip-pooled feature map; Step S5: Process the strip-pooled feature map through a fully connected layer, and construct a weighted fluctuation perception loss function to train the symmetric sliding window fluctuation perception CNN model to generate heat load forecasting data.

5. The energy storage management system for building energy conservation according to claim 4, characterized in that: Step S2 specifically includes: performing symmetric padding on the standard sample window tensor in the time dimension, and continuing to model the boundary structure information of the standard sample window tensor by copying the feature vectors of the first and last time steps of the standard sample window tensor to generate a symmetrically padded sample tensor.

6. The energy storage management system for building energy conservation according to claim 1, characterized in that: The process of the energy storage scheduling module generating an energy storage scheduling plan specifically includes the following steps: Step B1: Initialize the user Actor network, the opponent Actor network, and the experience replay pool of the adversarial dual-rail Actor-Critic model; define the state space, and input the heat load forecasting data and the HVAC state data into the state space; Step B2: Based on the current state of the state space, use the user Actor network to select a user action, use the opponent Actor network to generate a perturbation action, and construct a five-tuple and store it in the experience replay pool; Step B3: Sample data from the experience replay pool to obtain an updated Critic network; Step B4: Based on the updated Critic network, update the user Actor network and the opponent Actor network through an adversarial dual-rail optimization mechanism; the opponent Actor network maximizes the cost evaluated by the Critic along the ascending gradient direction, and then introduces a subgradient correction to accumulate and correct the update direction of the opponent Actor network; Step B5: Iteratively loop through Step B3 - Step B4 to obtain a trained user Actor network and generate an energy storage scheduling plan.

Images