An energy storage management system for building energy conservation

Abstract

The present invention relates to the field of reinforcement learning technology and provides an energy storage management system for building energy conservation. The system comprises a data acquisition module, a data processing module, a load forecasting module, an energy storage scheduling module, and an intelligent scheduling control module. In the load forecasting module, the system constructs a symmetric sliding window fluctuation-aware CNN model, which improves the timeliness and sensitivity of heat load forecasting. In the energy storage scheduling module, the system combines opponent disturbance modeling, a two-way adversarial update mechanism, and a sub-gradient cumulative correction technology to improve the Actor-Critic model, forming an adversarial dual-track Actor-Critic model, thereby enhancing the robustness and adaptability of the phase change energy storage scheduling strategy in complex dynamic environments. The system can achieve high-precision heat load forecasting and high-reliability energy storage charging and discharging scheduling under multi-source disturbance conditions, significantly improving the energy efficiency level of the building HVAC system and reducing operating energy consumption and costs.

Description

Technical Field

[0001] The present invention relates to the technical field of reinforcement learning, and in particular to an energy storage management system for building energy conservation. Background Art

[0002] With global energy shortages and increasing demands for energy conservation and emission reduction, energy consumption in the building HVAC sector is receiving increasing attention. More and more energy storage management systems are being introduced to improve energy utilization efficiency and reduce operating costs through environmental perception, load forecasting, and intelligent scheduling. However, existing energy storage management systems still have many shortcomings: First, traditional systems are still relatively extensive in load forecasting and thermal management strategy design. They lack a modeling mechanism for fine-grained time series fluctuations and complex interactions between features in building energy consumption data, and fail to fully explore the fine-grained laws of thermal load changes under different operating conditions, resulting in lags or deviations in energy release and storage regulation, affecting the overall energy-saving effect. Second, existing systems mainly conduct strategy training based on static or weakly dynamic environments, which can easily cause slow response of energy storage units and reduced heat storage and release efficiency, restricting the stability and economy of the system. Summary of the Invention

[0003] The present invention provides an energy storage management system for building energy conservation. Aiming at the problems of large thermal load fluctuation, frequent environmental disturbance and insufficient adaptability of intelligent prediction and scheduling systems in traditional energy storage management systems, a new system architecture based on symmetric sliding window fluctuation perception CNN load prediction and adversarial dual-track Actor-Critic energy storage scheduling optimization is proposed. The system can achieve higher-precision thermal load prediction and more robust phase change energy storage charging and discharging and thermal load intelligent scheduling, significantly improving the comprehensive energy efficiency and operational stability of building HVAC energy storage management systems. Specifically: by introducing the time dimension symmetric filling strategy, strip The feature pooling mechanism and weighted fluctuation-aware loss function improve the CNN model's ability to extract and predict fine-grained load fluctuation features, and can more accurately capture the changing patterns of heating and hot water demand in different operating areas and time periods. At the same time, through opponent disturbance modeling, two-way adversarial update and sub-gradient cumulative correction mechanism, the robustness and adaptability of the energy storage scheduling strategy in complex environments such as load mutations and meteorological anomalies are enhanced, and it can intelligently optimize the charging and discharging behavior of phase change energy storage equipment, flexible response to thermal loads and coordinated environmental control, thereby achieving overall energy efficiency improvement of the building HVAC system and effective reduction of operating costs.

[0004] The present invention provides an energy storage management system for building energy conservation, which includes a data acquisition module, a data processing module, a load forecasting module, an energy storage scheduling module and an intelligent scheduling control module;

[0005] The data acquisition module deploys environmental sensors and energy consumption monitoring equipment in the heating heat source room, domestic hot water equipment area, and terminal radiator area to collect supply and return water temperature, flow rate, terminal room temperature, hot water usage, and status parameters of the phase change energy storage unit. This data is then converted into raw energy storage time series data through edge computing, unified formatting, and preliminary denoising.

[0006] The data processing module cleans the original energy storage time series data, removes outliers, synchronizes time, and standardizes the format to generate the energy storage data feature matrix;

[0007] The load forecasting module establishes a CNN model and optimizes it through a time-dimensional symmetric filling strategy, a strip-based feature pooling mechanism, and a weighted fluctuation-aware loss function. A symmetric sliding-window fluctuation-aware CNN model is constructed and used to process the energy storage data feature matrix to generate thermal load forecast data. The symmetric sliding-window fluctuation-aware CNN model includes a multi-layer two-dimensional convolutional network and a fully connected layer.

[0008] The energy storage scheduling module establishes an actor-critic model. Through adversary perturbation modeling, a two-way adversarial update mechanism, and sub-gradient cumulative correction, the actor-critic model is improved to construct an adversarial dual-track actor-critic model. Combined with heat load forecast data, the adversarial dual-track actor-critic model generates an energy storage scheduling plan.

[0009] The intelligent scheduling control module generates and executes control instructions based on the energy storage scheduling plan, coordinates the charging and discharging process of the phase change energy storage unit, the start and stop and adjustment of the heat source equipment, and the flexible scheduling of the terminal load side, and generates an energy-saving evaluation report.

[0010] Furthermore, the process of generating heat load forecast data by the load forecast module specifically includes the following steps:

[0011] Step S1: Use a fixed-step sliding window to crop the energy storage data feature matrix to form a stable structure and generate a standard sample window tensor;

[0012] Step S2: performing symmetric padding on the standard sample window tensor to generate a symmetric padding sample tensor;

[0013] Step S3: A multi-layer two-dimensional convolutional network is used to extract the joint distribution characteristics of the time dimension and feature dimension of the symmetrically filled sample tensor, layer by layer, to form a high-dimensional convolutional feature map. Each layer uses a 2×2 convolution kernel, sliding along the time and feature directions to perceive the correlation pattern between local heat load changes and environmental variables.

[0014] Step S4: Strip pooling is performed on the high-dimensional convolutional feature map, and pooling compression is performed only in the feature dimension direction to generate a strip pooling feature map. This strategy avoids the destruction of the temporal structure by traditional maximum pooling and can compress the spatial redundancy between different energy consumption features while preserving the temporal dynamic trend.

[0015] Step S5: Process the strip pooling feature map through the fully connected layer to generate initial heat load prediction data, construct a mean square error loss function, optimize the mean square error loss function through error directionality penalty, time series fluctuation sensitive weight and amplification penalty mechanism, construct a weighted fluctuation perception loss function, train the symmetric sliding window fluctuation perception CNN model through the weighted fluctuation perception loss function, optimize the model parameters, and obtain the trained symmetric sliding window fluctuation perception CNN model. Use the trained symmetric sliding window fluctuation perception CNN model to optimize the initial heat load prediction data and generate heat load prediction data.

[0016] Furthermore, step S2 specifically includes: performing symmetrical filling on the standard sample window tensor in the time dimension, performing continuity modeling on the boundary structure information of the standard sample window tensor by copying the eigenvectors of the first and last time steps of the standard sample window tensor, and generating a symmetrically filled sample tensor.

[0017] Furthermore, the energy storage scheduling module generates an energy storage scheduling plan, which specifically includes the following steps:

[0018] Step B1: Initialize the user actor network, opponent actor network, critic network, and experience replay pool of the adversarial dual-track actor-critic model; define the state space and action space; input heat load prediction data and HVAC status data into the state space;

[0019] Step B2: Based on the current state in the state space, the user action in the action space selected by the user actor network is used, and Gaussian exploration noise is added to enhance exploration. The opponent actor network is used to generate a perturbation action based on the current state. The perturbation action and the opponent action interact with the environment to obtain a new state and reward. The five-tuple consisting of the current state, user action, perturbation action, reward, and new state is stored in the experience replay pool.

[0020] Step B3: Sample data from the experience replay pool, generate the next action through the user actor network, calculate the target value, optimize the critic network using least squares loss so that its output Q value is close to the target value, record the cumulative loss, and obtain the updated critic network; define the perturbation penalty coefficient and dynamically adjust the penalty for the perturbation action based on the cumulative loss;

[0021] Step B4: Based on the updated critic network, the user actor network and the opponent actor network are updated through an adversarial dual-track optimization mechanism. The user actor network minimizes the cost of the critic evaluation along the descending gradient direction; the opponent actor network maximizes the cost of the critic evaluation along the ascending gradient direction. Sub-gradient correction is then introduced to cumulatively correct the update direction of the opponent actor network to improve the perturbation coherence and attack strength of the opponent actor.

[0022] Step B5: Iterate steps B3 and B4, perform multiple rounds of adversarial training on the user actor network, obtain the trained user actor network, and generate an energy storage scheduling plan based on the trained user actor network.

[0023] By adopting the above scheme, the beneficial effects achieved by the present invention are as follows:

[0024] The present invention achieves high-precision heat load prediction in the complex heat load environment of the building HVAC system by introducing a symmetrical sliding window fluctuation perception CNN model; through the time dimension symmetrical filling strategy, the boundary characteristics of the heat load data sequence are effectively continued, solving the problems of boundary heat load information distortion and increased prediction error in the traditional convolution processing process; adopting a strip feature pooling mechanism, the temporal dynamic trend of heat load changes is accurately retained, and the model's perception of fine-grained fluctuations and sudden load changes is improved; at the same time, through the weighted fluctuation perception loss function, the learning weights of micro-fluctuations and short-term sudden heat load changes are emphasized, so that the prediction model can keenly capture the actual demand characteristics of heating and domestic hot water systems under different operating conditions, significantly improving the load prediction accuracy and response timeliness of the energy storage management system in the real building HVAC environment, and providing a highly reliable decision-making basis for subsequent phase change energy storage scheduling;

[0025] The present invention enhances the dynamic adaptability and scheduling robustness of the phase-change energy storage system in the face of multi-source disturbances (such as sudden load changes, weather fluctuations, changes in user heat usage behavior, etc.) during the operation of building HVAC by constructing an adversarial dual-track Actor-Critic energy storage scheduling optimization model; by introducing adversary disturbance modeling and a two-way adversarial update mechanism, the energy storage management system can continuously optimize the scheduling strategy under simulated most unfavorable operating conditions, effectively solving the problems of response lag and strategy degradation of existing intelligent scheduling systems in emergency scenarios; through the sub-gradient cumulative correction mechanism, the consistency and interference resistance of the anti-disturbance action are further improved, ensuring that the phase-change energy storage unit can still stably execute heat storage and release decisions and heat load response regulation under extreme working conditions, comprehensively improving the stability of the HVAC energy storage management system and its risk resistance in extreme environments;

[0026] In summary, the present invention achieves the comprehensive performance improvement of energy-saving management of building HVAC systems in complex dynamic environments through systematic innovation of heat load prediction and phase change energy storage scheduling optimization modules, significantly improves load prediction accuracy and energy storage scheduling reliability, and solves the problems of poor adaptability and insufficient stability of existing HVAC energy storage management systems; by accurately predicting the trend of heat load changes, flexibly optimizing heat storage and release management and terminal energy-consuming equipment control, effectively reducing the overall energy consumption and operating costs of building HVAC systems, improving indoor thermal comfort, and providing practical technical support for energy conservation, emission reduction and intelligent management in the building field. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a module diagram of an energy storage management system for building energy conservation proposed by the present invention;

[0028] Figure 2 This is a weight factor change diagram proposed in step S5 in embodiment three. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] Example 1, according to Figure 1 , the present invention provides an energy storage management system for building energy conservation, the system includes a data acquisition module, a data processing module, a load forecasting module, an energy storage scheduling module and an intelligent scheduling control module;

[0031] The data acquisition module deploys environmental sensors and energy consumption monitoring equipment in the heating heat source room, domestic hot water equipment area, and terminal radiator area to collect supply and return water temperature, flow rate, terminal room temperature, hot water usage, and status parameters of the phase change energy storage unit. This data is then converted into raw energy storage time series data through edge computing, unified formatting, and preliminary denoising.

[0032] The data processing module cleans the original energy storage time series data, removes outliers, synchronizes time, and standardizes the format to generate the energy storage data feature matrix;

[0033] The load forecasting module establishes a CNN model and optimizes it through a time-dimensional symmetric filling strategy, a strip-based feature pooling mechanism, and a weighted fluctuation-aware loss function. A symmetric sliding-window fluctuation-aware CNN model is constructed and used to process the energy storage data feature matrix to generate thermal load forecast data. The symmetric sliding-window fluctuation-aware CNN model includes a multi-layer two-dimensional convolutional network and a fully connected layer.

[0034] The energy storage scheduling module establishes an actor-critic model. Through adversary perturbation modeling, a two-way adversarial update mechanism, and sub-gradient cumulative correction, the actor-critic model is improved to construct an adversarial dual-track actor-critic model. Combined with heat load forecast data, the adversarial dual-track actor-critic model generates an energy storage scheduling plan.

[0035] The intelligent scheduling control module generates and executes control instructions based on the energy storage scheduling plan, coordinates the charging and discharging process of the phase change energy storage unit, the start and stop and adjustment of the heat source equipment, and the flexible scheduling of the terminal load side, and generates an energy-saving evaluation report.

[0036] Example 2, according to Figure 2 This embodiment is based on the first embodiment. In this embodiment, the process of generating heat load forecast data by the load forecast module specifically includes the following steps:

[0037] Step S1: Use a fixed-step sliding window to crop the energy storage data feature matrix to form a stable structure and generate a standard sample window tensor;

[0038] Step S2: performing symmetric padding on the standard sample window tensor to generate a symmetric padding sample tensor;

[0039] Step S3: A multi-layer two-dimensional convolutional network is used to extract the joint distribution characteristics of the time dimension and feature dimension of the symmetrically filled sample tensor, layer by layer, to form a high-dimensional convolutional feature map. Each layer uses a 2×2 convolution kernel, sliding along the time and feature directions to perceive the correlation pattern between local heat load changes and environmental variables.

[0040] Step S4: Strip pooling is performed on the high-dimensional convolutional feature map, and pooling compression is performed only in the feature dimension direction to generate a strip pooling feature map. This strategy avoids the destruction of the temporal structure by traditional maximum pooling and can compress the spatial redundancy between different energy consumption features while preserving the temporal dynamic trend.

[0041] Step S5: Process the strip pooling feature map through the fully connected layer to generate initial heat load prediction data, construct a mean square error loss function, optimize the mean square error loss function through error directionality penalty, time series fluctuation sensitive weight and amplification penalty mechanism, construct a weighted fluctuation perception loss function, train a symmetric sliding window fluctuation perception CNN model through the weighted fluctuation perception loss function, optimize the model parameters, and obtain a trained symmetric sliding window fluctuation perception CNN model. Use the trained symmetric sliding window fluctuation perception CNN model to optimize the initial heat load prediction data and generate heat load prediction data. The formula used is as follows:

[0042] ;

[0043] in, represents the weighted fluctuation-aware loss function, represents the length of the time series, represents the time step, represents the error direction weight factor, represents the volatility-sensitive weight factor, represents the amplification penalty coefficient, Indicates the true load value, represents the model prediction value, represents the squared error.

[0044] Embodiment 3: This embodiment is based on embodiment 1. In this embodiment, the process of generating heat load forecast data by the load forecast module specifically includes the following steps:

[0045] Step T1: Use a fixed-step sliding window to crop the energy storage data feature matrix to form a stable structure and generate a standard sample window tensor;

[0046] Step T2: perform symmetric padding on the standard sample window tensor to generate a symmetric padded sample tensor;

[0047] Step T3: A multi-layer 2D convolutional network is used to extract the joint distribution characteristics of the time and feature dimensions of the symmetrically filled sample tensor, layer by layer, to form a high-dimensional convolutional feature map. Each layer uses a 2×2 convolution kernel, sliding along the time and feature directions to perceive the correlation pattern between local heat load changes and environmental variables.

[0048] Step T4: Strip pooling is performed on the high-dimensional convolutional feature map, which performs pooling compression only in the feature dimension direction to generate a strip pooling feature map. This strategy avoids the destruction of the temporal structure by traditional maximum pooling and can compress the spatial redundancy between different energy consumption features while preserving the temporal dynamic trend.

[0049] Step T5: Process the strip pooling feature map through the fully connected layer to generate initial heat load prediction data, construct a mean square error loss function, train the symmetric sliding window fluctuation perception CNN model through the mean square error loss function, optimize the model parameters, and obtain the trained symmetric sliding window fluctuation perception CNN model. Use the trained symmetric sliding window fluctuation perception CNN model to optimize the initial heat load prediction data and generate heat load prediction data.

[0050] Embodiment 4. This embodiment is based on embodiment 2. In this embodiment, step S2 specifically includes: performing symmetrical padding on the standard sample window tensor in the time dimension, and performing continuity modeling on the boundary structure information of the standard sample window tensor by copying the feature vectors of the first and last time steps of the standard sample window tensor to generate a symmetrically filled sample tensor; this method is different from ordinary zero-padding, and can effectively retain feature boundary change information, preventing the model from misjudging the energy consumption pattern at both ends of the sample during convolution; and enabling the model to have context continuity when learning the energy consumption structure changes in the edge area.

[0051] Example 5: This example is based on Example 4. In this example, the energy storage scheduling module generates an energy storage scheduling plan, specifically including the following steps:

[0052] Step B1: Initialize the user actor network, opponent actor network, critic network, and experience replay pool of the adversarial dual-track actor-critic model; define the state space and action space; input heat load prediction data and HVAC status data into the state space;

[0053] Step B2: Based on the current state in the state space, the user action in the action space selected by the user actor network is used, and Gaussian exploration noise is added to enhance exploration. The opponent actor network is used to generate a perturbation action based on the current state. The perturbation action and the opponent action interact with the environment to obtain a new state and reward. The five-tuple consisting of the current state, user action, perturbation action, reward, and new state is stored in the experience replay pool.

[0054] Step B3: Sample data from the experience replay pool, generate the next action through the user actor network, calculate the target value, optimize the critic network using least squares loss so that its output Q value is close to the target value, record the cumulative loss, and obtain the updated critic network; define the perturbation penalty coefficient and dynamically adjust the penalty for the perturbation action based on the cumulative loss;

[0055] Step B4: Based on the updated critic network, the user actor network and the opponent actor network are updated through an adversarial dual-track optimization mechanism; the user actor network minimizes the cost of the critic evaluation along the descending gradient direction; the opponent actor network maximizes the cost of the critic evaluation along the ascending gradient direction. Sub-gradient correction is then introduced to cumulatively correct the update direction of the opponent actor network to improve the perturbation coherence and attack strength of the opponent actor. The formula used is as follows:

[0056] Gradient Descent:

[0057] ;

[0058] in, Represents the parameters of the user Actor network, that is, the parameters of the energy storage scheduling strategy, represents the learning rate of the user Actor network, Represents the sampled data in the experience replay pool, Represents the parameters of the opponent Actor network, Represents the Actor adversarial loss function; Relative to The loss function gradient;

[0059] Gradient Ascent:

[0060] ;

[0061] in, represents the learning rate of the opponent Actor network, Relative to The loss function gradient; represents the weight coefficient of the subgradient correction term, express The historical cumulative change is used to correct the current update direction;

[0062] Step B5: Iterate steps B3 and B4, perform multiple rounds of adversarial training on the user actor network, obtain the trained user actor network, and generate an energy storage scheduling plan based on the trained user actor network.

[0063] Example 6: This example is based on Example 4. In this example, the energy storage scheduling module generates an energy storage scheduling plan by specifically following the steps below:

[0064] Step R1: Initialize the user actor network, opponent actor network, critic network, and experience replay pool of the adversarial dual-track actor-critic model; define the state space and action space; the state space includes heat load prediction data and HVAC status data;

[0065] Step R2: Based on the current state in the state space, the user action in the action space selected by the user actor network is used, and Gaussian exploration noise is added to enhance exploration. The opponent actor network is used to generate a perturbation action based on the current state. The perturbation action and the opponent action interact with the environment to obtain a new state and reward. The five-tuple consisting of the current state, user action, perturbation action, reward, and new state is stored in the experience replay pool.

[0066] Step R3: Sample data from the experience replay pool, generate the next action through the user actor network, calculate the target value, optimize the critic network with least squares loss so that its output Q value is close to the target value, record the cumulative loss, and obtain the updated critic network; define the perturbation penalty coefficient and dynamically adjust the penalty for the perturbation action based on the cumulative loss;

[0067] Step R4: Based on the updated critic network, the user actor network and the opponent actor network are updated through an adversarial dual-track optimization mechanism; the user actor network minimizes the cost of the critic evaluation along the descending gradient direction; the opponent actor network maximizes the cost of the critic evaluation along the ascending gradient direction;

[0068] Step R5: Iterate steps R3 and R4, perform multiple rounds of adversarial training on the user actor network, obtain the trained user actor network, and generate an energy storage scheduling plan based on the trained user actor network.

[0069] The present invention and its embodiments are described above. Such description is not restrictive. What is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in this field are inspired by it and do not depart from the purpose of the invention, they can creatively design structural methods and embodiments similar to the technical solution, which should all fall within the scope of protection of the present invention.

Claims

1. An energy storage management system for building energy conservation, comprising a data acquisition module and a data processing module, wherein the data acquisition module acquires HVAC status data; the data processing module processes the HVAC status data to generate an energy storage data feature matrix; characterized in that: The system also includes a load forecasting module and an energy storage scheduling module; The load forecasting module constructs a symmetric sliding window fluctuation perception CNN model, processes the energy storage data feature matrix through the symmetric sliding window fluctuation perception CNN model, and generates thermal load forecast data; The energy storage scheduling module constructs an adversarial dual-track Actor-Critic model, combines it with heat load forecast data, and generates an energy storage scheduling plan through the adversarial dual-track Actor-Critic model; The method for constructing a symmetric sliding window fluctuation-aware CNN model is as follows: establishing a CNN model, and optimizing the CNN model through a time dimension symmetric filling strategy, a strip-like feature pooling mechanism, and a weighted fluctuation-aware loss function; The process of generating an energy storage scheduling plan by the energy storage scheduling module specifically includes the following steps: Step B1: Initialize the user actor network, opponent actor network, and experience replay pool of the adversarial dual-track actor-critic model; define the state space and input the heat load forecast data and HVAC status data into the state space; Step B2: Based on the current state of the state space, the user actor network is used to select the user action, the opponent actor network is used to generate the perturbation action, and a five-tuple is constructed and stored in the experience replay pool; Step B3: Sample data from the experience replay pool to obtain the updated critic network; Step B4: Based on the updated critic network, the user actor network and the opponent actor network are updated through an adversarial dual-track optimization mechanism; the opponent actor network maximizes the cost of the critic evaluation along the ascending gradient direction, and then introduces sub-gradient correction accumulation to correct the update direction of the opponent actor network; Step B5: Iterate steps B3 and B4 to obtain the trained user actor network and generate an energy storage scheduling plan.

2. The energy storage management system for building energy conservation according to claim 1, characterized in that: The symmetric sliding window fluctuation-aware CNN model consists of multiple layers of two-dimensional convolutional networks and fully connected layers.

3. The energy storage management system for building energy conservation according to claim 2, characterized in that: The process of generating heat load forecast data by the load forecast module specifically includes the following steps: Step S1: Use a fixed-step sliding window to crop the energy storage data feature matrix to generate a standard sample window tensor; Step S2: performing symmetric padding on the standard sample window tensor to generate a symmetric padding sample tensor; Step S3: extracting the joint distribution features of the symmetrically filled sample tensor through a multi-layer two-dimensional convolutional network to form a high-dimensional convolution feature map; Step S4: performing strip pooling on the high-dimensional convolutional feature map, performing pooling compression in the feature dimension direction, and generating a strip pooling feature map; Step S5: Process the strip pooling feature map through the fully connected layer, and construct a weighted fluctuation-aware loss function to train the symmetric sliding window fluctuation-aware CNN model to generate heat load prediction data.

4. The energy storage management system for building energy conservation according to claim 3, characterized in that: Step S2 specifically includes: performing symmetrical filling on the standard sample window tensor in the time dimension, performing continuity modeling on the boundary structure information of the standard sample window tensor by copying the eigenvectors of the first and last time steps of the standard sample window tensor, and generating a symmetrically filled sample tensor.

Images