Building energy consumption multi-scale prediction method based on CNN-double-path LSTM-Transform hybrid architecture

Abstract

A building energy consumption multi-scale prediction method based on a CNN-double-path LSTM-Transform hybrid architecture comprises the following steps: step S1, obtaining energy consumption data of a large-scale building group and daily data of various characteristic variables, forming a data set, and preprocessing the data set; s2, dividing the data in proportion to serve as a training set and a test set; s3, a CNN-parallel LSTM-Transform hybrid model is constructed, after features are extracted through the CNN, global long-term dependence and local short-term fluctuation features are processed by adopting double-path LSTM, and finally, the spatial-temporal feature interaction capability is enhanced by using Transform; s4, designing a cross attention mechanism, dynamically fusing double-path features, and combining residual connection; and S5, optimizing the model by using a mixed loss function, and finally outputting a non-negative prediction result through Softplus. The method is more effective and higher in prediction precision.

Description

Technical Field

[0001] This invention belongs to the field of building energy consumption prediction technology, specifically involving a multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture. Background Technology

[0002] Building energy consumption forecasting is a key technology for achieving energy conservation and intelligent building management. Against the backdrop of global energy scarcity and low-carbon development, this technology can provide data support for optimizing building energy systems, effectively reducing energy costs and carbon emissions. By accurately forecasting building energy demand, it can provide decision-making basis for intelligent control systems, thereby improving the overall energy efficiency of buildings. With the development of intelligent technologies, building energy consumption forecasting has become an important component of modern building energy management systems.

[0003] Traditional building energy consumption prediction methods primarily rely on physical modeling and statistical learning techniques. Physical model-based methods predict energy consumption by establishing building thermodynamic equations and energy balance models. While these methods have clear physical meaning, they require detailed building parameters as input, and the computational complexity increases exponentially with building size. Statistical learning methods, such as ARIMA and SRIMA, perform well in handling linear time series relationships, but real-world building energy consumption data is generally nonlinear. Support vector regression, gradient boosting decision trees, and random trees can handle some nonlinear problems, but they still have significant limitations when dealing with long-term time series data.

[0004] In recent years, the development of deep learning technology has brought new breakthroughs to building energy consumption prediction. Convolutional neural networks have shown advantages in local feature extraction, effectively capturing local patterns such as daily and weekly cycles in building energy consumption. RNNs, LSTMs, and GRUs have significant advantages in handling temporal dependencies. Transformer models have unique advantages in modeling long-distance dependencies; their attention mechanism can capture global relationships across time steps. However, single deep learning models still have many limitations in dealing with building energy consumption prediction. CNNs struggle to model long-term dependencies, LSTMs are computationally inefficient and suffer from the vanishing gradient problem, and Transformer models require large amounts of data for training and perform poorly in small-sample scenarios. Especially in handling multi-source data and multi-scale feature fusion challenges, existing methods still face difficulties.

[0005] Therefore, a more effective and accurate method for predicting building energy consumption is needed to solve the above problems. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention proposes a more effective and accurate multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture, the method comprising the following steps:

[0009] Step S1: Obtain daily data on energy consumption and related environmental characteristic variables of large-scale building complexes, form a dataset, and perform data standardization and missing value handling preprocessing operations on the data.

[0010] Step S2: Divide the dataset into training set and test set according to the time series characteristics and a set ratio;

[0011] Step S3: Construct a CNN-parallel LSTM-Transformer model. First, use a convolutional neural network to extract local features of energy consumption data. Then, use a dual-path LSTM parallel architecture to process global long-term trend features and local short-term fluctuation features respectively. On this basis, introduce a Transformer module to enhance the interactive modeling capability of spatiotemporal features.

[0012] Step S4: An enhanced cross-attention feature fusion mechanism was designed. This mechanism uses a query-key-value attention architecture, with global features as query vectors and local features as key-value pairs. Feature weights are dynamically generated through multi-head attention calculation.

[0013] Step S5: Optimize the model using a hybrid loss function, adopt an adaptive learning rate adjustment strategy, and finally output non-negative prediction results through the Softplus activation function.

[0014] Furthermore, the process of step S1 is as follows:

[0015] Raw data preprocessing includes padding missing data with the historical average for the same period and processing outlier data using quantile truncation; the processed data is then subjected to max-min normalization using the formula... Perform data preprocessing, where x i It is the actual value of historical data, x max It is a historical data value, x min It is the minimum value in historical data. It is data after normalization.

[0016] In step S3, the construction process of the CNN-parallel LSTM-Transformer hybrid prediction model is as follows:

[0017] Step S3.1: Convert the building energy consumption data into a three-dimensional tensor X∈R B×T×F Where B is the batch size, T is the time step, and F is the feature dimension;

[0018] Step S3.2: Use a residual convolutional network (Residual CNN) to extract multi-scale local features. The first convolutional layer (downsampling) uses a one-dimensional convolutional kernel to scan the time series and extract local features. Then, the SiLU activation function is used to introduce nonlinearity, and finally, max pooling is used to compress the time dimension to reduce it.

[0019] C1=MaxPool(SiLU(Conv1D(x,W1)))

[0020] The SiLU activation function formula is SiLU(x)=x·σ(x);

[0021] The second convolutional layer (adaptive pooling dimensionality reduction) introduces residual connections to alleviate the gradient vanishing problem, and finally, adaptive average pooling unifies the time step to a fixed value.

[0022] C2=AdaptiveAvgPool1D(ResBlock(C1))

[0023] Where C2∈R B×T′×C Where C is the number of channels, T ′ This refers to the compressed time step.

[0024] Step S3.3: Since a single LSTM model struggles to capture both global trends and local dynamics, this model employs a dual-path LSTM parallel structure. The complete sequence C2 output by the CNN is input into the global LSTM to capture long-term trends. The global LSTM uses a two-layer LSTM to enhance expressive power, with hidden layer dimension D:

[0025] H global =LSTM(C2)

[0026] Local LSTM expands the receptive field through dilated convolutions to capture short-term fluctuations. It uses a single-layer LSTM with a hidden layer dimension of D / 2.

[0027] H local =LSTM(DilatedConv(C2))

[0028] Step S3.4: To capture global dependencies across time steps, the model uses a Transformer architecture to perform deep encoding on the temporal features extracted by the parallel LSTM. First, temporal positional information is injected into the global features through positional encoding:

[0029] H′ global =H global +P :T′

[0030] Where P is generated using a sine-cosine function;

[0031] Then, the dynamic weights between time steps are calculated using a multi-head self-attention mechanism:

[0032] Z global =TransformerEncoder(H′) global )

[0033] Meanwhile, local features are also enhanced by independent Transformer encoding layers:

[0034] Z local =TransformerEncoder(H local ).

[0035] In step S4, a cross-attention mechanism is introduced, using global feature Z. global As a query, local feature Z local As key values, when calculating attention weights, the model automatically learns which local details are more important for global prediction. Simultaneously implementing residual connections and layer normalization preserves original global information and avoids feature degradation after fusion.

[0036] F = LayerNorm(Z) global +A).

[0037] The process of step S5 is as follows:

[0038] First, the fused temporal feature matrix F∈R B×T′×D Perform average pooling along the time dimension to obtain the global representation vector:

[0039] f = MeanPool(F) ∈ R B×D

[0040] Subsequently, a nonlinear mapping is performed using a two-layer multilayer perceptron: the first layer uses the SiLU activation function to compress the dimension to D / 2, enhancing feature representation; the second layer uses the Softplus activation function to ensure the predicted value is non-negative. The specific calculation process is as follows:

[0041]

[0042] Where W1∈R D×D / 2 and W1∈R D / 2×1 b1 and b2 are learnable weights, and b1 and b2 are bias terms.

[0043] The beneficial effects of this invention are that it is more effective and has higher prediction accuracy. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture.

[0045] Figure 2 This is a flowchart of a multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture.

[0046] Figure 3 This is a schematic diagram of the functional structure of the CNN-parallel LSTM-Transformer hybrid prediction model.

[0047] Figure 4 This is a comparison chart of the actual and predicted values ​​of the CNN-parallel LSTM-Transformer hybrid prediction model. Detailed Implementation

[0048] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. The embodiments described with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0049] Reference Figures 1-4 A multi-scale prediction method for building energy consumption using a CNN-dual-path LSTM-Transformer hybrid architecture includes the following steps:

[0050] Step S1: Obtain historical data for the large-scale building complex. This historical data includes time-series data, weather and temperature data, and location category data. The time-series data includes electricity consumption data at different times, describing how electricity demand changes over time. The historical data is preprocessed, including handling missing and outlier values, as follows:

[0051] Step 1.1: Missing data is processed by filling zero values ​​with the historical average energy consumption data for the same period;

[0052] Step 1.2: Abnormal data is processed using the quantile truncation method;

[0053] Step 1.3: Perform max-min normalization on the processed data: using the formula Perform data preprocessing, where xi It is the actual value of historical data, x max It is a historical data value, x min It is the minimum value in historical data. This is the data after normalization.

[0054] Step S2: Divide the dataset into training and test sets in an 8:2 ratio according to the characteristics of the time series.

[0055] Step S3: Construct a CNN-parallel LSTM-Transformer hybrid prediction model, such as... Figure 3 As shown, after extracting features through CNN, dual-path LSTM is used to process global long-term dependencies and local short-term fluctuation features respectively. Finally, Transformer is introduced to enhance the interactive modeling capability of spatiotemporal features.

[0056] The process of step S3 is as follows:

[0057] Step S3.1: Convert the building energy consumption data into a three-dimensional tensor X∈R B×T×F Where B is the batch size, T is the time step, and F is the feature dimension;

[0058] Step S3.2: To address the problem of prediction lag caused by insufficient memory capacity in the traditional LSTM model, the LSTM model is improved by using a convolutional neural network (CNN) to extract feature information before LSTM model processing, while removing useless information.

[0059] Therefore, a residual convolutional network (Residual CNN) is used to extract multi-scale local features. The first convolutional layer (downsampling) uses a one-dimensional convolutional kernel to scan the time series and extract local features. Then, the SiLU activation function is used to introduce nonlinearity, and finally, max pooling is used to compress the time dimension to reduce it.

[0060] C1=MaxPool(SiLU(Conv1D(X,W1)))

[0061] The SiLU activation function formula is SiLU(x)=x·σ(x);

[0062] The second convolutional layer (adaptive pooling dimensionality reduction) introduces residual connections to alleviate the gradient vanishing problem, and finally, adaptive average pooling unifies the time step to a fixed value.

[0063] C2=AdaptiveAvgPool1D(ResBlock(C1))

[0064] Where C2∈R B×T′×C , where C is the number of channels and T' is the compressed time step;

[0065] Step S3.3: LSTM (Long Short-Term Memory) addresses the vanishing or dilated gradient problem by adding a threshold. It adds three logic control units to the recurrent neural network: an input gate, an output gate, and a forget gate. The improved LSTM neural network formula at time t is as follows:

[0066] f t =(W f ·[h t-1 ,x t ]+b f )

[0067] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0068] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0069]

[0070] h t =o t *tanhc t

[0071] Where x t Let x be the input vector at time t. ' t This is the output vector after processing by a convolutional neural network. σ is the sigmoid function and tanh is the hyperbolic tangent function. Both σ and tanh are activation functions. The forget gate f... t Input gate i t and output gate o t The corresponding weighting matrix for each threshold is W. f W i and W o Each conversion deviation value b f b i and b o * indicates matrix multiplication, c t This refers to the network unit state output at time t, i.e., the memory unit. The weighting matrix of the memory unit is W. c Conversion deviation value b c c t-1 The forget gate determines the information discarded by memory units in the previous moment. It updates the information in the memory unit, h t It is the implicit information output by the memory unit, ht-1 It is the implicit information input into the memory unit;

[0072] Since a single LSTM model struggles to capture both global trends and local dynamics, this model employs a dual-path LSTM parallel structure. The complete sequence C2 output from the CNN is input into the global LSTM to capture long-term trends. The global LSTM uses a two-layer LSTM to enhance expressive power, with hidden layer dimension D:

[0073] H global =LSTM(C2)

[0074] Local LSTM expands the receptive field through dilated convolutions to capture short-term fluctuations. It uses a single-layer LSTM with a hidden layer dimension of D / 2.

[0075] H local =LSTM(DilatedConv(C2))

[0076] Step S3.4: To capture global dependencies across time steps, the model employs a Transformer architecture to perform deep encoding on the temporal features extracted by the parallel LSTM. First, temporal positional information is injected into the global features through positional encoding:

[0077] H′ global =H global +P :T′

[0078] Where P is generated using a sine-cosine function;

[0079] Then, the dynamic weights between time steps are calculated using a multi-head self-attention mechanism:

[0080] Z global =TransformerEncoder(H′) global )

[0081] Meanwhile, local features are also enhanced by independent Transformer encoding layers:

[0082] Z local =TransformerEncoder(H local );

[0083] Step S4: Design an enhanced cross-attention feature fusion mechanism to dynamically fuse dual-path features and combine residual connections to improve model stability;

[0084] The process in step S4 is as follows: a cross-attention mechanism is introduced, using global feature Z... global As a query, local feature Z localAs key values, when calculating attention weights, the model automatically learns which local details are more important for global prediction. Simultaneously, residual connections and layer normalization are implemented to preserve original global information and avoid feature degradation after fusion.

[0085] F = LayerNorm(Z) global +A);

[0086] Step S5: Optimize the model using a hybrid loss function, adopt an adaptive learning rate adjustment strategy, and finally output non-negative prediction results through the Softplus activation function;

[0087] The process of step S5 is as follows:

[0088] First, the fused temporal feature matrix F∈R B×T′×D Perform average pooling along the time dimension to obtain the global representation vector:

[0089] f = MeanPool(F) ∈ R B×D

[0090] Subsequently, a nonlinear mapping is performed using a two-layer multilayer perceptron: the first layer uses the SiLU activation function to compress the dimension to D / 2, enhancing feature representation; the second layer uses the Softplus activation function to ensure the predicted value is non-negative. The specific calculation process is as follows:

[0091]

[0092] Where W1∈R D×D / 2 and W1∈R D / 2×1 b1 and b2 are learnable weights, and b1 and b2 are bias terms.

[0093] The multi-scale prediction method for building energy consumption in this embodiment also includes the following steps:

[0094] Step S6, Model Training: Set the Epoch rounds to 150, batch_size to 128, training set to test set ratio to 8:2, training set to 133972 data points, test set to 33493 data points, and use sliding window prediction with window size set to 14.

[0095] Step S7: Output the prediction results: The prediction results are the energy consumption data for the next day. The goodness of fit and prediction performance of the combined model are measured and verified using the following two commonly used evaluation metrics: Root Mean Square Error (RMSE) and Mean Absolute Error (MAPE), as shown below:

[0096]

[0097] Where y i This represents the true value of the time series at time i. This represents the predicted value of the time series at the same point in time.

[0098] Step S8, the comparison chart of the true and predicted values ​​of the CNN-parallel LSTM-Transformer hybrid prediction model is shown below. Figure 4 As shown, there are two curves: the actual value and the predicted value from the CNN-parallel LSTM-Transformer hybrid prediction model.

[0099] This invention uses a novel CNN-parallel LSTM-Transformer hybrid prediction model to predict building energy consumption, which can improve the accuracy of the prediction model and provide a certain reference in the fields of time series analysis and energy consumption prediction.

[0100] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture, characterized in that, The method includes the following steps: Step S1: Obtain daily data on energy consumption and related environmental characteristic variables of large-scale building complexes, form a dataset, and perform data standardization and missing value handling preprocessing operations on the data. Step S2: Divide the dataset into training set and test set according to the time series characteristics and a set ratio; Step S3: Construct a CNN-parallel LSTM-Transformer model. First, use a convolutional neural network to extract local features of energy consumption data. Then, use a dual-path LSTM parallel architecture to process global long-term trend features and local short-term fluctuation features respectively. On this basis, introduce a Transformer module to enhance the interactive modeling capability of spatiotemporal features. Step S4: An enhanced cross-attention feature fusion mechanism was designed. This mechanism uses a query-key-value attention architecture, with global features as query vectors and local features as key-value pairs. Feature weights are dynamically generated through multi-head attention calculation. Step S5: Optimize the model using a hybrid loss function, adopt an adaptive learning rate adjustment strategy, and finally output non-negative prediction results through the Softplus activation function.

2. The multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture as described in claim 1, characterized in that, The process of step S1 is as follows: Raw data preprocessing includes padding missing data with the historical average for the same period and processing outlier data using quantile truncation; the processed data is then subjected to max-min normalization using the formula... Perform data preprocessing, where x i It is the actual value of historical data, x max It is a historical data value, x min It is the minimum value in historical data. It is data after normalization.

3. The multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture as described in claim 1 or 2, characterized in that, In step S3, the construction process of the CNN-parallel LSTM-Transformer hybrid prediction model is as follows: Step S3.1: Convert the building energy consumption data into a three-dimensional tensor X∈R B×T×F Where B is the batch size, T is the time step, and F is the feature dimension; Step S3.2: Use a residual convolutional network to extract multi-scale local features. In the first convolutional layer, a one-dimensional convolutional kernel is used to scan the time series and extract local features. Then, the SiLU activation function is used to introduce nonlinearity. Finally, max pooling is used to compress the time dimension and reduce it. C1 = MaxPool(SiLU(Conv1D(X,W1))) where the SiLU activation function is SiLU(x) = x·σ(x); The second convolutional layer introduces residual connections to alleviate the gradient vanishing problem, and finally, adaptive average pooling is used to unify the time step to a fixed value. C2=AdaptiveAvgPool1D(ResBlock(C1)) Where C2∈R B×T′×C , where C is the number of channels and T′ is the compressed time step; Step S3.3: Since a single LSTM model cannot simultaneously capture both global trends and local dynamics, this model employs a dual-path LSTM parallel structure. The complete sequence C2 output by the CNN is input into the global LSTM to capture long-term trends. The global LSTM uses a two-layer LSTM to enhance expressive power, with hidden layer dimension D: H global =LSTM(C2) Local LSTM expands the receptive field through dilated convolutions to capture short-term fluctuations. It uses a single-layer LSTM with a hidden layer dimension of D / 2. H local =LSTM(DilatedConv(C2)) Step S3.4: To capture global dependencies across time steps, the model uses a Transformer architecture to perform deep encoding on the temporal features extracted by the parallel LSTM. First, temporal positional information is injected into the global features through positional encoding: H′ global =H global +P :T′ Where P is generated using a sine-cosine function; Then, the dynamic weights between time steps are calculated using a multi-head self-attention mechanism: Z global =TransformerEncoder(H′ global ) Meanwhile, local features are also enhanced by independent Transformer encoding layers: Z local =TransformerEncoder(H local )。 4. The multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture as described in claim 1 or 2, characterized in that, In step S4, a cross-attention mechanism is introduced, using global feature Z. global As a query, local feature Z local As key values, when calculating attention weights, the model automatically learns which local details are more important for global prediction. Simultaneously, residual connections and layer normalization are implemented to preserve original global information and avoid feature degradation after fusion. F=LayerNorm(Z global +A).

5. The multi-scale prediction method for building energy consumption based on a CNN-dual-path LSTM-Transformer hybrid architecture as described in claim 1 or 2, characterized in that, The process of step S5 is as follows: First, the fused temporal feature matrix F∈R B×T′×D Perform average pooling along the time dimension to obtain the global representation vector: f=MeanPool(F)∈R B×D Subsequently, a nonlinear mapping is performed using a two-layer multilayer perceptron: the first layer uses the SiLU activation function to compress the dimension to D / 2, enhancing the feature representation capability; the second layer uses the Softplus activation function to ensure that the predicted value is non-negative. The calculation process is as follows: Where W1∈R D×D / 2 and W1∈R D / 2×1 b1 and b2 are learnable weights, and b1 and b2 are bias terms.

Images