A load prediction method and system based on decomposition and cross-correlation feature fusion

Abstract

The application discloses a load prediction method and system based on decomposition and cross-correlation feature fusion. The method firstly acquires power load historical data, and performs pretreatment. The pretreated power load historical data is input into a learnable trend mode decomposition module. A learned weighted adaptive trend mode is used to decompose each variable of the input data to obtain residual features. Secondly, the residual features are input into a feature fusion module based on cross-correlation feature fusion to obtain fused features. Then, the fused features are dimensionally eliminated and added with the weighted adaptive trend mode to predict a future trend mode and obtain a final power load prediction result. Finally, a hybrid loss function is used for control to perform reverse training, and the output predicted power load result is tested. The application can better fuse features, and make full use of weather and holiday data to predict future power load based on historical power load data.

Description

Technical Field

[0001] This invention relates to the field of power load forecasting technology based on time series analysis, specifically a load forecasting method and system based on the fusion of decomposition and cross-correlation features. Background Technology

[0002] In recent years, driven by deep learning, time series forecasting technology has developed rapidly, evolving from traditional statistical models and machine learning models to deep learning models represented by Transformer. It has successfully broken through the technical bottlenecks of long sequence dependency capture, multimodal data fusion and cross-domain generalization capabilities, and has performed particularly well in high-dimensional time series data processing and prediction accuracy improvement.

[0003] Current mainstream deep learning prediction methods use the trend-seasonal decomposition paradigm. However, this decomposition assumes that seasonal patterns remain constant over time. Therefore, when the influence of the seasonal components of some variables on other variables changes over time, using only trend-seasonal decomposition can easily mask these changes. Furthermore, historical data often shows frequent similar trend patterns, which seasonal trend decomposition struggles to highlight. In the original Transformer attention mechanism, the attention matrix is... , The dot product of vectors at the same time step in the matrix is ​​used for prediction, but this calculation method does not fully utilize the inherent similarity between vectors at the same time step. The primary value of the power load forecasting method proposed in this invention, based on the fusion of learnable trend patterns and cross-correlation features, lies in breaking the traditional trend seasonal decomposition paradigm. It utilizes adaptive learnable trend pattern decomposition, improves the original attention calculation method to enhance feature fusion capabilities, and uses historical power load data and related covariate data such as temperature, weather conditions, rainfall, and holidays to accurately predict electricity consumption over a certain period in the future. This guides power companies to generate electricity accurately and improves the power system's regulation and dispatch capabilities. Against this backdrop, a power load forecasting method based on the fusion of learnable trend patterns and cross-correlation features has emerged. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a load forecasting method and system based on decomposition and cross-correlation feature fusion. Electricity load data often exhibits strong periodicity and variable correlation. Many past deep learning methods for electricity load forecasting have used representative trend seasonal decomposition methods. These methods struggle to capture frequently occurring trend patterns in historical electricity load data and are difficult to simultaneously fuse temporal and feature correlations in time-series multivariate feature fusion methods. We employ learnable trend patterns to learn trend patterns throughout the entire historical electricity load data, and simultaneously utilize cross-correlation methods to improve the calculation of attention scores, thereby promoting feature fusion and enhancing the accuracy of electricity load forecasting.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] In one aspect, this invention provides a load forecasting method based on the fusion of decomposition and cross-correlation features, comprising the following five steps:

[0007] Step (1) Obtain historical power load data and perform preprocessing: merge historical power load data with historical weather data, and add holiday variables according to time. The integrated data is called historical data. Remove outliers and interpolate missing values.

[0008] Step (2) Input the preprocessed historical data into the learnable trend pattern decomposition module, and use the learned weighted adaptive trend pattern to decompose each variable of the input data to obtain residual features.

[0009] Step (3) inputs the obtained residual features into the feature fusion module based on cross-correlation feature fusion to obtain fused features.

[0010] Step (4) combines the fusion feature elimination patch dimension with the weighted adaptive trend pattern from step (2) to predict the future trend pattern and obtain the final power load prediction result.

[0011] Step (5) uses hybrid loss function control for inverse training and tests the output predicted power load results.

[0012] In another aspect, the present invention also provides a load forecasting system based on the fusion of decomposition and cross-correlation features, for implementing the aforementioned load forecasting method, comprising the following modules:

[0013] The power load data acquisition module is used to acquire historical power load data and perform preprocessing.

[0014] The residual feature module is used to input preprocessed historical data into the learnable trend pattern decomposition module. It uses the learned weighted adaptive trend pattern to decompose each variable of the input data to obtain residual features.

[0015] The feature fusion module is used to input residual features into the feature fusion module based on cross-correlation feature fusion to obtain fused features.

[0016] The power load forecasting output module is used to eliminate the block dimension of the fused features and add a weighted adaptive trend pattern to predict the future trend pattern and obtain the final power load forecasting result.

[0017] The training and testing module uses a hybrid loss function control for inverse training and tests the output of predicted power load results.

[0018] This invention has the following characteristics and beneficial effects:

[0019] The above technical solution offers the following advantages in using learnable trend patterns to decompose input historical data: First, decomposition has been proven by numerous studies to be an effective time series forecasting technique, with most models utilizing trend-seasonal decomposition methods. Trend-seasonal decomposition smooths the original time series using moving averages to obtain the trend component, then subtracts the trend from the original data to obtain the seasonal component. However, the trend obtained from moving averages may not truly separate the seasonal component from the original series, thus masking the changes in the seasonal component over time and making it susceptible to noise interference. Learning seasonal patterns from a large amount of noise is inherently difficult and meaningless. Therefore, using learnable trend patterns for decomposition helps retain some trend components that are helpful for model understanding in the decomposed residual features, aiding in the modeling of the residuals.

[0020] Secondly, feature fusion helps to incorporate features of variables such as weather and holidays into the power load features, enabling the model to better predict future power load. The cross-correlation feature fusion mechanism improves the attention calculation method of the original Transformer by utilizing cross-correlation calculation, which can better fuse features and allow historical power load data to fully utilize weather and holiday data to predict future power load. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is the overall network architecture of the present invention;

[0023] Figure 2 This is the learnable trend pattern decomposition module of the present invention;

[0024] Figure 3 This is the cross-correlation feature fusion module of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings.

[0026] The purpose of this method is to improve upon existing methods using deep learning models in the field of power load forecasting. For example... Figure 1 As shown, this paper proposes a load forecasting method based on decomposition and cross-correlation feature fusion. Electricity load forecasting is an important direction in the field of time series forecasting. Existing mainstream time series forecasting models based on the decomposition paradigm mainly use trend seasonal decomposition. This method will use a learnable trend pattern to decompose electricity load data and improve the attention calculation method of the original Transformer through a cross-correlation feature fusion mechanism, enabling better fusion between various dimensions of the hidden layers in the features and improving the accuracy of electricity load forecasting. Since the patch method can effectively improve semantic information and reduce computational complexity, this method will use the PatchTST model as the basic model. The PatchTST model is also the first model to introduce the patch method into time series forecasting.

[0027] First, define the input data. , ,in This represents the length of the historical data review window for the input model, including historical electricity load, weather, and holidays. This represents the number of variables in the historical data. , Representative input At time step A vector consisting of different variables at a given location. The Each component is denoted as ( This is the component index, and its value range is... ), Selected from feature set , .

[0028] Step (1): Due to issues such as equipment sampling omissions or transmission loss, power load data may contain missing values. Therefore, linear interpolation is first used to interpolate the missing values ​​in the power load data. At the same time, equipment errors or noise interference may cause extreme outliers in the power load data. Therefore, data exceeding three times the mean are replaced with data exceeding three times the mean.

[0029] To eliminate unit differences, smooth gradients between different batches and layers of data, and accelerate model convergence, z-score normalization was used to process continuous data such as electricity load, temperature, and rainfall. This method scales the data to a standard normal distribution, eliminating dimensional differences and ensuring fair calculation and comparison of the weights of different features. Min-Max normalization was used to process discrete data such as weather codes and time.

[0030] Step (2): As Figure 2 As shown, the preprocessed historical data is divided into two categories: one category includes continuous variables such as power load, temperature, and rainfall. To indicate, This represents the number of continuous variables; another type of non-continuous variable, such as weather, holidays, and time, is represented by... To indicate, This indicates the number of non-continuous variables. Continuous variables first pass through a gate, which uses a linear layer to... Review window dimension size Mapped to , This represents the number of learnable trend patterns. The intermediate results obtained after gating are transformed from numerical values ​​to dimensionality using a normalized exponential (softmax) function. The probability representation, and the use of the expanded dimension of the probability representation. express, . The learnable trend pattern is represented by pre-set learnable parameters that are initialized to all zeros. . It uses continuous variable input The number of input channels is The number of output channels is The kernel size is Fill one on each side The effect obtained after a one-dimensional convolutional layer (Conv1D) is... The weight, Next, we will... , , Perform element-wise multiplication, in , , Subscript This represents the index of the dimension representing the number of learnable trend patterns. Then, the dimension representing the number of learnable trend patterns (this dimension has a size of...) is... Summing yields the weighted adaptive trend pattern. , Finally, minus Decomposition results , The above process can be formalized as follows:

[0031]

[0032]

[0033]

[0034]

[0035] Step (3): As Figure 3 As shown, the final output of step (2) With non-continuous variables Perform tensor concatenation (concat) and then instance normalization. get Then on Perform block embedding The operation yields a patched result. Following the steps of the original PatchTST model in dividing patches, that is, in The time dimension is padded with repeated values ​​at the end, and the length of these repeated values ​​is [length missing]. The value is the last value in the time dimension, and the resulting number of patches is... , Indicates the patch length. This represents the non-overlapping length between two consecutive patches. The temporal embedding and value embedding in `Patch_embedding` follow the operations of the original `PatchTST` model. Then, the results of the patch division are... , where h represents the size of the hidden layer, and the input is a Transformer module improved by a cross-correlation feature fusion mechanism for feature fusion.

[0036] The cross-correlation feature fusion mechanism improves the way the attention matrix is ​​calculated in the Transformer module. During the attention calculation process... First mapped Cheng matrix, The number of heads representing multi-head attention. Then first... Adjust the dimensions of the matrices to After adjusting the dimensions The matrix has a size of 1 in the last position. The dimensions are normalized; the next step is to use RFFT (Fast Fourier Transform) to obtain the frequency domain representation. The next step will be and complex conjugate Element-wise multiplication; for Using the Inverse Real Fast Fourier Transform (IRFFT), the result dimension is... Take the absolute value of the inverse real fast Fourier transform result. ;right Sum the last dimension (the size of this dimension is large). ,exist middle (the traversal index of the last dimension), then divide by Cross-correlation results were obtained ; The normalized cross-correlation attention score is obtained after softmax operation. Finally, normalized cross-correlation attention scores were used. With matrix Using Einstein's summation convention The output of the Transformer module with improved cross-correlation feature fusion mechanism is obtained. The above process can be formally represented as:

[0037]

[0038]

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] The improved cross-correlation feature fusion mechanism enhances the calculation of the attention matrix in the Transformer module. The original Transformer's attention score is derived from the inner product of feature vectors; different feature dimensions at the same time point only interact with each other through product with their own dimension, while interactions between different feature dimensions are achieved through addition. The improved attention calculation method, however, implements average cross-correlation between different feature dimensions at the same time point through Fourier transform and inverse Fourier transform. Since cross-correlation is essentially the degree of correlation of offsets, this significantly strengthens the interaction between different feature dimensions, leading to better fusion of these dimensions.

[0045] Step (4) will fuse features After mapping to eliminate the patch dimension and adding a weighted adaptive trend pattern through a linear layer. Finally, inverse instance normalization is performed. To obtain the final prediction result This can be formalized as:

[0046]

[0047] in It is a mapping layer in the PatchTST model used to remove patched features. It is a function that takes the input length as an input. Mapping to output length The trend pattern prediction layer aims to utilize weighted adaptive trend patterns. Predicting future trend patterns. This is the final prediction result. This represents the length of the prediction window.

[0048] Step (5) Train the network using the designed hybrid loss function. To make the learnable trend patterns in the learnable trend pattern decomposition module smoother and more representative of the trend patterns learned from the global training samples, a hybrid loss function is used to train the model. This hybrid loss function is as follows:

[0049]

[0050] The first term of the loss function These are common predicted values. and its corresponding true value The mean square loss function between them This represents the number of historical data variables. (Second item) It calculates the variance of the weighted learnable trend. These are variance-constrained weights. It is the number of continuous variables. yes The mean obtained along the review window dimension is used to reduce the volatility of the learnable trend pattern by optimizing the variance. (Third item) This involves adding a first-order difference penalty term to the weighted learnable trend, with the aim of reducing the rate of change of the weighted learnable trend and making it smoother. To smooth the constraint weights, It is the length of the review window. This represents a difference sequence consisting of the differences between adjacent elements in the time dimension. The subscript index range in the time dimension is from arrive , Subscript represents The index is based on the time dimension. Starting from the position to The total index length is All variable dimensions are retained; , represent The index is based on the time dimension. Starting from the position to The total index length is , .

[0051] In another aspect, the present invention also provides a load forecasting system based on the fusion of decomposition and cross-correlation features, for implementing the aforementioned load forecasting method, comprising the following modules:

[0052] The power load data acquisition module is used to acquire historical power load data and perform preprocessing.

[0053] The residual feature module is used to input preprocessed historical data into the learnable trend pattern decomposition module. It uses the learned weighted adaptive trend pattern to decompose each variable of the input data to obtain residual features.

[0054] The feature fusion module is used to input residual features into the feature fusion module based on cross-correlation feature fusion to obtain fused features.

[0055] The power load forecasting output module is used to eliminate the block dimension of the fused features and add a weighted adaptive trend pattern to predict the future trend pattern and obtain the final power load forecasting result.

[0056] The training and testing module uses a hybrid loss function control for inverse training and tests the output of predicted power load results.

[0057] experiment:

[0058] As shown in Table 1, the test dataset used four datasets widely used for long-term forecasting: ETTh1, ETTh2, ETTm1, and ETTm2. The ETT dataset is a classic benchmark dataset for time series forecasting in power systems, containing seven variables: high available load, high unavailable load, medium available load, medium unavailable load, low available load, low unavailable load, and transformer oil temperature. The ratio of the training set, validation set, and test set is 3:1:1.

[0059] Table 1

[0060]

[0061] Mean squared loss (MSE) and absolute value loss (MAE) were used as evaluation metrics for the prediction results on the test set. The models compared were PatchTST (a block time series model) and Autoformer (a long-term series prediction model based on a deep decomposition architecture and autocorrelation mechanism). The experimental parameters for the power load prediction method based on the fusion of learnable trend patterns and cross-correlation features were set as follows: batch size of 32, maximum number of training rounds of 10, early shutdown strategy enabled, learning rate of 0.0001, patch size of 16, number of learnable trend patterns K of 7, improved attention layer in the cross-correlation feature fusion module set to 2, and the learning rate progressively adjusted using the Adam optimizer. The model was trained using the aforementioned hybrid loss function. Set to 0.001, The value was set to 0.01. Each experimental result was the average of the results from 5 different random seed experiments, as shown in Table 2.

[0062] Table 2

[0063]

[0064] The best results are highlighted in bold. Experimental results show that our model improves prediction accuracy compared to the PatchTST model, even though our model is an improvement on PatchTST. The added modules clearly enhance prediction accuracy. Our model also comprehensively outperforms Autoformer, a model based on average moving decomposition. This demonstrates the effectiveness of our proposed learnable trend-based decomposition module and cross-correlation feature fusion module.

[0065] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A load forecasting method based on the fusion of decomposition and cross-correlation features, characterized in that, Includes the following steps: Step 1: Obtain historical power load data and perform preprocessing; Step 2: Input the preprocessed historical data into the learnable trend pattern decomposition module. Use the learned weighted adaptive trend pattern to decompose each variable of the input data to obtain residual features. The specific implementation process of the learnable trend pattern decomposition module is as follows: The preprocessed historical data is divided into two categories: one is continuous variables... express, The number of continuous variables; another type of non-continuous variable is... express, Indicates the number of non-continuous variables; Continuous variables first pass through a gating system, and then a linear layer is used to... Review window dimension size Mapped to , The number of learnable trend patterns is represented; the intermediate results obtained after gating are transformed from numerical values ​​to dimensional values ​​through a normalized exponential function. The probability representation, and the use of the expanded dimension of the probability representation. express, ; Learnable parameters initialized to all zeros represent learnable trend patterns. ; It uses continuous variable input The effect obtained after a one-dimensional convolutional layer is The weight, ; Will , , Perform element-wise multiplication, in , , Subscript The index represents the dimension of the number of learnable trend patterns; then, the weighted adaptive trend pattern is obtained by summing the dimensions of the number of learnable trend patterns. , Finally, minus Decomposition results , ; Step 3: Input the residual features into the feature fusion module based on cross-correlation feature fusion to obtain the fused features; Step 4: Eliminate the block dimension of the fused features and add a weighted adaptive trend pattern to predict the future trend pattern to obtain the final power load forecast result; Step 5: Perform inverse training using hybrid loss function control and test the output predicted power load results.

2. The load forecasting method based on decomposition and cross-correlation feature fusion according to claim 1, characterized in that, The historical data includes merging historical power load data with historical weather data, and adding holiday variables based on time. The preprocessing includes removing outliers and interpolating missing values.

3. The load forecasting method based on decomposition and cross-correlation feature fusion according to claim 2, characterized in that, The specific implementation process of the feature fusion module is as follows: Will With non-continuous variables Perform tensor concatenation, then perform instance normalization to obtain... Then on Perform block embedding operations to obtain the block patch results. ;Following the steps of the original PatchTST model in dividing patches, that is, in The time dimension is padded with repeated values ​​at the end, and the length of these repeated values ​​is [length missing]. The value is the last value in the time dimension, and the resulting number of patches is... , Indicates the patch length. This represents the non-overlapping length between two consecutive patches; then the results of patching... The input is processed by the Transformer module, which has been improved by the cross-correlation feature fusion mechanism, to perform feature fusion.

4. The load forecasting method based on the fusion of decomposition and cross-correlation features according to claim 3, characterized in that, The specific implementation process of the cross-correlation feature fusion mechanism is as follows: During attention calculation First mapped to matrix, The number of heads representing multi-head attention. ;Will Adjust the dimensions of the matrices to After adjusting the dimensions The matrix has a size of 1 in the last position. The dimensions are normalized; then the frequency domain representation is obtained using RFFT (Fast Fourier Transform for Real Numbers). ;Will and complex conjugate Element-wise multiplication; for Using the Inverse Real Fast Fourier Transform (IRFFT), the result dimension is... ; Taking the absolute value of the inverse real fast Fourier transform result ;right Sum the last dimension and then divide by Cross-correlation results were obtained ; The normalized cross-correlation attention score is obtained after softmax operation. Finally, normalized cross-correlation attention scores were used. With matrix The output of the Transformer module, improved by using Einstein's summation convention, is obtained through a cross-correlation feature fusion mechanism. .

5. The load forecasting method based on the fusion of decomposition and cross-correlation features according to claim 4, characterized in that, The process for step 4 is as follows: Fusion features After mapping to eliminate the patch dimension and adding a weighted adaptive trend pattern through a linear layer. Finally, inverse instance normalization is performed. Obtain the final power load forecast results .

6. The load forecasting method based on the fusion of decomposition and cross-correlation features according to claim 5, characterized in that, The hybrid loss function includes three additive terms: First item It is a predicted value. and its corresponding true value The mean square loss function between them The second item represents the number of historical data variables. It calculates the variance of the weighted learnable trend. These are variance-constrained weights. It is the number of continuous variables. yes The mean obtained along the review window dimension is used to reduce the volatility of the learnable trend pattern by optimizing the variance; the third term It adds a first-order difference penalty term to the weighted learnable tendency. To smooth the constraint weights, It is the length of the review window. This represents a difference sequence consisting of the differences between adjacent elements in the time dimension. The subscript index range in the time dimension is from arrive , Subscript represents The index is based on the time dimension. Starting from the position to The total index length is All variable dimensions are retained; , represent The index is based on the time dimension. Starting from the position to The total index length is , .

7. A load forecasting system based on the fusion of decomposition and cross-correlation features, used to implement the load forecasting method according to any one of claims 1 to 6, characterized in that, Includes the following modules: The power load data acquisition module is used to acquire historical power load data and perform preprocessing. The residual feature module is used to input the preprocessed historical data into the learnable trend pattern decomposition module, and use the learned weighted adaptive trend pattern to decompose each variable of the input data to obtain residual features. The feature fusion module is used to input the residual features into the feature fusion module based on cross-correlation feature fusion to obtain the fused features; The power load forecasting output module is used to eliminate the block dimension of the fused features and add a weighted adaptive trend pattern to predict the future trend pattern and obtain the final power load forecasting result. The training and testing module uses a hybrid loss function control for inverse training and tests the output of predicted power load results.

Images