Power load prediction method based on dual-channel cross attention network

Abstract

The invention discloses a power load prediction method based on a two-channel cross attention network, and belongs to the technical field of time sequence prediction in deep learning. The method comprises the following steps of obtaining a power load data set, performing preprocessing on the power load data set, decomposing power load data into a plurality of local components by applying VMD, iteratively optimizing VMD parameters through an APSO algorithm, and screening a subsequence with the lowest complexity to be combined with original features to serve as model input; and then constructing a dual-channel cross attention network model, processing time sequence data by using a parallel mechanism of a BiLSTM channel and a Transform channel, enhancing the feature expression ability through an improved cross attention layer, and performing power load prediction by using an MLP fused KAN network. According to the method, the time sequence prediction of the power load is carried out by constructing the two-channel cross attention network, the prediction precision and robustness are high, and the accuracy of load time sequence prediction is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction in deep learning, and in particular relates to a power load prediction method based on a dual-channel cross-attention network. Background Art

[0002] Time series data prediction has important application value in the field of power load, especially in the prediction of power transformer temperature. By modeling and analyzing historical temperature, load, voltage and other related data, time series data prediction can accurately predict the future temperature change trend of the transformer and identify potential overheating risks or abnormal conditions in advance. This not only helps with equipment health management and fault warning, but also optimizes the operation and maintenance strategy of the power system, extends equipment life, reduces downtime risks and operation and maintenance costs, thereby ensuring the safe, stable and efficient operation of the power system.

[0003] There are many methods for predicting load time series data. According to different prediction principles, they can be divided into two categories: traditional prediction methods and prediction methods based on deep learning. Traditional prediction methods often have low prediction accuracy and high data requirements, and are mainly suitable for stable and smooth data. Recently developed deep learning-based methods have achieved great success in many application fields of time series prediction. The model architecture used is mostly GRU / LSTM model, but this type of model can only extract information between time dimensions, but cannot obtain information of time series data from multiple dimensions in parallel. In addition, when facing complex, nonlinear and non-stationary time series data, existing research still has problems such as non-stationarity, feature extraction limitations and insufficient nonlinear mapping capabilities.

[0004] Therefore, a new technical solution is urgently needed in the prior art to solve this problem. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a power load forecasting method based on a dual-channel cross-attention network to solve the technical problems that the existing load time series data forecasting methods still have non-stationarity problems, feature extraction limitations and insufficient nonlinear mapping capabilities.

[0006] A method for power load forecasting based on a dual-channel cross attention network comprises the following steps, and the following steps are performed in sequence: Step S1: Obtain a power load data set, pre-process the power load data therein, then apply variational mode decomposition VMD to decompose the power load data time series into data components of different modes, iteratively optimize VMD parameters through adaptive particle swarm algorithm APSO, obtain the optimized decomposition sequence and combine it with the original power load feature as a new feature to form mixed feature data, and sequentially process the mixed feature data through sliding window technology and Z-Score standardization, and divide it into training set and test set, and use the data in the training set and test set as the input vector of the model, and the vector is a three-dimensional feature vector containing a set time step; Step S2: construct a deep learning network model based on a dual-channel cross-attention mechanism, using a bidirectional long short-term memory network BiLSTM model and a Transformer model as BiLSTM channels and Transformer channels for parallel calculation to achieve feature extraction, and then combining an improved cross-attention mechanism and a multi-layer perceptron MLP weighted fusion Kolmogorov-Arnold network KAN network model for feature enhancement; Step S3: Use the data in the training set as input to train the constructed dual-channel cross-attention network model, and continue to optimize the model until convergence meets the set requirements; Step S4: Use the trained network model to predict power load data.

[0007] The preprocessing in step S1 includes filling missing values ​​with mean values, replacing abnormal values ​​with mean values, and then normalizing.

[0008] In the optimization of VMD decomposition parameters in step S1, sample entropy is selected as a fitness function to quantify the quality of parameter combinations, and the global optimal parameters are found by iterating continuously with the goal of minimizing sample entropy. The load value is then subjected to VMD decomposition using the global optimal parameters, and a subsequence with the lowest complexity is screened. The sample entropy of the subsequence is then calculated and enhanced by adaptive threshold screening to obtain an optimized decomposition sequence. The sample entropy calculation formula is as follows: (1); In the formula, Represents the total number of data points; represents the similarity threshold; and They are respectively represented in the embedding dimension and When the distance between vectors is less than The probability of Indicates the length The sample entropy of a time subseries.

[0009] The formula for Z-Score standardization in step S1 is: (2); In the formula, For the current data points, is the average value of the entire data set, is the standard deviation of the entire data set, Z For the The normalized scores of the data points.

[0010] In step S2, the BiLSTM channel uses a bidirectional long short-term memory network BiLSTM and residual connection technology to convert the sequence data X t =(x1, x2,..., x T ) as input data X forward , passed to the positive long short-term memory network LSTM, and a positive output result is obtained , and the original sequence data X t Reverse and get X backward =(x T , x T-1 ,...,x1), and input it into the reverse LSTM to get a reverse output result Then, the forward and reverse output results are spliced ​​to achieve feature integration, so as to fully extract the time dependency between long-term and short-term, and the integrated results are combined with the original sequence data X t Add them to realize residual connection to further optimize the feature expression, and finally obtain the feature extraction result vector of the channel through full connection layer mapping ; The Transformer channel first transforms the sequence data X t =(x1, x2,..., x T ) is passed as input to a 1D convolutional neural network CNN to obtain local semantic information , then Combined with the positional encoding, it is input into the Transformer encoder. The self-attention mechanism of the Transformer encoder is used to calculate the attention score. Then, it is further processed through residual connection and hierarchical normalization, and then through the feedforward neural network to obtain a time-dependent feature sequence combining global and local temporal features. Finally, the result vector of the channel feature extraction is obtained through the fully connected layer mapping. ; The feature sequence obtained by the dual channels and The improved cross-attention mechanism is used to perform feature fusion, cross-calculate and obtain the attention weights between different positions, and the different attention weights are weighted summed up. The obtained sequence is the key feature to be extracted.

[0011] The method for performing feature enhancement on the improved cross attention mechanism and the network model of the multi-layer perceptron MLP weighted fusion Kolmogorov-Arnold network KAN in step S2 is: firstly, the Transformer channel feature is used as the query vector matrix , with BiLSTM channel features as key vector matrices Sum vector matrix , the attention weight based on the Transformer channel features is calculated through the attention mechanism, and the weighted temporal feature representation is generated ; The BiLSTM channel features are then used as the query vector matrix , with Transformer channel features as key vector matrices Sum vector matrix , the attention weight based on BiLSTM channel features is calculated again through the attention mechanism, and the weighted temporal feature representation is generated , and then the two attention weights are fused using the contextual dynamic weight method to obtain the fused cross-attention feature representation, which is finally input into the network model of the multi-layer perceptron MLP fused with the Kolmogorov-Arnold network KAN for prediction. The calculation formula is as follows: (3); (4); (5); (6); (7); (8); In the formula, is the feature vector extracted by the Transformer channel; is the feature vector extracted by the BiLSTM channel; , and They are respectively used in the Transformer channel to generate the query vector matrix , key vector matrix and the value vector matrix The learnable weight matrix of , and They are used to generate query vector matrices in the BiLSTM channel. , the key vector matrix and the value vector matrix The learnable weight matrix of is the dimension size of the key vector; It is a normalization operation; The sequence length is Context features; FC stands for fully connected mapping; is the activation function; represents the context weight; crossAttention is the output of the improved cross attention mechanism; matrix It represents a nonlinear mapping. Matrix of layers It is composed of a linear transformation matrix and a set of trainable univariate spline functions Composition, among which It is The first ( i , j ) elements corresponding to the spline function, different layers are operated by composite operations Connect to form a hierarchical mapping relationship; x represents the output feature of the cross attention mechanism; and Respectively represent the results predicted by KAN and MLP; in MLP calculation, represents the weight matrix of the Lth layer, and Represent the bias terms of the first layer and the Lth layer respectively; is the weight item for the fusion of prediction results, It is the final output time series prediction result.

[0012] The convergence judgment of the model training in step S3 is based on the loss function setting. The loss function uses huber_loss to calculate the parameter gradient of the model and reversely updates the parameters of the model. The training process is expressed as: (9); In the formula, Indicates The true value of the data; Indicates The predicted value of data; Represents the threshold used to switch the range of square loss and absolute loss; and The difference between the two is less than , then the loss function uses square loss; and The difference between the two is greater than , linear loss is used to avoid outliers from having too much impact on the loss function.

[0013] Through the above design scheme, the present invention can bring the following beneficial effects: The present invention adopts improved variational mode decomposition technology to decompose and reconstruct the power load time series, uses variational mode decomposition VMD optimized by adaptive particle swarm algorithm APSO to decompose the load data, and then calculates the sample entropy of different modes, adaptively sets the threshold to screen the decomposition sequence that meets the conditions, so as to generate an enhanced decomposition sequence RIMF ,This method effectively extracts the key features in the power load data, ,smoothes the non-stationary components, and improves the model’s ,ability to handle complex time series tasks.

[0014] The present invention designs an improved cross-attention mechanism for the dual-channel hybrid model to fuse the dual-channel features, thereby enhancing the feature fusion capability of the model and effectively improving the accuracy of load time series prediction.

[0015] The present invention realizes prediction through the weighted fusion design of the multi-layer perceptron MLP network and the Kolmogorov-Arnold network KAN, enhances the nonlinear feature mapping capability of the model, and effectively improves the accuracy of load time series prediction and the generalization ability of the model.

[0016] In summary, the present invention improves the accuracy of existing time series prediction and can be promoted in multiple time series prediction fields such as power load prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention is further described below with reference to the accompanying drawings and specific embodiments: Figure 1 It is a flow chart of a power load forecasting method based on a dual-channel cross attention network of the present invention; Figure 2 It is a decomposition diagram of data in a power load forecasting method based on a dual-channel cross attention network of the present invention after original variational mode decomposition VMD; Figure 3 It is a decomposition diagram of variational mode decomposition (VMD) of data in a power load forecasting method based on a dual-channel cross attention network of the present invention after being optimized by an adaptive particle swarm algorithm (APSO); Figure 4 A schematic diagram of the model structure of a dual-channel cross-attention network model VBTCKN in a power load forecasting method based on a dual-channel cross-attention network of the present invention; Figure 5 It is an application flow chart of a power load forecasting method based on a dual-channel cross attention network of the present invention; Figure 6 A network architecture diagram of a Kolmogorov-Arnold network KAN used in a power load forecasting method based on a dual-channel cross attention network of the present invention; Figure 7 A training loss diagram of a dual-channel cross-attention network model VBTCKN compared with other models in a power load forecasting method based on a dual-channel cross-attention network of the present invention; Figure 8 A comparison diagram of the predicted value and the true value of the dual-channel cross-attention network model VBTCKN in the power load forecasting method based on the dual-channel cross-attention network of the present invention; Fig. 9 This is a graph showing the RMSE variation results of the dual-channel cross-attention network model VBTCKN and other models in a power load forecasting method based on a dual-channel cross-attention network in the present invention under different prediction step sizes in the ETTh1 data set; Fig.10 This is a graph showing the RMSE variation results of the dual-channel cross-attention network model VBTCKN and other models in a power load forecasting method based on a dual-channel cross-attention network in the present invention under different prediction step sizes in the ETTh2 data set; Fig.11 It is a graph showing the RMSE variation results of the dual-channel cross-attention network model VBTCKN and other models in a power load forecasting method based on a dual-channel cross-attention network in the present invention under different prediction step sizes in the ETTm1 data set; Fig.12 This is a graph showing the RMSE variation results of the dual-channel cross-attention network model VBTCKN and other models in a power load forecasting method based on a dual-channel cross-attention network in the present invention under different prediction step sizes in the ETTh2 data set; Fig.13 This is an ablation comparison result diagram of a dual-channel cross-attention network model VBTCKN in a power load forecasting method based on a dual-channel cross-attention network of the present invention; Fig.14 A loss comparison diagram of a dual-channel cross-attention network model VBTCKN and other models in a power load forecasting method based on a dual-channel cross-attention network of the present invention under fixed length; Fig.15 It is a graph showing the RMSE variation results of the dual-channel cross-attention network model VBTCKN and other models in the power load forecasting method based on the dual-channel cross-attention network in the GEFCom2014-E data set at different prediction step sizes; Fig.16This is a graph showing the RMSE changes of the dual-channel cross-attention network model VBTCKN and other models in a power load forecasting method based on a dual-channel cross-attention network in the present invention at different prediction steps in the Weather dataset. DETAILED DESCRIPTION

[0018] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a power load forecasting method based on a dual-channel cross-attention network of the present invention in conjunction with the accompanying drawings. Example

[0019] Figure 1 A flow chart of a method for power load forecasting based on a dual-channel cross attention network according to an embodiment of the present invention is provided. The specific steps of the method are as follows: Step S1: Obtain a power load data set, preprocess and enhance the power load data in the power load data set, and then divide and generate a training set and a test set. The specific steps are as follows: Step S1-1, preprocessing the acquired power load data, including filling the missing values ​​with the mean, replacing the abnormal values ​​with the mean, and constructing a data set suitable for power load forecasting.

[0020] Step S1-2, perform variational mode decomposition VMD on the preprocessed data set, decompose the power load time series into data components of different modes, use adaptive particle swarm algorithm APSO to optimize VMD decomposition parameters, select sample entropy as fitness function to quantify the quality of parameter combination, and iterate continuously with the goal of minimizing sample entropy to find the global optimal parameters, and then use the global optimal parameters to perform VMD decomposition on the load value, screen the subsequence with the lowest complexity, and then calculate the sample entropy of the subsequence and enhance it by adaptive threshold screening to obtain the optimized decomposition sequence.

[0021] The sample entropy calculation formula is as follows: (1); In the formula, Represents the total number of data points; represents the similarity threshold; and They are respectively represented in the embedding dimension and When the distance between vectors is less than The probability of Indicates the length The sample entropy of a time subseries.

[0022] The search strategy of the adaptive particle swarm algorithm APSO is to imitate the group behavior of bird and fish flocks, guide particles to search for the global optimal solution, dynamically adjust the inertia weight and learning factor to balance global exploration and local development, and adaptively adjust the parameters according to the evolutionary state of the population to ensure that the population evolves in a better direction in each iteration. The update formula of the adaptive particle swarm algorithm APSO in the iteration process is as follows: (10); (11); In the formula, Represents particles In the The speed at the iteration, Indicates the inertia weight, which is used to control the influence of the particle's current velocity on the new velocity. and is the learning factor, and is a random number between 0 and 1. For particles The best location currently being searched for, is the optimal position of the entire particle group, For particles At the current spatial position, For particles In the The current position at iteration .

[0023] For the VMD decomposition sequence optimized by the adaptive particle swarm algorithm APSO, the sample entropy is calculated and enhanced by adaptive threshold screening to obtain the enhanced decomposition sequence. RIMF , the calculation formula is as follows: (12); (13); (14); (15); In the formula, the similarity threshold is the adaptive parameter, / are the maximum and minimum threshold factors, which are set to 0.1 and 0.4 respectively in the present invention. is the base of natural logarithms, is the sample entropy of the current decomposition sequence IMF, For IMF The standard deviation of is the total number of data, For the current data points, is the average value of the entire data set, is the adaptive threshold for screening signal x. The original variational mode decomposition VMD decomposition sequence and the variational mode decomposition VMD decomposition sequence after adaptive particle swarm algorithm APSO optimization are as follows: Figure 2 and Figure 3 shown.

[0024] Step S1-3, obtain the decomposition sequence after optimization by the adaptive particle swarm algorithm APSO and combine it with the original power load characteristics as a new feature to form mixed feature data. The mixed feature data set is processed using the sliding window technology, and the window size is set to 60. Each batch contains a three-dimensional feature vector B*T*C of 10 time steps, where B represents the number of batch samples, T represents the time step, and C represents the feature dimension, and the input vector of the model is constructed.

[0025] Step S1-4, the input vectors at different times are normalized by the Z-Score function, and the normalized power load data are divided to obtain a training set and a test set; wherein the Z-Score function is as follows: (2); In the formula, is the standard deviation of the entire data set, For the The normalized scores of the data points.

[0026] Step S2: Construct a deep learning network model based on a dual-channel cross-attention mechanism. The deep learning network model of the dual-channel cross-attention mechanism of the present invention is as follows: Figure 4 As shown in the figure, it is composed of a parallel architecture of BiLSTM channels and Transformer channels, combined with an improved cross-attention mechanism and a network model of multi-layer perceptron MLP fused with Kolmogorov-Arnold network KAN. Among them, the BiLSTM channel consists of a bidirectional long short-term memory network BiLSTM layer, a random dropout layer, a residual connection and a fully connected module. First, the three-dimensional vector sequence data X processed by the improved variational mode decomposition VMD and sliding window technology is t =(x1, x2,..., x T), input to the bidirectional long short-term memory network BiLSTM layer to extract features, the data will pass through a 64-unit bidirectional long short-term memory network BiLSTM layer, and apply a random dropout layer to prevent overfitting. Then, the data will pass through a residual connection to fuse the extracted time series features with the original time series features. Finally, the data will pass through a fully connected layer with 64 neurons to map the feature dimensions extracted by the bidirectional long short-term memory network BiLSTM layer to 32 to extract the key time series information. In addition, the three-dimensional vector sequence data X processed by the improved VMD and sliding window technology is t =(x1, x2,..., x T ) will also be input into the Transformer channel at the same time to achieve parallel processing. The Transformer channel consists of a 1D convolution layer, a Transformer encoder Transformer Encoder layer and a fully connected layer. First, the time series data will undergo an upsampling process to map the feature dimension to 32. Then, the data will enter a 1D convolution layer with a convolution kernel size of 1 and a hidden layer dimension of 64 to obtain local information of the time series data. Then the data will enter the Transformer Encoder layer with 2 heads, 2 layers and a hidden dimension of 64. The Transformer Encoder layer contains the self-attention mechanism Self-Attention, residual connection Add and hierarchical normalization Norm, as well as the feedforward neural network Feed Forward to extract rich global time series information. Then the data will pass through a fully connected layer with 64 neurons to map the feature dimension extracted by the channel to 32 to extract the key time series information. The features extracted by the two channels are then input into the improved cross attention mechanism. The Transformer channel features are first used as the query vector matrix , with BiLSTM channel features as key vector matrices Sum vector matrix , the attention weight based on the Transformer channel features is calculated through the attention mechanism, and the weighted temporal feature representation is generated ; The BiLSTM channel features are then used as the query vector matrix , with Transformer channel features as key vector matrices Sum vector matrix , the attention weight based on BiLSTM channel features is calculated again through the attention mechanism, and the weighted temporal feature representation is generated , and then the two attention weights are weightedly fused using the contextual dynamic weight method to obtain a fused cross-attention feature representation. Finally, the output of the cross-attention mechanism will enter the multi-layer perceptron MLP and the Kolmogorov-Arnold network KAN respectively to obtain the predicted value and perform weighted fusion, thereby obtaining the output predicted value of the network model of the multi-layer perceptron MLP fused with the Kolmogorov-Arnold network KAN. The KAN network in the present invention has 2 layers, the multi-layer perceptron MLP has 1 layer, and β is 0.5, such as Figure 6 Shown is a diagram of the KAN network architecture used in the present invention.

[0027] The BiLSTM channel uses a bidirectional long short-term memory network BiLSTM and residual connection technology to convert the sequence data X t =(x1, x2,..., x T ) as input data X forward , passed to the positive long short-term memory network LSTM, and a positive output result is obtained , and the original sequence data X t Reverse and get X backward =(x T , x T-1 ,...,x1), and input it into the reverse LSTM to get a reverse output result Then, the forward and reverse output results are concatenated to achieve feature integration, so as to fully extract the time dependency between long-term and short-term, and the integrated results are combined with the original sequence data X t Add them to realize residual connection to further optimize the feature expression, and finally map FC through the fully connected layer to obtain the feature extraction result vector of the channel .

[0028] The Transformer channel first transforms the sequence data X t =(x1, x2,..., x T ) is passed as input to a 1D convolutional neural network CNN to obtain local semantic information , then Combined with the positional encoding, it is input into the Transformer Encoder. The self-attention mechanism of the Transformer Encoder is used to calculate the attention score. Then, it is further processed through the residual connection Add and hierarchical normalization Norm, and then through the feedforward neural network Feed Forward to obtain a time-dependent feature sequence combining global and local temporal features. Finally, the fully connected layer FC is used to map the result vector of the channel feature extraction. .

[0029] The feature sequence obtained by the dual channels and The feature fusion is performed through the improved cross attention mechanism to extract the time series information, which is then input into the network model of the multi-layer perceptron MLP fusion Kolmogorov-Arnold network KAN for prediction. The calculation formula is as follows: (3); (4); (5); (6); (7); (8); In the formula, is the feature vector extracted by the Transformer channel; is the feature vector extracted by the BiLSTM channel; , and They are respectively used in the Transformer channel to generate the query vector matrix , key vector matrix and the value vector matrix The learnable weight matrix of , and They are used to generate query vector matrices in the BiLSTM channel. , the key vector matrix and the value vector matrix The learnable weight matrix of is the dimension size of the key vector; It is a normalization operation; The sequence length is Context features; FC stands for fully connected mapping; is the activation function; represents the context weight; crossAttention is the output of the improved cross attention mechanism; matrix It represents a nonlinear mapping. Matrix of layers It is composed of a linear transformation matrix and a set of trainable univariate spline functions Composition, among which It is The first ( i , j ) elements corresponding to the spline function, different layers are operated by composite operations Connect to form a hierarchical mapping relationship; x represents the output feature of the cross attention mechanism; and Respectively represent the results predicted by KAN and MLP; in MLP calculation, represents the weight matrix of the Lth layer, and Represent the bias terms of the first layer and the Lth layer respectively; is the weight item for the fusion of prediction results, It is the final output time series prediction result.

[0030] Step S3: The generated training set and test set are used as model inputs to train the constructed dual-channel cross-attention network model. Considering the different extraction requirements of long-term and short-term time series information and global time series information, the power load data is respectively input into the BiLSTM channel and the Transformer channel for parallel calculation to extract features, and then the improved cross-attention mechanism and the network model of the multi-layer perceptron MLP weighted fusion Kolmogorov-Arnold network KAN are combined for feature enhancement, and the dual-channel cross-attention network model is continuously optimized until convergence and stable performance is achieved, wherein the stable performance refers to the dual-channel cross-attention network model reaching an indicator of low error, i.e., high accuracy; In this embodiment, the constructed dual-channel cross-attention network model is trained, the number of iterations is set to 100, the ratio of training set to test set is 9:1, the learning rate of Adam defaults to 0.001, and the batch size is Set to 128, use sliding window mechanism, window size is set to 60, and use regression problem loss function huber_loss. During the training process, the feature extraction part of the dual-channel cross attention mechanism and the time series prediction part of the multi-layer perceptron MLP weighted fusion Kolmogorov-Arnold network KAN can be expressed as: (9); In the formula, Indicates The true value of the data; Indicates The predicted value of data; Represents the threshold used to switch the range of square loss and absolute loss; and The difference between the two is less than , then the loss function uses square loss; and The difference between the two is greater than , linear loss is used to avoid outliers from having too much impact on the loss function.

[0031] The training loss of the present invention and other model training loss results are as follows Figure 7 As shown, the prediction results of the present invention are as follows Figure 8 As shown in the figure, it can be seen that the convergence speed of this model training is very good, and the prediction accuracy on the test set is very close to the true value.

[0032] Step S4: Use the trained deep learning network model based on the dual-channel cross-attention mechanism to predict power load data. Figure 5 A flow chart of the application of the method used according to the present invention is presented.

[0033] The present invention uses the following common performance indicators to measure the prediction performance of the deep learning network model based on the dual-channel cross-attention mechanism: Mean Absolute Error (MAE): is a statistical indicator that measures the average difference between the predicted value and the actual value. It is the average of the absolute values ​​of the deviations of all individual observations from the arithmetic mean. It describes the average of the absolute values ​​of the difference between the predicted value and the true value. That is, the smaller the MAE, the more accurate the model's prediction. The calculation formula is as follows: (16); In the formula, Indicates the total amount of data.

[0034] Mean Squared Error (MSE): A measure of the difference between the estimator and the estimated value, describing the difference between the predicted value and the true value. Similar to MAE, the smaller the MSE, the more accurate the model's prediction. The calculation formula is as follows: (17).

[0035] Root Mean Square Error (RMSE): An indicator used to measure the prediction accuracy of a prediction model on continuous data, indicating the average deviation between the predicted value and the true value. Similar to MAE, the smaller the RMSE, the more accurate the model's prediction. The calculation formula is as follows: (18).

[0036] Coefficient of Determination ( ) : It is an indicator to evaluate the consistency between the predicted value and the actual value. Its value range is from 0 to 1. The closer it is to 1, the better the model fits the data. The calculation formula is as follows: (19); In the formula, Represents the average of all real data.

[0037] Set up comparative experiments and ablation experiments, and predict evaluation indicators: Use the following models CNN[1], CNN-BiGRU[2], LSTM, BiLSTM, EMD-BiLSTM[3], VMD-BiLSTM[4], and BiLSTM-attention-LSTM[5] as comparative experiments. The results are as follows: Fig. 9 , Fig.10 , Fig.11 , Fig.12 As shown, it can be seen that the dual-channel cross-attention network model VBTCKN proposed in the present invention has the lowest RMSE for multi-step prediction in the four sub-datasets under the power transformer dataset ETT.

[0038] BiLSTM, BiLSTM-Transformer, BiLSTM-Transformer-CrossAttention, BiLSTM-Transformer-CrossAttention-KAN, VMD-BiLSTM, and VMD-Transformer were used for ablation experiments. The results are as follows: Fig.13 As shown, the rationality and effectiveness of the model design of the present invention are verified.

[0039] To further verify the applicability of the dual-channel cross-attention network, the dual-channel cross-attention network model VBTCKN was compared with other models in the GEFCom2014-E dataset under the conditions of prediction lengths of 10 and 30, including eight curves. The results are as follows Fig.14 It can be seen that the dual-channel cross-attention network model VBTCKN proposed in the present invention is still better than other models, and the specific values ​​are shown in Table 1.

[0040] Table 1 is a comparison table of this model and the existing mainstream time series prediction model The details of these comparison models are published at: [1] K. Wang et al., "Multiple convolutional neural networks for multivariate time series prediction," Neurocomputing, vol. 360, pp. 107-119, 2019. [2] He, J. Liu, F. Yang, X. Yang, S. Guo, and G. Sun, "Short-termPrediction of Distribution Voltage Based on CNN-BiGRU," 2023 4thInternational Conference on Advanced Electrical and Energy Systems (AEES), Shanghai, China, 2023, pp. 361-365. [3] L. Thi Minh Lien, V. Quoc Anh, N. Duc Tuyen and G. Fujita, "Prediction of State-of-Health and Remaining-Useful-Life of Battery Based onHybrid Neural Network Model," in IEEE Access, vol. 12, pp. 129022-129039, 2024. [4] S. Ghimire, RC Deo, D. Casillas-Pérez, and S. Salcedo-Sanz, "Electricity demand error corrections with attention bi-directional neural networks," Energy, vol. 291, p. 129938, 2024. In order to verify the generalization performance of the model of the present invention, the present invention is compared with other mainstream models in the GEFCom2014-E and Weather datasets. The results are as follows: Fig.15 , Fig.16 As shown, it can be seen that in the multi-step prediction in different data sets, the RMSE of the model proposed in the present invention is the lowest.

[0041] In summary, the prediction accuracy of the power load forecasting method based on the dual-channel cross-attention network of the present invention is higher than that of the existing time series forecasting method.

[0042] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A power load forecasting method based on a dual-channel cross-attention network, characterized by: The method comprises the following steps, and the following steps are performed in sequence: Step S1: Obtain a power load data set, pre-process the power load data therein, then apply variational mode decomposition VMD to decompose the power load data time series into data components of different modes, iteratively optimize VMD parameters through adaptive particle swarm algorithm APSO, obtain the optimized decomposition sequence and combine it with the original power load feature as a new feature to form mixed feature data, and sequentially process the mixed feature data through sliding window technology and Z-Score standardization, and divide it into training set and test set, and use the data in the training set and test set as the input vector of the model, and the vector is a three-dimensional feature vector containing a set time step; Step S2: construct a deep learning network model based on a dual-channel cross-attention mechanism, using a bidirectional long short-term memory network BiLSTM model and a Transformer model as BiLSTM channels and Transformer channels for parallel calculation to achieve feature extraction, and then combining an improved cross-attention mechanism and a multi-layer perceptron MLP weighted fusion Kolmogorov-Arnold network KAN network model for feature enhancement; Step S3: Use the data in the training set as input to train the constructed dual-channel cross-attention network model, and continue to optimize the model until convergence meets the set requirements; Step S4: Use the trained network model to predict power load data.

2. The method for power load forecasting based on a dual-channel cross attention network according to claim 1 is characterized in that: The preprocessing in step S1 includes filling missing values ​​with mean values, replacing abnormal values ​​with mean values, and then normalizing.

3. The power load forecasting method based on a dual-channel cross attention network according to claim 1 is characterized in that: In the optimization of VMD decomposition parameters in step S1, sample entropy is selected as a fitness function to quantify the quality of parameter combinations, and the global optimal parameters are found by iterating continuously with the goal of minimizing sample entropy. The load value is then subjected to VMD decomposition using the global optimal parameters, and a subsequence with the lowest complexity is screened. The sample entropy of the subsequence is then calculated and enhanced by adaptive threshold screening to obtain an optimized decomposition sequence. The sample entropy calculation formula is as follows: (1)； In the formula, N Represents the total number of data points; represents the similarity threshold; and They are respectively represented in the embedding dimension and When the distance between vectors is less than probability; Indicates the length The sample entropy of a time subseries.

4. The method for power load forecasting based on a dual-channel cross attention network according to claim 1 is characterized in that: The formula for Z-Score standardization in step S1 is: (2)； In the formula, For the current data points, is the average value of the entire data set, is the standard deviation of the entire data set, Z For the The normalized scores of the data points.

5. The method for power load forecasting based on a dual-channel cross attention network according to claim 1 is characterized in that: In step S2, the BiLSTM channel uses a bidirectional long short-term memory network BiLSTM and residual connection technology to convert the sequence data X t =(x1, x2,..., x T ) as input data X forward , passed to the positive long short-term memory network LSTM, and a positive output result is obtained , and the original sequence data X t Reverse and get X backward =(x T , x T-1 ,...,x1), and input it into the reverse LSTM to get a reverse output result Then, the forward and reverse output results are spliced ​​to achieve feature integration, so as to fully extract the time dependency between long-term and short-term, and the integrated results are combined with the original sequence data X t Add them to realize residual connection to further optimize the feature expression, and finally obtain the feature extraction result vector of the channel through full connection layer mapping ; The Transformer channel first transforms the sequence data X t =(x1, x2,..., x T ) is passed as input to a 1D convolutional neural network CNN to obtain local semantic information , then Combined with the positional encoding, it is input into the Transformer encoder. The self-attention mechanism of the Transformer encoder is used to calculate the attention score. Then, it is further processed through residual connection and hierarchical normalization, and then through the feedforward neural network to obtain a time-dependent feature sequence combining global and local temporal features. Finally, the result vector of the channel feature extraction is obtained through the fully connected layer mapping. ; The feature sequence obtained by the dual channels and The improved cross-attention mechanism is used to perform feature fusion, cross-calculate and obtain the attention weights between different positions, and the different attention weights are weighted summed up. The obtained sequence is the key feature to be extracted.

6. The method for power load forecasting based on a dual-channel cross attention network according to claim 1 is characterized in that: The method for performing feature enhancement on the improved cross attention mechanism and the network model of the multi-layer perceptron MLP weighted fusion Kolmogorov-Arnold network KAN in step S2 is: firstly, the Transformer channel feature is used as the query vector matrix , with BiLSTM channel features as key vector matrices Sum vector matrix , the attention weight based on the Transformer channel features is calculated through the attention mechanism, and the weighted temporal feature representation is generated ; The BiLSTM channel features are then used as the query vector matrix , with Transformer channel features as key vector matrices Sum vector matrix , the attention weight based on BiLSTM channel features is calculated again through the attention mechanism, and the weighted temporal feature representation is generated , and then the two attention weights are fused using the context dynamic weight method to obtain the fused cross-attention feature representation, which is finally input into the network model of multi-layer perceptron MLP fused Kolmogorov-Arnold network KAN for prediction. The calculation formula is as follows: （3）； （4）； （5）； (6)； (7)； (8)； In the formula, is the feature vector extracted by the Transformer channel; is the feature vector extracted by the BiLSTM channel; , and They are respectively used in the Transformer channel to generate the query vector matrix , key vector matrix and the value vector matrix The learnable weight matrix of , and They are used to generate query vector matrices in the BiLSTM channel. , the key vector matrix and the value vector matrix The learnable weight matrix of is the dimension size of the key vector; It is a normalization operation; The sequence length is contextual features; FC stands for fully connected mapping; is the activation function; represents the context weight; crossAttention is the output of the improved cross attention mechanism; matrix It represents a nonlinear mapping. Matrix of layers It is composed of a linear transformation matrix and a set of trainable univariate spline functions Composition, among which It is The first ( i , j ) elements corresponding to the spline function, different layers are operated by composite operations Connect to form a hierarchical mapping relationship; x represents the output feature of the cross attention mechanism; and Respectively represent the results predicted by KAN and MLP; in MLP calculation, represents the weight matrix of the Lth layer, and Represent the bias terms of the first layer and the Lth layer respectively; is the weight item for the fusion of prediction results, It is the final output time series prediction result.

7. The method for power load forecasting based on a dual-channel cross attention network according to claim 1 is characterized in that: The convergence judgment of the model training in step S3 is based on the loss function setting. The loss function uses huber_loss to calculate the parameter gradient of the model and reversely updates the parameters of the model. The training process is expressed as: (9)； In the formula, Indicates The true value of the data; Indicates The predicted value of data; Represents the threshold used to switch the range of square loss and absolute loss; and The difference between the two is less than , then the loss function uses square loss; and The difference between the two is greater than , linear loss is used to avoid outliers from having too much impact on the loss function.

Images