Power load forecasting method and system

Abstract

The present invention provides a method and system for power load forecasting, and relates to the technical field of power load forecasting. Power load data is collected and preprocessed to generate a target data set; the target data set is decomposed into three parts: seasonality, trend, and residual using a seasonal trend decomposition strategy; the three parts of seasonality, trend, and residual are respectively input into an encoder for multi-head sparse self-attention calculation to obtain a first attention integral; the three parts of seasonality, trend, and residual are respectively fast Fourier transformed and input into a coding layer for multi-head sparse self-attention calculation to obtain a second attention integral; the first attention integral is fused with the second attention integral to generate a target fused integral; the target fused integral is sent to a decoder and an output result is obtained to obtain a prediction sequence. The technical effect of accurately predicting long sequences is achieved in the power load forecasting problem.

Description

Technical Field

[0001] The present application relates to the technical field of power load forecasting, and in particular to a power load forecasting method and system. Background Art

[0002] Power load forecasting plays a vital role in power system management and planning. Accurate load forecasting is the key to ensuring stable operation of the power grid, optimizing resource allocation, improving energy efficiency and reducing operating costs.

[0003] Traditional prediction models, such as LSTM and RNN, perform well in short-term predictions, but their accuracy drops significantly when dealing with long-term dependencies. They cannot provide good assistance for decision-making in the subsequent period of time, consume a lot of computing resources, and have limited prediction speed.

[0004] Traditional forecasting methods often ignore the characteristics of data in the frequency domain when processing time series data. Power load data not only shows periodicity and trend in the time domain, but also contains rich information in the frequency domain, such as harmonic components, periodic fluctuations, etc., which are also important for forecasting. However, traditional time domain forecasting models lack effective frequency domain analysis methods and are difficult to fully utilize frequency domain information, which may affect the forecasting accuracy.

[0005] In recent years, the Transformer model has attracted widespread attention due to its powerful performance in processing sequence data. Through the self-attention mechanism, the Transformer can effectively capture the long-term dependencies in the sequence, and has made significant progress in sequence modeling compared to traditional RNN and LSTM models. However, the application of the Transformer model in the field of power load forecasting still has some limitations: (1) High computational complexity, especially when processing large-scale data sets, the computational cost of the full attention mechanism is high, which limits the practical application of the model; (2) Because it uses point-wise attention, each time point is independently calculated and predicted, so the overall and global feature information is difficult to capture; (3) It pays more attention to the relationship between the sequence in the time dimension. Some interference information may be better identified in the frequency domain, and the model lacks information about the frequency domain of the data.

[0006] Therefore, how to achieve the technical effect of accurately predicting long sequences in the problem of power load forecasting has become a technical problem that needs to be solved urgently. Summary of the invention

[0007] In order to achieve the technical effect of accurately predicting long sequences in the problem of power load forecasting, the present application provides a power load forecasting method and system.

[0008] In the first aspect, the present application provides a method for predicting power load using the following technical solution:

[0009] A method for predicting electric load, comprising:

[0010] Collecting power load data and preprocessing the power load data to generate a target data set including a time series;

[0011] Decomposing the target data set into three parts: seasonality, trend and residual using seasonal trend decomposition strategy;

[0012] Input the seasonality, the trend and the residual into the encoder respectively to perform multi-head sparse self-attention calculation to obtain a first attention integral containing time domain information;

[0013] The seasonality, the trend and the residual are respectively subjected to fast Fourier transformation and passed to the encoding layer for multi-head sparse self-attention calculation to obtain a second attention integral containing frequency domain information;

[0014] Fusing the first attention integral with the second attention integral to generate a target fused integral;

[0015] Sending the target fusion integral to a decoder and obtaining an output result;

[0016] A predicted sequence is obtained according to the output results.

[0017] Optionally, the step of collecting power load data and preprocessing the power load data to generate a target data set containing a time series includes:

[0018] Collecting power load data, and performing missing value filling strategy, anomaly detection strategy, data cleaning strategy and normalization strategy processing on the power load data to generate a first data set;

[0019] The first data set is embedded and position encoding is added to generate a target data set.

[0020] Optionally, the step of embedding the first data set and adding position coding to generate a target data set includes:

[0021] Embed the first data set and add position encoding to generate a target data set

[0022] .

[0023] Optionally, the step of decomposing the target data set into three parts, namely seasonality, trend and residual, using a seasonal trend decomposition strategy comprises:

[0024] The target data set Series decomposition by seasonal trend decomposition method

[0025] ;

[0026] Among them, the is a trend item, For seasonal items, is the residual term.

[0027] Optionally, the step of inputting the seasonality, the trend and the residual into an encoder respectively to perform multi-head sparse self-attention calculation to obtain a first attention integral containing time domain information includes:

[0028] The seasonal The trend and the residual As A input to the encoder for calculation and obtain the calculation result

[0029]

[0030] in, represents the queries matrix, represents the keys matrix, represents the values ​​matrix, T represents the transpose of the keys matrix, and d is the dimension of K;

[0031] The first attention score containing time domain information is obtained according to the calculation result

[0032]

[0033] in, 1 represents the weight of seasonal integral, 2 represents the weight of trend integral, and 3 represents the weight of residual attention integral. Represents the first attention score.

[0034] Optionally, the seasonal The trend and the residual The steps of inputting A to the encoder for calculation and obtaining the calculation result include:

[0035] Set the encoder input sequence ,in is the sequence length, It is the feature dimension and is a multi-head attention mechanism to calculate Q, K, V;

[0036] For each head i, three weight matrices are initially defined , each is A matrix of dimension , where is the feature dimension of each head, , where h is the number of longs:

[0037]

[0038] in yes A matrix of dimension d k is the dimension of K;

[0039] Calculate the attention weights. For each head i, calculate the attention score matrix As and The dot product of

[0040] ,

[0041] Apply the Softmax function to each score to obtain the attention weight matrix ,in is a sparse matrix that contains only sparse metrics Top-u q under;

[0042]

[0043] represents the mth q vector, represents the nth k vector, d represents the dimension of the K matrix, represents the number of k vectors, The n in represents the position of k that makes the calculation reach the maximum value;

[0044] Among them, u is controlled by a sampling factor, , c is a fixed parameter, is the sequence length, which means that only one query-key lookup calculation is required Dot product;

[0045] u Q values ​​are selected, and the other Q values ​​are replaced by the mean value of V;

[0046] The attention weight matrix and Multiply to get the attention-weighted output ;

[0047] The outputs of all heads are concatenated and passed through an additional linear transformation Get the final output, .

[0048] Optionally, the outputs of all heads are concatenated and passed through an additional linear transformation After the final output step, it also includes:

[0049] Perform attention distillation

[0050]

[0051] in, represents the attention block, use The activation function performs a one-dimensional convolution filter in the temporal dimension. A maximum pooling layer with a stride of 2 is added after the stacked layer to reduce the spatial dimension of the input data to half of its original size, reducing the overall memory usage to , is a negligible number.

[0052] Optionally, the step of fusing the first attention score with the second attention score to generate a target fused score includes:

[0053] Learning weights ,in represents the Sigmoid activation function, is the weight matrix, is the bias term, is the vector of the time domain attention integral and the frequency domain attention integral concatenated by column, and the fused attention integral .

[0054] Optionally, the step of sending the target fusion integral to a decoder and obtaining an output result includes:

[0055] Input sequence to the decoder ,in To mark the start, is a placeholder for the target sequence. The target sequence is set to 0. token is the length of the labeled sequence in the first half of the input, L y is the length of the sequence marked as 0 in the second half, d mod is the model dimension, wherein the structure of the decoder includes a masked multi-head sparse attention mechanism and a multi-head attention mechanism;

[0056] The output of the decoder and the fully connected layer is obtained as an output result.

[0057] In a second aspect, the present application provides a power load forecasting system, the power load forecasting system comprising:

[0058] A data acquisition module, used for acquiring power load data and preprocessing the power load data to generate a target data set including a time series;

[0059] A data decomposition module, used for decomposing the target data set into three parts: seasonality, trend and residual using a seasonal trend decomposition strategy;

[0060] A first attention integral module, used for inputting the seasonality, the trend and the residual into an encoder respectively to perform multi-head sparse self-attention calculation to obtain a first attention integral containing time domain information;

[0061] The second attention integral module is used to perform fast Fourier transform on the seasonality, the trend and the residual respectively and pass them into the encoding layer for multi-head sparse self-attention calculation to obtain a second attention integral containing frequency domain information;

[0062] a fusion integration module, configured to fuse the first attention integration with the second attention integration to generate a target fusion integration;

[0063] An output result module, used for sending the target fusion integral to a decoder and obtaining an output result;

[0064] The prediction sequence module is used to obtain the prediction sequence according to the output result.

[0065] In summary, the present application includes the following beneficial technical effects:

[0066] This application collects power load data and preprocesses the power load data to generate a target data set; uses a seasonal trend decomposition strategy to decompose the target data set into three parts: seasonality, trend, and residual; inputs the three parts of seasonality, trend, and residual into the encoder for multi-head sparse self-attention calculation to obtain the first attention integral; performs fast Fourier transformation on the three parts of seasonality, trend, and residual and inputs them into the encoding layer for multi-head sparse self-attention calculation to obtain the second attention integral; fuses the first attention integral with the second attention integral to generate a target fusion integral; sends the target fusion integral to the decoder and obtains the output result to obtain a prediction sequence. The technical effect of accurately predicting long sequences is achieved in the power load prediction problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flowchart of the first embodiment of the power load forecasting method of the present application;

[0068] Figure 2 This is a data embedding principle diagram of the first embodiment of the power load forecasting method of the present application;

[0069] Figure 3 is a schematic diagram of the attention distillation mechanism of the first embodiment of the power load forecasting method of the present application;

[0070] Figure 4 is a fast Fourier transform schematic diagram of the first embodiment of the power load forecasting method of the present application;

[0071] Figure 5 It is a structural block diagram of the first embodiment of the power load forecasting system of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0073] The present application provides a method for predicting power load. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the power load forecasting method of the present application.

[0074] In this embodiment, the power load prediction method includes the following steps:

[0075] Step S10: Collect power load data and pre-process the power load data to generate a target data set containing a time series.

[0076] It should be noted that in this embodiment, an Informer-based power load forecasting model is used. After preprocessing the data, the seasonal trend decomposition method is used to decompose the data into three parts: seasonality, trend, and residual. The attention integral containing time domain and frequency domain information is obtained by multi-head sparse self-attention calculation, and the two are fused to obtain a fusion integral, which is then sent to the decoder for calculation to obtain a prediction result. The present invention is not only suitable for short-term prediction but also for long-sequence prediction. At the same time, it reduces the computational complexity, can well capture the overall and global feature information of the data to improve the prediction accuracy and applicability, and add the processing of the frequency domain information of the data to analyze the data more comprehensively.

[0077] In a specific implementation, the steps of collecting power load data and preprocessing the power load data to generate a target data set containing a time series include: collecting power load data, processing the power load data using a missing value filling strategy, anomaly detection strategy, data cleaning strategy, and normalization strategy to generate a first data set; embedding the first data set and adding position coding to generate a target data set.

[0078] It can be understood that the step of embedding and adding position coding to the first data set to generate the target data set includes: embedding and adding position coding to the first data set to generate the target data set

[0079] .

[0080] It can be understood that in this embodiment, power load data is collected; data preprocessing includes missing value filling, anomaly detection, cleaning, and normalization. The time series power load training set, verification set, and test set are constructed in a ratio of 7:1:2; the normalized input data is embedded and position coding is added. The specific data embedding principle is as follows Figure 2 shown.

[0081] Step S20: Use the seasonal trend decomposition strategy to decompose the target data set into three parts: seasonality, trend and residual.

[0082] In the specific implementation, the seasonal trend decomposition strategy is used to decompose the target data set into three parts: seasonality, trend and residual, including: Series decomposition by seasonal trend decomposition method

[0083] ;

[0084] in, is the trend item, is the seasonal term, is the residual term.

[0085] Step S30: Input the seasonality, trend and residual parts into the encoder respectively for multi-head sparse self-attention calculation to obtain the first attention integral containing time domain information.

[0086] In a specific implementation, the seasonality, trend and residual are respectively input into the encoder for multi-head sparse self-attention calculation to obtain the first attention integral containing time domain information, including: seasonality , Trend and residual As A input to the encoder for calculation and obtain the calculation result

[0087]

[0088] in, represents the queries matrix, represents the keys matrix, represents the values ​​matrix, T represents the transpose of the keys matrix, and d is the dimension of K; the first attention score containing time domain information is obtained according to the calculation results

[0089]

[0090] in, 1 represents the weight of seasonal integral, 2 represents the weight of trend integral, and 3 represents the weight of residual attention integral. Represents the first attention score.

[0091] It should be noted that seasonal , Trend and residual The steps of inputting A to the encoder for calculation and obtaining the calculation result include:

[0092] Set encoder input sequence ,in is the sequence length, It is the feature dimension and is a multi-head attention mechanism to calculate Q, K, V;

[0093] For each head i, three weight matrices are initially defined , each is A matrix of dimension , where is the feature dimension of each head, , where h is the number of heads. Here is a multi-head attention mechanism. h=8 means eight heads, or eight parts:

[0094]

[0095] in yes A matrix of dimension d k is the dimension of K;

[0096] Calculate the attention weights. For each head i, calculate the attention score matrix As and The dot product of

[0097] ,

[0098] Apply the Softmax function to each score to obtain the attention weight matrix ,in is a sparse matrix that contains only sparse metrics Top-u q under;

[0099]

[0100] represents the mth q vector, represents the nth k vector, d represents the dimension of the K matrix, represents the number of k vectors, The n in represents the position of k that makes the calculation reach the maximum value;

[0101] Among them, u is controlled by a sampling factor, c is a fixed parameter, and as an empirical parameter, it is set to 5 in the code in this embodiment. , is the sequence length, which means that only one query-key lookup calculation is required Dot product;

[0102] represents the mth q vector; It represents the number of k vectors. The first part finds the maximum value, and the second part is the average value. The difference between the two is obtained, and then the Top-U q with the largest difference is selected through the u parameter.

[0103] Used to describe computational complexity. "O" is a notation used to describe the time complexity of an algorithm, even if the input sequence length is It is very long. The method of this embodiment can also effectively reduce the amount of calculation, after all, it grows logarithmically.

[0104] u Q values ​​are selected, and the other Q values ​​are replaced by the mean value of V;

[0105] The attention weight matrix and Multiply to get the attention-weighted output ;

[0106] The outputs of all heads are concatenated and passed through an additional linear transformation Get the final output, .

[0107] It is understandable that the outputs of all heads are concatenated and passed through an additional linear transformation After the final output step, it also includes:

[0108] Perform attention distillation

[0109]

[0110] in, represents the attention block, use The activation function performs a one-dimensional convolution filter in the temporal dimension. A maximum pooling layer with a stride of 2 is added after the stacked layer to reduce the spatial dimension of the input data to half of its original size, reducing the overall memory usage to , is a negligible number.

[0111] It should be noted that in each layer stack in the encoder, the required matrix Q is calculated, some QK products are calculated, important Q is selected according to similarity, and the remaining Q is replaced by the mean of V, and then an attention distillation is performed. After multiple calculations, the original feature map is obtained. The attention distillation mechanism in this embodiment is as follows Figure 3 shown.

[0112] Step S40: Perform fast Fourier transform on the seasonality, trend and residual parts respectively and pass them into the encoding layer for multi-head sparse self-attention calculation to obtain the second attention integral containing frequency domain information.

[0113] In the specific implementation, Figure 4 As shown, the three parts of the decomposed data are subjected to fast Fourier transform respectively, and are passed to the encoding layer for multi-head sparse self-attention calculation to obtain the attention integral containing frequency domain information.

[0114] Step S50: Fuse the first attention integral with the second attention integral to generate a target fused integral.

[0115] In a specific implementation, the step of fusing the first attention integral with the second attention integral to generate a target fusion integral includes: learning weights ,in represents the Sigmoid activation function, is the weight matrix, is the bias term, is the vector of the time domain attention integral and the frequency domain attention integral concatenated by column, and the fused attention integral .

[0116] In a specific implementation, if the number of layers of the encoder is N, the following steps are performed N times:

[0117] 1) Assume the encoder input sequence ,in is the sequence length, is the feature dimension, and is a multi-head attention mechanism (h-head), calculating Q, K, V:

[0118] For each head i, three weight matrices are initially defined , each is A matrix of dimension , where is the feature dimension of each head, , where h is the number of longs, so:

[0119]

[0120] in Both A matrix of dimension d k is the dimension of K;

[0121] 2) Calculate the attention weight. For each head i, calculate the attention score matrix As and The dot product of :

[0122]

[0123] 3) Then apply the Softmax function to each score to obtain the attention weight matrix ,in is a sparse matrix that contains only sparse metrics Top-u q under,

[0124] ,

[0125] 4) u is controlled by a sampling factor, , c is a fixed parameter, is the sequence length, which means that only one query-key lookup calculation is required The dot product greatly reduces the computing resources. In the case of multiple heads, different sparse query-key pairs are generated for each head, thereby avoiding information loss. u Q values ​​are selected, and the other Q values ​​are replaced by the mean of V. Then the attention weight matrix and Multiply to get the attention-weighted output Finally, the outputs of all heads are concatenated and passed through an additional linear transformation Get the final output, ;

[0126] 5) Calculate residual connections and perform layer normalization;

[0127] 6) Enter the feedforward neural network;

[0128] 7) Calculate residual connections and perform layer normalization;

[0129] 8) Perform attention distillation:

[0130]

[0131] in Represents the attention block, which contains multi-head attention and basic operations, use The activation function performs a one-dimensional convolution filter (kernel width = 3) in the temporal dimension. A max-pooling layer with a stride of 2 is added after the stacked layer to reduce the spatial dimension of the input data to half, thus reducing the overall memory usage to , For a very small number.

[0132] Step S60: Send the target fusion integral to the decoder and obtain the output result.

[0133] In a specific implementation, the step of feeding the target fusion integral into a decoder and obtaining an output result includes: inputting a sequence into the decoder

[0134]

[0135] in To mark the start, is a placeholder for the target sequence. The target sequence is set to 0. token is the length of the labeled sequence in the first half of the input, L y is the length of the sequence marked as 0 in the second half, d mod is the model dimension, where the structure of the decoder includes a masked multi-head sparse attention mechanism and a multi-head attention mechanism; the output of the decoder and the fully connected layer is obtained as the output result.

[0136] In a specific implementation, the number of layers of the decoder is M, then the following steps are performed M times, specifically:

[0137] 1) Embed the normalized sequence;

[0138] 2) Decompose the input data into three parts: trend, season and residual;

[0139] 3) Perform masked multi-head sparse attention calculation;

[0140] 4) Calculate residual connections and layer normalization;

[0141] 5) Calculate the multi-head attention, and calculate the Q matrix obtained by the decoder with the K and V matrices passed in by the encoder;

[0142] 6) Calculate residual connections and layer normalization;

[0143] 7) Enter the feedforward neural network;

[0144] 8) Calculate residual connections and layer normalization.

[0145] Step S70: Obtain a prediction sequence according to the output result.

[0146] In the specific implementation, the prediction result is obtained through the fully connected layer, and the prediction result is denormalized to obtain the final prediction sequence.

[0147] This embodiment collects power load data and preprocesses the power load data to generate a target data set; uses a seasonal trend decomposition strategy to decompose the target data set into three parts: seasonality, trend, and residual; inputs the three parts of seasonality, trend, and residual into the encoder for multi-head sparse self-attention calculation to obtain the first attention integral; performs fast Fourier transform on the three parts of seasonality, trend, and residual and inputs them into the encoding layer for multi-head sparse self-attention calculation to obtain the second attention integral; fuses the first attention integral with the second attention integral to generate a target fusion integral; sends the target fusion integral to the decoder and obtains the output result to obtain a prediction sequence. The technical effect of accurately predicting long sequences is achieved in the power load prediction problem.

[0148] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the power load forecasting system of the present application.

[0149] like Figure 5 As shown, the power load forecasting system proposed in the embodiment of the present application includes:

[0150] The data acquisition module 10 is used to collect power load data and pre-process the power load data to generate a target data set including a time series;

[0151] A data decomposition module 20 is used to decompose the target data set into three parts: seasonality, trend and residual using a seasonal trend decomposition strategy;

[0152] The first attention integral module 30 is used to input the seasonality, trend and residual parts into the encoder respectively to perform multi-head sparse self-attention calculation to obtain the first attention integral containing time domain information;

[0153] The second attention integral module 40 is used to perform fast Fourier transform on the seasonality, trend and residual parts respectively and pass them into the encoding layer for multi-head sparse self-attention calculation to obtain the second attention integral containing frequency domain information;

[0154] A fusion integration module 50, configured to fuse the first attention integral with the second attention integral to generate a target fusion integral;

[0155] An output result module 60 is used to send the target fusion integral to the decoder and obtain an output result;

[0156] The prediction sequence module 70 is used to obtain a prediction sequence according to the output result.

[0157] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.

[0158] This embodiment collects power load data and preprocesses the power load data to generate a target data set; uses a seasonal trend decomposition strategy to decompose the target data set into three parts: seasonality, trend, and residual; inputs the three parts of seasonality, trend, and residual into the encoder for multi-head sparse self-attention calculation to obtain the first attention integral; performs fast Fourier transform on the three parts of seasonality, trend, and residual and inputs them into the encoding layer for multi-head sparse self-attention calculation to obtain the second attention integral; fuses the first attention integral with the second attention integral to generate a target fusion integral; sends the target fusion integral to the decoder and obtains the output result to obtain a prediction sequence. The technical effect of accurately predicting long sequences is achieved in the power load prediction problem.

[0159] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0160] In addition, for technical details not described in detail in this embodiment, please refer to the method for power load forecasting provided in any embodiment of the present application, which will not be repeated here.

[0161] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0162] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0164] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for predicting power load, characterized in that: include: Collecting power load data and preprocessing the power load data to generate a target data set including a time series; Decomposing the target data set into three parts: seasonality, trend and residual using seasonal trend decomposition strategy; Input the seasonality, the trend and the residual into the encoder respectively to perform multi-head sparse self-attention calculation to obtain a first attention integral containing time domain information; The seasonality, the trend and the residual are respectively subjected to fast Fourier transformation and passed to the encoding layer for multi-head sparse self-attention calculation to obtain a second attention integral containing frequency domain information; Fusing the first attention integral with the second attention integral to generate a target fused integral; Sending the target fusion integral to a decoder and obtaining an output result; Obtain a predicted sequence according to the output result; The step of using the seasonal trend decomposition strategy to decompose the target data set into three parts: seasonality, trend and residual includes: The target data set Series decomposition by seasonal trend decomposition method ； Among them, the For trend items, is the seasonal term, is the residual term; The step of inputting the seasonality, the trend and the residual into the encoder to perform multi-head sparse self-attention calculation to obtain a first attention integral containing time domain information includes: The seasonal The trend and the residual As A Input to the encoder for calculation and obtain the calculation result: in, represents the queries matrix, represents the keys matrix, represents the values ​​matrix, T represents the transpose of the keys matrix, and d is the dimension of K; The first attention score containing time domain information is obtained according to the calculation result in, 1 represents the weight of seasonal integral, 2 represents the weight of trend integral, and 3 represents the weight of residual attention integral. represents the first attention score; Wherein, the seasonal The trend and the residual As A The steps of inputting to the encoder for calculation and obtaining the calculation results include: Set the encoder input sequence ,in is the sequence length, It is the feature dimension and is a multi-head attention mechanism to calculate Q, K, V; For each head i, three weight matrices are initially defined , each is A matrix of dimension , where is the feature dimension of each head, , where h is the number of longs: in, yes A matrix of dimension d k is the dimension of K; Calculate the attention weights. For each head i, calculate the attention score matrix As and The dot product of ， Apply the Softmax function to each score to obtain the attention weight matrix ,in is a sparse matrix that contains only sparse metrics Top- q; represents the mth q vector, represents the nth k vector, d represents the dimension of the K matrix, represents the number of k vectors, The n in represents the position of k that makes the calculation reach the maximum value; in, Controlled by a sampling factor, , c is a fixed parameter, is the sequence length, which means that only one query-key lookup calculation is required Dot product; Selected Q is taken out, and the other Q values ​​are replaced by the mean value of V; The attention weight matrix and Multiply to get the attention-weighted output ; The outputs of all heads are concatenated and passed through an additional linear transformation Get the final output, .

2. The power load forecasting method according to claim 1, characterized in that: The step of collecting power load data and preprocessing the power load data to generate a target data set containing a time series includes: Collecting power load data, and performing missing value filling strategy, anomaly detection strategy, data cleaning strategy and normalization strategy processing on the power load data to generate a first data set; The first data set is embedded and position encoding is added to generate a target data set.

3. The power load forecasting method according to claim 2, characterized in that: The step of embedding the first data set and adding position coding to generate a target data set comprises: Embed the first data set and add position encoding to generate a target data set 。 4. The power load forecasting method according to claim 1, characterized in that: The outputs of all heads are concatenated and passed through an additional linear transformation After the final output step, it also includes: Perform attention distillation in, represents the attention block, use The activation function performs a one-dimensional convolution filter in the temporal dimension. A maximum pooling layer with a stride of 2 is added after the stacked layer to reduce the spatial dimension of the input data to half of its original size, reducing the overall memory usage to , is a negligible number.

5. The power load forecasting method according to claim 1, characterized in that: The step of fusing the first attention score with the second attention score to generate a target fused score comprises: Learning weights ,in represents the Sigmoid activation function, is the weight matrix, is the bias term, is the vector of the time domain attention integral and the frequency domain attention integral concatenated by column, and the fused attention integral .

6. The power load forecasting method according to claim 1, characterized in that: The step of sending the target fusion integral to a decoder and obtaining an output result comprises: Input sequence to the decoder ,in To mark the start, is a placeholder for the target sequence. The target sequence is set to 0. token is the length of the labeled sequence in the first half of the input, L y is the length of the sequence marked as 0 in the second half, d mod is the model dimension, wherein the structure of the decoder includes a masked multi-head sparse attention mechanism and a multi-head attention mechanism; The output of the decoder and the fully connected layer is obtained as an output result.

7. A power load forecasting system, characterized in that: Executing the method according to claim 1, the power load forecasting system comprises: A data acquisition module, used for acquiring power load data and preprocessing the power load data to generate a target data set including a time series; A data decomposition module, used for decomposing the target data set into three parts: seasonality, trend and residual using a seasonal trend decomposition strategy; A first attention integral module, used for inputting the seasonality, the trend and the residual into an encoder respectively to perform multi-head sparse self-attention calculation to obtain a first attention integral containing time domain information; The second attention integral module is used to perform fast Fourier transform on the seasonality, the trend and the residual respectively and pass them into the encoding layer for multi-head sparse self-attention calculation to obtain a second attention integral containing frequency domain information; a fusion integration module, configured to fuse the first attention integration with the second attention integration to generate a target fusion integration; An output result module, used for sending the target fusion integral to a decoder and obtaining an output result; The prediction sequence module is used to obtain the prediction sequence according to the output result.

Images