Transform-based multi-factor load prediction method

Abstract

The invention provides a Transform-based multi-factor load prediction method, and belongs to the technical field of load prediction. The method comprises the following steps: acquiring source data to be subjected to multi-factor load prediction, preprocessing the source data, and extracting timestamp information, load data and external factor data from the preprocessed source data; and carrying out position coding on the timestamp information, training a Transform prediction model based on the load data, the external factor data and the timestamp information subjected to position coding, and carrying out multi-factor load prediction by utilizing the trained Transform prediction model. The load data refers to data representing power consumption conditions, the external factor data refers to environment data and electricity price data, the environment data comprises humidity and temperature, and the timestamp information refers to sampling time points of the load data and the external factor data. The method is used for improving the time sequence modeling capability of the model and improving the prediction precision.

Description

Technical Field

[0001] The present invention belongs to the technical field of load forecasting, and in particular relates to a Transformer-based multi-factor load forecasting method. Background Art

[0002] In the existing technology, recurrent neural networks (RNNs) are often used for load forecasting, which can accurately predict the future load change trend and have strong stability and adaptability. However, when processing long sequences, gradient vanishing and gradient explosion problems are prone to occur; for example, when making load forecasts, it is easy to pay too much attention to abnormal data, resulting in prediction deviations. Summary of the invention

[0003] In response to the above technical problems, the present invention proposes a multi-factor load forecasting solution based on Transformer. The beneficial technical effects of the present invention include: improving the model's modeling ability for local context information, reducing the self-attention mechanism's attention to abnormal data, thereby reducing its impact on the prediction results; at the same time, the present invention extracts the timestamp information separately and embeds it into the input data for position encoding, aiming to improve the model's modeling ability for time series and improve prediction accuracy.

[0004] The first aspect of the present invention discloses a multi-factor load forecasting method based on Transformer, the method comprising:

[0005] Acquire source data for multi-factor load forecasting, preprocess the source data, and extract timestamp information, load data, and external factor data from the preprocessed source data;

[0006] The timestamp information is position-encoded, a Transformer prediction model is trained based on the load data, the external factor data, and the position-encoded timestamp information, and multi-factor load prediction is performed using the trained Transformer prediction model.

[0007] The load data refers to data characterizing power consumption, the external factor data refers to environmental data and electricity price data, the environmental data includes humidity and temperature, and the timestamp information refers to sampling time points of the load data and the external factor data.

[0008] Specifically, the preprocessing includes missing value processing, outlier processing and normalization processing.

[0009] Specifically, the timestamp information is position-coded, and the position coding is expressed as:

[0010]

[0011] Among them, pos represents the position of the data in the sequence, i represents the mapping column index, and d is the step size of the data in the sequence.

[0012] Specifically, the position encoding vector, the load data vector, and the time series data matrix of the external factors are combined as the input of the Transformer prediction model; where:

[0013] The vector representation of timestamp information is:

[0014] T=(t1,…,t d )

[0015] t i is the timestamp of the i-th step in the sequence;

[0016] The position encoding vector after position encoding is:

[0017] P = PositionEmbedding(T) = (p i ,…,p d )

[0018] p i is the position encoding vector of the i-th step in the sequence;

[0019] The load data vector is:

[0020] L=(l1,…,l d )

[0021] l i is the load data of the i-th step in the sequence;

[0022] The time series data matrix of external factors is:

[0023]

[0024] e mi is the mth external factor data in the i-th step in the sequence;

[0025] Then the input of the Transformer prediction model is:

[0026] I=P T +Embedding((L T ,E)).

[0027] Specifically, the input of the Transformer prediction model is divided into a training set and a test set, the Transformer prediction model is trained using the training set, and the trained Transformer prediction model is tested using the test set; wherein the training process specifically includes:

[0028] For any set of data in the training set, count the number of sampling points K according to the timestamp information it contains, sort the data of all sampling points in order from front to back according to the timestamp information, select the data from the L1th to the L2th sampling points as the input of the Transformer prediction model to predict the data from the L2+1th to the Kth sampling points; where 1≤L1<0.3K, 0.7K<L2<0.9K, and L1, L2, and K are all positive integers; calculate the predicted data and real data from the L2+1th to the Kth sampling points to calculate the loss function of the Transformer prediction model;

[0029] Multiple groups of data in the training set are trained, and the loss function is continuously optimized until the prediction accuracy of the Transformer prediction model reaches the optimal value.

[0030] Specifically, the Transformer prediction model includes an encoder and a decoder; wherein:

[0031] The encoder includes a position encoding module PC1, a multi-head self-attention module A1, a residual connection and normalization module N1, a feedforward neural network F1, and a residual connection and normalization module N2; wherein:

[0032] The data from the L1th to the L2th sampling points are used as encoder input data, and the position encoding module PC1 performs position encoding on the timestamp information in the encoder input data, while other data in the encoder input data remain unchanged;

[0033] The data output by the position encoding module PC1 is sent to the multi-head attention module A1 and the residual connection and normalization module N1 respectively. The multi-head attention module A1 adopts a multi-channel processing method, and its output is also transmitted to the residual connection and normalization module N1;

[0034] The residual connection and normalization module N1 integrates the data output by the position encoding module PC1 and the data output by the multi-head attention module A1, and sends the integration results to the feedforward neural network F1 and the residual connection and normalization module N2 respectively;

[0035] The feedforward neural network F1 processes the integration result and sends the processing result to the residual connection and normalization module N2. The residual connection and normalization module N2 integrates the data output by the residual connection and normalization module N1 and the data output by the feedforward neural network F1 as the encoding result of the encoder;

[0036] The encoded result is sent to the second multi-head self-attention module of the decoder.

[0037] Specifically, the decoder includes a position encoding module PC2, a first multi-head self-attention module, a residual connection and normalization module N3, a second multi-head self-attention module, a residual connection and normalization module N4, a feedforward neural network F2, and a residual connection and normalization module N5; wherein:

[0038] The data from the L2+1th to the Kth sampling points are used as the decoder input data, and the position encoding module PC2 performs position encoding on the timestamp information in the decoder input data, and other data in the decoder input data remain unchanged;

[0039] The data output by the position encoding module PC2 is sent to the first multi-head attention module and the residual connection and normalization module N3 respectively. The first multi-head attention module adopts a multi-channel processing method, and its output is also transmitted to the residual connection and normalization module N3;

[0040] The residual connection and normalization module N3 integrates the data output by the position encoding module PC2 and the data output by the first multi-head attention module, and sends the integration results to the second multi-head attention module and the residual connection and normalization module N4 respectively;

[0041] The second multi-head attention module processes the integration result. The second multi-head attention module adopts a multi-channel processing method, in which one channel is used to process the data from the residual connection and normalization module N3, and the other channels are used to process the encoding results from the encoder; the processing result of the second multi-head attention module is sent to the residual connection and normalization module N4;

[0042] The residual connection and normalization module N4 integrates the data output by the residual connection and normalization module N3 and the data output by the second multi-head attention module, and sends the integration results to the feedforward neural network F2 and the residual connection and normalization module N5 respectively;

[0043] The feedforward neural network F1 processes the integration result and sends the processing result to the residual connection and normalization module N5. The residual connection and normalization module N5 integrates the data output by the residual connection and normalization module N4 and the data output by the feedforward neural network F2 as the decoding result of the decoder;

[0044] The encoding result is sent to the linear transformation module and the Softmax layer respectively, and the prediction results of the data from the L2+1th to the Kth sampling points are output.

[0045] Specifically, a convolutional layer is configured before each channel of the multi-head self-attention module A1, the first multi-head attention module, and the second multi-head attention module.

[0046] The second aspect of the present invention discloses an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the multi-factor load forecasting method based on Transformer described in the first aspect is implemented.

[0047] The third aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the Transformer-based multi-factor load forecasting method described in the first aspect is implemented.

[0048] The beneficial technical effects of the present invention include: improving the model's ability to model local context information, reducing the self-attention mechanism's attention to abnormal data, thereby reducing its impact on the prediction results; at the same time, the present invention extracts the timestamp information separately and embeds it into the input data for position encoding, aiming to improve the model's ability to model time series and improve prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a flowchart of the multi-factor load forecasting method based on Transformer;

[0051] Figure 2 This is the structural diagram of the Transformer prediction model;

[0052] Figure 3 Another structural diagram of the Transformer prediction model;

[0053] Figure 4 This is the prediction result graph output by the Transformer prediction model. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0055] First embodiment

[0056] The process of this embodiment is as follows Figure 1 As shown:

[0057] S1: Preprocess the source data, including missing value processing, outlier processing and normalization processing, to obtain timestamp, load data and external factor data.

[0058] Source data refers to the original unprocessed data, which includes timestamp, load data and external factor data. Load data refers to the power consumption recorded at a certain timestamp.

[0059] S2: Position-encode the timestamps obtained in S1, and combine the encoded timestamp vector with the load data vector and the time series data matrix of the external factors as the input of the model. The detailed steps are as follows:

[0060] S2.1: Position encoding is performed on the timestamps obtained in S1. Position encoding is used to help the Transformer model capture the position information in the time series data, that is, the order information of the time series load data. Its definition formula is as follows:

[0061]

[0062]

[0063] pos is the position of the data in the input sequence, i is used to map the column index (the final output of position encoding is a matrix), d is the dimension of the data in the input sequence, and in load forecasting it is the step size of the input data.

[0064] Through position encoding, the information of each time step in the input data can be encoded into a unique code. In addition, position encoding through the sine function can also make the model pay attention to the relative position information of the input data, thereby giving the model the ability to understand time series information.

[0065] S2.2: Combine the position encoding vector with the load data vector and the time series data matrix of external factors as the input of the model.

[0066] The timestamp vector extracted from the original time series data is:

[0067] T=(t1,…,t d )

[0068] t i is the timestamp of the i-th step in the input sequence.

[0069] Then the position encoding vector after position encoding is:

[0070] P = PositionEmbedding(T) = (p i ,…,p d )

[0071] p i is the position encoding vector of the i-th step in the input sequence.

[0072] The load data vector is:

[0073] L=(l1,…,l d )

[0074] l i is the load data for the i-th step in the input sequence.

[0075] The external factor matrix is:

[0076]

[0077] e mi is the mth external factor data in the i-th step in the input sequence.

[0078] Then the input matrix of the final model is:

[0079] I=P T +Embedding((L T ,E))

[0080] S3: Input the input matrix obtained in S2 into the multivariate load forecasting model based on Transformer for training and forecasting to obtain the forecast results. The detailed steps are as follows:

[0081] S3.1: Divide the input matrix obtained in S2 into training set and test set in a ratio of 8:2.

[0082] S3.2: Build a Transformer-based multivariate load forecasting model model, use the training set obtained in S3.1 to train the model, and obtain a trained model.

[0083] S3.3: Use the trained model to predict the test set and obtain the final prediction result.

[0084] The encoder is responsible for receiving input data, i.e., multivariate time series data, and its output is related to the context-dependent representation of the input sequence. The decoder is responsible for outputting the generated predicted data and comparing it with the real data in the training set for learning training.

[0085] The multi-factor load forecasting method based on Transformer proposed in the first aspect of the present invention includes:

[0086] Acquire source data for multi-factor load forecasting, preprocess the source data, and extract timestamp information, load data, and external factor data from the preprocessed source data;

[0087] The timestamp information is position-encoded, a Transformer prediction model is trained based on the load data, the external factor data, and the position-encoded timestamp information, and multi-factor load prediction is performed using the trained Transformer prediction model.

[0088] The load data refers to data characterizing power consumption, the external factor data refers to environmental data and electricity price data, the environmental data includes humidity and temperature, and the timestamp information refers to sampling time points of the load data and the external factor data.

[0089] Specifically, the preprocessing includes missing value processing, outlier processing and normalization processing.

[0090] Specifically, the timestamp information is position-coded, and the position coding is expressed as:

[0091]

[0092] Among them, pos represents the position of the data in the sequence, i represents the mapping column index, and d is the step size of the data in the sequence.

[0093] Specifically, the position encoding vector, the load data vector, and the time series data matrix of the external factors are combined as the input of the Transformer prediction model; where:

[0094] The vector representation of timestamp information is:

[0095] T=(t1,…,t d )

[0096] t i is the timestamp of the i-th step in the sequence;

[0097] The position encoding vector after position encoding is:

[0098] P = PositionEmbedding(T) = (p i ,…,p d )

[0099] p i is the position encoding vector of the i-th step in the sequence;

[0100] The load data vector is:

[0101] L=(l1,…,l d )

[0102] l i is the load data of the i-th step in the sequence;

[0103] The time series data matrix of external factors is:

[0104]

[0105] e mi is the mth external factor data in the i-th step in the sequence;

[0106] Then the input of the Transformer prediction model is:

[0107] I=P T +Embedding((L T ,E)).

[0108] Specifically, the input of the Transformer prediction model is divided into a training set and a test set, the Transformer prediction model is trained using the training set, and the trained Transformer prediction model is tested using the test set; wherein the training process specifically includes:

[0109] For any set of data in the training set, count the number of sampling points K according to the timestamp information it contains, sort the data of all sampling points in order from front to back according to the timestamp information, select the data from the L1th to the L2th sampling points as the input of the Transformer prediction model to predict the data from the L2+1th to the Kth sampling points; where 1≤L1<0.3K, 0.7K<L2<0.9K, and L1, L2, and K are all positive integers; calculate the predicted data and real data from the L2+1th to the Kth sampling points to calculate the loss function of the Transformer prediction model;

[0110] Multiple groups of data in the training set are trained, and the loss function is continuously optimized until the prediction accuracy of the Transformer prediction model reaches the optimal value.

[0111] Specifically, Figure 2 As shown, the Transformer prediction model includes an encoder and a decoder; wherein:

[0112] The encoder includes a position encoding module PC1, a multi-head self-attention module A1, a residual connection and normalization module N1, a feedforward neural network F1, and a residual connection and normalization module N2; wherein:

[0113] The data from the L1th to the L2th sampling points are used as encoder input data, and the position encoding module PC1 performs position encoding on the timestamp information in the encoder input data, while other data in the encoder input data remain unchanged;

[0114] The data output by the position encoding module PC1 is sent to the multi-head attention module A1 and the residual connection and normalization module N1 respectively. The multi-head attention module A1 adopts a multi-channel processing method, and its output is also transmitted to the residual connection and normalization module N1;

[0115] The residual connection and normalization module N1 integrates the data output by the position encoding module PC1 and the data output by the multi-head attention module A1, and sends the integration results to the feedforward neural network F1 and the residual connection and normalization module N2 respectively;

[0116] The feedforward neural network F1 processes the integration result and sends the processing result to the residual connection and normalization module N2. The residual connection and normalization module N2 integrates the data output by the residual connection and normalization module N1 and the data output by the feedforward neural network F1 as the encoding result of the encoder;

[0117] The encoded result is sent to the second multi-head self-attention module of the decoder.

[0118] Specifically, the decoder includes a position encoding module PC2, a first multi-head self-attention module, a residual connection and normalization module N3, a second multi-head self-attention module, a residual connection and normalization module N4, a feedforward neural network F2, and a residual connection and normalization module N5; wherein:

[0119] The data from the L2+1th to the Kth sampling points are used as the decoder input data, and the position encoding module PC2 performs position encoding on the timestamp information in the decoder input data, and other data in the decoder input data remain unchanged;

[0120] The data output by the position encoding module PC2 is sent to the first multi-head attention module and the residual connection and normalization module N3 respectively. The first multi-head attention module adopts a multi-channel processing method, and its output is also transmitted to the residual connection and normalization module N3;

[0121] The residual connection and normalization module N3 integrates the data output by the position encoding module PC2 and the data output by the first multi-head attention module, and sends the integration results to the second multi-head attention module and the residual connection and normalization module N4 respectively;

[0122] The second multi-head attention module processes the integration result. The second multi-head attention module adopts a multi-channel processing method, in which one channel is used to process the data from the residual connection and normalization module N3, and the other channels are used to process the encoding results from the encoder; the processing result of the second multi-head attention module is sent to the residual connection and normalization module N4;

[0123] The residual connection and normalization module N4 integrates the data output by the residual connection and normalization module N3 and the data output by the second multi-head attention module, and sends the integration results to the feedforward neural network F2 and the residual connection and normalization module N5 respectively;

[0124] The feedforward neural network F1 processes the integration result and sends the processing result to the residual connection and normalization module N5. The residual connection and normalization module N5 integrates the data output by the residual connection and normalization module N4 and the data output by the feedforward neural network F2 as the decoding result of the decoder;

[0125] The encoding result is sent to the linear transformation module and the Softmax layer respectively, and the prediction results of the data from the L2+1th to the Kth sampling points are output.

[0126] Specifically, Figure 3 As shown, a convolutional layer is configured before each channel of the multi-head self-attention module A1, the first multi-head attention module, and the second multi-head attention module.

[0127] Second embodiment

[0128] The load data of a certain area in 2018 and external factors such as temperature, humidity, and electricity price are used as the data set. The sampling frequency of the data is once every hour, with 24 sampling points per day. The data set is divided into a training set and a test set, with the data from January 1, 2018 to October 19, 2018 as the training set and the data from October 20, 2018 to December 31, 2018 as the test set, with a ratio of approximately 8 to 2. In training and testing, the model will use the data of the past week, that is, 7 days, as the input independent variable, and the 24-point data of the next day as the dependent variable (label) for supervised training.

[0129] The mean absolute percentage error (MAPE), root mean square error (RMSE) and determination coefficient (R2_score) are selected as the evaluation indicators of the model. The calculation formula is as follows:

[0130]

[0131] Where P represents the predicted value sequence, R represents the true value sequence, Represents the average value of the true value, and n is the number of data points in the sequence. MAPE and RMSE are used to measure the error between the predicted value and the true value. The smaller the value, the more accurate the prediction. R2_score reflects the degree of fit of the model to the data. Its value range is [0,1]. The closer to 1, the better the model fit.

[0132] The results of prediction using the technology of the present invention are as follows Figure 4 The comparison results are shown in Table 1.

[0133] Table 1: Comparison of results

[0134] Indicators\Algorithm Model RMSE MAPE R2_Score Single Factor LSTM 493.711 0.039 0.814 Multi-factor LSTM 410.677 0.029 0.872 The technology of the present invention 298.620 0.021 0.932

[0135] The second aspect of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the multi-factor load forecasting method based on Transformer described in the first aspect is implemented.

[0136] The third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the Transformer-based multi-factor load forecasting method described in the first aspect is implemented.

[0137] The beneficial technical effects of the present invention include: improving the model's ability to model local context information, reducing the self-attention mechanism's attention to abnormal data, thereby reducing its impact on the prediction results; at the same time, the present invention extracts the timestamp information separately and embeds it into the input data for position encoding, aiming to improve the model's ability to model time series and improve prediction accuracy.

[0138] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above-mentioned embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. A multi-factor load forecasting method based on Transformer, characterized in that: The method comprises: Acquire source data for multi-factor load forecasting, preprocess the source data, and extract timestamp information, load data, and external factor data from the preprocessed source data; The timestamp information is position-encoded, a Transformer prediction model is trained based on the load data, the external factor data, and the position-encoded timestamp information, and multi-factor load prediction is performed using the trained Transformer prediction model. The load data refers to data characterizing power consumption, the external factor data refers to environmental data and electricity price data, the environmental data includes humidity and temperature, and the timestamp information refers to sampling time points of the load data and the external factor data.

2. According to claim 1, a Transformer-based multi-factor load forecasting method is characterized in that: The preprocessing includes missing value processing, outlier processing and normalization processing.

3. The multi-factor load forecasting method based on Transformer according to claim 1 is characterized in that: The timestamp information is position-coded, and the position coding is expressed as: Among them, pos represents the position of the data in the sequence, i represents the mapping column index, and d is the step size of the data in the sequence.

4. The multi-factor load forecasting method based on Transformer according to claim 3 is characterized in that: The position encoding vector, load data vector, and time series data matrix of external factors are combined as the input of the Transformer prediction model; where: The vector representation of timestamp information is: T=(t1,…,t d ) t i is the timestamp of the i-th step in the sequence; The position encoding vector after position encoding is: P=PositionEmbedding(T)=(p i ,…,p d p i is the position encoding vector of the i-th step in the sequence; The load data vector is: L＝(l1,…,l d ) l i is the load data of the i-th step in the sequence; The time series data matrix of external factors is: e mi is the mth external factor data in the i-th step in the sequence; Then the input of the Transformer prediction model is: I=P T +Embedding((L T ,E))。 5. The multi-factor load forecasting method based on Transformer according to claim 4 is characterized in that: The input of the Transformer prediction model is divided into a training set and a test set, the Transformer prediction model is trained using the training set, and the trained Transformer prediction model is tested using the test set; wherein the training process specifically includes: For any set of data in the training set, count the number of sampling points K according to the timestamp information it contains, sort the data of all sampling points in order from front to back according to the timestamp information, select the data from the L1th to the L2th sampling points as the input of the Transformer prediction model to predict the data from the L2+1th to the Kth sampling points; where 1≤L1<0.3K, 0.7K<L2<0.9K, and L1, L2, and K are all positive integers; calculate the predicted data and real data from the L2+1th to the Kth sampling points to calculate the loss function of the Transformer prediction model; Multiple groups of data in the training set are trained, and the loss function is continuously optimized until the prediction accuracy of the Transformer prediction model reaches the optimal value.

6. The multi-factor load forecasting method based on Transformer according to claim 5 is characterized in that: The Transformer prediction model includes an encoder and a decoder; wherein: The encoder includes a position encoding module PC1, a multi-head self-attention module A1, a residual connection and normalization module N1, a feedforward neural network F1, and a residual connection and normalization module N2; wherein: The data from the L1th to the L2th sampling points are used as encoder input data, and the position encoding module PC1 performs position encoding on the timestamp information in the encoder input data, while other data in the encoder input data remain unchanged; The data output by the position encoding module PC1 is sent to the multi-head attention module A1 and the residual connection and normalization module N1 respectively. The multi-head attention module A1 adopts a multi-channel processing method, and its output is also transmitted to the residual connection and normalization module N1; The residual connection and normalization module N1 integrates the data output by the position encoding module PC1 and the data output by the multi-head attention module A1, and sends the integration results to the feedforward neural network F1 and the residual connection and normalization module N2 respectively; The feedforward neural network F1 processes the integration result and sends the processing result to the residual connection and normalization module N2. The residual connection and normalization module N2 integrates the data output by the residual connection and normalization module N1 and the data output by the feedforward neural network F1 as the encoding result of the encoder; The encoded result is sent to the second multi-head self-attention module of the decoder.

7. The Transformer-based multi-factor load forecasting method according to claim 6, characterized in that: The decoder includes a position encoding module PC2, a first multi-head self-attention module, a residual connection and normalization module N3, a second multi-head self-attention module, a residual connection and normalization module N4, a feedforward neural network F2, and a residual connection and normalization module N5; wherein: The data from the L2+1th to the Kth sampling points are used as the decoder input data, and the position encoding module PC2 performs position encoding on the timestamp information in the decoder input data, and other data in the decoder input data remain unchanged; The data output by the position encoding module PC2 is sent to the first multi-head attention module and the residual connection and normalization module N3 respectively. The first multi-head attention module adopts a multi-channel processing method, and its output is also transmitted to the residual connection and normalization module N3; The residual connection and normalization module N3 integrates the data output by the position encoding module PC2 and the data output by the first multi-head attention module, and sends the integration results to the second multi-head attention module and the residual connection and normalization module N4 respectively; The second multi-head attention module processes the integration result. The second multi-head attention module adopts a multi-channel processing method, in which one channel is used to process the data from the residual connection and normalization module N3, and the other channels are used to process the encoding results from the encoder; the processing result of the second multi-head attention module is sent to the residual connection and normalization module N4; The residual connection and normalization module N4 integrates the data output by the residual connection and normalization module N3 and the data output by the second multi-head attention module, and sends the integration results to the feedforward neural network F2 and the residual connection and normalization module N5 respectively; The feedforward neural network F1 processes the integration result and sends the processing result to the residual connection and normalization module N5. The residual connection and normalization module N5 integrates the data output by the residual connection and normalization module N4 and the data output by the feedforward neural network F2 as the decoding result of the decoder; The encoding result is sent to the linear transformation module and the Softmax layer respectively, and the prediction results of the data from the L2+1th to the Kth sampling points are output.

8. The Transformer-based multi-factor load forecasting method according to claim 7, characterized in that: A convolutional layer is configured before each channel of the multi-head self-attention module A1, the first multi-head attention module, and the second multi-head attention module.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, a Transformer-based multi-factor load forecasting method as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the Transformer-based multi-factor load forecasting method according to any one of claims 1 to 8 is implemented.

Images