A Power Load Forecasting Method Based on Multi-Scale Context-Enhanced Self-Attention

Abstract

This invention discloses a power load forecasting method based on multi-scale context-enhanced self-attention, comprising the following steps: S1, preprocessing the raw power load data; S2, extracting discrete time features at four time scales (hourly, daily, weekly, and monthly) based on the periodic structure of the time series to capture the changing patterns of power load at different time scales; S3, using a context-enhanced location encoding method to introduce multi-scale time features into the location representation; S4, constructing a power load forecasting model and training it using a training set, saving the optimal power load forecasting model; S5, forecasting the power load based on the optimal power load forecasting model. This invention upgrades location information to composite features including time-series coordinates, semantic associations, and periodic attributes, improving the power load forecasting model's ability to model non-obvious location dependencies in complex time series.

Description

Technical Field

[0001] This invention relates to the field of power system technology, and in particular to a power load forecasting method based on multi-scale context-enhanced self-attention. Background Technology

[0002] With the continuous growth of the proportion of new energy power generation and the improvement of the intelligence level of power grid operation and dispatch, the power system load exhibits stronger dynamic variability and multi-periodic characteristics, posing a severe challenge to traditional load forecasting methods. To model the temporal characteristics of power load, recurrent neural networks (RNNs) and their variants (such as LSTM and GRU) are widely used in this field. However, RNN structures have certain limitations in modeling long-term dependent information and complex periodic changes, and are prone to gradient vanishing or exploding problems when stacked deeply.

[0003] Self-attention, due to its ability to model global dependencies, has been introduced into load forecasting tasks in recent years. Its core advantage lies in its ability to establish a direct correlation between any two time steps in the input sequence, enhancing the model's ability to represent complex temporal structures. However, existing methods often employ static absolute or relative positional encoding, failing to effectively incorporate the actual temporal context. This results in insufficient modeling capabilities for complex situations such as load abrupt changes, holiday disturbances, and periodic overlaps.

[0004] To address the multi-timescale structure of power loads, some studies have introduced multi-scale temporal features during data preprocessing, such as hour corresponding to day, day to week, month to day, and year to week, to express periodic information at different time granularities. These features help reveal the multi-scale patterns of load data; however, traditional methods often use these features only as input vector concatenation, failing to participate in the generation and adjustment of location encoding, resulting in insufficient utilization of temporal structure information. Furthermore, existing models lack structured contextual enhancement designs within their attention mechanisms, failing to adjust attention weights based on the context of the input sequence, thus limiting the model's ability to model complex behaviors such as load abrupt changes and periodic variations. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a power load forecasting method based on multi-scale context-enhanced self-attention, which improves the forecasting accuracy and stability under complex time-series structures and enhances the adaptability to extreme fluctuations and uncertainties in power load time series.

[0006] Technical solution: A power load forecasting method based on multi-scale context-enhanced self-attention, comprising the following steps:

[0007] S1, preprocesses the raw power load data;

[0008] S2, based on the periodic structure of time series, extracts discrete time features including four time scales: hour, day, week and month, to capture the changing patterns of power load at different time scales;

[0009] S3 employs a context-enhanced location coding method to introduce multi-scale temporal features into the location representation;

[0010] S4. Construct a power load prediction model and train the power load prediction model using a training set, and save the best power load prediction model.

[0011] S5, based on the optimal power load forecasting model, completes the forecasting of power load.

[0012] Furthermore, the implementation steps of incorporating multi-scale temporal features into the location representation using a context-enhanced location encoding method are as follows:

[0013] S31, construct the relative position distance matrix Each term in R Indicates position and Relative offset:

[0014] ,

[0015] in, Indicates the length of the input sequence. Represents the set of integers;

[0016] Define a relative position embedding function Map the relative offset to 3D embedding vectors are used to capture the structural relationships between elements:

[0017] ,

[0018] in, express The embedding results For relative embedding at all positions, Includes all , Represent real numbers;

[0019] S32 extracts contextual features through a one-dimensional convolution operation with a kernel size of n:

[0020] ,

[0021] in, Represents the position in the sequence Local contextual features extracted by one-dimensional convolution; The input sequence features are given by m, where m is the dimension of the feature vector at each position in the sequence; Conv1D() represents a one-dimensional convolution operation.

[0022] all Arranged according to sequence position, the context encoding matrix is ​​obtained. The context encoding matrix C is used to represent the information of the local neighborhood at each location.

[0023] S33, Define a mapping function Used to generate the corresponding context vector from the tag. :

[0024] ,

[0025] Introduce a learnable gating unit And activated by the Sigmoid function. Obtain the context gate for each position. , means as follows:

[0026] ,

[0027] in, , Represents the input vector Feature dimensions, For learnable parameters, For the first Input vectors at each position;

[0028] Output using gating unit By controlling the degree of involvement of the control context at position i, the gating context representation is obtained as follows:

[0029] ,

[0030] in, Indicates the gating context encoding;

[0031] S34, relative embedding With Gating Context Coding Perform fusion to generate context-enhanced positional codes. The fusion operation is represented as follows:

[0032] ,

[0033] in, This represents a function that combines location encoding with contextual information.

[0034] Furthermore, the gating unit adopts an extended RNN unit DRNN, the basic structure of which is represented as follows:

[0035] ,

[0036] in, This represents the input at the current time step, including time information and feature information; , These represent the hidden state and cell state at the current time step, respectively. , These represent the hidden state and cell state at the previous time step, respectively. , These represent the delayed states of the hidden state and the cell state, respectively;

[0037] The input is transformed by linear transformation Compared to the previous hidden state Delayed state Perform concatenation to obtain the input concatenated vector. If there is no delayed state, then a zero-initialized hidden state is used; the input connections are represented as follows:

[0038] ,

[0039] The forget gate controls the retention ratio of the previous state and updates the current cell state in conjunction with the new state; if a delayed state exists, the current state and the delayed state are weighted and combined, and the state update is represented as follows:

[0040] ,

[0041] in, Indicates the output of the forget gate; The candidate cell state representing the current time step is an encoding of the current information;

[0042] During the output calculation process, the output of the current time step is processed through the output gate. The relationship between the hidden state of the time step and the output gate is as follows:

[0043] ,

[0044] in, This indicates the output of the output gate.

[0045] Furthermore, the structure of the power load prediction model includes: an input layer, an embedding layer, an RNN stack layer, a linear mapping layer, and an output layer. The RNN stack layer includes multiple RNN blocks. A self-attention module is introduced after each RNN structure, and the self-attention module's ability to perceive the location structure is enhanced through context-enhanced positional encoding.

[0046] The output of the current time step in the RNN stack is dynamically weighted using a self-attention module to incorporate information from other time steps in the sequence.

[0047] Furthermore, during the training process of the power load forecasting model, a dynamic scaling gradient pruning method is adopted to control parameter updates by adaptively adjusting the pruning threshold.

[0048] Gradient after clipping The expression is as follows:

[0049] ,

[0050] ,

[0051] in, The gradient of the current parameters; k is a hyperparameter; It is a median function. It is a tiny positive number.

[0052] Compared with the prior art, the significant advantages of this invention are as follows:

[0053] 1. The context-enhanced location encoding designed in this invention breaks through the limitation of traditional location encoding that only marks the temporal order. By dynamically adapting the context, fusing multi-scale periodic features, and providing semantic guidance for self-attention, the location information is upgraded into a composite feature that includes temporal coordinates, semantic associations, and periodic attributes. This fundamentally improves the ability of power load forecasting models to model non-obvious location dependencies (such as cross-period associations and context-sensitive dependencies) in complex time series, laying the foundation for subsequent combination with extended recurrent neural networks to jointly capture long-term dependencies and local disturbances.

[0054] 2. This invention constructs a self-attention mechanism based on context-enhanced positional encoding, combined with a multi-scale temporal feature guidance strategy, which improves the model's ability to model positional dependencies in complex temporal structures; the self-attention mechanism is introduced into multiple layers of the expanded recurrent neural network, and periodic features such as hours, days, weeks, and months are integrated as inputs and encoding references, which effectively enhances the model's ability to jointly model long-term dependencies and local perturbations.

[0055] 3. In the multi-quantization objective function optimization stage, this invention introduces weighted pinball loss to construct a multi-quantile loss function, which effectively improves the prediction accuracy and stability under different confidence intervals, enhances the model's adaptability to extreme fluctuations and uncertainties in power load time series, and provides a more robust and generalizable modeling framework for practical applications, especially suitable for regional power system prediction tasks with high fluctuations and high uncertainties. Attached Figure Description

[0056] Figure 1 This is the overall flowchart of the present invention;

[0057] Figure 2 A schematic diagram of the context-enhanced positional encoding structure;

[0058] Figure 3 This is a schematic diagram of the power load prediction model of the present invention;

[0059] Figure 4 A comparison chart showing the electricity load forecast results for Belgium at different hourly intervals using different models;

[0060] Figure 5 A comparison chart showing the electricity load forecast results for Hungary at different hourly intervals using different models;

[0061] Figure 6 This is a graph showing the power load forecasting effect of the power load forecasting model of the present invention on Belgium;

[0062] Figure 7 This is a graph showing the power load forecasting effect of the power load forecasting model of this invention on Slovenia. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0064] like Figure 1 The diagram shows a flowchart of a power load forecasting method based on multi-scale context-enhanced self-attention according to the present invention, which mainly includes the following steps:

[0065] Step 1, Data Preprocessing;

[0066] In this embodiment, the original power load data of the study area is format-checked and missing value is processed. A continuous sample input is constructed through a sliding window to enhance the consistency of the sequence, with a duration of 168 hours.

[0067] Step 2: Extract multi-scale temporal features;

[0068] Based on the periodic structure of time series data, discrete time features at multiple time scales, including hourly, daily, weekly, monthly, and yearly, are extracted to capture the changing patterns of electricity load across different time scales. This process incorporates Holt exponential smoothing to remove and stabilize trend and seasonal factors, reducing the randomness and non-stationarity of the data.

[0069] Step 3: Use the context-enhanced location coding method to introduce multi-scale temporal features into the location representation;

[0070] This invention proposes a context-enhanced positional encoding method that combines multi-scale temporal information. It utilizes convolutional structures to extract neighborhood contextual features and incorporates these multi-scale temporal features into the positional representation generation process, enabling dynamic adjustment of the positional encoding and thus more accurately capturing the relative structure between positions in a time series. This context-enhanced positional encoding method not only captures local features but also effectively preserves the relative positional information of elements in the sequence, taking into account both absolute position and dynamic changes in the input data. The contextual features are centered on 24-hour (daytime) and 7-day (weektime) timeframes, referring to the positional attributes and relationships of the current moment within these two short-term time periods. For example, the original positional encoding only contains information such as the 1st hour, 2nd hour, etc., but with the addition of 24-hour and 7-day information, it includes information such as 8 AM on Monday, 12 PM on Wednesday, etc., helping the model find similar moments and enhancing its understanding of cyclical patterns.

[0071] This contextual feature is extracted through a one-dimensional convolution operation to reflect the state changes of the current time point in the local temporal environment. In this way, the model can not only perceive the location of a single point, but also identify the local dynamic connections between time points, thereby enhancing its ability to model short-term dependencies. This dynamic perception capability enables the encoding method to more realistically represent the changing features in complex temporal structures, thus effectively improving the model's performance in tasks involving predicting variable time series.

[0072] The process of incorporating multi-scale temporal features into the generation of location representations is as follows:

[0073] Step 31: By constructing a relative position distance matrix, the relative embeddings of all position pairs are obtained;

[0074] Let the length of the input sequence be... Construct a relative position distance matrix Each item in R Indicates position and Relative offset:

[0075] (1)

[0076] in, Represents the set of integers, representing Each element in the array is an integer.

[0077] Based on this, define a relative position embedding function. Mapping relative distance to 3D embedding vectors are used to capture the structural relationships between elements:

[0078] (2)

[0079] in, express The embedding results; For the relative embedding of all position pairs, Includes all , Represents a real number.

[0080] Step 32: Extract contextual features to obtain the context encoding matrix;

[0081] Unlike traditional fixed-position coding, context-enhanced positional coding further introduces a context-aware module based on one-dimensional convolution, thereby combining the local dynamic features of each position in the sequence. Let the features of the input sequence be... Let m be the dimension of the feature vector at each position in the sequence. Then, context features are extracted through a one-dimensional convolution operation with a kernel size of n.

[0082] (3)

[0083] in, This represents the local context features extracted from position i in the sequence after a one-dimensional convolution; Conv1D() represents the one-dimensional convolution operation.

[0084] all Arranged according to sequence position, the context encoding matrix is ​​obtained. The context encoding matrix C is used to represent the information of the local neighborhood at each location.

[0085] Step 33: Obtain the gating context representation;

[0086] Figure 2 In this context, y1, y2, y3… represent descriptive labels for different contextual information, including hourly information and the day within the week. Define a mapping function. Used to generate the corresponding context vector from the tag. :

[0087] (4)

[0088] In addition, to flexibly control whether to introduce contextual representations at each position, a learnable gating unit is introduced. And through the Sigmoid activation function Obtain the context gate for each position. :

[0089] (5)

[0090] in, For the learnable weight matrix used for linear transformation, Represents the input vector Feature dimensions, For learnable parameters, For the first An input vector at each position.

[0091] Context gate value output by the gating unit This is used to control the degree of contextual involvement at position i, thereby obtaining a gated context representation:

[0092] (6)

[0093] in, This indicates the gating context encoding.

[0094] To address long sequence learning tasks, this embodiment introduces a novel gating unit—the Dilated Recurrent Neural Network (DRNN). Compared to traditional LSTM and GRU, DRNN combines the input, the previous hidden state, and the delayed state, employing a simple gating mechanism to dynamically update and regulate the cell state. Through forgetting and output gate regulation, DRNN achieves fine-grained management of the cell state. Compared to the complex structures of LSTM and GRU, DRNN offers greater flexibility and fewer parameters, improving training efficiency and effectively capturing long-term dependencies.

[0095] The basic structure of DRNN is shown in equation (7):

[0096] (7)

[0097] in This represents the input at the current time step (including time information and feature information). , These represent the hidden state and cell state at the current time step, respectively. , These represent the hidden state and cell state at the previous time step, respectively. , These represent the delayed states of the hidden state and the cell state, respectively. At each time step, the DRNN uses a gating mechanism to determine how to combine these states to compute the new output and state.

[0098] The input is transformed by linear transformation Compared to the previous hidden state and delayed state Perform concatenation to obtain the input concatenated vector. If there is no delayed state, then a zero-initialized hidden state is used; the input connection is as shown in equation (8):

[0099] (8)

[0100] The forget gate is used to control the retention ratio of the previous state and to update the current cell state in combination with the new state; if a delayed state exists, the current state and the delayed state are weighted and combined; the state update is shown in equation (9):

[0101] (9)

[0102] in, The output of the forget gate is a vector between 0 and 1. The candidate cell state representing the current time step is an encoding of the current information.

[0103] Output gate output It is a vector of 0s and 1s, which is output through the gate to the current time step. The output is processed to ensure effective information transmission. The relationship between the hidden state of the time step and the output gate is as follows: .

[0104] Step 34: Generate context-enhanced location codes;

[0105] relative embedding With Gating Context Coding Perform fusion to generate context-enhanced positional codes. The fusion operation can be performed using vector addition or concatenation, etc.

[0106] (10)

[0107] in, This represents a function that combines location encoding with contextual information.

[0108] Step 4: Construct a power load forecasting model, train the power load forecasting model using a training set, and save the best power load forecasting model.

[0109] like Figure 3 The diagram illustrates a power load forecasting model that integrates a deep recurrent neural network (DRNN) with a multi-layer self-attention module. After each RNN layer, a self-attention mechanism is introduced to model long-range dependencies, and context-enhanced positional encoding strengthens the self-attention module's ability to perceive positional structures. This context-enhanced positional encoding integrates multi-scale temporal features and local contextual information, effectively improving the self-attention mechanism's ability to model complex temporal patterns, thereby enhancing the overall temporal representation capability and prediction accuracy of the power load forecasting model.

[0110] By using progressively layered RNN blocks with expansion factors (i.e., inflation coefficients) of 2, 4, and 7, the temporal features of the input are processed layer by layer to capture dependencies at different time scales. A custom-defined gated unit (DRNN) is used as the core unit to ensure that the model fully exploits the temporal correlations of the feature sequences. The inflation mechanism effectively reduces information redundancy and improves the ability to capture long sequences. A self-attention module dynamically weights the output of the RNN block at the current time step to incorporate information from other time steps in the sequence. This mechanism effectively captures long-range dependencies and contextual relationships, enhances the representational power of the output information, and maintains the continuity and consistency between time steps.

[0111] The self-attention module processes data as follows:

[0112] Step 41: First, the query key is used to calculate the original association score s1 / s2 / s3 with each key using F(Q,K);

[0113] Step 42: Normalize the original association scores s1, s2, and s3 using the SoftMax function to obtain attention weights a1, a2, and a3.

[0114] Step 43: Multiply the attention weights a1 and Value1, a2 and Value2, and a3 and Value3, and then sum them to obtain preliminary attention feature information;

[0115] Step 44: The initial attention feature information is processed using the SoftMax function, and the output is a probability distribution vector.

[0116] The weighted features processed by the self-attention module are incorporated into subsequent layers of the model, allowing the model to focus on key information and enhance the expression of important content.

[0117] like Figure 3 As shown, the power load forecasting model includes an input layer, an embedding layer, an RNN stack layer, a linear mapping layer, an output layer, a self-attention mechanism layer, and a context-enhanced positional encoding layer.

[0118] Input layer: Receives and preprocesses historical load data and time characteristics to ensure data validity and consistency, and standardizes it to a uniform scale.

[0119] Embedding layer: Maps features to a four-dimensional space to capture the relationships between features, enhances the model's understanding of time series data, and provides rich semantic representations for subsequent prediction tasks.

[0120] The self-attention mechanism layer dynamically adjusts the attention weights at different positions in the input sequence, enhancing the model's focus on key information. This allows the model to autonomously select the input most relevant to the current time step, effectively capturing long-distance dependencies. This mechanism flexibly integrates information from different time steps, improving prediction accuracy and overall performance.

[0121] Context-enhanced positional encoding layer: This layer incorporates positional information into the input data by encoding the relative position of each time step. This helps the model understand the order and structure of the data, thereby better capturing dependencies in time series. Especially when dealing with long series, positional encoding provides the necessary contextual information, optimizing the model's ability to identify time dependencies and predict time series data.

[0122] RNN Stack Layers: A progressive RNN layer with expansion factors of 2, 4, and 7 processes the temporal features of the input layer by layer to capture dependencies at different time scales. A custom-gated DRNN is used as the core unit to ensure the model fully exploits the temporal correlations of the feature sequences. The expansion mechanism effectively reduces information redundancy and improves the ability to capture long sequences.

[0123] Linear mapping layer: This layer transforms high-dimensional features into the target output dimension, ensuring the output meets the requirements of subsequent tasks. It achieves dimensionality reduction of temporal features while preserving key information for further prediction.

[0124] Output layer: Maps the high-dimensional features processed by the RNN stack to predicted values ​​and state parameters of future loads. Quantization of forecasts helps capture uncertainties in load forecasting, combining historical loads, time features, and future forecast windows to ensure the accuracy and reliability of the forecasts.

[0125] Figure 3 In the middle, x1,x2,x3,…,x n It represents 168 hours of time-series data and includes information such as load value, features extracted from the time series (binary code of which hour of the day, binary code of which day of the week), etc.

[0126] The power load forecasting model was trained using a dataset from the European Electricity Transmission System Operator (ENTSO-E), covering hourly power demand data for 35 European countries from 2006 to 2018. The dataset from 2006 to 2017 served as the training set, and 2018 as the test set. This dataset is a univariate time series dataset with hourly intervals.

[0127] An adaptive gradient clipping strategy is introduced to dynamically adjust the gradient norm, preventing gradient explosion and improving training stability and optimization efficiency. Gradient explosion often leads to model non-convergence or unstable convergence during the training of complex deep learning models. Therefore, this embodiment employs a dynamic scaling gradient clipping method, adaptively adjusting the clipping threshold to effectively control parameter updates, thereby improving training stability and efficiency.

[0128] Set the gradient of the current parameter as Define the threshold for clipping. As shown in equation (11):

[0129] (11)

[0130] Where k is a hyperparameter, which is a small constant; It is the median function, which calculates the median of the input data; It is a tiny positive number used to ensure numerical smoothness.

[0131] The gradient after clipping is shown in equation (12):

[0132] (12)

[0133] in, This represents the gradient after clipping.

[0134] The introduction of dynamic scaling and pruning enhances the stability of power load forecasting model training. By adaptively adjusting the pruning threshold, it effectively prevents gradient explosion and reduces fluctuations during training. This dynamic scaling gradient pruning method accelerates the convergence speed of the power load forecasting model while maintaining a reasonable gradient update range, helping the model better capture the inherent patterns of complex time-series data.

[0135] During model training, in the multi-quantization objective function optimization stage, this invention introduces weighted pinball loss to construct a multi-quantile loss function, which effectively improves the prediction accuracy and stability under different confidence intervals, enhances the model's adaptability to extreme fluctuations and uncertainties in power load time series, and provides a more robust and generalizable modeling framework for practical applications, especially suitable for regional power system prediction tasks with high fluctuations and high uncertainties.

[0136] Step 5: Based on the constructed power load forecasting model, complete the forecasting of power load;

[0137] Figure 4 , Figure 5The paper presents a comparison of the electricity load prediction results for Belgium (BE) and Hungary (HU) in different hourly segments using different models. It also shows the changes in predicted electricity load over time for each country. The different curves represent the electricity load prediction model of this invention, ES_drm, CNN, LSTM, GAN, and the actual electricity load value, respectively. These two countries represent different electricity load scales.

[0138] In summary, the power load forecasting model of this invention demonstrates superior forecasting advantages under various power load scales, exhibiting high fitting accuracy and fluctuation capture capabilities, particularly during peak power load periods and phases of rapid power load changes. This indicates that the model possesses strong generalization ability, can adapt to the forecasting needs of power loads under different scales and fluctuation patterns, and has high application value.

[0139] Figure 6 , Figure 7 The power load forecasting model of this invention is shown to predict the power load in Belgium (BE) and Slovenia (SI). The figures show the comparison between the forecast interval and the actual power load data. The data is in hourly intervals and covers a two-week time span to verify the predictive performance of the model under different countries and power load levels.

[0140] Depend on Figure 6 , Figure 7 The effectiveness of the power load forecasting model of this invention under different countries and power load levels is demonstrated. The forecast interval generated by the power load forecasting model of this invention not only accurately reflects the fluctuation trend of actual power load data, but also exhibits a reasonable interval width in various power load change scenarios, effectively balancing forecast accuracy and uncertainty quantification. These results fully verify that the method proposed in this invention has strong adaptability and robustness in power load forecasting tasks, providing a reliable basis for high-precision power load forecasting. Table 1 shows the comparison results of the forecasting indicators of the power load forecasting model of this invention with other models.

[0141] Table 1. Comparison of Predictive Indicators between the Method of the Present Invention and Other Methods

[0142]

[0143] As shown in Table 1, compared with ES_drnn, the power load forecasting model of this invention reduces RMSE by approximately 11.97%, MAPE by approximately 9.84%, and MASE by approximately 11.01%. Furthermore, the power load forecasting model of this invention also demonstrates significant advantages in comparison with other models: compared with GAN, the power load forecasting model of this invention reduces RMSE by approximately 49.01%, MAPE by approximately 45.95%, and MASE by approximately 47.08%; compared with CNN, RMSE is reduced by approximately 33.01%, MAPE by approximately 29.94%, and MASE by approximately 25.96%; and compared with LSTM, RMSE is reduced by approximately 24.00%, MAPE by approximately 20.86%, and MASE by approximately 22.01%.

[0144] Furthermore, as a crucial indicator for measuring model prediction bias, the power load forecasting model of this invention exhibits minimal bias in this aspect. These results further validate the effectiveness of this invention in improving prediction accuracy.

Claims

1. A power load forecasting method based on multi-scale context-enhanced self-attention, characterized in that, Includes the following steps: S1, preprocesses the raw power load data; S2, based on the periodic structure of time series, extracts discrete time features including four time scales: hour, day, week and month, to capture the changing patterns of power load at different time scales; S3 employs a context-enhanced location coding method to introduce multi-scale temporal features into the location representation; The implementation steps are as follows: S31, construct the relative position distance matrix Each term in R Indicates position and Relative offset: ， in, Indicates the length of the input sequence. Represents the set of integers; Define a relative position embedding function Map the relative offset to 3D embedding vectors are used to capture the structural relationships between elements: ， in, express The embedding results For relative embedding at all positions, Includes all , Represent real numbers; S32, using a convolution kernel size of One-dimensional convolution operations extract contextual features: ， in, Represents the position in the sequence Local contextual features extracted by one-dimensional convolution; For the input sequence features, Let be the dimension of the feature vector at each position in the sequence; Conv1D() represents a one-dimensional convolution operation; all Arranged according to sequence position, the context encoding matrix is ​​obtained. The context encoding matrix C is used to represent the information of the local neighborhood at each location. S33, Define a mapping function Used to generate the corresponding context vector from the tag. : ， Introduce a learnable gating unit And activated by the Sigmoid function. Obtain the context gate for each position. , means as follows: ， in, , Represents the input vector Feature dimensions, For learnable parameters, For the first Input vectors at each position; Output using gating unit By controlling the degree of involvement of the control context at position i, the gating context representation is obtained as follows: ， in, Indicates the gating context encoding; S34, relative embedding With Gating Context Coding Perform fusion to generate context-enhanced positional codes. The fusion operation is represented as follows: ， in, This represents a function that fuses location encoding with contextual information. S4. Construct an electricity load prediction model and train the model using a training set, and save the best electricity load prediction model. The structure of the electricity load prediction model includes: an input layer, an embedding layer, an RNN stack layer, a linear mapping layer, and an output layer. The RNN stack layer includes multiple RNN blocks. A self-attention module is introduced after each RNN structure, and the self-attention module's ability to perceive the position structure is enhanced through context-enhanced position encoding. Among them, the output of the current time step of the RNN stack layer is dynamically weighted using a self-attention module to incorporate information from other time steps in the sequence; S5, based on the optimal power load forecasting model, completes the forecasting of power load.

2. The power load forecasting method based on multi-scale context-enhanced self-attention as described in claim 1, characterized in that, The gating unit uses an extended RNN unit DRNN, and the basic structure of DRNN is as follows: ， in, This represents the input at the current time step, including time information and feature information; , These represent the hidden state and cell state at the current time step, respectively. , These represent the hidden state and cell state at the previous time step, respectively. , These represent the delayed states of the hidden state and the cell state, respectively; The input is transformed by linear transformation Compared to the previous hidden state Delayed state Perform concatenation to obtain the input concatenated vector. If there is no delayed state, then a zero-initialized hidden state is used; the input connections are represented as follows: ， The forget gate controls the retention ratio of the previous state and updates the current cell state in conjunction with the new state; if a delayed state exists, the current state and the delayed state are weighted and combined, and the state update is represented as follows: ， in, Indicates the output of the forget gate; The candidate cell state representing the current time step is an encoding of the current information; During the output calculation process, the output of the current time step is processed through the output gate. The relationship between the hidden state of the time step and the output gate is as follows: ， in, This indicates the output of the output gate.

3. The power load forecasting method based on multi-scale context-enhanced self-attention as described in claim 1, characterized in that, During the training process of the power load forecasting model, a dynamic scaling gradient pruning method is adopted to control parameter updates by adaptively adjusting the pruning threshold. Clipped gradient The expression is as follows: ， ， in, The gradient of the current parameters; It is a hyperparameter; It is a median function. It is a tiny positive number.

Images