Mamba-based endogenous and exogenous variable fusion power load prediction method

Abstract

The present application belongs to the technical field of power load prediction, and particularly relates to a power load prediction method based on Mamba and fusing endogenous and exogenous variables, comprising the following steps: data set acquisition and preprocessing; endogenous variable embedding; dividing the endogenous sequence of power load into N non-overlapping Patch blocks, adding position coding to each block, and then generating block tokens P of sub-block fine granularity through linear projection and position coding en , and introducing learnable global tokens G en to represent the macro state of the sequence; global embedding of exogenous variables; cross-granularity fusion of endogenous and exogenous tokens; Mamba time series dependency modeling; multi-step prediction output, mapping the time series features to future multi-step load prediction values through a fully connected layer, restoring the original data scale after inverse normalization, and outputting the final prediction result. Through the block embedding strategy and dynamic parameter selection mechanism, the adaptability of the model to complex power scenarios is significantly improved, and reliable technical support is provided for efficient dispatching and market operation of smart grids.

Description

Technical Field

[0001] This invention belongs to the field of power load forecasting technology, specifically relating to a power load forecasting method based on the fusion of endogenous and exogenous variables using Mamba. Background Technology

[0002] As the global energy structure transitions towards cleaner and smarter energy, power load forecasting has become a core technological support for smart grid dispatching, demand-side management, and electricity market transactions. Accurate load forecasting requires a comprehensive understanding of the complex interactions between historical load data (endogenous variables) and external factors such as weather, electricity prices, and renewable energy output (exogenous variables). However, existing technologies face significant bottlenecks in model architecture, computational efficiency, and multi-source data fusion, which limit the accuracy and practicality of forecasts.

[0003] Early load forecasting primarily relied on statistical models such as the Autoregressive Integral Moving Average (ARIMA) model, which models time-series trends through linear assumptions but cannot capture nonlinear fluctuations (such as load surges during holidays). Recurrent Neural Networks (RNNs) and their variants LSTM and GRU alleviated the vanishing gradient problem through gating mechanisms, making progress in short-term load forecasting. However, the serial computation characteristic of LSTM leads to slow training speeds and insufficient ability to model long-term dependencies exceeding 24 hours. The Transformer model, with its self-attention mechanism, overcomes the temporal constraints of sequence modeling and shows potential in long-sequence forecasting. However, the high complexity caused by the attention mechanism makes it difficult to meet the needs of practical power grid applications. Subsequently, the Structured State Space Model (S4) achieves sequence modeling with linear complexity O(T) through hidden state propagation, significantly outperforming the Transformer in long-sequence scenarios. However, S4 cannot dynamically enhance feature weights, resulting in low prediction efficiency. The Mamba model, proposed in 2023, introduces a selective scanning mechanism that can adjust SSM parameters based on the input content, but still lacks cross-dimensional feature selection capabilities in multivariate scenarios.

[0004] Existing fusion methods often rely on shallow attention or simple splicing, making it difficult to distinguish between the direct effects and indirect confounding factors of exogenous variables. For example, the Tide model developed by Tencent AI Lab inputs temperature, humidity, and load data into a Transformer in parallel, but it cannot identify the direct driving effect (causal relationship) of high temperature on air conditioning load and the indirect effect (confounding factor) of high temperature simultaneously inhibiting industrial electricity consumption, resulting in systematic bias in summer data from the Guangdong power grid. Summary of the Invention

[0005] To address the technical problem that existing fusion methods often rely on shallow attention or simple splicing, making it difficult to distinguish between the direct effects and indirect confounding of exogenous variables, this invention provides a power load forecasting method based on Mamba that fuses exogenous and exogenous variables. Through block embedding, global token bridging, bidirectional cross-attention, and dynamic parameter selection, it achieves efficient, accurate, and interpretable power load forecasting, providing key technical support for new power systems.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A power load forecasting method based on Mamba, which integrates endogenous and exogenous variables, includes the following steps:

[0008] S1. Dataset collection and preprocessing: obtain endogenous and exogenous variables, fill in missing values, normalize, generate standardized time series tensors, and divide the dataset into training, validation, and test sets.

[0009] S2. Endogenous variable embedding: The endogenous sequence of power load is divided into N non-overlapping patches. After adding a position code to each patch, a fine-grained block token P is generated through linear projection and position coding. en And introduce a learnable global token G en Characterizes the macroscopic state of the sequence;

[0010] S3. Global embedding of exogenous variables: directly mapping each exogenous sequence to a single global token V. ex To avoid fine-grained segmentation and reduce noise interference;

[0011] S4. Cross-granularity fusion of endogenous and exogenous tokens, combining the block tokens P of the endogenous variables obtained above. en Global Token G en and the global token V of exogenous variables ex Three types of attention are employed: inter-block self-attention (block-level tokens capture local temporal patterns through self-attention) and global-block bidirectional attention (global tokens). Integrating macroscopic and local information with block-level tokens: using endogenous global tokens as queries and exogenous tokens as keys and values ​​for cross-attention interaction; achieving cross-granularity fusion of endogenous and exogenous variables;

[0012] S5 and Mamba temporal dependency modeling maps fused features to the latent space through a linear layer, iteratively updates the latent state based on the discretized state space model, and accelerates the extraction of long sequence temporal dependency features through global convolution.

[0013] S6. Multi-step prediction output: The time series features are mapped to future multi-step load prediction values ​​through a fully connected layer. After inverse normalization, the original data scale is restored, and the final prediction result is output.

[0014] The method for data collection and preprocessing in S1 is as follows:

[0015] First, define historical electricity load data as an endogenous variable L = {l1, l2, ..., l...} T}, and obtain the time-aligned exogenous variables as E = {e1, e2, ..., e T} where T is the length of the time series; for missing values ​​in the data, linear interpolation is used to fill in the missing values ​​to obtain the complete endogenous variable sequence. Exogenous variable sequence Next, the padded sequences are standardized: for endogenous variables: Where μ L and σ L They are respectively The mean and standard deviation; for exogenous variables: Where μ E and σ E They are respectively The mean and standard deviation of the endogenous variable L were then calculated; subsequently, the standardized endogenous variable L was... norm With exogenous variable E norm Concatenate into a unified temporal tensor Finally, the training set, validation set, and test set were divided in a ratio of 7:1:2.

[0016] The method for embedding endogenous variables in S2 is as follows:

[0017] The endogenous power load sequence L of length T obtained from S1 norm Divided into A non-overlapping block {s1,s2,…,s} N}, where P is the block length; subsequently, for each block s i Add position encoding And mapped to fine-grained time tokens through a linear projection layer. To avoid information granularity mismatch caused by the direct fusion of endogenous variable block-level features and exogenous variable macro-level features, an additional learnable global token G is generated for the endogenous sequence. en =Learnable(x), which represents the overall macroscopic state of the sequence; it transmits the causal information of exogenous variables to the fine-grained features of endogenous blocks, realizing cross-granularity information alignment and fusion.

[0018] The method for global embedding of exogenous variables in S3 is as follows:

[0019] Each exogenous sequence Directly mapped to a single global token V via the variable embedding module. ex,i =VariateEmbed(z (i)), i∈{1,…,C},T ex The length of the exogenous sequence backtracking, where VaritaeEmbed: For linear projection layers; ultimately, the global token set B of all exogenous variables. ex ={V ex,1 ,…,V ex,C Block tokens with endogenous variables and global token Common input model.

[0020] The method for cross-granularity fusion of endogenous and exogenous tokens in S4 is as follows:

[0021] S4-1. First, perform inter-block attention on the l-th block-level tokens. By capturing local temporal patterns between blocks through self-attention, the following update is made:

[0022] S4-2, using a bidirectional attention mechanism between global and block and block and global variables, with endogenous global variable tokens. With block tokens Perform cross-attention interaction and update to Integrate local information between blocks with global context.

[0023] Simultaneously capture inter-block dependencies and global-local associations;

[0024] S4-3. Perform cross-attention between exogenous and endogenous variables to generate an endogenous global token. For query Query, exogenous variable token V ex For keys and values, cross attention is used. Integrate cross-variable information and update the global token.

[0025] S4-4, Updated Global Token The input feedforward layer further extracts features through nonlinear transformations, generating the input for the next layer and extracting features from endogenous and exogenous variables.

[0026] The method for modeling Mamba time-series dependencies in S5 is as follows:

[0027] First, the time series X = [X1, X2, ..., x] obtained from S4 is processed through a linear embedding layer. T Mapping to the latent space generates a hidden state sequence H0 = W e X+b e , where b eThe parameters are then learned; discretized state propagation is then performed, and based on the zero-order preservation rule, the quotient of the continuous SSM parameters {A,b,C} is discretized into... The latent state h is updated iteratively according to time steps. t : Finally, global convolution is performed for acceleration: to efficiently compute long sequence outputs, convolution kernels are constructed. And the final temporal features are generated through convolution operations:

[0028] The prediction method in S6 is evaluated using mean absolute error (MAE), root mean square error (RMSE), symmetric mean percentage error (SMAPE), and correlation coefficient.

[0029] Compared with the prior art, the beneficial effects of this invention are:

[0030] This invention proposes an innovative Mamba-based power load forecasting method that significantly improves prediction accuracy and computational efficiency through simultaneous extraction of variable and temporal features and multi-granularity cross-modal fusion. First, without altering the core Transformer architecture, historical power load sequences are segmented into fine-grained time blocks. Time tokens are generated through causal convolutional embedding and positional encoding. Simultaneously, the global sequences of exogenous variables (such as weather and electricity prices) are independently mapped to coarse-grained variable tokens, achieving multi-scale feature representation. Second, in the feature extraction module, local fluctuations, long-term trends, and the dynamic influence of exogenous variables in the load sequence are captured hierarchically through time block self-attention and cross-variable cross-attention mechanisms. Combined with Mamba's linear complexity state-space modeling, key temporal dependencies are adaptively selected. Finally, cross-granularity information is bridged through global tokens, and future multi-step load forecasts are output via linear projection. This invention significantly improves the model's adaptability to complex power scenarios through a segmented embedding strategy and dynamic parameter selection mechanism, providing reliable technical support for the efficient scheduling and market-oriented operation of smart grids. Attached Figure Description

[0031] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0032] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0033] Figure 1 This is a schematic diagram of the prediction method of the present invention;

[0034] Figure 2 This is a block diagram of the present invention;

[0035] Figure 3 This is a structural diagram of the Mamba of the present invention;

[0036] Figure 4 This is a diagram of the attention mechanism of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. These descriptions are only for further illustrating the features and advantages of the present invention, and not for limiting the claims of the present invention. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0039] The Mamba-based power load forecasting method based on the fusion of endogenous and exogenous variables in this invention, as described in the embodiments of the present invention, is as follows: Figure 1-4 As shown, the process includes:

[0040] Step 1: Dataset collection and preprocessing. Obtain endogenous variables (historical power load values) and exogenous variables (electricity prices, temperature, wind power generation values, photovoltaic power generation values, etc.), perform missing value imputation and normalization processing, generate standardized time series tensors, and divide them into training sets, validation sets, and test sets.

[0041] In step 1, data collection and preprocessing: First, historical power load data is defined as an endogenous variable L = {l1, l2, ..., l...} T}, and obtain the time-aligned exogenous variables as E = {e1, e2, ..., e T} where T is the length of the time series. For missing values ​​in the data, linear interpolation is used to fill in the missing values, resulting in a complete sequence of endogenous variables. Exogenous variable sequence Next, the padded sequences are standardized: for endogenous variables: Where μ L and σ L They are respectively The mean and standard deviation. For exogenous variables: Where μ E and σ E They are respectively The mean and standard deviation of the endogenous variable L were then calculated. norm With exogenous variable E norm Concatenate into a unified temporal tensor Finally, the training set, validation set, and test set were divided in a ratio of 7:1:2.

[0042] Step 2, Endogenous Variable Embedding: The endogenous sequence of power load is divided into N non-overlapping patch blocks. After adding position encoding to each block, fine-grained block tokens P are generated through linear projection and position encoding. en And introduce a learnable global token G en It represents the macroscopic state of the sequence.

[0043] In step 2, the endogenous power load sequence L of length T obtained in step 1 is... norm Divided into A non-overlapping block {s1,s2,…,s} N}, where P is the block length; subsequently, for each block s i Add position encoding And mapped to fine-grained time tokens through a linear projection layer. To avoid information granularity mismatch caused by the direct fusion of endogenous variable block-level features and exogenous variable macro-level features, an additional learnable global token G is generated for the endogenous sequence. en =Learnable(x), representing the overall macroscopic state of the sequence; it transmits the causal information of exogenous variables (such as electricity prices and meteorological data) to endogenous block-level fine-grained features, realizing cross-granularity information alignment and fusion.

[0044] Step 3: Global embedding of exogenous variables, directly mapping each exogenous sequence (such as temperature, wind power generation value) to a single global token V. exTo avoid fine-grained segmentation and reduce noise interference, since exogenous variables (such as electricity prices and meteorological data) assist in the prediction of endogenous variables through cross-variable interactions, using the same fine-grained embedding as the endogenous variables would introduce noise and increase computational redundancy. Therefore, this embodiment segments each exogenous sequence... T ex The exogenous sequence backtracking length is directly mapped to a single global token V through the variable embedding module. ex,i =VariateEmbed(z (i) ), where Varitae Embed: For linear projection layers; ultimately, the global token set V of all exogenous variables. ex ={V ex,1 ,…,V ex,C Block tokens with endogenous variables and global token Common input model.

[0045] Step 4: Merge endogenous and exogenous tokens across granularity, combining the block token P of the endogenous variables obtained above. en Global Token G en and the global token V of exogenous variables ex Three types of attention are employed: inter-block self-attention (block-level tokens capture local temporal patterns through self-attention) and global-block bidirectional attention (global tokens). Integrating macroscopic and local information with block-level tokens: Cross-attention is achieved by using the endogenous global token as the query and the exogenous token as the key and value. This enables cross-granularity fusion of endogenous and exogenous variables.

[0046] Step 4-1: First, perform inter-block attention on the l-th block-level tokens. By capturing local temporal patterns between blocks through self-attention, the following update is made:

[0047] Step 4-2: Using a bidirectional attention mechanism between global and block and block and global variables, an endogenous global variable token is generated. With block tokens Perform cross-attention interaction and update to It integrates local information between blocks with global context, while capturing inter-block dependencies and global-local associations.

[0048] Step 4-3: Perform cross-attention between exogenous and endogenous variables to generate an endogenous global token. For query Query, exogenous variable token V ex For keys and values, cross attention is used. Integrate cross-variable information and update the global token.

[0049] Step 4-4: Updated Global Token The input feedforward layer further extracts features through nonlinear transformations, generating the input for the next layer and extracting features from endogenous and exogenous variables.

[0050] Step 5: Mamba temporal dependency modeling. The fused features are mapped to the latent space via a linear layer. The latent states are iteratively updated based on the discretized state space model, and global convolution accelerates the extraction of long-sequence temporal dependency features. First, the time series X = [x1, x2, ..., x...] obtained in Step 4 is processed through a linear embedding layer. T Mapping to the latent space generates a hidden state sequence H0 = W e X+b e , where b e These are learnable parameters. Then, discretized state propagation is performed, and based on the zeroth-order preservation rule, the quotients of the continuous SSM parameters {A,B,C} are discretized into... The latent state h is updated iteratively according to time steps. t : Finally, global convolution is performed for acceleration: to efficiently compute long sequence outputs, convolution kernels are constructed. And the final temporal features are generated through convolution operations:

[0051] Step 6: Multi-step prediction output. The time series features are mapped to future multi-step load prediction values ​​through a fully connected layer. After inverse normalization, the original data scale is restored, and the final prediction result is output.

[0052] The loss function used is cross-entropy loss, and the formula is:

[0053]

[0054] The method is implemented using PyTorch and trained using the Adam optimizer. After multiple experiments and parameter adjustments, this invention sets the learning rate to 0.0001, the batch size to 32, and each experiment to run for 100 cycles, allowing early stopping when the results are good.

[0055] This embodiment significantly improves prediction accuracy and computational efficiency by simultaneously extracting variable and temporal features and fusing multi-granularity cross-modal data. First, without altering the core Transformer architecture, historical power load sequences are segmented into fine-grained time blocks. Time tokens are generated through causal convolutional embedding and positional encoding. Simultaneously, global sequences of exogenous variables (such as weather and electricity prices) are independently mapped to coarse-grained variable tokens, achieving multi-scale feature representation. Second, in the feature extraction module, local fluctuations, long-term trends, and the dynamic impact of exogenous variables in the load sequence are captured hierarchically through time block self-attention and cross-variable cross-attention mechanisms. Combined with Mamba's linear complexity state-space modeling, key temporal dependencies are adaptively selected. Finally, cross-granularity information is bridged through global tokens, and future multi-step load predictions are output via linear projection. This embodiment significantly improves the model's adaptability to complex power scenarios through a segmented embedding strategy and dynamic parameter selection mechanism, providing reliable technical support for the efficient scheduling and market-oriented operation of smart grids.

[0056] The above description only illustrates the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention, and all such changes should be included within the protection scope of the present invention.

Claims

1. A power load forecasting method based on Mamba, which integrates endogenous and exogenous variables, characterized in that, Includes the following steps: S1. Dataset collection and preprocessing: obtain endogenous and exogenous variables, fill in missing values, normalize, generate standardized time series tensors, and divide the dataset into training, validation, and test sets. S2. Endogenous variable embedding: The endogenous sequence of power load is divided into N non-overlapping patch blocks. After adding position encoding to each block, fine-grained block tokens are generated through linear projection and position encoding. And introduce learnable global tokens Characterizes the macroscopic state of the sequence; S3. Global embedding of exogenous variables: directly mapping each exogenous sequence to a global token of the exogenous variable. To avoid fine-grained segmentation and reduce noise interference; S4. Cross-granularity fusion of endogenous and exogenous tokens: This involves fusing the block tokens of the endogenous variables obtained above. Global token and global tokens of exogenous variables Three types of attention are employed: inter-block self-attention (block-level tokens capture local temporal patterns through self-attention) and global-block bidirectional attention (global tokens). Integrating macroscopic and local information with block-level tokens: using the endogenous global token as the query and the exogenous token as the key and value for cross-attention interaction; Complete cross-granularity fusion of endogenous and exogenous variables; S5 and Mamba temporal dependency modeling maps fused features to the latent space through a linear layer, iteratively updates the latent state based on the discretized state space model, and accelerates the extraction of long sequence temporal dependency features through global convolution. S6. Multi-step prediction output: The time series features are mapped to future multi-step load prediction values ​​through a fully connected layer. After inverse normalization, the original data scale is restored, and the final prediction result is output.

2. The power load forecasting method based on Mamba and its fusion of endogenous and exogenous variables as described in claim 1, characterized in that, The method for data collection and preprocessing in S1 is as follows: First, we define historical electricity load data as an endogenous variable. And obtain the time-aligned exogenous variables as ,in The time series length is given; for missing values ​​in the data, linear interpolation is used to fill in the gaps, resulting in a complete endogenous variable sequence. exogenous variable sequence Next, the padded sequences are standardized: for endogenous variables: ,in and They are respectively The mean and standard deviation; for exogenous variables: ,in and They are respectively The mean and standard deviation of the endogenous variables were then calculated; subsequently, the standardized endogenous variables were... With exogenous variables Concatenate into a unified temporal tensor Finally, the training set, validation set, and test set are divided in a ratio of 7:1:

2.

3. The Mamba-based method for fusing endogenous and exogenous variables in power load forecasting according to claim 1, characterized in that, The method for embedding endogenous variables in S2 is as follows: The length obtained from S1 is Endogenous sequence of power load Divided into Non-overlapping blocks ,in The length of the block; subsequently, for each block Add position encoding And mapped to fine-grained time tokens through a linear projection layer. To avoid information granularity mismatch caused by the direct fusion of endogenous variable block-level features and exogenous variable macro-level features, learnable global tokens are additionally generated for the endogenous sequence. It represents the overall macroscopic state of the sequence; it transmits the causal information of exogenous variables to the fine-grained features of endogenous blocks, realizing cross-granularity information alignment and fusion.

4. The Mamba-based method for fusing endogenous and exogenous variables in power load forecasting according to claim 1, characterized in that, The method for global embedding of exogenous variables in S3 is as follows: Each exogenous sequence Directly mapped to a single global token via the variable embedding module. , The length of the exogenous sequence backtracking, where VaritaeEmbed: For linear projection layers; ultimately, the global token set of all exogenous variables. Block tokens with endogenous variables and global token Common input model.

5. The Mamba-based method for fusing endogenous and exogenous variables in power load forecasting according to claim 1, characterized in that, The method for cross-granularity fusion of endogenous and exogenous tokens in S4 is as follows: S4-1, First, perform inter-block attention, for the first... Block-level tokens By capturing local temporal patterns between blocks through self-attention, the following update is made: ; S4-2, using a bidirectional attention mechanism between global and block and block and global variables, with endogenous global variable tokens. With block tokens Perform cross-attention interaction and update to Integrate local information between blocks with global context, while capturing inter-block dependencies and global-local associations; S4-3. Perform cross-attention between exogenous and endogenous variables to generate an endogenous global token. For querying the global token of the exogenous variable. For keys and values, cross attention is used. Integrate cross-variable information and update the global token. ; S4-4, Updated Global Token The input feedforward layer further extracts features through nonlinear transformations, generating the input for the next layer and extracting features from endogenous and exogenous variables.

6. The Mamba-based method for fusing endogenous and exogenous variables in power load forecasting according to claim 1, characterized in that, The method for modeling Mamba time-series dependencies in S5 is as follows: First, the time series obtained from S4 is processed through a linear embedding layer. Mapping to the latent space generates a sequence of latent states. ,in The parameters are then learned; discretized state propagation is then performed, and based on the zero-order preservation rule, the continuous SSM parameters are... Discretize into Iterate and update the latent state according to time steps. : Finally, global convolution is performed to accelerate the process: to efficiently compute long sequence outputs, convolution kernels are constructed. And the final temporal features are generated through convolution operations: .

7. The Mamba-based method for fusing endogenous and exogenous variables in power load forecasting according to claim 1, characterized in that, The prediction method in S6 is evaluated using mean absolute error (MAE), root mean square error (RMSE), symmetric mean percentage error (SMAPE), and correlation coefficient.

Images