Mama-based endogenous and endogenous variable fusion power load prediction method

Abstract

The invention belongs to the technical field of power load prediction, and particularly relates to a Mama-based endogenous and endogenous variable fusion power load prediction method, which comprises the following steps of: collecting and preprocessing a data set; embedding an endogenous variable, dividing a power load endogenous sequence into N non-overlapping Patch blocks, adding position codes to each block, generating a block token Pen of block fine granularity through linear projection and position codes, and introducing a learnable global token Gen to represent the macroscopic state of the sequence; global embedding of exogenous variables; performing cross-granularity fusion on internally and externally generated tokens; carrying out Mama time sequence dependence modeling; and performing multi-step prediction output, mapping the time sequence characteristics into future multi-step load prediction values through a full connection layer, restoring the original data scale after reverse normalization, and outputting a final prediction result. According to the method, through a block embedding strategy and a dynamic parameter selection mechanism, the adaptability of the model to a complex power scene is remarkably improved, and reliable technical support is provided for efficient scheduling and marketization operation of an intelligent power grid.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric load forecasting, and particularly relates to a method for fusing endogenous and exogenous variables for electric load forecasting based on Mamba. Background Art

[0002] With the transformation of the global energy structure towards cleaner and more intelligent forms, electric load forecasting has become a core technical support for smart grid dispatching, demand-side management, and electricity market trading. Accurate load forecasting requires comprehensive consideration of the complex interaction between historical load data (endogenous variables) and external factors such as weather, electricity prices, and new energy output (exogenous variables). However, existing technologies have significant bottlenecks in aspects such as model architecture, computational efficiency, and multi-source data fusion, restricting the forecasting accuracy and practicality.

[0003] Early load forecasting mainly relied on statistical models such as the autoregressive integrated moving average model (ARIMA), which modeled the time series trend through linear assumptions but could not capture non-linear fluctuations (such as sudden increases in load during holidays). Recurrent neural networks (RNNs) and their variants LSTM and GRU alleviated the problem of gradient vanishing through gating mechanisms and made progress in short-term load forecasting. However, the serial computational characteristics of LSTM led to slow training speed and insufficient modeling ability for long-term dependencies exceeding 24 hours. The Transformer model, with its self-attention mechanism, broke through the temporal constraints of sequence modeling and showed potential in long-sequence forecasting. However, the complexity caused by the attention mechanism was extremely high and difficult to meet the actual power grid applications. Subsequently, the Structured State Space Model (S4) achieved sequence modeling with linear complexity O(T) through hidden state transfer and had significantly better computational efficiency than the Transformer in long-sequence scenarios. However, S4 could not dynamically enhance feature weights, resulting in low forecasting efficiency. The Mamba model proposed in 2023 introduced a selective scanning mechanism that could adjust the SSM parameters according to the input content, but still lacked the ability to select cross-dimensional features in multi-variable scenarios.

[0004] Existing fusion methods mostly rely on shallow attention or simple concatenation and are difficult to distinguish the direct effects and indirect confusions of exogenous variables. For example, the Tide model developed by Tencent AI Lab inputs temperature, humidity, and load data in parallel into the Transformer, but cannot identify the direct driving effect (causal relationship) of high temperature on air-conditioning load and the indirect impact (confounding factor) of high temperature simultaneously suppressing industrial electricity consumption, resulting in systematic biases in the summer data of the Guangdong power grid. Summary of the Invention

[0005] Aiming at the technical problems that the existing fusion methods mentioned above mostly rely on shallow attention or simple splicing and it is difficult to distinguish the direct effect of exogenous variables from indirect confounding, the present invention provides a method for fusing endogenous and exogenous variables for power load forecasting based on Mamba, which realizes efficient, accurate and interpretable power load forecasting through block embedding, global token bridging, bidirectional cross-attention and dynamic parameter selection, and provides key technical support for the new power system.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A method for fusing endogenous and exogenous variables for power load forecasting based on Mamba, comprising the following steps:

[0008] S1. Data set collection and preprocessing, obtaining endogenous variables and exogenous variables, performing missing value filling and normalization processing, generating a standardized time series tensor, and dividing it into a training set, a validation set and a test set;

[0009] S2. Endogenous variable embedding, dividing the power load endogenous sequence into N non-overlapping Patch blocks, adding positional encoding to each block, and generating block-level fine-grained block tokens P en , and introducing a learnable global token G en to represent the macroscopic state of the sequence;

[0010] S3. Exogenous variable global embedding, directly mapping each exogenous sequence into a single global token V ex to avoid fine-grained partitioning to reduce noise interference;

[0011] S4. Cross-granularity fusion of endogenous and exogenous tokens, using the above-obtained block tokens P en of endogenous variables, global token G en and global token V ex of exogenous variables through three attention mechanisms: inter-block self-attention: block-level tokens capture local time series patterns through self-attention; global-block bidirectional attention: the global token integrates macroscopic and local information with block-level tokens; endogenous-exogenous cross-attention: using the endogenous global token as the Query, and the exogenous tokens as the Key and Value for cross-attention interaction; to complete the cross-granularity fusion of endogenous and exogenous variables;

[0012] S5. Mamba time series dependence modeling, mapping the fused features to the hidden space through a linear layer, iteratively updating the latent state based on the discretized state space model, and accelerating the extraction of long sequence time dependence features through global convolution;

[0013] S6. Multi-step prediction output, mapping the time series features to future multi-step load prediction values through a fully connected layer, and restoring the original data scale after anti-normalization to output the final prediction result.

[0014] The method for data set collection and preprocessing in S1 is as follows:

[0015] First, define the historical power load data as the endogenous variable L = {l1, l2, …, l T}, and obtain the time-aligned exogenous variables as E = {e1, e2, …, e T}, where T is the time series length; for the missing values in the data, use the linear interpolation method to fill them, and obtain the complete endogenous variable sequence exogenous variable sequence Next, perform standardization processing on the filled sequences respectively: For the endogenous variable: where μ L and σ L are respectively the mean and standard deviation of; for the exogenous variable: where μ E and σ E are respectively the mean and standard deviation of; subsequently, splice the standardized endogenous variable L norm with the exogenous variable E norm into a unified time series tensor Finally, divide the training set, validation set, and test set according to the ratio of 7:1:2.

[0016] The method for endogenous variable embedding in S2 is as follows:

[0017] Divide the endogenous sequence L of power load with length T obtained in S1 norm into non-overlapping blocks {s1, s2, …, s N}, where P is the block length; subsequently, add positional encoding i to each block s and map it to fine-grained time tokens through a linear projection layer To avoid the information granularity mismatch caused by the direct fusion of the block-level features of endogenous variables and the macro features of exogenous variables, an additional learnable global token G en = Learnable(x) is generated for the endogenous sequence, representing the overall macro state of the sequence; transfer the causal information of the exogenous variable to the block-level fine-grained features of the endogenous variable to achieve cross-granularity information alignment and fusion.

[0018] The method for exogenous variable global embedding in S3 is as follows:

[0019] Map each exogenous sequence directly to a single global token V ex,i = VariateEmbed(z (i)), i ∈ {1, …, C}, T ex is the length of the exogenous sequence backtracking, where VariateEmbed: is a linear projection layer; finally, the global token set B ex = {V ex,1 , …, V ex,C} and the block-level tokens of the endogenous variables and the global tokens are jointly input into the model.

[0020] The method for cross-granularity fusion of endogenous and exogenous tokens in S4 is as follows:

[0021] S4-1. First, perform inter-block attention. For the block-level tokens at the l-th layer capture the local temporal pattern between blocks through self-attention and update to:

[0022] S4-2. Through the bidirectional attention mechanism of global-block and block-global, use the endogenous global variable token and the block-level token to perform cross-attention interaction and update to Integrate the local information between blocks and the global context,

[0023] while capturing the dependencies between blocks and the global-local associations;

[0024] S4-3. Perform cross-attention between exogenous and endogenous variables. Use the endogenous global token as the query Query, the exogenous variable token V ex as the key Key and value Value, and fuse the cross-variable information through cross-attention to update the global token to

[0025] S4-4. The updated global token is input into the feed-forward layer to further extract features through non-linear transformation, generate the input for the next layer, and extract the features of endogenous and exogenous variables.

[0026] The method for Mamba temporal dependence modeling in S5 is as follows:

[0027] First, map the time series X = [X1, X2, …, x T obtained in S4 to the hidden space through a linear embedding layer to generate the hidden state sequence H0 = W e X + b e , where b eare learnable parameters; then discrete state propagation is performed, and based on the zero-order hold rule, the continuous SSM parameters {A, b, C} are discretized into and the latent state h is iteratively updated in time steps t : Finally, global convolution acceleration is performed: to efficiently compute the long sequence output, a convolutional kernel is constructed and the final temporal features are generated through convolution operations:

[0028] In S6, the prediction method uses the mean absolute error MAE, root mean square error RMSE, symmetric mean absolute percentage error SMAPE, and correlation coefficient for evaluation.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] The present invention proposes an innovative Mamba-based power load forecasting method, which significantly improves the forecasting accuracy and computational efficiency through synchronous extraction of variable features and time features and multi-granularity cross-modal fusion. First, without changing the core architecture of the Transformer, the historical power load sequence is segmented into fine-grained time blocks, and time tokens are generated through causal convolution embedding and positional encoding. At the same time, the global sequence of exogenous variables (such as meteorology, electricity price) is independently mapped into coarse-grained variable tokens to achieve multi-scale feature representation. Second, in the feature extraction module, the local fluctuations, long-term trends of the load sequence and the dynamic effects of exogenous variables are captured layer by layer through the time block self-attention and cross-variable cross-attention mechanisms, and combined with the linear complexity state space modeling of Mamba to adaptively screen key temporal dependencies. Finally, cross-granularity information is bridged through global tokens, and the future multi-step load prediction values are output through linear projection. The present invention significantly improves the adaptability of the model to complex power scenarios through the block embedding strategy and the dynamic parameter selection mechanism, providing reliable technical support for the efficient scheduling and market operation of smart grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained according to the provided drawings.

[0032] The structures, proportions, sizes, etc. shown in this specification are only used to match the content disclosed in the specification for those familiar with this technology to understand and read, and are not used to limit the implementation conditions of the present invention. Therefore, they do not have substantial technical significance. Any modification of the structure, change in the proportional relationship, or adjustment of the size, without affecting the effects that the present invention can produce and the purposes that can be achieved, should still fall within the scope covered by the technical content disclosed in the present invention.

[0033] Figure 1 It is a schematic flowchart of the prediction method of the present invention;

[0034] Figure 2 It is a module diagram of the present invention;

[0035] Figure 3 It is a Mamba structure diagram of the present invention;

[0036] Figure 4 It is an attention mechanism diagram of the present invention. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than a limitation on the claims of the present invention; based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0038] The following will further describe in detail the specific implementation manners of the present invention in combination with the drawings and embodiments. The following embodiments are used to illustrate the present invention but are not used to limit the scope of the present invention.

[0039] The method for predicting power load by fusing endogenous and exogenous variables based on Mamba in the embodiments of the present invention, as Figures 1-4 shown, the process includes:

[0040] Step 1: Dataset collection and preprocessing, obtaining endogenous variables (historical power load values) and exogenous variables (related to power price, temperature, wind power generation value, photovoltaic power generation value, etc.), performing missing value filling and normalization processing, generating a standardized time series tensor, and dividing it into a training set, a validation set, and a test set.

[0041] In Step 1, data collection and preprocessing: First, define the historical power load data as endogenous variable L = {l1, l2,..., l T}, and obtain the time-aligned exogenous variable as E = {e1, e2,..., e T}, where T is the length of the time series. For the missing values in the data, linear interpolation is used for filling to obtain a complete sequence of endogenous variables Exogenous variable sequence Next, the filled sequences are respectively standardized: For the endogenous variables: where μ L and σ L are respectively the mean and standard deviation of For the exogenous variables: E where μ E and σ are respectively norm the mean and standard deviation of norm Subsequently, the standardized endogenous variable L is concatenated with the exogenous variable E

[0042] Finally, the training set, validation set, and test set are divided according to the ratio of 7:1:2. en Step 2, Endogenous variable embedding. The endogenous sequence of power load is divided into N non-overlapping Patch blocks. After adding positional encoding to each block, fine-grained block tokens P en are generated through linear projection and positional encoding, and a learnable global token G

[0043] is introduced to represent the macroscopic state of the sequence. norm In step 2, the endogenous sequence of power load L with length T obtained in step 1 is divided into N non-overlapping blocks {s1, s2, …, s i}, where P is the block length; Subsequently, positional encoding is added to each block s To avoid the information granularity mismatch caused by the direct fusion of the block-level features of the endogenous variables and the macroscopic features of the exogenous variables, a learnable global token G en = Learnable(x) is additionally generated for the endogenous sequence to represent the overall macroscopic state of the sequence; The causal information of the exogenous variables (such as electricity price, meteorological data) is transmitted to the block-level fine-grained features of the endogenous variables to achieve cross-granularity information alignment and fusion.

[0044] Step 3, Exogenous variable global embedding. Each exogenous sequence (such as temperature, wind power generation value) is directly mapped to a single global token V ex, avoid fine-grained chunking to reduce noise interference. Since the role of exogenous variables (such as electricity prices, meteorological data) is to assist in predicting endogenous variables through cross-variable interactions, using the same fine-grained embedding as endogenous variables will introduce noise and increase computational redundancy; therefore, in this embodiment, each exogenous sequence T ex (where is the exogenous sequence backtracking length) is directly mapped to a single global token V through the variable embedding module ex,i =VariateEmbed(z (i) ), where Variate Embed: is a linear projection layer; finally, the set of global tokens V ex ={V ex,1 ,…,V ex,C} of all exogenous variables, the block-level tokens of endogenous variables, and the global tokens are jointly input into the model.

[0045] Step 4, cross-granularity fusion of endogenous and exogenous tokens. The block tokens P en , global tokens G en of endogenous variables, and the global tokens V ex of exogenous variables obtained above are passed through three types of attention: inter-block self-attention: block-level tokens capture local temporal patterns through self-attention; global-block bidirectional attention: global tokens and block-level tokens integrate macro and local information; endogenous-exogenous cross-attention: using the endogenous global token as the Query, and the exogenous tokens as the Key and Value for cross-attention interaction. Complete the cross-granularity fusion of endogenous and exogenous variables.

[0046] Step 4-1: First, perform inter-block attention. For the block-level tokens at the l-th layer, capture the local temporal patterns between blocks through self-attention and update to:

[0047] Step 4-2: Through the global-block and block-global bidirectional attention mechanism, perform cross-attention interaction between the endogenous global variable token and the block-level tokens and update to Integrate the local information between blocks and the global context, and simultaneously capture the dependencies between blocks and the global-local associations.

[0048] Step 4-3: Perform cross-attention between exogenous and endogenous variables. Using the endogenous global token as the query Query, the exogenous variable token V ex as the key (Key) and value (Value), and perform cross-attention Fuse cross-variable information and update the global token to

[0049] Step 4-4: The updated global token The input feedforward layer further extracts features through non-linear transformation, generates the input for the next layer, and extracts the features of endogenous and exogenous variables.

[0050] Step 5, Mamba temporal dependence modeling. Map the fused features to the latent space through a linear layer, iteratively update the latent state based on the discretized state space model, and accelerate the extraction of long-sequence time dependence features through global convolution. First, map the time series X = [x1, x2, …, x T obtained in Step 4 to the latent space through a linear embedding layer to generate the latent state sequence H0 = W e X + b e where b e is a learnable parameter. Then perform discretized state propagation. Based on the zero-order hold rule, discretize the continuous SSM parameters {A, B, C} into and iteratively update the latent state h t : Finally, perform global convolution acceleration: To efficiently calculate the long-sequence output, construct a convolution kernel and generate the final time features through convolution operations:

[0051] Step 6, multi-step prediction output. Map the temporal features to the future multi-step load prediction values through a fully connected layer, restore the original data scale after inverse normalization, and output the final prediction result.

[0052] The loss function used is cross-entropy loss, and the formula is:

[0053]

[0054] The method is implemented using Pytorch and trained using the Adam optimizer. After adjusting the parameters through multiple experiments, the present invention sets the learning rate to 0.0001, the batch size to 32, each experiment runs for 100 epochs, and early stop is allowed when the effect is good.

[0055] In this embodiment, by synchronously extracting variable features and time features and performing multi-granularity cross-modal fusion, the prediction accuracy and computational efficiency are significantly improved. First, without changing the core architecture of the Transformer, the historical power load sequence is segmented into fine-grained time blocks, and time tokens are generated through causal convolution embedding and positional encoding. At the same time, the global sequences of exogenous variables (such as meteorology and electricity price) are independently mapped into coarse-grained variable tokens to achieve multi-scale feature representation. Second, in the feature extraction module, the local fluctuations, long-term trends of the load sequence and the dynamic impacts of exogenous variables are captured hierarchically through the time-block self-attention and cross-variable cross-attention mechanisms, and combined with the linear complexity state space modeling of Mamba to adaptively screen key temporal dependencies. Finally, cross-granularity information is bridged through global tokens, and the future multi-step load prediction values are output through linear projection. In this embodiment, through the block embedding strategy and the dynamic parameter selection mechanism, the adaptability of the model to complex power scenarios is significantly improved, providing reliable technical support for the efficient scheduling and market-oriented operation of smart grids.

[0056] The above only elaborates on the preferred embodiments of the present invention in detail. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various changes can be made without departing from the purpose of the present invention, and all such changes should be included within the protection scope of the present invention.

Claims

1. A method for fusing endogenous and exogenous variables in power load forecasting based on Mamba, characterized in that It includes the following steps: S1. Dataset collection and preprocessing, obtaining endogenous variables and exogenous variables, performing missing value filling and normalization processing, generating a standardized time series tensor, and dividing it into a training set, a validation set, and a test set; S2. Endogenous variable embedding: The endogenous sequence of power load is divided into N non-overlapping Patch blocks. After adding positional encoding to each block, fine-grained block tokens P are generated through linear projection and positional encoding. en And a learnable global token G is introduced en to represent the macroscopic state of the sequence. S3. Global embedding of exogenous variables, directly mapping each exogenous sequence to a single global token V ex , avoiding fine-grained chunking to reduce noise interference; S4. Cross-granularity fusion of endogenous and exogenous tokens. The block token P of the endogenous variable obtained above en , the global token G en , and the global token V of the exogenous variable ex are fused through three types of attention: self-attention between blocks: the block-level tokens capture local temporal patterns through self-attention; global-block bidirectional attention: the global token is integrated with the block-level tokens to combine macro and local information; endogenous-exogenous cross-attention: the endogenous global token is used as the Query, and the exogenous tokens are used as the Key and Value for cross-attention interaction; Complete the cross-granularity fusion of endogenous and exogenous variables; S5. Mamba time series dependence modeling, mapping the fused features to the latent space through a linear layer, iteratively updating the latent state based on the discretized state space model, and accelerating the extraction of long sequence time dependence features through global convolution; S6. Multi-step prediction output, mapping the time series features to future multi-step load prediction values through a fully connected layer, restoring the original data scale after inverse normalization, and outputting the final prediction result.

2. The method for predicting power load by fusing endogenous and exogenous variables based on Mamba according to claim 1, wherein The method for dataset collection and preprocessing in S1 is: First, define the historical power load data as the endogenous variable \(L = \{l_1, l_2, \ldots, l\) T \}, and obtain the time-aligned exogenous variables as \(E = \{e_1, e_2, \ldots, e\) T \}, where \(T\) is the length of the time series; for the missing values in the data, use linear interpolation to fill them, obtaining the complete endogenous variable sequence exogenous variable sequence Next, perform standardization processing on the filled sequences respectively: For the endogenous variable: where \(\mu\) L and \(\sigma\) L are respectively the mean and standard deviation of; for the exogenous variable: where \(\mu\) E and \(\sigma\) E are respectively the mean and standard deviation of; Subsequently, concatenate the standardized endogenous variable \(L\) norm with the exogenous variable \(E\) norm to form a unified time series tensor Finally, divide the training set, validation set, and test set in a ratio of 7:1:

2.

3. The method for predicting power load by fusing endogenous and exogenous variables based on Mamba according to claim 1, wherein The method for endogenous variable embedding in S2 is: The endogenous power load sequence L of length T obtained in S1 norm is divided into non - overlapping blocks {s1, s2, …, s N}, where P is the block length; subsequently, for each block s i adds positional encoding and maps it to fine - grained time tokens through a linear projection layer To avoid the information granularity mismatch caused by the direct fusion of the block - level features of endogenous variables and the macro - level features of exogenous variables, a learnable global token G en = Learnable(x) is additionally generated for the endogenous sequence, representing the overall macro - state of the sequence; Transfer the causal information of exogenous variables to the endogenous block-level fine-grained features to achieve cross-granularity information alignment and fusion.

4. A method for predicting power load by fusing endogenous and exogenous variables based on Mamba according to claim 1, characterized in that, The method for exogenous variable global embedding in S3 is: Map each exogenous sequence directly to a single global token V through the variate embedding module ex,i = VariateEmbed(z (i) ), i ∈ {1, …, C}, T ex is the exogenous sequence backtracking length, where VariateEmbed: is a linear projection layer; finally, the set of global tokens V ex = {V ex,1 , …, V ex,C} of all exogenous variables, the block-level tokens of the endogenous variables, and the global tokens are jointly input into the model.

5. A method for predicting power load by fusing endogenous and exogenous variables based on Mamba according to claim 1, characterized in that, The method for cross-granularity fusion of endogenous and exogenous tokens in S4 is: S4-1. First, perform inter-block attention on the block-level tokens of the l-th layer Capture the local temporal patterns between blocks through self-attention and update them to: S4-2. Through the global-block and block-global bidirectional attention mechanism, update the endogenous global variable token through cross-attention interaction with the block-level token to integrate the local information between blocks and the global context, while capturing the dependencies between blocks and the global-local correlation; ​ S4-3. Perform cross-attention on exogenous and endogenous variables to obtain the endogenous global token G l as the query Query, the exogenous variable token V ex as the key Key and value Value, and fuse cross-variable information through cross-attention to update the global token to S4-4. Updated Global Token The input feedforward layer further extracts features through non-linear transformation to generate the input of the next layer and extract the features of endogenous and exogenous variables.

6. The method for predicting power load by fusing endogenous and exogenous variables based on Mamba according to claim 1, wherein, The method for Mamba time series dependence modeling in S5 is: First, the time series X = [x1, x2, …, x T obtained from S4 is mapped to the latent space through a linear embedding layer, generating the latent state sequence H0 = W e X + b e , where b e is a learnable parameter; then, discrete state propagation is performed. Based on the zero-order hold rule, the continuous SSM parameters {A, B, C} are discretized into and the latent state h t is iteratively updated in time steps: Finally, global convolution acceleration is performed: to efficiently compute the long sequence output, a convolution kernel is constructed and the final temporal features are generated through convolution operations:

7. A method for predicting power load by fusing endogenous and exogenous variables based on Mamba according to claim 1, characterized in that In S6, the prediction method is evaluated using the mean absolute error MAE, the root mean square error RMSE, the symmetric mean absolute percentage error SMAPE, and the correlation coefficient.

Images