Electric power prediction method for long-context multivariable time series

Abstract

The invention provides an electric power prediction method for a long-context multivariable time sequence. The electric power prediction method comprises the following steps: step 1, taking the multivariable time sequence in an electric power system as input; step 2, carrying out marking processing on the input time sequence data, and enabling the marked data to obtain embedded representation through an embedded network layer; 3, enabling the embedded representation obtained in the step 2 to pass through a mixed attention decoder, and enabling the mixed attention decoder to efficiently capture time dynamics and cross-variable interaction in the power data by alternately using variable independent attention, variable related attention and a multivariable relative position coding strategy; 4, performing prediction according to an output result of the mixed attention decoder in the step 3 by using a variational level prediction strategy; and 5, selecting a prediction result consistent with the expected prediction length from the variational level prediction results in the step 4 as an output prediction result.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and machine learning, and particularly to a power prediction method for long-context multi-variable time series. Background Art

[0002] Foundation Models have revolutionized artificial intelligence and achieved remarkable success in the fields of natural language processing and computer vision, thanks to their unprecedented generalization ability. However, in the critical field of time series prediction, especially power prediction, the development of general prediction models remains a major challenge. Power prediction not only needs to consider the changes in historical data but also cope with multi-dimensional variable interactions and long-time-span dependencies. Existing models are mainly limited by the training methods of specific datasets and rigid prediction frameworks, so they face many problems in practical applications: (1) The heterogeneity of time data, including varying context lengths, prediction horizons, and complex dependencies between variables; (2) In practical applications, it is necessary to balance the expressive power and computational efficiency of the model. Although the decoder-only architecture has shown potential in modeling long-term dependencies and ensuring causal consistency, existing methods still have limitations.

[0003] Multi-variable time series prediction methods attempt to unify different prediction scenarios into long-context generation problems, but their sequence flattening strategies convert two-dimensional token sequences into one-dimensional formats, which leads to quadratic computational complexity when dealing with long contexts and multi-variable data, severely limiting the scalability of the model. In power prediction, this flattening strategy often fails to effectively utilize the structural features of the data, resulting in waste of computational resources. Traditional autoregressive decoding strategies show low computational efficiency in long-term prediction and may cause error accumulation during sequence generation, thus affecting the prediction accuracy and bringing risks to the scheduling and operation of power systems.

[0004] Therefore, how to fully consider the multi-dimensional dependencies between variables and long-time-span dependencies in power prediction tasks while improving the computational efficiency of the model is an urgent problem to be solved in the current power prediction field. Summary of the Invention

[0005] Object of the Invention: The technical problem to be solved by the present invention is to provide a power prediction method for long-context multi-variable time series in view of the deficiencies of the prior art.

[0006] To solve the above technical problem, the present invention discloses a power prediction method for long-context multi-variable time series, including:

[0007] Step 1, taking the multi-variable time series X in the power system as an input;

[0008] Step 2: Tokenize the input time series data, and then embed the tokenized data through the embedding network layer to obtain the embedded representation.

[0009] Step 3: The embedding representation obtained in step 2 is passed through a hybrid attention decoder. The hybrid attention decoder efficiently captures the temporal dynamics and cross-variable interactions in the power data by alternating between variable-independent attention and variable-dependent attention, as well as a multivariate relative position encoding strategy.

[0010] Step 4: Use the variational level prediction strategy to make predictions based on the output of the hybrid attention decoder in step 3. The variational level prediction strategy supports prediction requirements at different time scales in the power system. By setting a parallel projection layer for each prediction range, it enhances the training signal and improves prediction robustness.

[0011] Step 5: Select the prediction result that matches the expected prediction length from the variational level prediction results in step 4 as the output prediction result.

[0012] In step 1:

[0013] Since the variable time series data in the power system includes multiple variables such as power load, power generation, meteorological data, etc., the multivariate time series X∈R N×S , where N represents the number of variables and S represents the length of the time series;

[0014] The variables include load, power generation, temperature and meteorological data.

[0015] Step 2 is as follows:

[0016] Step 2-1: Transform the multivariate time series X∈R N×S Each variable in is divided into multiple tags of length P, and the tokenized multivariate time tag sequence x∈R is obtained N×T×P , where S = T × P, T is the number of labels for each variable; for the label x of the mth variable at position i m,i , which consists of the sequence X∈R N×S The mth variable starts from position (i-1)P+1 and ends at position iP, with a total of P data:

[0017] x m,i =[X m,(i-1)P+1 ,…,X m,iP ]∈R P

[0018] where m∈{1,…,N} and i∈{1,…,T};

[0019] Step 2-2: Encode the tokenized multivariate time token sequence \(x\) through the initial linear embedding network layer to obtain the embedding representation \(H\) of the multivariate time token sequence (0) ∈R N×T×d , where \(d\) is the feature dimension of the embedding representation, and the embedding representation m,i of the sequence token \(x\) of the \(m\) -th variable at position \(i\) is calculated as follows:

[0020]

[0021] where \(m\in\{1,\ldots,N\}\) and \(i\in\{1,\ldots,T\}\), \(W E ∈R P×d is the weight parameter of the linear embedding network layer, and \(b E ∈R d is the offset parameter of the linear embedding network layer.

[0022] In step 3, the hybrid attention decoder is stacked by multiple Transformer layers, which are alternately stacked by variable - independent attention Transformer layers and variable - dependent attention Transformer layers; the input of each layer is the output of the previous layer to form an alternating architecture. The hybrid attention decoder alternates between variable - dependent attention and variable - independent attention layers, where each coupled variable - dependent attention layer (as shown in Figure 2 ) alternates with the subsequent variable - independent attention layer. This design effectively combines iterative optimization with variable - dependent attention capturing cross - power - variable dependencies and variable - independent attention processing time patterns, thus better understanding the computational impact of interactions in power data.

[0023] The hybrid attention decoder contains a pairwise sharing mechanism that enables the key - value projection to share parameters in the variable - independent attention Transformer layer and the variable - dependent attention Transformer layer of the alternating architecture. This approach captures patterns in power data through specialized query transformations while maintaining unique attention functions in the shared key - value projection.

[0024] Step 3 is specifically as follows:

[0025] Step 3-1: Each layer obtains the projection representations of the query, key, and value through linear mapping based on the embedding representation \(H\) of the previous layer. For the token of the \(m\) -th variable at position \(i\), its projection representation of the query (l-1) , projection representation of the key , and projection representation of the value are calculated as follows:

[0026]

[0027] ​where \(m\in\{1,\ldots,N\}\) and \(i\in\{1,\ldots,T\}\), \(l\) represents the level, and represent the weight parameter and the offset parameter of the linear transformation network of the query, and represent the weight parameter and the offset parameter of the linear transformation network of the key, and represent the weight parameter and the offset parameter of the linear transformation network of the value.

[0028] This method has two significant advantages. First, it significantly improves parameter efficiency by eliminating redundant key-value projection parameters, reducing the projection parameters of each variable-related attention-variable-independent attention pair by 33%. In addition, as an implicit regularization mechanism, it promotes more robust and transferable representation learning, through shared key-value projections across complex attention mechanisms, suitable for the characteristics of power data.

[0029] To effectively model the complex multivariate time series dynamics in the power system, a hybrid attention decoder is proposed, which takes the multivariate time series embedding representation \(H\) obtained in step 2 (0) as the input of the hybrid attention decoder. Based on independent attention (variable-independent attention) to capture the temporal dynamics within power variables and variable-dependent attention (variable-related attention) to extract the interaction relationships between variables (such as load and meteorological data), so as to achieve long-context power sequence understanding while maintaining computational efficiency.

[0030] Step 3-2. In the power system, the causality and variable invariance of multivariate time series need to maintain time invariance. For this purpose, the present invention proposes a multivariate relative position encoding strategy, which considers the relative position of variable dimensions on the basis of the relative position encoding (ALiBi) scheme to efficiently handle the temporal and cross-variable dependencies in power data.

[0031] Calculate the query for the key attention scores The calculation process simultaneously considers and the relative positions of and whether the two come from the same variable, and its formula is:

[0032]

[0033] where \(m,n\in\{1,\ldots,N\}\) and \(i,j\in\{1,\ldots,T\}\); the query is the embedding representation of variable \(m\) at time \(i\) and layer \(l\), and the key is the embedded representation of variable n at layer l at time j; sgn is the sign function, where sgn(i - j) = 1 when i > j, sgn(i - j) = 0 when i = j, and sgn(i - j) = -1 when i < j; 1 m≠n is an indicator function that, when the query and the key come from the same variable, i.e., 1 when m = n m≠n = 0, when the query and the key come from different variables, i.e., 1 when m ≠ n m≠n = 1.

[0034] This formula maintains the permutation equivalence across variables while preserving temporal dependencies through relative positions, without the need for additional learnable parameters;

[0035] The normalized attention score α is calculated from the attention scores specifically as:

[0036]

[0037] This formula normalizes the attention scores to a probability distribution through the softmax function;

[0038] The normalized attention score α mn,ij contains two types of key dependencies. Content-based interaction is through the query-key pair q m,i and k n,j , and positional dependency, which comes from the position indices (m, i) and (n, j);

[0039] Step 3-3, In power data, the correlation between variables is crucial, such as the relationship between load and temperature. Define the required position set S for each query which can not only capture the correlation between tokens within a variable but also effectively model multi-variable correlations; m,i The hybrid design focuses on developing the optimal position set S

[0040] to maximize computational efficiency while maintaining power prediction accuracy; m,i

[0041] Step 3-4, For each position (m, i), calculate the weighted sum of the position value embedded representations from the position set S m,i according to the attention scores:

[0042]

[0043] where m, n ∈ {1, …, N} and i, j ∈ {1, …, T};

[0044] The result after the attention mechanism calculation passes through the output linear mapping network layer to obtain the output of the attention layer

[0045]

[0046] where m ∈ {1, …, N}, i ∈ {1, …, T}, is the bias parameter of the output flavor mapping network layer, is the linear transformation parameter of the output flavor mapping network layer;

[0047] Step 3-5: The output of the attention layer calculated in Step 3-4 passes through a feed-forward neural network for further feature extraction and enhancement, obtaining the sequence label x of the m-th variable at position i m,i The embedding representation of the l-th layer

[0048]

[0049] where m ∈ {1, …, N} and i ∈ {1, …, T}; is the linear transformation parameter of the first linear mapping network layer in the feed-forward neural network, is the bias parameter of the first linear mapping network layer in the feed-forward neural network, is the linear transformation parameter of the second linear mapping network layer in the feed-forward neural network, is the bias parameter of the second linear mapping network layer in the feed-forward neural network, and the nonlinear activation function ReLU is used in the middle and a residual connection is used to ensure a stable training process.

[0050] The position set S described in Step 3-3 m,i Specifically:

[0051] For the Transformer layer using the variable-independent attention mechanism, the variable-independent attention mechanism focuses on the temporal patterns within each variable to achieve a causal self-attention pattern. Therefore, for each selected query position (m, i), the position set S it uses m,i is defined as:

[0052] S m,i = {(n, j)| |(n = m) and (j ≤ i)}

[0053] This attention connection pattern enables all previous positions within the same variable to maintain temporal causality.

[0054] For the Transformer layer using the variable - related attention mechanism, the variable - related attention mechanism extends the function of variable - independent attention by explicitly modeling cross - variable interactions. It selects the positions that need to perform attention connections for the query at position i to achieve efficient cross - variable modeling, which is suitable for the complex dependencies between variables in the power system. Through strategic sparsification, specifically, the attention queries are only calculated at the stride - time positions, denoted as (i mod k = 0), where k represents the stride. For each selected query position (m, i), the set of positions S it uses m,i is defined as:

[0055] S m,i ={(n, j)∣n ∈ {1, …, N} and j ≤ i and ((i - j) mod k = 0)}

[0056] where m ∈ {1, …, N} and i ∈ {1, …, T}; this formula enables each query to focus on the stride - time positions of all variables, creating a sparse but effective attention structure that preserves the time patterns and cross - variable dependencies.

[0057] In the variable - independent attention layer, the queries with stride k require a time complexity of O(N · T 2 ), while in the variable - related attention layer, it requires a time complexity of O(N 2 ·(T / k) 2 ), presenting a significant improvement of O(N · T 2 +N 2 ·(T / k) 2 ) in total. Compared with the complexity of O(N 2 ·T 2 ) of the traditional full - attention mechanism, especially in the case of high - dimensional power variables N or long sequences T.

[0058] Step 4

[0059] Effective multivariate time - series prediction needs to be able to make predictions at different levels, such as short - term load forecasting and long - term energy planning, while maintaining accuracy and computational efficiency. The present invention introduces a variational level prediction strategy that enables the model to make multi - level predictions simultaneously while maintaining accuracy and computational efficiency. This method enhances the training signal and develops more robust representations through parallel multi - level learning.

[0060] The prediction through the variational level prediction strategy is specifically as follows:

[0061] Use K parallel projection layers for power predictions at different levels, and each projection layer corresponds to a specific power prediction level τ k , for example, when the predicted power level is four days, τ k = 96, when the predicted power level is eight days, τk = 8 × 24 = 192 when the predicted power level is two weeks, τ k = 24 × 7 × 2 = 336. The parallel projection layers share a common learned representation H out ∈R N×T×d and generate predictions in the following way

[0062]

[0063] where and represent learnable parameters specific to level τ k and each prediction represents a τ k -step prediction;

[0064] where the learned representation H out is calculated as follows:

[0065]

[0066] where L is the number of Transformer layers, the input to each layer is the output representation of the last Transformer layer in step 3. For each variable m, the embedding representations at all its T positions are concatenated in order, and then the concatenated vectors of N variables are stacked into a matrix H out .

[0067] The joint loss for variational level prediction optimization is calculated as:

[0068]

[0069] where K is the number of parallel projection layers, represents the τ k -step prediction given by the model, represents the true τ k -step power value, and the mean squared error between the two is used as the loss function. During training, the model optimizes the joint loss for all power prediction levels i.e., calculates the mean of the mean squared errors at each power prediction level.

[0070] Step 5 is specifically as follows:

[0071] For the power prediction level τ of the expected output, select the prediction k with τ = τ in step 4 as the final output prediction result.

[0072] Beneficial effects:

[0073] 1. By introducing a hybrid attention decoder (including variable-independent attention VIA and variable-dependent attention VDA), it can efficiently capture the long-term dynamics and cross-variable dependencies in time series of power systems, such as the complex interactions between power loads and meteorological conditions, overcoming the limitations of traditional basic models under the variable independence assumption. Combining multi-variable ALiBi positional encoding and variational horizontal forecasting (VHF) strategy, the model exhibits higher accuracy and robustness under different forecasting horizons in power forecasting, and is applicable to complex scenarios of multi-variable and high-dimensional data in power systems.

[0074] 2. Compared with the traditional full attention mechanism with a time complexity of (O(N 2 ·T 2 )), the hybrid attention mechanism of the present invention optimizes the computational complexity to (O(N·T 2 +N 2 ·(T / k) 2 )) and significantly reduces the computational resource requirements through stride sparsification and pairwise shared KV projection. This design is particularly efficient in scenarios of long sequences (such as historical data of several months) and high-dimensional variables (such as loads and power generations in multiple regions) in power forecasting, and significantly improves the scalability of the model.

[0075] 3. The present invention unifies the multi-variable time series forecasting problem of power systems into a long context generation task, and through a flexible attention mechanism and positional encoding scheme, adapts to the heterogeneity of time data in power data (including varying context lengths, forecasting horizons, and variable dependencies). BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a flow chart of the present invention.

[0077] Figure 2 It is a structural diagram of the hybrid attention decoder.

[0078] Figure 3 It is a schematic diagram of the variational horizontal forecasting strategy. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0079] The present invention proposes a long-context multivariate time series prediction method based on hybrid attention. Through innovative model design and optimization strategies, the performance of multivariate time series prediction is significantly improved, and many beneficial effects have been achieved in practical applications. The power prediction method of the long-context multivariate time series based on hybrid attention of the present invention shows significant advantages in improving prediction performance, reducing computational costs, enhancing multi-range prediction capabilities, etc. This method can not only accurately predict power load and power generation, but also provide an efficient, robust and practical solution for power dispatching, energy management and renewable energy grid connection, providing important technical support for the intelligent and sustainable development of the power system.

[0080] As shown in this embodiment Figure 1 as follows:

[0081] Step 1: Take the multivariate time series X in the power system as the input;

[0082] Step 2: Tokenize the input time series data, and the tokenized data obtains an embedding representation through the embedding network layer;

[0083] Step 3: As Figure 2 shown, pass the embedding representation obtained in Step 2 through the hybrid attention decoder. The hybrid attention decoder efficiently captures the temporal dynamics and cross-variable interactions in the power data by alternately using variable-independent attention and variable-dependent attention, as well as the multivariate relative position encoding strategy;

[0084] Step 4: As Figure 3 shown, use the variational horizontal prediction strategy to make a prediction based on the output result of the hybrid attention decoder in Step 3;

[0085] Step 5: Select the prediction result that matches the expected prediction length from the variational horizontal prediction results in Step 4 as the output prediction result.

[0086] Specifically, Step 1 is as follows:

[0087] In the analysis scenario of the power system, based on the hourly power consumption data of 321 household users in the past two years, we construct the multivariate time series X:

[0088] Step 1-1: Data collection and collation. The data comes from the smart meters of the power system, recording the power consumption (unit: kilowatt-hour, kWh) of 321 household users. The time span is two years, covering a continuous time series from the 1st hour to the S = 365×2×24 = 17520th hour. The power consumption data of each household user forms a time series variable, and there are a total of N = 321 variables.

[0089] Step 1-2: Format the data. Arrange the power consumption data of each household user in chronological order to form a time series with a length of S = 17520. Integrate the data of all household users into a matrix X ∈ R N×S , where the number of rows N = 321 represents 321 household users, and the number of columns S = 17520 represents the power consumption records per hour.

[0090] Step 1-3: Preprocess the data. First, fill in the missing values, check and fill in the possible missing data (such as the power consumption in some time periods not being recorded). Second, perform normalization processing, normalize or standardize the power consumption data to eliminate the magnitude differences between different users.

[0091] Step 5 is specifically as follows:

[0092] For the expected power prediction level τ, select the prediction where τ k = τ as the final output prediction result. When τ = 96, predicting the power consumption for the next 96 hours (4 days) can help power companies prepare in advance for medium- and short-term power generation plans, load dispatching, and electricity price adjustment; when v = 192, predicting the power consumption for the next 192 hours (8 days) helps in the allocation of power resources, especially applicable to the management of electricity demand during holidays or special weather conditions. When τ = 336, predicting the power consumption for the next 336 hours (two weeks) can provide a longer-term planning basis for power companies, such as formulating bi-weekly power generation plans, arranging grid maintenance times, and evaluating the access schemes for new energy generation; when τ = 720, it is applicable to seasonal forecasting or long-term planning, such as analyzing the changing trends of electricity demand during summer peak periods or winter, and providing support for policy-making and power market transactions.

[0093] This embodiment conducts experiments on the standard publicly available dataset Electricity. Table 1 shows the mean squared error (MSE) and mean absolute error (MAE) of the present invention under the predicted power levels of 96, 192, 336, and 720

[0094]

[0095] Table 1. Power Prediction Results

[0096] The present invention provides a long-context multi-variable time series prediction method. There are many methods and ways to specifically implement this technical solution. The above description is only a preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by existing technologies.

Claims

1. A power prediction method for long-context multi-variable time series, characterized in that, Including: Step 1: Take the multivariate time series X in the power system as the input; Step 2: Tokenize the input time series data, and the tokenized data obtains an embedding representation through the embedding network layer; Step 3: Pass the embedding representation obtained in Step 2 through the hybrid attention decoder. The hybrid attention decoder efficiently captures the temporal dynamics and cross-variable interactions in the power data by alternately using variable-independent attention and variable-dependent attention, as well as the multivariate relative position encoding strategy; Step 4: Use the variational level prediction strategy to make predictions based on the output results of the hybrid attention decoder in Step 3; Step 5: Select the prediction result that matches the expected prediction length from the variational level prediction results in Step 4 as the output prediction result.

2. The power prediction method for long context multi-variable time series according to claim 1, characterized in that In Step 1: The multivariate time series X ∈ RN×S N×S , where N represents the number of variables and S represents the length of the time series; The variables include load, power generation, temperature, and meteorological data.

3. A power prediction method for long context multi-variable time series according to claim 1, characterized in that, Step 2 is specifically: Step 2-1: Divide each variable in the multivariate time series \(X\in\mathbb{R}\) N×S into multiple tokens of length \(P\), obtaining the tokenized multivariate time token sequence \(x\in\mathbb{R}\) N×T×P , where \(S = T\times P\) and \(T\) is the number of tokens for each variable; The tag x of the m-th variable at position i m,i includes a sequence x ∈ R N×S of a total of P data of the m-th variable starting from position (i - 1)P + 1 to position iP in the sequence: x m,i = [X m,(i-1)P+1 , …, X m,iP ∈ R P where m ∈ {1, …, N} and i ∈ {1, …, T}; Step 2-2: Encode the tokenized multi-variable time token sequence x through the initial linear embedding network layer to obtain the embedding representation H of the multi-variable time token sequence (0) ∈R N×T×d , where d is the feature dimension of the embedding representation, and the embedding representation of the sequence token x m,i at the position i of the m-th variable is calculated as follows: where \(m\in\{1,\ldots,N\}\) and \(i\in\{1,\ldots,T\}\), \(W\) E \(\in\mathbb{R}\) P×d is the weight parameter of the linear embedding network layer, \(b\) E \(\in\mathbb{R}\) d is the offset parameter of the linear embedding network layer.

4. A power prediction method for long context multi-variable time series according to claim 1, characterized in that In Step 3, the hybrid attention decoder is stacked by multiple Transformer layers, and is alternately stacked by variable-independent attention Transformer layers and variable-dependent attention Transformer layers; the input of each layer is the output of the previous layer to form an alternating architecture.

5. A power prediction method for long context multi-variable time series according to claim 4, characterized in that, The hybrid attention decoder includes a pairwise sharing mechanism, which enables the key-value projection to share parameters in the variable-independent attention Transformer layer and the variable-dependent attention Transformer layer of the alternating architecture.

6. A power prediction method for long context multi-variable time series according to claim 5, characterized in that, Step 3 is specifically: Step 3-1: For each layer, based on the embedding representation H of the previous layer (l-1) obtain the projected representations of the query, key, and value through linear mapping. For the token at position i of the m-th variable, the projected representation of its query the projected representation of the key and the projected representation of the value are calculated as follows: where \(m\in\{1,\ldots,N\}\) and \(i\in\{1,\ldots,T\}\), \(l\) represents the level, and represent the weight parameter and the offset parameter of the linear transformation network of the query, and represent the weight parameter and the offset parameter of the linear transformation network of the key, and represent the weight parameter and the offset parameter of the linear transformation network of the value; Step 3-2: Calculate the query for the key attention scores The calculation process simultaneously considers and their relative positions and whether they come from the same variable. The formula is as follows: where m, n ∈ {1, …, N} and i, j ∈ {1, …, T}; the query is the embedding representation of variable m at the l-th layer at time i, and the key is the embedding representation of variable n at the l-th layer at time j; sgn is the sign function, sgn(i - j) = 1 when i > j, sgn(i - j) = 0 when i = j, and sgn(i - j) = -1 when i < j; 1 m≠n is an indicator function, which is 1 when the query and the key come from the same variable, i.e., 1 m≠n = 0 when m = n, and 1 is 1 when the query and the key m≠n come from different variables, i.e., 1 = 1 when m ≠ n; From the attention score Calculate the normalized attention score α, specifically as follows: The formula normalizes the attention scores into a probability distribution through the softmax function; Normalized attention score α mn,ij Contains two types of key dependencies. Content-based interaction is through the query-key pair q m,i and k n,j as well as position dependencies, which come from the position indices (m, i) and (n, j); Step 3-3: For each query define the set S of positions it requires m,i ; Step 3-4: For each position (m, i), calculate the weighted sum of the position value embeddings from the position set S m,i according to the attention scores: where m, n ∈ {1, …, N} and i, j ∈ {1, …, T}; The result obtained after the attention mechanism calculation passes through the output linear mapping network layer to obtain the output of the attention layer where m ∈ {1, …, N} and i ∈ {1, …, T}, is the bias parameter of the output flavor mapping network layer, and is the linear transformation parameter of the output flavor mapping network layer; Step 3-5: The output of the attention layer obtained in Step 3-4 passes through a feed-forward neural network for further feature extraction and enhancement to obtain the sequence label x of the m-th variable at position i m,i The embedding representation of the l-th layer where \(m\in\{1,\ldots,N\}\) and \(i\in\{1,\ldots,T\}\); are the linear transformation parameters of the first linear mapping network layer in the feedforward neural network, are the bias parameters of the first linear mapping network layer in the feedforward neural network, are the linear transformation parameters of the second linear mapping network layer in the feedforward neural network, are the bias parameters of the second linear mapping network layer in the feedforward neural network, with the ReLU non - linear activation function used in the middle and a residual connection to ensure a stable training process.

7. A power prediction method for long context multi-variable time series according to claim 6, characterized in that, The position set S described in step 3-3 m,i Specifically: For a Transformer layer using the variable-independent attention mechanism, for each selected query position (m, i), the set of positions S it uses m,i is defined as: S m,i = {(n, j) | (n = m) and (j ≤ i)} For a Transformer layer using a variable-related attention mechanism, for each selected query position (m, i), the set of positions S it uses m,i is defined as: S m,i = {(n, j) | n ∈ {1, …, N} and j ≤ i and ((i - j) mod k = 0)} where, m ∈ {1, …, N} and i ∈ {1, …, T}, k represents the calculation stride, and mod is the modulo operation.

8. A power prediction method for long context multi-variable time series according to claim 1, characterized in that, The prediction in Step 4 through the variational level prediction strategy is specifically: Power prediction is performed using K parallel projection layers, and for the power prediction level τ of each layer k , the parallel projection layers share a common learned representation H out ∈R N×T×d , and generate predictions in the following way where and represent learnable parameters specific to level τ k and each prediction represents a τ k -step prediction; Among them, the learning representation is H out The calculation formula is as follows: Among them, L is the number of Transformer layers. The input of each layer is the output representation of the last Transformer layer in step 3. For each variable m, the embedding representations at all its T positions are concatenated in order, and then the concatenated vectors of N variables are stacked into a matrix H out .

9. A power prediction method for long context multi-variable time series according to claim 8, characterized in that, The combined loss optimized by the variational level prediction is calculated as: where K is the number of parallel projection layers, represents the τ given by the model k -step prediction, represents the true τ k -step power value, and the mean square error between the two is used as the loss function.

10. A power prediction method for long context multi-variable time series according to claim 9, characterized in that, Step 5 is specifically: For the power prediction level τ of the expected output, select the prediction of τ j = τ in step 4 as the final output prediction result.

Images