A Multivariate Power Time Series Forecasting Method Based on Hierarchical Channel Strategy

Abstract

This invention discloses a multivariate power time series forecasting method based on a hierarchical channel strategy, belonging to the field of time series forecasting technology. By employing a modeling strategy of first fusing, then independent analysis, and finally fusing again, the forecasting performance is effectively improved. The method includes the following steps: Step 1, problem definition and data preprocessing; Step 2, performing bimodal Gaussian convolution processing on the input historical time series; Step 3, performing depthwise grouped dilated convolution processing on the output of Step 2; Step 4, performing inverted bottleneck pointwise convolution processing on the output of Step 3; Step 5, transforming and denormalizing the data output from the inverted bottleneck pointwise convolution, and then outputting the result.

Description

Technical Field

[0001] This invention belongs to the field of power time series forecasting technology, specifically relating to a multivariate power time series forecasting method based on a hierarchical channel strategy. Background Technology

[0002] Multivariate time series forecasting techniques have evolved significantly over time. Early on, Recurrent Neural Networks (RNNs) were widely used for time series forecasting tasks. RNNs could handle dependencies between sequences, but they faced challenges such as long-range dependencies and vanishing and exploding gradients. Subsequently, Convolutional Neural Networks (CNNs) were introduced, effectively extracting local features from time series data through their local awareness and weight sharing characteristics. In recent years, the Transformer architecture has shone brightly in multivariate time series forecasting tasks. Thanks to its powerful attention mechanism, the Transformer can flexibly capture long-range dependencies and global information, significantly improving prediction accuracy and generalization ability, making it a popular choice for current research and applications.

[0003] Despite the emergence of many efficient models in recent years, the fundamental question of whether to employ a channel-independent or channel-dependent strategy for multivariate time series modeling remains highly controversial and unresolved. For models using a channel-independent strategy (PatchTST, DLinear), the primary aim is to avoid oversmoothing in specific dimensions. While this approach does reduce the risk of overfitting to some extent, it clearly neglects the exploration and analysis of correlations between different time series variables. This neglect becomes an obstacle, limiting further improvements in model performance.

[0004] While some models do employ channel-dependent strategies, their architectures often rely on two-stage attention mechanisms (Crossformer) with high temporal and spatial complexity, or they pay insufficient attention to modeling the time dimension (itransformer), or they are difficult to train end-to-end (Softs). Therefore, they face various challenges and limitations in practical applications.

[0005] Furthermore, existing research lacks an efficient balance between the dual requirements of "cross-channel correlation mining" and "single-channel feature preservation," and the inherent limitations of a single strategy make it difficult to meet the accurate prediction needs in complex scenarios. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the present invention aims to provide a multivariate power time series forecasting method based on a hierarchical channel strategy, which improves forecasting performance by adopting a modeling strategy of first fusing, then isolating, and finally fusing again.

[0007] The technical problem solved by this invention can be achieved through the following specific technical solutions: The multivariate power time series forecasting method based on a hierarchical channel strategy includes the following steps: Step 1: Problem Definition and Data Preprocessing; Step 2: Perform bimodal Gaussian convolution processing on the input historical time series; Step 3: Perform depthwise grouped dilated convolution processing on the output of Step 2; Step 4: Perform a bottleneck-reversed pointwise convolution on the output of Step 3; Step 5: Transform and denormalize the data output from the pointwise convolution of the inverted bottleneck, and then output it.

[0008] Furthermore, in step 1, the problem definition is used to clarify the forecasting task, given a historical electricity time series. Predicting future electricity time series Where D is the number of variables, L is the historical window size, T is the vector transpose sign, and L' is the prediction length. This represents the input time series of the i-th channel. This indicates that the time series of the i-th channel will be output. The data is standardized using the z-score method. This process first converts the power time data into a form with a mean of 0 and a variance of 1. In the prediction stage, the output results are then denormalized to obtain the final prediction results.

[0009] Furthermore, in step 2, the bimodal Gaussian convolution method performs channel segmentation and convolution processing, specifically including: First, the input sequence is processed along the channel dimension. Divided into independent sequences where 1≤i≤N; Secondly, perform a one-dimensional convolution operation on each independent sequence, with 1 input and 1 output channel, a stride s=1, and a padding length of... This ensures that the lengths of the input and output sequences are consistent.

[0010] Furthermore, the convolution kernel parameters are initialized using a bimodal Gaussian distribution: , , ,, in, and Narrow Gaussian peaks and the wide Gauss peak The initialization parameters, w represents the parameters for bimodal Gaussian initialization. , , represents the standard deviation of the Gaussian distribution, k is the kernel size; narrow Gaussian peaks focus on capturing instantaneous features, while wide Gaussian peaks focus on capturing long-term trends, and i represents the kernel position index.

[0011] The convolutional output is fused with the residual connection through layer normalization: , , , , Where BGC(·) represents a bimodal Gaussian convolution operation, Let D represent the input of the i-th channel in the original timing sequence, where D is the number of channels. express The output of the bimodal Gaussian convolution operation is represented by X, where X represents the original input timing and LayerNorm represents the layer normalization operation. This is the final output result.

[0012] Furthermore, in step 3, the depth-group dilated convolution method performs channel-independent modeling: To address the problem that traditional one-dimensional convolution uses D×k convolution kernels to treat multivariate dependencies as depth dependencies, this method uses depthwise convolution to separate multivariate time series channels and independently capture the time information of each channel; Construct D distinct 1×k convolution kernels and establish a one-to-one mapping between the convolution kernels and channels: , , , , in, This represents the dilated convolutional layer corresponding to channel i. Represents the input sequence. Let X represent the input timing of the i-th channel, and let X represent the original input timing. The LayerNorm indicates the layer normalization operation, representing the output result. By introducing a dilation rate *r*, gaps are inserted between the elements of the convolution kernel, expanding the receptive field without increasing the number of parameters. The dilated convolution output of the *i*th channel is: ,in, is the kernel weight of the i-th channel, r is the dilation rate, and when r=1, it degenerates into a standard convolution; t is the time step index, and zero padding is used at the boundary to maintain the sequence length; s represents the position index of the convolution kernel.

[0013] A 7-layer depthwise separable dilated convolutional module is used to construct channel-independent modeling. The dilation coefficient of each layer increases geometrically: r=1,2,4,8,16,32,64. Small dilation rates capture short-term temporal correlations, while large dilation rates extract long-term temporal correlations, thereby improving the network's fitting ability while ensuring the channel independence strategy.

[0014] Furthermore, in step 4, the bottleneck-reversed pointwise convolution method performs feature re-fusion: For the input time series Pointwise convolution is used to achieve channel-dimensional feature fusion, employing N convolution kernels of size 1×D (kernel size 1, fusing information only along the channel dimension) to obtain the output. This operation is equivalent to applying a linear projection to the multivariate time series at each time step: .

[0015] To enhance feature representation capabilities and avoid information loss, an inverted bottleneck structure is designed: , , , , First, the number of channels is expanded to 4D through pointwise convolution. The GELU activation function introduces non-linearity, and then pointwise convolution reduces the number of channels back to the original dimension D to obtain the desired result. Finally, through layer normalization and residual connections, it is connected to the original input. The final output is obtained by fusion. Channel expansion effectively breaks the limitations of low-dimensional feature representation, while channel reduction ensures the availability of residual structure.

[0016] Furthermore, in step 5, the formula for the inverse normalization process is as follows: , in, and represents the standard deviation and mean of the input time series, respectively, and Z is the model output prediction.

[0017] Compared with the prior art, the present invention has the following advantages: This invention relates to FIFNet, a hierarchical channel strategy model specifically designed for multivariate power time series tasks. This model implements a fusion-independence-fusion hierarchical channel strategy by introducing a bimodal Gaussian convolution module, a depthwise grouped dilated convolution module, and a bottleneck-inverted pointwise convolution module. This fully exploits the correlation between the time domain and the channel domain, effectively overcoming the inherent limitations of existing models that employ a single strategy. Experiments show that the proposed method achieves performance improvements compared to seven mainstream state-of-the-art methods in the field of multivariate time series prediction, including iTransformer and Crossformer. Specifically, in 28 test metrics across four public datasets, almost all results rank in the top two, with most even ranking first. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] like Figure 1 The diagram shows the overall flowchart of this invention, with the middle section illustrating the forward propagation process of the model. The input multivariate time series data is sequentially passed through a bimodal Gaussian convolutional layer, a depthwise grouped dilated convolutional layer, and a bottleneck-inverted pointwise convolutional layer to obtain the final output. These three modules employ channel-dependent, channel-independent, and channel-dependent channel strategies, respectively (where CD represents channel-dependent and CI represents channel-independent).

[0021] This invention provides a multivariate power time series forecasting method based on a hierarchical channel strategy, the specific steps of which are as follows: Step 1: Problem definition and data preprocessing.

[0022] The problem definition clarifies the forecasting task, given a historical electricity time series. Predicting future electricity time series Where D is the number of variables, L is the historical window size, T is the vector transpose sign, and L' is the prediction length. This represents the input time series of the i-th channel. This indicates that the time series of the i-th channel is output.

[0023] Data preprocessing employs normalization, specifically z-score normalization, which converts the electricity time data into a form with a mean of 0 and a variance of 1. During the forecasting phase, the output is then denormalized to obtain the final forecast result. This sequential normalization and denormalization process ensures the validity and reliability of the forecast results.

[0024] Step 2: Perform bimodal Gaussian convolution on the input historical time series.

[0025] The bimodal Gaussian convolution method performs channel segmentation and convolution processing.

[0026] First, the input sequence is processed along the channel dimension. Divided into independent sequences where 1≤i≤N; Secondly, perform a one-dimensional convolution operation on each independent sequence, with 1 input and 1 output channel, a stride s=1, and a padding length of [missing information]. This ensures that the lengths of the input and output sequences are consistent.

[0027] The convolution kernel parameters are initialized using a bimodal Gaussian distribution: , , , in, and Narrow Gaussian peaks and the wide Gauss peak The initialization parameters, w represents the parameters for bimodal Gaussian initialization. , , represents the standard deviation of the Gaussian distribution, k is the kernel size; narrow Gaussian peaks focus on capturing instantaneous features, while wide Gaussian peaks focus on capturing long-term trends, and i represents the kernel position index.

[0028] The convolutional output is fused with the residual connection through layer normalization: , , , , Where BGC(·) represents a bimodal Gaussian convolution operation, Let D represent the input of the i-th channel in the original timing sequence, where D is the number of channels. express The output of the bimodal Gaussian convolution operation is represented by X, where X represents the original input timing and LayerNorm represents the layer normalization operation. This is the final output result.

[0029] Step 3: Perform depth-group dilated convolution processing on the output of Step 2.

[0030] To address the issue that traditional one-dimensional convolution uses D×k convolution kernels to treat multivariate dependencies as depth dependencies, this method employs depthwise convolution to separate multivariate time series channels and independently capture the time information of each channel.

[0031] Construct D distinct 1×k convolution kernels and establish a one-to-one mapping between the convolution kernels and channels: , , , , in, This represents the dilated convolutional layer corresponding to channel i. Represents the input sequence. Let X represent the input timing of the i-th channel, and let X represent the original input timing. The LayerNorm indicates the layer normalization operation, representing the output result. By introducing a dilation rate *r*, gaps are inserted between the elements of the convolution kernel, expanding the receptive field without increasing the number of parameters. The dilated convolution output of the *i*th channel is: ;in, is the kernel weight of the i-th channel, r is the dilation rate, and when r=1, it degenerates into a standard convolution; t is the time step index, and zero padding is used at the boundary to maintain the sequence length; s represents the position index of the convolution kernel.

[0032] A 7-layer depthwise separable dilated convolutional module is used to construct channel-independent modeling. The dilation coefficient of each layer increases geometrically: r=1,2,4,8,16,32,64. Small dilation rates capture short-term temporal correlations, while large dilation rates extract long-term temporal correlations, thereby improving the network's fitting ability while ensuring the channel independence strategy.

[0033] Step 4: Perform a bottleneck-reversed pointwise convolution on the output of Step 3.

[0034] For the input time series Pointwise convolution is used to achieve channel-dimensional feature fusion, employing N convolution kernels of size 1×D (kernel size 1, fusing information only along the channel dimension) to obtain the output. This operation is equivalent to applying a linear projection to the multivariate time series at each time step: .

[0035] To enhance feature representation capabilities and avoid information loss, an inverted bottleneck structure is designed: , , , , That is, firstly, the number of channels is expanded to 4D through pointwise convolution to obtain... The GELU activation function introduces non-linearity, and then pointwise convolution reduces the number of channels back to the original dimension D to obtain the desired result. Finally, through layer normalization and residual connections, it is connected to the original input. The final output is obtained by fusion. Channel expansion effectively breaks the limitations of low-dimensional feature representation, while channel reduction ensures the availability of residual structure.

[0036] Step 5: Transform and denormalize the data output from the pointwise convolution of the inverted bottleneck, and then output it.

[0037] The formula for the inverse normalization process is as follows: , in, and represents the standard deviation and mean of the input time series, respectively, and Z is the model output prediction.

[0038] Example This section details the specific implementation methods of the present invention: To verify the effectiveness of the proposed FIFNet (hierarchical channel strategy) model, extensive experiments were conducted on various datasets from the power industry: It includes four ETT datasets (ETTm1, ETTm2, ETTh1, ETTh2), and some statistical properties of the datasets can be seen in Table 1.

[0039] Table 1

[0040] It uses some state-of-the-art (SOTA) models from recent years as benchmarks, including Informer, Autoformer, PatchTST, Pyraformer, Fedformer, Crossformer, and iTransformer.

[0041] These models are not only the most advanced models of the past three years, but also encompass both channel-independent and channel-dependent paradigms, making them highly suitable as benchmarks. To ensure fair and objective comparisons among all models, a fixed backtracking window size of 𝐿=96 was uniformly adopted. This approach aims to maintain a level playing field and eliminate any potential biases.

[0042] To accurately evaluate model performance, this invention calculates mean squared error (MSE) and mean absolute error (MAE) as key evaluation metrics. These metrics were chosen because they can comprehensively and accurately assess the effectiveness of the model.

[0043] The ConvMixer model uses the Adam optimizer with a carefully selected learning rate of 0.0001; the default loss function is L2 loss, and the number of training epochs is fixed at 10 throughout the entire training phase.

[0044] All experiments were carefully conducted on an advanced and high-performance NVIDIA GeForce RTX 3090 graphics card, and were performed using Python 3.8 and Torch 1.7.1.

[0045] In the bimodal Gaussian convolution module, the key parameters are set as follows: kernel size = 25, input channel = 1, output channel = 1, padding = 12, stride = 1. Details regarding the initialization settings of the bimodal Gaussian parameters have been described in detail in previous texts, so they will not be repeated here.

[0046] In a depthwise separable dilated convolutional model, time series of different dimensions are decoupled, and a different convolutional kernel is applied to each series. For the convolutional layer corresponding to each channel of the time series, the parameters are set as follows: number of layers = 7, kernel size = 7, input channel = 1, output channel = 1, dilation size = (Layers=0, 1, ..., 6), Fill = (kernel size - 1) × expansion size.

[0047] In the anti-bottleneck pointwise convolution module, as mentioned above, it is essentially an MLP layer with a hidden layer size of 4 × DIM, where DIM is the number of channels in the time series.

[0048] Table 2 shows the comprehensive performance pairs of FIFNet and various benchmark models on four datasets.

[0049] It is important to note that, in terms of evaluation metrics, Mean Squared Error (MSE) and Mean Absolute Error (MAE) are used as quantitative standards. Smaller values ​​for these two metrics indicate better predictive performance. To highlight performance comparisons, the best results are indicated in bold in the table, and the second-best results are indicated by underlines.

[0050] Overall, FIFNet achieves state-of-the-art performance on all four datasets. Notably, almost all results rank in the top two, with most even ranking first. Compared to classic channel fusion and channel-independent models, FIFNet demonstrates a significant performance improvement.

[0051] Table 2

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multivariate power time series forecasting method based on a hierarchical channel strategy, characterized in that, Includes the following steps: Step 1: Problem Definition and Data Preprocessing; Step 2: Perform bimodal Gaussian convolution processing on the input historical time series; Step 3: Perform depthwise grouped dilated convolution processing on the output of Step 2; Step 4: Perform a bottleneck-reversed pointwise convolution on the output of Step 3; Step 5: Transform and denormalize the data output from the pointwise convolution of the inverted bottleneck, and then output it.

2. The multivariate power time series forecasting method based on a hierarchical channel strategy according to claim 1, characterized in that, In step 1, given a historical electricity time series Predicting future electricity time series Where D is the number of variables, L is the historical window size, T is the vector transpose sign, and L' is the prediction length. This represents the input time series of the i-th channel. This indicates that the time series of the i-th channel will be output. The data is standardized using the z-score method. This process first converts the power time data into a form with a mean of 0 and a variance of 1. In the prediction stage, the output results are then denormalized to obtain the final prediction results.

3. The multivariate power time series forecasting method based on a hierarchical channel strategy according to claim 1, characterized in that, In step 2, bimodal Gaussian convolution is used to perform channel segmentation and convolution processing: First, the input sequence is processed along the channel dimension. Divided into independent sequences Where 1≤i≤N; Secondly, perform a one-dimensional convolution operation on each independent sequence, with 1 input and 1 output channel, a stride s=1, and a padding length of... This ensures that the lengths of the input and output sequences are consistent.

4. The multivariate power time series forecasting method based on a hierarchical channel strategy according to claim 3, characterized in that, The convolution kernel parameters are initialized using a bimodal Gaussian distribution: ， ， ， in, and Narrow Gaussian peaks and the wide Gauss peak The initialization parameters, w represents the parameters for bimodal Gaussian initialization. , , represents the standard deviation of the Gaussian distribution, k is the kernel size; narrow Gaussian peaks focus on capturing instantaneous features, while wide Gaussian peaks focus on capturing long-term trends, and i represents the kernel position index; The convolutional output is fused with the residual connection through layer normalization: ， ， ， ， Where BGC(·) represents a bimodal Gaussian convolution operation, Let D represent the input of the i-th channel in the original timing sequence, where D is the number of channels. express The output of the bimodal Gaussian convolution operation is represented by X, where X represents the original input timing and LayerNorm represents the layer normalization operation. This is the final output result.

5. The multivariate power time series forecasting method based on a hierarchical channel strategy according to claim 1, characterized in that, In step 3, depthwise convolution is used to separate multivariate time series channels and independently capture the time information of each channel; D different 1×k convolution kernels are constructed, and a one-to-one mapping relationship between the convolution kernels and channels is established: ， ， ， ， in, This represents the dilated convolutional layer corresponding to channel i. Represents the input sequence. Let X represent the input timing of the i-th channel, and let X represent the original input timing. The LayerNorm indicates the layer normalization operation, representing the output result. By introducing a dilation rate *r*, gaps are inserted between the elements of the convolution kernel, expanding the receptive field without increasing the number of parameters. The dilated convolution output of the *i*th channel is: ,in, is the kernel weight of the i-th channel, r is the dilation rate, and when r=1, it degenerates into a standard convolution; t is the time step index, and zero padding is used at the boundary to maintain the sequence length; s represents the position index of the convolution kernel.

6. The multivariate power time series forecasting method based on a hierarchical channel strategy according to claim 1, characterized in that, In step 4, for the input time series Pointwise convolution is used to achieve channel-dimensional feature fusion, employing N convolution kernels of size 1×D to obtain the output. This operation is equivalent to applying a linear projection to the multivariate time series at each time step: ; To enhance feature representation capabilities and avoid information loss, a bottleneck inversion structure is designed: First, the number of channels is expanded to 4D through pointwise convolution, nonlinearity is introduced through the GELU activation function, and then the number of channels is reduced back to the original dimension D through pointwise convolution. Finally, it is fused with the original input through layer normalization and residual connections. Channel expansion effectively breaks the limitations of low-dimensional feature representation, and channel reduction ensures the usability of the residual structure.

7. The multivariate power time series forecasting method based on a hierarchical channel strategy according to claim 1, characterized in that, In step 5, the formula for the inverse normalization process is as follows: ， in, and represents the standard deviation and mean of the input time series, respectively, and Z is the model output prediction.

Images