A time series prediction method, device and medium based on an MPDLinear model

Abstract

The present application relates to the technical field of weather prediction, and particularly relates to a time series prediction method, device and medium based on an MPDLinear model; in the method, a multi-component decomposition module decomposes a time series into different sequence components, a feature-time dimension hybrid self-attention mechanism module dynamically adjusts the attention score of different time steps and features in the prediction output by combining the self-attention mechanism of the feature and time dimension, a multi-channel independent modeling and weighted prediction module models an independent channel for each attentive different time series component and outputs the component, feature dimension weight of the different time series component, and the multi-component decomposition module obtains a final prediction result through reverse summation by using the weight parameter; the device and the medium are used for saving a computer program, and when the program is executed, the time series prediction method is completed; the method not only overcomes the limitations of the LTSF-Linear-based model in processing complex multi-dimensional data, but also maintains the complexity lower than the Transformer-based model, and has higher generalization and prediction ability for complex time series.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of weather prediction, and particularly relates to a time series prediction method, device and medium based on an MPDLinear model. BACKGROUND

[0002] In the current data-driven era, time series data exists widely in various fields, and its prediction task (TSF) is also widely applied in various scenarios, including but not limited to traffic flow prediction, energy scheduling, financial investment, smart wear, intelligent weather, etc.; for a time series containing N variables (features), given the historical data where L is the size of the lookback window (input time step), is the value of variable (feature) i at time step t, X represents the values of features 1 to N from the 1st time step to the Lth time step, and the time series prediction task is to predict the value of the sequence, which represents the values of features 1 to N at future T time steps (prediction time steps), when T>1, it is multi-step prediction, and the iterative multi-step prediction (IMS) method learns a single-step prediction and iteratively applies the single-step prediction to obtain multi-step prediction, while the direct multi-step prediction (DMS) directly predicts the values of the prediction target at multiple time steps at one time. Compared with the prediction result of the direct multi-step prediction DMS, the mean square error of the IMS prediction is smaller, but in the iterative process, each step of the IMS prediction depends on the prediction result of the last step, and is inevitably affected by error accumulation; therefore, when there is a very accurate single-step predictor, and T is relatively small, the accuracy of the IMS prediction is the best, on the contrary, when it is difficult to obtain an unbiased single-step predictor, or T is very large, the accuracy of the DMS prediction is better.

[0003] Informer, Autoformer, FEDformer represent the latest progress of Transformer-based time series forecasting models; Informer solves the computational efficiency problem of ultra-long time series through sparse attention mechanism, Autoformer captures the trend and periodic characteristics in time series through automatic decomposition mechanism, and FEDformer combines frequency domain and time domain modeling to further improve performance and efficiency in complex long sequence prediction; LTSF-Linear model is a new group of linear models proposed in recent years, and its prediction method is DMS direct multi-step prediction. The core working method of Dlinear is to directly linearly map the historical time series data through weighted operation to predict the value of the future time step, but the design defects and limitations of the DLinear model in LTSF-Linear variant mainly manifest in that it cannot effectively capture complex nonlinear trends and medium-long periodic trends and ignores the mutual relationship between different time steps and features: First, the ability to capture complex multivariate time series change patterns is limited, and more complex components that may exist in the time series are ignored; Second, the way the model output prediction result is generated by multiple components is not reasonable and robust, and the flexibility is low; Third, the DLinear model shares modeling in multiple channels and cannot handle the differences between multiple channels; Fourth, DLinear cannot effectively extract the complex mutual relationship between the time-feature dimensions of the dataset, resulting in that the prediction accuracy of DLinear is too dependent on long input sequences, and when the input sequence is short, the prediction accuracy will be low due to the inability to capture complex change patterns.

[0004] In the prior art, the paper entitled "Are Transformers Effective for Time Series Forecasting" (Zeng, A., Proceedings of the 39th International Conference on Machine Learning (ICML) (2022)) discloses a time series forecasting model based on a simple linear layer, which realizes prediction performance exceeding Transformer-based time series forecasting models through simple component decomposition of input sequences and mapping of a single linear layer, and has the advantages of simplicity and efficiency, but has problems such as limited ability to capture time series change patterns, unreasonable and non-robust multi-component joint effect generation method, etc.

[0005] In the prior art, the patent with the publication number CN202411439254 and the name of "ultra-short-term wind power prediction method based on improved loss function and patch time sequence Transformer network" discloses a method for ultra-short-term wind power prediction, which mainly improves the Transformer network structure and the multivariate nonlinear loss function to perform high-precision prediction of wind power. However, when processing multi-dimensional input data, the method has insufficient modeling capability for complex interactions between input features, especially when processing high-dimensional and noisy time series data, the prediction accuracy is low and the stability is poor. SUMMARY

[0006] In order to overcome the shortcomings of the above prior art, the purpose of the present application is to provide a time series prediction method, device and medium based on MPDLinear model, wherein the time series prediction method decomposes time series data into different time series components through a multi-component decomposition module, a feature-time dimension hybrid self-attention mechanism module and a multi-channel independent modeling weighted prediction module are used to complete and obtain the preliminary prediction result of the different time series components after attention is applied, the self-learning weight vector module uses the above preliminary prediction result, and then obtains the component dimension weight and the feature dimension weight of the different time series components through back propagation calculation and loop traversal. Finally, the multi-component decomposition module performs weighted calculation and decomposition on the preliminary prediction result according to the component dimension weight and the feature dimension weight of the different time series components to obtain the final prediction result. This method overcomes the limitations of the LTSF-Linear-based model when processing complex multi-dimensional data, while maintaining the complexity of the Transformer-based model, and overall improves the prediction accuracy and stability of the model on time series.

[0007] To achieve the above purpose, the technical solutions adopted by the present application are as follows:

[0008] In a first aspect, a time series prediction method based on an MPDLinear model includes the following steps:

[0009] S1: After the multi-component decomposition module in the MPDLinear model receives the binary data stream of the weather data set, the binary data stream is deserialized and standardized into a structure that can be received by the MPDLinear model. The multi-component decomposition module decomposes the receivable structure into different time series components and outputs them to the multi-channel independent modeling weighted prediction module, the self-learning weight vector module and the feature-time dimension hybrid self-attention mechanism module in the MPDLinear model respectively;

[0010] S2: The feature-time dimension hybrid self-attention mechanism module receives the different time sequence components in step S1, performs dimension rearrangement processing, and generates dimension-rearranged different time sequence components. Then, the dimension-rearranged different time sequence components are processed through the self-attention mechanism layer of the feature-time dimension hybrid self-attention mechanism module to obtain the attention scores of the different time sequence components. The attention scores of the different time sequence components are further processed to obtain the attention weights of the different time sequence components and output to the multi-channel independent modeling weighted prediction module.

[0011] S3: The multi-channel independent modeling weighted prediction module receives the different time sequence components in step S1 and applies the attention weights of the different time sequence components in step S2. After matrix multiplication, the attentioned different time sequence components are obtained. The multi-channel independent modeling weighted prediction module models the independent channel for each attentioned different time sequence component and outputs the preliminary result sequence to the feature-time dimension hybrid self-attention mechanism module.

[0012] S4: The feature-time dimension hybrid self-attention mechanism module obtains the preliminary result sequence in step S3. The feature dimension self-attention mechanism layer and the prediction window time dimension self-attention mechanism layer of the module are processed in turn to obtain the preliminary prediction result of the different time sequence components after applying attention and output to the self-learning weight vector module and the multi-component decomposition module.

[0013] S5: The self-learning weight vector module receives the different time sequence components in step S1 and the preliminary prediction result in step S4. The parameters of the different time sequence component weights and the different time sequence feature weights in the self-learning weight vector module are calculated and updated when the preliminary prediction result is back propagated. After the parameters of the different time sequence component weights and the different time sequence feature weights are looped, the component dimension weight and the feature dimension weight of the different time sequence components are finally obtained and output to the multi-component decomposition module.

[0014] S6: The multi-component decomposition module receives the preliminary prediction result in step S4. The preliminary prediction result is weighted according to the component dimension weight and the feature dimension weight of the different time sequence components in step S5, and the weighted calculation result is summed in reverse according to the decomposition method of the multi-component decomposition module to obtain the final prediction result of the weather data set.

[0015] Further, the different time sequence components in step S1 include: linear trend component (Trend), nonlinear trend component (Nonlinear Trend), short cycle component (Short-Cycle-Seasonality), long cycle component (Long-Cycle-Periodic), and residual noise component (Residual). The different time sequence component extraction steps are as follows:

[0016] S11: The Simple Moving Average (SMA) based on a sliding window smooths the acceptable structure described in step S1. It achieves a simple moving average by defining a moving average (moving_avg) and obtains the linear trend component.

[0017] S12: The nonlinear trend component in the acceptable structure described in step S1 is extracted by using a dual nonlinear mapping. Specifically, a fully connected Linear layer is first used to perform linear mapping on the time dimension or the feature dimension respectively. Then, the ReLU nonlinear activation function is applied. Finally, a fully connected Linear layer is used to perform linear transformation on the remaining dimensions, and the result of the dual nonlinear mapping is obtained, which is the nonlinear trend component.

[0018] S13: First, remove the linear trend components and nonlinear trend components extracted in steps S11 and S12 from the acceptable structure (i.e., the original sequence) to obtain the remaining residual sequence. Then, apply the simple moving average (SMA) operation to the remaining residual sequence to extract the short-period components. The value of the sliding window used in this step is smaller than the size of the sliding window used to extract the linear trend components in step S11.

[0019] S14: Remove the short-period component from the remaining residual sequence obtained in step S13 to obtain the remaining sequence. Use the Discrete Fourier Transform (DFT) method to extract the medium- and long-period components from the remaining sequence.

[0020] S15: Remove the medium- and long-period components from the remaining sequence described in step S14 to obtain the residual noise components.

[0021] Furthermore, in step S3, the multi-channel independent modeling weighted prediction module completes the modeling of independent channels through a feature-independent linear layer. The formula for the feature-independent linear layer is as follows:

[0022] (Y (bs,pl) =X (bs,sl) ·W (sl,pl) +bias (bs,1) *channels

[0023] Among them, Y (bs,pl) The output of the model for each channel is independent, X (bs,c,sl) W is the tensor of the input linear layer. (sl,pl) The bias is used to map the weight matrix to the input time step. (bs,1) This is the bias vector for the linear model.

[0024] Further, the feature dimension self-attention mechanism layer and the prediction window time dimension self-attention mechanism layer in step S4 are processed as follows: the input is divided into the time dimension and the feature dimension, the time dimension is divided into the input time step (the look-back window) time dimension before entering the multi-channel independent modeling weighted prediction module and the prediction time step time dimension output from the multi-channel independent modeling weighted prediction module:

[0025] S41: The feature dimension, the input time step time dimension and the prediction time step time dimension are respectively generated into a query (Q) vector, a key (K) vector and a value (V) vector after linear transformation;

[0026] S42: The key vector in step S42 is transposed to obtain K T , and the dot product (Q·K T ) of the query vector and K T is calculated to obtain the preliminary attention score;

[0027] S43: The output dimension of the dot product in step S42 is scaled by a scaling factor to obtain the scaled dot product result, that is, the attention score;

[0028] S44: The attention score in step S43 is subjected to a SoftMax operation to convert it into an attention weight, which represents the attention of the query vector of each element on all element key vectors;

[0029] wherein, for each query position in the sequence, the SoftMax converts the attention score into a probability distribution of all key vector positions, and for the input vector as (i,j) = [as (i,1), as (i,2) ,...,as (i,n) ], the formula of the SoftMax is:

[0030]

[0031] wherein, as = [as (i,1), as (i,2) ,...,as (i,n) ] is an attention score containing n elements, which represents the correlation degree of q i and [k1, k2,..., k n ], exp(as (i,j) ) is the exponential of the jth element in the input attention score vector, and the sum of all input attention score exponents is

[0032] S45: Apply the attention weights of step S44 to the value vector using matrix multiplication (MatMul) to obtain preliminary prediction results of the different timing components after applying attention, and output the preliminary prediction results of the different timing components after applying attention to the learning weight vector module and the multi-component decomposition module through an aggregation operation.

[0033] Further, the specific process of step S5 is as follows:

[0034] S51: Define the preliminary prediction results of the different timing components of step S4 as S i , where i = 1, 2,..., n, and assign a component dimension weight w i to each of the different timing components in the component dimension. i , where j = 1, 2,..., channels, and assign a feature dimension weight V[i,j] to each feature in S

[0035] S52: Update w i and V[i,j] of step S51 through the backpropagation mechanism during the backpropagation training process, and the gradients of w and V[i,j] are calculated according to the loss function Loss. Thus, the component dimension weight and the feature dimension weight are continuously adjusted during the training process, and the component dimension weight and the feature dimension weight are as shown in the following formula:

[0036]

[0037] where η is the learning rate, w i and V[i,j] are the component dimension weight and the feature dimension weight of the different timing components, respectively.

[0038] Further, the specific process of step S6 is as follows:

[0039] S61: Calculate the different timing components after weighting by the component dimension weight w i using the preliminary prediction results of the different timing components of step S4 and the component dimension weight w of step S52. series ,

[0040] S62: Weight each feature of the preliminary prediction results of the different timing components of step S4 according to the feature dimension weight of step S52, and the specific formula is as follows:

[0041]

[0042] where, is the weighted output of the jth feature channel of the ith different time sequence component, and V[i,j] is the corresponding feature dimension weight;

[0043] All the feature dimension weighted channels are summarized to obtain the comprehensive output F channels ,

[0044]

[0045] S63: The comprehensive output F series and the comprehensive output F channels of the component dimension and the feature dimension are obtained respectively. final The final prediction result F i of the weather data set is obtained after weighting in the component dimension and the feature dimension, and the specific formula is as follows:

[0046]

[0047] where, S i [:,j,:] represents the value of the ith different time sequence component on the jth channel.

[0048] Further, the specific process of step S14 is as follows:

[0049] S141: The remaining sequence in step S14 is taken as the input of extracting the medium-long period component, and the remaining sequence is a time sequence {x1,x2,...,x n} with a length of N, which is split into even and odd parts:

[0050] Even part: x 2m , where

[0051] Odd part: x 2m+1 , where

[0052] The even and odd parts are expanded by the formula of discrete Fourier transform (DFT), and the following formula can be obtained:

[0053]

[0054] Further simplifying the formula of the previous step, we get:

[0055]

[0056] where,

[0057] that is, the DFT

[0058] DFT of odd part

[0059] Thus, the following equation can be obtained:

[0060]

[0061] By the above splitting, the discrete Fourier transform of the remaining sequence is decomposed into the discrete Fourier transforms of two sub-sequences of length This decomposition can continue recursively until the length of the sequence is reduced to one (the smallest unit), so the discrete Fourier transform can be expressed by the following equation:

[0062]

[0063] where is a twiddle factor (or called "twiddle kernel"), the sequence is recursively split into two sub-sequences of length : even part and odd part, at each recursive level, the discrete Fourier transform combines the results of even and odd parts by addition and multiplication of twiddle factors to form the frequency domain data;

[0064] S142: high frequency filtering is performed on the frequency domain data of step S141 to obtain low frequency components;

[0065] S143: inverse Fourier transform (IFT) is used to convert the low frequency components of step S142 back to time domain to generate final medium-long period components, where the inverse Fourier transform (IFT) formula is as follows:

[0066]

[0067] where x n is the value of the time domain signal at the nth time step, X k is the complex value of the kth frequency component of the frequency domain data, N is the total length of the signal (i.e. the number of frequency domain points), is the twiddle factor of inverse Fourier transform, and j is the imaginary unit (j 2 = -1).

[0068] In a second aspect, an electronic device includes a memory and a processor:

[0069] The memory is configured to store a computer program for implementing the MPDLinear model-based time series prediction method.

[0070] The processor is configured to execute the computer program to implement the MPDLinear model-based time series prediction method.

[0071] In a third aspect, a computer readable storage medium stores a computer program, which, when executed by a processor, implements the time series prediction method based on the MPDLinear model.

[0072] Compared with the prior art, the present application has the following beneficial effects:

[0073] 1. The multi-component decomposition module in the present application decomposes the time series into linear trend, nonlinear trend, short period, medium-long period, residual noise and other components, enhancing the adaptability of the model to diversified patterns in the time series;

[0074] 2. The self-learning weight vector module in the present application introduces a self-learning weight vector in the sequence and feature dimensions, realizes weighted summation of components and features, and can dynamically and accurately balance the contributions of each sequence and feature;

[0075] 3. The multi-channel independent modeling and weighted prediction module in the present application solves the homogenization problem of traditional models in feature modeling by independently constructing a linear layer for each feature channel, effectively improving the ability of the model to model different channel characteristics;

[0076] 4. The feature-time dimension hybrid self-attention mechanism module in the present application combines the self-attention mechanisms of the feature and time dimensions, dynamically adjusts the attention of the prediction output to different time steps and features, and further improves the ability of the model to capture complex patterns.

[0077] In summary, the present application not only overcomes the limitations of the LTSF-Linear-based model in processing complex multi-dimensional data, but also maintains a lower complexity than the Transformer-based model. Through the functions of the four modules and their synergistic cooperation, the precision and stability of the entire model in time series prediction are improved, the modeling and understanding ability of the model for complex time series patterns is enhanced, and the model has high generalization and prediction ability for complex time series. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 is a flowchart of the time series prediction method based on the MPDLinear model of the present application.

[0079] Figure 2 is a structure diagram of the MPDLinear full model of the present application.

[0080] Figure 3 is a structure diagram of the double nonlinear mapping single layer of the present application.

[0081] Figure 4 is a structure diagram of the self-learning weight vector module of the present application.

[0082] Figure 5 is a schematic diagram of a multi-channel independent modeling weighted prediction module structure of the present application.

[0083] Figure 6 is a schematic diagram of internal mapping structure of a feature independent linear layer in a multi-channel independent modeling weighted prediction module of the present application.

[0084] Figure 7 is a schematic diagram of internal structure of a feature-time dimension hybrid self-attention mechanism module of the present application.

[0085] Figure 8 is a decomposition diagram of an original sequence on a humidity feature of a Beijing weather data set in embodiment one of the present application.

[0086] Figure 9 is a distribution diagram of a linear trend component on a humidity feature of a Beijing weather data set in embodiment one of the present application.

[0087] Figure 10 is a distribution diagram of a nonlinear trend component on a humidity feature of a Beijing weather data set in embodiment one of the present application.

[0088] Figure 11 is a distribution diagram of a short-period component on a humidity feature of a Beijing weather data set in embodiment one of the present application.

[0089] Figure 12 is a distribution diagram of a medium-long period component on a humidity feature of a Beijing weather data set in embodiment one of the present application.

[0090] Figure 13 is a distribution diagram of a residual noise component on a humidity feature of a Beijing weather data set in embodiment one of the present application. DETAILED DESCRIPTION

[0091] The present application will be further described in detail below in combination with Figures 1 to 13 and embodiment one:

[0092] In a first aspect, a time series prediction method based on an MPDLinear model is provided, wherein the MPDLinear model comprises a multi-component decomposition module, a self-learning weight vector module, a multi-channel independent modeling weighted prediction module, and a feature-time dimension hybrid self-attention mechanism module.

[0093] The multi-component decomposition module deserializes the binary data stream of the weather data set and normalizes it into a structure acceptable to the MPDLinear model, and the multi-component decomposition module further decomposes the acceptable structure into different time sequence components, while the module can separately calculate the component dimension weight and the feature dimension weight of the different time sequence components, and obtain the final prediction result of the weather data set by reverse summation of the weighted calculation result according to the decomposition method of the multi-component decomposition module;

[0094] The self-learning weight vector module calculates the parameters of the different time sequence component weights and the different time sequence feature weights in the self-learning weight vector module when the different time sequence component preliminary prediction results are back propagated, and obtains the component dimension weight and the feature dimension weight of the different time sequence components after cyclically traversing the parameters of the different time sequence component weights and the different time sequence feature weights.

[0095] The feature-time dimension hybrid self-attention mechanism module can obtain the attention score of the different time sequence components and the attention weight of the different time sequence components by processing the different time sequence components through the self-attention mechanism layer of the window time dimension in the module, and can obtain the preliminary prediction result of the different time sequence components after applying attention by sequentially processing the different time sequence components through the feature dimension self-attention mechanism layer and the prediction window time dimension self-attention mechanism layer of the module.

[0096] The multi-channel independent modeling weighted prediction module obtains the attentioned different time sequence components by matrix multiplication after using the different time sequence components and the attention weight of the different time sequence components, and can model the independent channels for each attentioned different time sequence component.

[0097] Figure 2 The overall structure of the MPDLinear full-quantity model is shown, wherein the multi-component decomposition module decomposes the time sequence data into different time sequence components, the self-learning weight vector module introduces the self-learning weight vector in the component and feature dimensions of the different time sequence components to realize weighted summation of the component and the feature, the multi-channel independent modeling weighted prediction module independently constructs a linear layer for each feature channel, and the feature-time dimension hybrid self-attention mechanism module combines the self-attention mechanisms of the feature and the time dimension to dynamically adjust the attention of the prediction output to different time steps and features.

[0098] As Figure 1 The time sequence prediction method according to the present application comprises the following steps:

[0099] Preprocessing: The multivariate time series data of the weather dataset, which contains multiple features (i.e., channels) in multiple time steps, is processed into a binary data stream through the Jackson framework and output to the multi-component decomposition module. Each feature represents a time-dependent variable.

[0100] S1: After receiving the binary data stream of the weather dataset, the multi-component decomposition module in the MPDLinear model deserializes and normalizes the binary data stream into a structure that the MPDLinear model can receive. The multi-component decomposition module decomposes the received structure into different time series components and outputs them to the multi-channel independent modeling weighted prediction module, the self-learning weight vector module, and the feature-time dimension hybrid self-attention mechanism module in the MPDLinear model. The decomposed different time series components are used for the calculation of the self-learning weight vector module, the multi-channel independent modeling weighted prediction module, and the feature-time dimension hybrid self-attention mechanism module.

[0101] S2: After receiving the different time series components from step S1, the feature-time dimension hybrid self-attention mechanism module performs dimension rearrangement processing. Specifically, the dimension rearrangement function in pytorch is used for simple dimension rearrangement (permute) to generate dimension-rearranged different time series components. Then, the dimension-rearranged different time series components are processed through the self-attention mechanism layer in the feature-time dimension hybrid self-attention mechanism module to obtain the attention scores of the different time series components. The attention scores of the different time series components are the attention scores of the different time series components in the look-back window time dimension. The attention scores of the different time series components are further processed through the Softmax function in this module to obtain the attention weights of the different time series components and output them to the multi-channel independent modeling weighted prediction module. The attention weights of the different time series components are the attention weights of the same channel in the same batch and different time steps.

[0102] S3: After receiving the different time series components from step S1 and applying the attention weights of the different time series components from step S2, the multi-channel independent modeling weighted prediction module performs matrix multiplication to obtain attentioned different time series components. The multi-channel independent modeling weighted prediction module models each independent channel for each attentioned different time series component and outputs the preliminary result sequence to the feature-time dimension hybrid self-attention mechanism module.

[0103] Figure 2 The medium purple superimposed box represents that each channel (feature) in each time series component is modeled and processed through an independent linear layer (e.g., Trend_Channel[i]_Linear, Nonlinear_Trend_Channel[i]_Linear, etc., where i is the channel number).

[0104] S4: The hybrid self-attention mechanism module in the feature-time dimension acquires the preliminary result sequence in step S3, sequentially passes through the feature dimension self-attention mechanism layer and the prediction window time dimension self-attention mechanism layer of the module, obtains the preliminary prediction results of different time sequence components after applying attention, and outputs to the self-learning weight vector module and the multi-component decomposition module;

[0105] S5: After the self-learning weight vector module receives the different time sequence components in step S1 and the preliminary prediction results in step S4, the parameters of the different time sequence component weights and the different time sequence feature weights in the self-learning weight vector module are calculated using the preliminary prediction results during back propagation, the component dimension weights and the feature dimension weights of the different time sequence components are obtained through the cyclic traversal of the parameters of the different time sequence component weights and the different time sequence feature weights, and are output to the multi-component decomposition module; wherein, in the model training stage, the module will update the initialization parameters during back propagation, and dynamically adjust the effective weights of different components and channels according to the output during forward propagation;

[0106] S6: The multi-component decomposition module receives the preliminary prediction results in step S4, and respectively performs weighted calculation on the preliminary prediction results according to the component dimension weights and the feature dimension weights of the different time sequence components in step S5, and obtains the final prediction results of the weather data set by reverse summation according to the decomposition method of the multi-component decomposition module.

[0107] The application designs a multi-component decomposition module to extract and model each component by different decomposition methods, so as to better capture multiple change patterns in complex time series.

[0108] Further, the different time sequence components in step S1 include: linear trend component (Trend), nonlinear trend component (Nonlinear Trend), short cycle component (Short-Cycle-Seasonality), long cycle component (Long-Cycle-Periodic) and residual noise component (Residual), and the different time sequence component extraction steps are as follows:

[0109] S11: Linear trend component extraction, which aims to extract the long-term trend in the time series and eliminate short-term fluctuations; the input receivable structure is smoothed by a simple moving average (SMA) based on a sliding window, and the linear trend component is obtained by defining a moving average (moving_avg) to realize the simple moving average;

[0110] The one-dimensional average pooling layer nn.AvgPool1d is used to calculate the average value by applying a sliding window on the time series, and the calculation frequency is controlled by adjusting the stride. The calculation frequency determines the calculation time points to be included in the moving average. The formula is:

[0111]

[0112] wherein SMA t represents the moving average at the t-th time step; X t represents the value of the time series at the t-th time step; k is the size of the sliding window, representing the number of time steps to be considered when calculating the moving average;

[0113] S12: Nonlinear trend component extraction, which aims to further capture complex long-term nonlinear changes based on the trend component, to fit the mutation points or irregular points in the data as much as possible. The core of this part is double nonlinear mapping;

[0114] As shown in Figure 3 , the double nonlinear mapping is used to extract the nonlinear trend component in the receivable structure described in step S1. Specifically, a fully connected layer Linear is used to perform linear mapping on the time dimension or the feature dimension respectively, learning the linear relationship between each time step or feature. Then, a ReLU nonlinear activation function is applied to enable the model to fit more complex relationships. Finally, a fully connected layer Linear is used for linear transformation of the remaining dimensions, and the result of the double nonlinear mapping, i.e., the nonlinear trend component, is obtained.

[0115] In actual calculation, the trend sequence is first nonlinearly mapped in the time step dimension to obtain the nonlinear trend component in the time step dimension, and then the nonlinear trend component in the time step dimension is nonlinearly mapped in the feature dimension, so as to finally obtain the result of the double nonlinear mapping, i.e., the nonlinear trend component. This component can model from both time and feature dimensions, and is suitable for time series data with complex multi-dimensional relationships and nonlinear trends.

[0116] S13: For the identification and extraction of short periodic fluctuations in the time series, the Season Series, i.e. the Short-Cycle-Seasonality, will be extracted; first, the linear trend component and the nonlinear trend component extracted in steps S11 and S12 are removed from the receivable structure (i.e. the original sequence) to obtain the remaining residual sequence, because we extract the long-term trend in the time series and eliminate the short-term fluctuations in the trend sequence, then the reverse subtraction naturally obtains the short-term fluctuations and other components; then the remaining residual sequence is applied to the simple moving average (SMA) operation to extract the short-cycle component, and the value of the sliding window used in this step is smaller than the size of the sliding window used in step S11 to extract the linear trend component;

[0117] For the extraction of short-term and stable seasonal components, the step size stride is still 1 to extract a complete component, and padding operation will also be performed before moving average to fill the data; in this process, we extract the short-cycle component from the residual sequence after removing the long-term trend component;

[0118] S14: Next is the extraction of the medium-long periodic component (Periodic Series), which aims to capture the medium-long periodic fluctuations in the time series, together with the long-term trend, to model and extract the long-term patterns in the time series;

[0119] The remaining sequence is obtained by removing the short-cycle component in step S13 from the residual sequence in step S13, and the medium-long periodic component is extracted from the remaining sequence by using the Discrete Fourier Transform (DFT) method;

[0120] The core of the medium-long periodic component extraction is the Discrete Fourier Transform (DFT) method. Fourier transform can convert time domain signal to frequency domain data, and use frequency domain data to reveal the amplitude and phase characteristics of the signal to extract the periodic characteristics of the time series.

[0121] Further, the specific process of step S14 is as follows:

[0122] S141: The remaining sequence in step S14 is used as the input for extracting the medium-long periodic component, and the remaining sequence is a time series {x1, x2,..., xN} of length N, which is split into even and odd parts: n

[0123] Even part: x 2m , where

[0124] Odd part: x​2m+1 , wherein

[0125] By expanding the formula of the discrete Fourier transform (DFT) into even and odd parts, the following formula can be obtained:

[0126]

[0127] Further simplifying the formula of the previous step, we get:

[0128]

[0129] Here:

[0130] i.e. DFT of even part

[0131] i.e. DFT of odd part

[0132] Therefore, the following formula can be obtained:

[0133]

[0134] By the above splitting, the discrete Fourier transform of the remaining sequence is decomposed into the discrete Fourier transforms of two sub-sequences of length This decomposition process can continue recursively until the length of the sequence is reduced to 1 (the smallest unit), so the discrete Fourier transform can be represented by the following formula:

[0135]

[0136] where, is the rotation factor (or "rotation kernel"), and the recursion splits the sequence into two sub-sequences of length : even and odd parts. At each recursive level, the discrete Fourier transform combines the results of the even and odd parts by addition and multiplication by the rotation factor to form the frequency domain data;

[0137] S142: Perform high-frequency filtering on the frequency domain data described in step S141 and obtain low-frequency components. This module filters out high-frequency noise by setting a frequency filtering threshold, and only retains low-frequency components. These low-frequency components correspond to medium and long-term periodic fluctuations;

[0138] S143: Convert the low-frequency components described in step S142 back to the time domain using inverse Fourier transform (IFT) to generate the final medium and long-term components. The inverse Fourier transform (IFT) (also known as inverse discrete Fourier transform (IDFT)) formula is as follows:

[0139]

[0140] Where, x n X is the value of the time-domain signal at the nth time step. k It is the complex value of the k-th frequency component of the frequency domain data, and N is the total length of the signal (i.e., the number of frequency domain points). It is the twitch factor of the inverse Fourier transform, where j is the imaginary unit (j 2 =-1);

[0141] S15: Remove the medium- and long-period components from the remaining sequence described in step S14 to obtain the residual noise components.

[0142] The residual noise component represents random fluctuations and high-frequency noise in the sequence after removing nonlinear trends. It can identify high-frequency noise and irregular fluctuations in the time series that cannot be explained by other components. It provides information about data uncertainty during the prediction process and helps to improve the robustness of the model.

[0143] The MPDLinear model introduces a self-learning weight vector module, enabling it to capture more information through this cross-dimensional adaptive mechanism. By introducing self-learning weight vectors in both the sequence and feature dimensions, the model can dynamically adjust these weights based on data characteristics during training. It continuously learns the importance of different time-series components (such as trends, seasonality, periodicity, and residuals) and different features, allowing the model to dynamically adjust the output contribution of each channel for finer-grained weight allocation. This results in more refined weighted combinations, which are then integrated to achieve the final output. This reduces errors caused by single dimensions, enabling the model to better handle multi-dimensional, multi-channel time-series features and providing stronger modeling capabilities for complex time-series data, especially channel-dense time-series data.

[0144] like Figure 4 As shown, the multiple input sequences in the leftmost part are multiple sequences decomposed by the multi-component decomposition module, i.e., different time series components.

[0145] Furthermore, the specific process of step S5 is as follows:

[0146] S51: Define the preliminary prediction results of the different time-series components described in step S4 as S i Where i = 1, 2, ..., n, a component dimension weight w is assigned to different temporal components in the component dimension. i In the feature dimension, it is S i Each feature in the dataset is assigned a feature dimension weight V[i,j], where j = 1, 2, ..., channels;

[0147] S52: During the backpropagation training process, update w described in step S51 through the backpropagation mechanism. iand V[i,j], whose gradients are calculated according to the loss function Loss and Thus, the component dimension weight and the feature dimension weight are adjusted during the training process, and the component dimension weight and the feature dimension weight are as shown in the following formula:

[0148]

[0149] wherein η is a learning rate, w i and V[i,j] are the component dimension weight and the feature dimension weight of different time sequence components, respectively.

[0150] Further, the specific process of step S6 is as follows:

[0151] S61: using the preliminary prediction results of different time sequence components in step S4 and the component dimension weight w i , the component dimension weight weighted different time sequence components By adding all the component dimension weight weighted different time sequence components, the comprehensive output F of the component dimension is obtained series ,

[0152] After summarizing the contribution of each component sequence, the next step is to weight in the feature dimension. We introduce a feature dimension weight V[i,j] in the feature (channels) dimension, which has a shape of (n, channels). The first dimension represents the number of different time sequence components, and the second dimension represents the number of features of each different time sequence component, i.e. the number of feature channels.

[0153] S62: using the preliminary prediction results of different time sequence components in step S4, and weighting each feature according to the feature dimension weight in step S52, the specific formula is as follows:

[0154]

[0155] wherein, is the weighted output of the jth feature channel of the ith different time sequence component, and V[i,j] is the corresponding feature dimension weight;

[0156] After learning through step S52 in the training process, after calculating the feature channel weight of all different time sequence components, we summarize all the feature dimension weighted channels to obtain the comprehensive output F of the feature dimension

[0157] F channels ,

[0158] S63: The comprehensive output F of the component dimension and the feature dimension in steps S61 and S62, respectively series and the comprehensive output F of the feature dimension channels The final prediction result F of the weather data set after weighting in the component dimension and the feature dimension final , and the specific formula is as follows:

[0159] wherein, S i [:,j,:] represents the value of the ith different time sequence component on the jth channel.

[0160] Specifically, each different time sequence component is weighted w i in the component dimension, then each feature is weighted V[i,j] in the feature dimension, and finally the multi-dimensional dynamic adjustment and combination of the time series data are realized through double-weighted summation.

[0161] In the DLinear model, all input channels (features) share the same linear layer, and this design fails to distinguish the differences between different features, limiting the model's ability to utilize unique information of each channel. In actual time series data, different channels (features) may exhibit completely different patterns, and the way that all channels share the same linear layer cannot fully adapt to the differences between different channels, resulting in the model failing to utilize the information of all channels, but instead performing a unified fitting on all channels. The fitting effect is also universal and average, because the model does not have enough parameters to learn the patterns of changes of multiple features over time, but only performs a general "rough" mapping process. The model prediction performance is poor.

[0162] The multi-channel independent modeling and weighting prediction module in the method better captures the features of each independent channel and improves the performance of the model through multi-channel independent modeling and weighting prediction in cooperation with the self-learning weight vector module.

[0163] For example Figure 5The diagram shows the internal structure of the multi-channel independent modeling weighted prediction module. The core design feature of this module lies in its flexibility and independent channel processing capability. Experimental parameters allow the model to choose between independent and shared modeling. The model can allow all input channels (features) to share a single linear layer, or it can build a dedicated independent linear layer for each feature channel, thus providing a customized prediction model for each feature. Feature-shared linear layer modeling is suitable for datasets with small numbers of features and relatively homogeneous relationships between features. All channels share the same linear layer to reduce model complexity and improve computational efficiency; this is indicated by a red cross in the diagram. Feature-independent linear layer modeling is suitable for datasets with large numbers of features and complex relationships between features; this is indicated by a green checkmark in the diagram. In this paper's experiments with complex datasets, the feature-independent modeling approach of the multi-channel independent modeling weighted prediction module will be used.

[0164] Similar to the nonlinear trend component extraction part in the multi-component decomposition module, this module uses PyTorch's nn.Linear module for multi-channel independent linear layer modeling to define fully connected layers and perform linear transformations, except that it is specifically designed for...

[0165] The shape variations of the All_Channels feature-shared linear layers and feature-independent linear layers are slightly different. In addition, the different linear layers for each sequence are loaded separately by the nn.ModuleList container in PyTorch.

[0166] Furthermore, in step S3, the multi-channel independent modeling weighted prediction module completes the modeling of independent channels through a feature-independent linear layer. The formula for the feature-independent linear layer is as follows:

[0167] (Y (bs,pl) =X (bs,sl) ·W (sl,pl) +bias (bs,1) )*channels

[0168] Among them, Y (bs,pl) The output of the model for each channel is independent, X (bs,c,sl) The input tensor of the linear layer, X. (bs,sl) The shape is (batch_size, seq_len), and the direct input is a decomposed sequence (such as trend_series) of a certain input (containing batch_size batches) in train_loader. W (sl,pl) The bias is used to map the weight matrix to the input time step. (bs,1) This is the bias vector for the linear model.

[0169] The module container list (Module0List) stores a plurality of independent linear layers of certain features of a sequence (for example, the independent linear layer ModuleList of the seasonal sequence is Linear_Seasonal), and when we input the data of a certain channel of a certain sequence (seasonal_init) to the linear layer, it is

[0170] Linear_Seasonal[i](seasonal_init[:,i,:]), which represents the Linear layer of the i-th channel, has a shape of (batch_size, seq_len) and will be mapped to the output Y (bs,pl) , which has a shape of (batch_size, pred_len).

[0171] Figure 6 The mapping structure of each scalar entering the feature-independent linear layer of the multi-channel independent modeling weighted prediction module is shown in the process. The linear layer performs linear transformation on the input, uses a weight matrix W (12, 5) to map the input of the window size (seq_len) to the output of the prediction length (pred_len), and adds a bias term (bias) after the linear transformation to adjust the value of each output node. The bias term helps the model to have an output benchmark without input data, enhances the expression ability of MPDLinear, and finally, the output layer contains the prediction results of the model for the next 5 time steps.

[0172] DLinear performs uniform weighting processing on all input features and time steps, without distinguishing the importance of different features at different time steps or the importance of different time steps at different features. The lack of such a mechanism that can simultaneously process the interaction between features and time steps makes DLinear only learn more time series features by inputting more time series data (larger window size) for each inference to make better predictions. In addition, this fixed and uniform way makes the model unable to capture the complex feature dependencies in the time series and the importance differences of certain features over time, so it may miss some key patterns and information.

[0173] In order to solve the above problems, the feature-time dimension hybrid self-attention mechanism module is designed, the self-attention mechanism of the module uses the scaled dot-product attention, meanwhile, the corresponding attention weights are grabbed in the two time step dimensions (input time step, prediction time step) and the feature (channel) dimension, and the hybrid is used, and is respectively applied to the input and output sequences, so that the model can learn the features and time relationship in the data with bias, and the complex patterns in the multi-dimensional time sequence can be more flexibly captured.

[0174] Figure 7 The mixed calculation workflow of the hybrid self-attention mechanism in the feature-time dimension hybrid self-attention mechanism module is shown.

[0175] Further, the feature dimension self-attention mechanism layer and the prediction window time dimension self-attention mechanism layer in step S4 are processed as follows, wherein the input is divided into the time dimension and the feature dimension, the time dimension is divided into the input time step (the look-back window) time dimension before entering the multi-channel independent modeling weighted prediction module, and the prediction time step time dimension output from the multi-channel independent modeling weighted prediction module:

[0176] S41: The feature dimension, the input time step time dimension and the prediction time step time dimension generate respective query (Q) vectors, key (K) vectors and value (V) vectors after linear transformation; wherein the query vector represents the query of the current element, is used to obtain the relevant attention information, the key vector represents the features of all elements in the sequence, is used to match the query vector, so as to calculate the similarity between the features, obtain the attention score and the attention weight, and the value vector contains the actual information of the element, is used to perform weighted average matrix multiplication according to the attention weight, so as to generate the final output;

[0177] S42: The Q, K and V of each dimension enter the scaled dot-product attention part, and the structure of the part is as shown in Figure 7 In this part, the transpose of each key vector in step S42 is obtained as K T , then the dot product (Q·K T ) of the query vector and K T is calculated, so as to capture the relationship between different time steps and features, and obtain the preliminary attention score;

[0178] S43: In order to make the attention score distribution in a smaller range, the output dimension of the dot product in step S42 is scaled by a scaling factor , so as to obtain the scaled dot product result, that is, the attention score;

[0179] S44: SoftMax operation is performed on the attention scores in step S43 to convert them into attention weights, which represent the attention of each element's query vector on all element key vectors;

[0180] where, for each query position in the sequence, SoftMax converts the attention scores into a probability distribution over all key vector positions, for the input vector as (i,j) = [as (i,1), as (i,2) ,...,as (i,n) ], the formula of SoftMax is:

[0181]

[0182] where as = [as (i,1), as (i,2) ,...,as (i,n) ] is an attention score containing n elements, representing the relevance of q i to [k1, k2,..., k n ], exp(as (i,j) ) is the exponential of the jth element in the input attention score vector, is the sum of all input attention score exponents;

[0183] This means that for each query Q, the attention of this position to other positions is converted into a probability value, and all probability values (attention) sum to 1. In addition, in order to eliminate the influence of the previous padding data, we set the attention weight of the padding part of the input to 0 in advance (the attention score is set to -inf in advance);

[0184] S45: using matrix multiplication (MatMul) to apply the attention weights (of all query vectors on all key vectors) in step S44 to the value vector, weighting and summing the information of different positions through the attention weights to extract important information in the value vector, thereby obtaining a more focused feature representation, dynamically selecting and emphasizing important features and time steps related to the current prediction task, and weakening the attention to unimportant features and time steps, obtaining the preliminary prediction results of the different time sequence components after applying attention, and outputting the preliminary prediction results of the different time sequence components after applying attention to the learning weight vector module and the multi-component decomposition module through the aggregation operation.

[0185] Embodiment one

[0186] This embodiment one uses the Beijing weather data set processed in the China weather data set (weather_2013_2017_china) to decompose and visualize the data using the multi-component decomposition module. This paper uses the Pytorch framework to build the model required data loader (DataLoader) for the training set in the Beijing weather data set, and mixes the data set, randomly selects all time steps in a batch (batch), here, when the experiment of the China weather data set is carried out, because the best known configuration (Best Known Config, BKC) of the batch size (batch_sizebatch_size), the lookback window size (seq_len), and the prediction window size (pred_len) of MPDLinear on this data set has been obtained through the comparison experiment, and the lookback window size (seq_len) = 365 is the best, so the lookback window size (seq_len) is 365 when decomposing, and the sampling interval of this data set is day.

[0187] In embodiment one, the convolution kernel size (kernel_size) is uniformly taken as 25, so that the smoothing window contains enough data points, which is suitable for various sampling frequencies (such as minutes, hours, days, months), and this window size is most suitable for removing short-term fluctuations and retaining long-term trends; the stride is uniformly taken as 1, when stride = 1, each time step t has a corresponding average value, to obtain a smoothed complete time series as a trend component, without skipping any intermediate time steps. At the same time, in order to ensure that the calculation of the sliding window at the sequence boundary will not lose data, we add internal padding when calculating the moving average, fill part of the data at both ends of the sequence, so that the sliding window can be smoothed from the first time step to the last time step, and the number of elements of the internal padding (padding) is:

[0188] In the extraction process of the medium and long period component (Periodic Series), the frequency filtering threshold is set to filter out high-frequency noise and only retain low-frequency components, which correspond to medium and long-term periodic fluctuations. In this embodiment one, the first 10% of the frequency domain components are retained as low-frequency components, and the remaining 90% of the high-frequency components are set to zero.

[0189] Figure 8 The multi-component decomposition module is shown in the original sequence decomposition of the humidity feature of the Beijing weather data set, Figure 8 The blue part in the middle represents the change of the humidity feature of Beijing in the entire time range (365 days).Figure 9 The linear trend component distribution of the humidity feature of the Beijing weather dataset is shown in FIG. 6A, Figure 9 The middle orange part is the linear trend component, which is extracted by the moving average method, and it shows the overall change trend of humidity over time, revealing the rising and falling rules of humidity within a year (365 days); Figure 10 The nonlinear trend component distribution of the humidity feature of the Beijing weather dataset is shown in FIG. 6B, Figure 10 The middle green part is the nonlinear trend component, which is extracted by using nonlinear mapping on the basis of the linear trend component, and it reveals the complex patterns that may exist in the nonlinear trend component from Figure 10 As can be seen from FIG. 6B, the nonlinear trend component has both small amplitude changes and large amplitude changes, and these changes are often some subtle trend fluctuations and nonlinear fluctuations, reflecting the details that are difficult to capture by simple linear methods in the change of humidity over time; Figure 11 The short-period component distribution of the humidity feature of the Beijing weather dataset is shown in FIG. 6C, Figure 11 The middle red part is the short-period component, which mainly extracts short-term and repetitive fluctuations in the original sequence, and it shows the Figure 11 As can be seen from FIG. 6C, the short-period component captures the high-frequency periodic changes of humidity in a short time range, and the fluctuation curve has many spikes, representing the different changes of humidity in each short time period, and each spike represents a possible change pattern in three days or a week or a month; Figure 12 The medium-long period component distribution of the humidity feature of the Beijing weather dataset is shown in FIG. 6D, Figure 12 The middle purple part is the medium-long period component, which is mainly obtained by discrete Fourier transform (DFT) and high-frequency filtering and low-frequency extraction, and it shows a clear periodic fluctuation, which repeats many times within a year. Compared with the short-period fluctuation of the seasonal component, the periodic component has a longer period and a larger amplitude, and is smoother, because the medium-long period component pays more attention to the medium-long term changes of the sequence, and ignores the high-frequency short-term changes, which are obtained by the seasonal component; Figure 13 The residual noise component distribution of the humidity feature of the Beijing weather dataset is shown in FIG. 6E, Figure 13 The middle brown part is the residual noise component, which is the final remaining component after removing all the above components, and it contains relatively complex high-frequency fluctuations, which may be caused by noise, data measurement error or some short-term and unpredictable sudden events. Similarly, this component is obtained for subsequent learning of the model.

[0190] Through the multi-component decomposition module, we split the complex original sequence into multiple fine-grained components, including linear trend component (Trend), nonlinear trend component (Nonlinear Trend), short-cycle component (Short-Cycle-Seasonality), long-cycle component (Long-Cycle-Periodic), and residual noise component (Residual), each of which can represent a specific time series feature, enabling the model to understand the change pattern of time series at different time scales. The model not only accurately captures long-term and short-term changes in data, but also effectively models nonlinear and complex structures in data, thereby improving the overall prediction performance of multivariate time series.

[0191] The self-learning weight vector module introduces a self-learning weight vector, achieving dynamic weighting in both component and feature (channel) dimensions, fully utilizing information from multiple time series components, and adjusting weights across dimensions within the same component. The dynamic weight adjustment and self-learning adaptive capability of the self-learning weight vector module can automatically learn and adjust prediction weights based on the characteristics of time series data. By introducing a multi-dimensional self-learning weight vector, the model can weight different components and channel features, thereby improving prediction accuracy and flexibility. The dynamic weight adjustment mechanism adaptively allocates weights, ensuring that the contribution of each component in multivariate time series is reasonably measured. For example, some components have a greater impact on the prediction result, while others have a smaller impact. Dynamic adjustment can amplify the weight of important features while reducing the impact of noise or irrelevant features, thereby enhancing the ability to capture key trends. The self-learning adaptive capability further improves the adaptability of the model. The weight vector does not rely on empirical values, but is automatically adjusted through backpropagation and optimization algorithms. This not only reduces the workload of hyperparameter tuning, but also ensures that the model maintains high robustness in different datasets and tasks. As time series change, the self-learning mechanism can continuously optimize weight allocation, ensuring that the model remains sensitive to changes in data distribution, thereby improving prediction accuracy and stability. Through dynamic weight allocation and self-learning adjustment, the model can better extract important information from time series, fully utilize the value of each feature, and significantly improve prediction performance and generalization ability.

[0192] The multi-channel independent modeling weighted prediction module differentiates the characteristics of different channels by constructing independent linear layers for each feature channel, more accurately capturing the unique change patterns of each channel, solving the "averaging" defect of DLinear that cannot distinguish different features, making MPDLinear have higher adaptability and performance in the context of significant heterogeneity or complex interaction of each feature in multivariate time series data. The multi-channel independent modeling weighted prediction module uses an independent modeling method to model the characteristics of each channel respectively, thereby significantly improving the model's ability to capture complex multivariate time series. In actual time series data, the feature distribution and dynamic changes of different channels often differ significantly. Compared with the traditional method of sharing a unified model weight, independent modeling can model each channel's characteristics in more detail, ensuring that the model makes full use of the unique information of each channel. In this way, the model can not only more accurately fit the pattern of each channel, but also avoid interference between different channels. The advantage of independent modeling is higher flexibility and adaptability. When the feature pattern of one channel is relatively simple and the other is relatively complex, the model can adjust the structure and parameters for the two cases respectively, so that the complex pattern gets more attention, while the simple pattern maintains a lower modeling complexity. This differentiated modeling strategy can reduce unnecessary computational cost and improve the overall generalization ability and prediction performance of the model. In addition, through independent modeling, the multi-channel independent modeling weighted prediction module can better capture the complex interaction patterns between channels. Although each channel is independently modeled, the model can consider the features of these independent channels through a flexible weighting mechanism or interaction module in the later stage, ensuring that the dependence between channels and complex patterns are fully mined and utilized. This design effectively enhances the model's performance when faced with multi-dimensional and high-complexity time series data, allowing it to retain channel independence while fully utilizing the interaction information between channels to improve prediction accuracy and modeling stability.

[0193] The feature-time dimension hybrid self-attention mechanism module realizes more fine modeling of multivariate time series data by introducing scaled dot-product self-attention mechanism in both feature and time dimensions, so that the MPDLinear model can process complex interactions between feature and time dimensions. Compared with DLinear, MPDLinear no longer excessively relies on long input sequences (lookback window) to obtain more accurate prediction results, and the change in prediction accuracy of MPDLinear under different input sequence lengths is reduced. The feature-time dimension hybrid self-attention mechanism module focuses on effectively matching the heterogeneity of time-feature dimensions and improves the adaptability of the model to complex time series data. In multivariate time series, the correlation between features and time steps is often uneven and constantly changing, and traditional methods usually use uniform weights or fixed processing methods for all dimensions, which is difficult to fully capture the subtle dynamics in the data. To solve this problem, the feature-time dimension hybrid self-attention mechanism module introduces a dynamic hybrid attention mechanism that can dynamically adjust the attention weights in the time dimension and the feature dimension, so as to allocate different attention degrees between dimensions according to the changes in input data, mine short-term shocks and long-term trends in time series, and capture complex interaction patterns between features. This dynamic attention method enables the model to flexibly adapt to the diverse changes in data characteristics, improving the ability to capture potential structures in time series. In addition, the feature-time dimension hybrid self-attention mechanism module identifies situations where different features are more important in certain time periods through a self-learning heterogeneity matching scheme and dynamically adjusts the prediction weights. This method reduces the dependence on hyperparameter tuning, improving the adaptability and stability of the model. In processing multivariate time series data, the feature-time dimension hybrid self-attention mechanism module exhibits stronger flexibility and accuracy, effectively capturing complex patterns and adapting to different task requirements, improving the prediction accuracy and generalization ability of the model.

[0194] In a second aspect, an electronic device includes a memory and a processor:

[0195] The memory is configured to store a computer program for implementing the MPDLinear model-based time series prediction method.

[0196] The processor is configured to execute the computer program to implement the MPDLinear model-based time series prediction method.

[0197] In a third aspect, a computer readable storage medium storing a computer program is provided. The computer program, when executed by a processor, implements the time series prediction method based on the MPDLinear model. The computer readable storage medium includes a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and the like, which can store program codes.

[0198] The working principle of the present application is as follows:

[0199] In the present method, a high-precision time series prediction model (MPDLinear) is proposed, which includes a multi-component decomposition module, a self-learning weight vector module, a multi-channel independent modeling and weighted prediction module, and a feature-time dimension hybrid self-attention mechanism module. The multi-component decomposition module decomposes the time series into four components, namely linear trend, short cycle, medium-long cycle, residual error and nonlinear trend, thereby enhancing the adaptability of the model to diversified patterns in the time series. Secondly, the self-learning weight vector module introduces a self-learning weight vector in the time series and feature dimensions, realizes the weighted summation of the components and features, and dynamically and accurately balances the contributions of each sequence and feature. The multi-channel independent modeling and weighted prediction module independently constructs a linear layer for each feature channel, thereby solving the homogenization problem of traditional models in feature modeling and effectively improving the ability of the model to model different channel characteristics. The feature-time dimension hybrid self-attention mechanism module combines the self-attention mechanisms of the feature and time dimensions, dynamically adjusts the attention of the prediction output to different time steps and features, and further improves the ability of the model to capture complex patterns. The above modules work together to make the MPDLinear have high generalization and prediction ability.

Claims

1. A time series forecasting method based on multi-dimension decomposition linear (MPDLinear) model, characterized in that, Comprising the following steps: S1: After the multi-component decomposition module in the MPDLinear model receives the binary data stream, the binary data stream is deserialized and standardized into a structure that the MPDLinear model can receive, which represents the changes of the humidity feature of the weather data set over the entire time range. The multi-component decomposition module decomposes the humidity feature of the weather data set into different time series components and outputs them to the multi-channel independent modeling weighted prediction module, the self-learning weight vector module, and the feature-time dimension hybrid self-attention mechanism module in the MPDLinear model, respectively. S2: After the feature-time dimension hybrid self-attention mechanism module receives the different time series components in step S1, it processes them through dimension rearrangement and generates dimension-rearranged different time series components. Then, the dimension-rearranged different time series components are processed through the self-attention mechanism layer of the feature-time dimension hybrid self-attention mechanism module to obtain the attention scores of the different time series components. The attention scores of the different time series components are further processed to obtain the attention weights of the different time series components and output to the multi-channel independent modeling weighted prediction module. S3: After the multi-channel independent modeling weighted prediction module receives the different time series components in step S1 and applies the attention weights of the different time series components in step S2, it obtains the attentioned different time series components through matrix multiplication. The multi-channel independent modeling weighted prediction module models each attentioned different time series component in an independent channel and outputs a preliminary result sequence to the feature-time dimension hybrid self-attention mechanism module. S4: The feature-time dimension hybrid self-attention mechanism module obtains the preliminary result sequence in step S3, and processes it through the feature dimension self-attention mechanism layer and the prediction window time dimension self-attention mechanism layer of the module in turn to obtain the preliminary prediction results of the attentioned different time series components and output them to the self-learning weight vector module and the multi-component decomposition module. S5: After the self-learning weight vector module receives the different time series components in step S1 and the preliminary prediction results in step S4, it calculates the parameters of the different time series component weights and the different time series feature weights in the self-learning weight vector module using the preliminary prediction results during backpropagation. After iterating through the parameters of the different time series component weights and the different time series feature weights, the component dimension weights and the feature dimension weights of the different time series components are obtained and output to the multi-component decomposition module. S6: The multi-component decomposition module receives the preliminary prediction results in step S4, and performs weighted calculation on the preliminary prediction results according to the component dimension weights and the feature dimension weights of the different time series components in step S5, respectively. The weighted calculation results are summed in reverse according to the decomposition method of the multi-component decomposition module to obtain the final prediction results of the weather data set.

2. The temporal prediction method of claim 1, wherein, The different time series components in step S1 include linear trend components, nonlinear trend components, short-period components, medium-long-period components, and residual noise components. The steps for extracting different time series components are as follows: S11: Simple moving average (SMA) based on a sliding window is used to smooth the input structure described in step S1, and a linear trend component is obtained by defining a moving average; S12: A double nonlinear mapping is used to extract the nonlinear trend component in the input structure described in step S1. Specifically, a linear mapping is first performed on the time dimension or the feature dimension through a fully connected layer Linear, then a ReLU nonlinear activation function is applied, and finally a linear transformation is performed on the remaining dimensions through another fully connected layer Linear. The final result of the double nonlinear mapping is the nonlinear trend component; S13: The linear trend component and the nonlinear trend component extracted in steps S11 and S12 are first removed from the input structure to obtain a residual sequence, and then a simple moving average (SMA) operation is applied to the residual sequence to extract a short-period component. The value of the sliding window used in this step is smaller than the size of the sliding window used to extract the linear trend component in step S11; S14: The residual sequence described in step S13 is further removed from the short-period component described in step S13 to obtain a remaining sequence. A discrete Fourier transform (DFT) method is used to extract a medium-long period component from the remaining sequence; S15: The residual sequence described in step S14 is further removed from the medium-long period component described in step S14 to obtain a residual noise component.

3. The temporal prediction method of claim 1, wherein, The multi-channel independent modeling and weighted prediction module in step S3 completes independent channel modeling through a feature-independent linear layer. The formula of the feature-independent linear layer is as follows: (Y (bs,pl) = X (bs,sl) * W (sl,pl) + bias (bs,1) )* channels where Y (bs,pl) is the output of the independent model for each channel, X (bs,c,sl) is the tensor of input linear layer, W (sl,pl) is the weight matrix mapping the input time step, bias (bs,1) is the bias vector of the linear model.

4. The temporal prediction method of claim 1, wherein, The feature dimension self-attention mechanism layer and the prediction window time dimension self-attention mechanism layer in step S4 process as follows. The input is divided into the time dimension and the feature dimension. The time dimension is further divided into the input time step time dimension before entering the multi-channel independent modeling and weighted prediction module and the prediction time step time dimension output from the multi-channel independent modeling and weighted prediction module: S41: The feature dimension, the input time step time dimension, and the prediction time step time dimension generate their respective query vectors Q, key vectors K, and value vectors V after linear transformation; S42: Transpose each key vector of step S41 to get K T Then calculate the dot product Q·K of the query vector and K T T to get the preliminary attention score;​ S43: scaling the output dimension of the dot product with a scaling factor The output dimension of the dot product described in step S42 is scaled to obtain a scaled dot product result, i.e., an attention score. S44: Perform a SoftMax operation on the attention score described in step S43 to convert it into an attention weight. The attention weight represents the attention of each element's query vector on all element key vectors; where, for each query position in the sequence, SoftMax converts the attention scores into a probability distribution over all key vector positions, for the input vector as (i,j) = [as (i,1), as (i,2) ,...,as (i,n) ], the formula for SoftMax is: where as = [as (i,1), as (i,2) ,...,as (i,n) ] is an attention score containing n elements, representing the relevance of q i to [k1, k2,..., k n ], exp(as (i,j) ) is the exponential of the jth element of the input attention score vector, and is the sum of all input attention score exponents. S45: Apply the attention weight described in step S44 to the value vector using matrix multiplication to obtain the preliminary prediction result of the different time sequence components after applying attention. The preliminary prediction result of the different time sequence components after applying attention is output to the learning weight vector module and the multi-component decomposition module through an aggregation operation.

5. The temporal prediction method of claim 1, wherein, The specific process of step S5 is as follows: S51: Defining the preliminary prediction results of the different timing components as S i where i = 1, 2,..., n, assigning a component dimension weight w to each of the different timing components in the component dimension i S4, assigning a feature dimension weight V[i,j] to each feature in S i where j = 1, 2,..., channels; S52: In the back propagation training process, the w in step S51 is updated by the back propagation mechanism i and V[i,j] whose gradient is calculated according to the loss function Loss and Thus, the component dimension weight and the feature dimension weight are constantly adjusted in the training process, and the component dimension weight and the feature dimension weight are as shown in the following formula: where η is the learning rate, w i and V[i,j] are the component dimension weight and feature dimension weight of different time sequence components, respectively.

6. The temporal prediction method of claim 5, wherein, The specific process of step S6 is as follows: S61: using the preliminary prediction results of the different timing components in step S4 and the component dimension weight w in step S52 i , calculate the component dimension weight weighted different timing components By adding all the component dimension weight weighted different timing components, the comprehensive output F of the component dimension is obtained series , S62: Use the preliminary prediction result of the different time sequence components described in step S4 to weight each feature according to the feature dimension weight described in step S52. The specific formula is as follows: wherein, is the weighted output of the jth feature channel of the ith different timing component, V[i,j] is the corresponding feature dimension weight; The channels of all the weighted feature dimensions are aggregated to obtain a comprehensive output F of the feature dimension channels , S63: The comprehensive output F of the component dimension and the feature dimension in steps S61 and S62 respectively series and the comprehensive output F of the feature dimension channels The final prediction result F of the weather data set after weighting in the component dimension and the feature dimension final The specific formula is as follows: where S i [:,j,:] denotes the value of the i-th time-independent component on the j-th channel.

7. The temporal prediction method of claim 2, wherein, The specific process of step S14 is as follows: S141: taking the residual sequence as the input of extracting the medium-long period component, the residual sequence is a time sequence {x1, x2,..., xN} of length N, which is obtained by the following formula: n We split it into even and odd parts: Even part: x 2m , wherein Odd part: x 2m+1 , wherein The formula of the discrete Fourier transform (DFT) is expanded into even and odd parts as follows: Further simplifying the formula of the last step, we get: wherein, i.e. even part of DFT DFT of odd part Therefore, the following formula can be obtained: By the above splitting, the discrete Fourier transform of the remaining sequence is decomposed into the discrete Fourier transforms of two sub-sequences of length This decomposition can be continued recursively until the length of the sequence is reduced to one, so that the discrete Fourier transform can be expressed by the following equation: wherein, is a rotation factor, the recursion splits the sequence into two sub-sequences of length : even and odd parts, at each recursion level, the discrete Fourier transform combines the results of the even and odd parts by addition and multiplication by a rotation factor, forming the frequency domain data; S142: High frequency filtering is performed on the frequency domain data of step S141 to obtain low frequency components; S143: The low frequency components of step S142 are converted back to time domain by inverse Fourier transform IFT to generate final medium-long period components, wherein the inverse Fourier transform IFT formula is as follows: where x n is the value of the time domain signal at the n-th time step, X k is the complex value of the k-th frequency component of the frequency domain data, N is the total length of the signal, is the rotation factor of the inverse Fourier transform, j is the imaginary unit.

8. An electronic device comprising a memory and a processor, characterized in that The memory stores a computer program capable of running on the processor, and the computer program is executed by the processor to implement the time series prediction method based on the multi-dimension decomposition linear MPDLinear model according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the time series prediction method based on the multi-dimension decomposition linear MPDLinear model according to any one of claims 1 to 7.

Images