Transform-based multivariate time series prediction method and computer program product

Abstract

The invention discloses a multivariate time sequence prediction method based on Transform and a computer program product. The method comprises the following steps: firstly, constructing a mixed model fusing a trend item and a season item, abandoning single mode decomposition, adopting a divide-and-conquer strategy, and executing customization operation for data features; secondly, a dynamic partitioning mechanism is provided to capture multi-level feature attributes, and the limitation of traditional fixed length partitioning is broken through, so that the method can flexibly adapt to multi-scale features of data; and then, integrating multiple channels by using a Granger causal relationship to supplement the limitation of a single channel and a mixed channel, and constructing richer and three-dimensional feature representation. And finally, the integrated features are sent to a Transform module so as to obtain a prediction result. Therefore, the features of different time scales can be effectively captured, information redundancy is reduced, and calculation efficiency is optimized. Experimental results show that the accuracy rates of the method in six data sets in four fields are all superior to those of similar prediction models based on Transformer.

Description

Technical Field

[0001] The present invention relates to a prediction method, and in particular to a Transformer-based multivariate time series prediction method and a computer program product. Background Art

[0002] Time series forecasting is an important research field with wide applications in various fields. From financial markets to weather forecasts, from healthcare systems to traffic flow management, the development of time series forecasting technology has a huge impact on daily life and decision-making processes. Such forecasts are critical for making strategic decisions, resource allocation, and planning. For example, in the financial field, correctly predicting stock price fluctuations can help investors develop effective investment strategies. In the field of transportation, accurately predicting changes in traffic flow can help urban planners improve traffic management systems. But with the continuous advancement of data acquisition and storage technology, a large amount of time series data is collected, which makes time series forecasting more important and challenging.

[0003] Recently, inspired by the great success of Transformer in fields such as computer vision and natural language processing, a series of Transformer-based time series prediction models have emerged to address the above challenges, such as Informer, LogTrans, Fedformer, PatchTST, Difformer, etc. These studies usually adopt a modified sequence decomposition attention mechanism to be compatible with the seasonality / periodicity of time series, or use sparse self-attention and equal-length patch blocks to reduce the secondary computational cost of long sequences, or use independent channels to alleviate the distribution drift problem. Although these methods have been successful, if the data does not have obvious seasonal or periodic patterns, decomposition may lead to inaccurate results and may not provide useful information; compared with channel independence, channel mixing is less effective in dealing with channel noise and distribution drift problems, but channel mixing still has some unique advantages; aggregating time steps into sub-series-level patches can enhance locality and capture comprehensive semantic information that cannot be obtained at the point level, but there are also some potential disadvantages: information loss, context loss, imbalanced distribution, and boundary effects. Summary of the invention

[0004] The present invention provides a multivariate time series prediction method based on Transformer to address the problems that the prediction model based on Transformer mechanism has the problem that the data is not compatible with the traditional seasonal periodic decomposition method, and the hybrid channel is not effective in processing channel noise and distribution drift.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a multivariate time series prediction method, comprising:

[0007] Input the target to be predicted into the multivariate time series prediction model to obtain the prediction result;

[0008] The training method of the multivariate time series prediction model includes:

[0009] Establishing a multivariate time series prediction data set; the multivariate time series prediction data set includes historical data of the target to be predicted and has several attributes;

[0010] The multivariate time series prediction data set is used to train the multivariate time series prediction model; the multivariate time series prediction model includes a hybrid decomposition module, a dynamic block module, a similar channel merging module, and a Transformer module;

[0011] The hybrid decomposition module is used to decompose the multivariate time series prediction data set to obtain data features including seasonal items and trend items;

[0012] The dynamic block module is used to dynamically block the seasonal item features to obtain subsequences of different scales;

[0013] The similar channel merging module is used to merge the single channels after block division to obtain the input data required by the Transformer encoder;

[0014] The Transformer module is used to extract features in the previous module synchronously, obtain extracted features, and predict seasonal items for the extracted features, and then combine the decomposed trend items to output the final prediction results.

[0015] Furthermore, after establishing the multivariate time series prediction data set, the method further includes:

[0016] The multivariate time series prediction data set is completed to fill the missing values ​​in the multivariate time series prediction data set with the average of the two values ​​before and after the missing value;

[0017] The multivariate time series prediction data set after completion is standardized so that the mean of the multivariate time series prediction data set is 0 and the variance is 1.

[0018] Furthermore, the multivariate time series prediction dataset meets the following requirements:

[0019] The time difference between adjacent data in the multivariate time series prediction data set is a constant;

[0020] The feature type and feature quantity at each moment in the multivariate time series prediction data set are consistent;

[0021] The multivariate time series prediction data set includes the date and time stamps of each data collection, and the date and time stamps corresponding to each data have the same format.

[0022] Furthermore, the specific process of the dynamic block module includes:

[0023] (1) The optimal number of blocks is obtained through experiments.

[0024] They are: S1=2, S2=4, S3=8, S4=16 respectively.

[0025] The total number of blocks is:

[0026] n / (S1+S2+S3+S4)×4+[n%(S1+S2+S3+S4)]

[0027] Where n represents the lookback window and % represents the remainder.

[0028] (2) Each input unit time series is divided into subsequences of unequal lengths through the dilated causal convolutional patch model.

[0029] This module can be simplified to:

[0030] F(t)=Select(W[j]*X[td*j])

[0031] Where d is the expansion factor, which is also the patch length; W[j] is the block size, that is, the weight; X(td*j) represents the value at time step td*j.

[0032] (3) Calculate the fitting score.

[0033] SSE(θ0,θ1)=∑[y i -(θ0×x i +θ1)] 2

[0034] Where θ(0, 1) represents the objective function parameter, x i and i Represents the predicted value and the true value.

[0035] (4) Conduct a comprehensive comparison and select the best block partitioning scheme.

[0036] (5) The loop continues to search for the optimal solution until all data are processed in blocks.

[0037] Furthermore, the specific process of the similar channel merging module includes:

[0038] Preprocess the data. Introduce a channel identifier, a learnable tensor V∈R (M×D) After linear projection of the patch, the labeling adds positional encoding and channel identifiers:

[0039] x token =x patch W+U+V,x token ∈R M×PN×D

[0040] Where W represents linear projection, U represents position encoding, M represents the dimension of each time step vector, PN represents the number of patches, and D represents the model dimension.

[0041] Then, x token is fed into the Transformer encoder for a high-level representation in a channel-independent manner:

[0042]

[0043] The channel identifier represents the unique characteristics of each channel, which enables the model to distinguish channels and capture unique representations of channels.

[0044] Consider the internal factors (GC) and external factors (FDTW) of different channels to verify whether the two channels should be merged. Channels can only be merged when both the internal and external factors meet the conditions.

[0045] Internal factors use Granger causality test to establish two linear regression models:

[0046]

[0047] Where P is the maximum number of lagged observations included in the model, that is, the autoregressive order; A is the coefficient matrix, that is, the contribution of each lagged observation to X; E is the residual of the time series.

[0048] The external factor uses fast dynamic time planning to align the two channels through dynamic planning so that the distance between them is minimized.

[0049] Furthermore, the specific process of the hybrid decomposition module includes:

[0050] Using the mixed decomposition formula:

[0051] Y t =(T t +C t )*(S t +I t )

[0052] where Y t represents the original sequence, T tIndicates trend, S t Indicates seasonality, C t Indicates periodicity, I t Indicates offset.

[0053] The series is decomposed into seasonal terms and trend terms, and the seasonal terms are used as the main input data part of the model.

[0054] Furthermore, the loss function used when training the multivariate time series prediction model includes:

[0055]

[0056] Where MSE is the loss value obtained by the MSE loss function, output a Indicates the a-th prediction result, True a Representation and output a The corresponding a-th true value, a=1, 2, ..., L.

[0057] In a second aspect, the present invention provides a computer program product, wherein the computer program product comprises a computer program or instructions, and when the computer program or instructions are executed by a communication device, the above method is executed.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] The present invention proposes a multivariate time series prediction method. First, a hybrid decomposition module is constructed, which abandons the seasonal periodic decomposition of all data in the traditional method, and adopts a divide-and-conquer strategy to perform different operations on different data to better process data with different change patterns; then a dynamic block strategy is added to achieve a dynamic effect, which can flexibly divide subsequences of different scales according to the differences in data, so as to capture the characteristics of different time scales, reduce information overlap and save computing resources; then, through an effective similar channel merging strategy, it aims to overcome the limitations of the single channel method, cleverly realize the deep mining and comprehensive understanding of data features, and thus construct a richer and more three-dimensional feature representation.

[0060] The present invention also proposes a computer program product, which has all the advantages of the above prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 It is a flow chart of the multivariate time series prediction model training method of the present invention.

[0063] Figure 2 It is a schematic diagram of the principle of the multivariate time series prediction model of the present invention.

[0064] Figure 3 It is a schematic diagram of the principle of the dynamic block module of the present invention.

[0065] Figure 4 It is a schematic diagram of the principle of merging similar channels of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0067] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0069] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0070] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0071] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0072] Multivariate time series forecasting is the use of historical data of multiple time series variables to predict future values. In this forecast, each observation point contains the values ​​of multiple features, and these observation points are arranged in chronological order. The goal is to use the historical data of these features to predict the future values ​​of the target series.

[0073] Multivariate time series prediction has a wide range of application scenarios, such as transportation, medical, energy, meteorology, and finance. Let's take a specific example to illustrate: suppose you are predicting the weather conditions for a certain day. You can predict the future weather based on historical humidity, temperature, wind speed, season, air quality, etc.

[0074] In order to perform multivariate time series forecasting, it is usually necessary to use some statistical models or machine learning algorithms. The models in these algorithms can capture the linear and nonlinear relationships in time series data and use these relationships to make predictions.

[0075] At present, the Transformer model-based method is often used for long sequence prediction, which mainly has three problems:

[0076] (1) About series decomposition. It is a standard approach in time series analysis to perform seasonal and cyclical decomposition on all data. Not all time series data meet this assumption. If the data do not have obvious seasonal or cyclical patterns, decomposition may lead to inaccurate results and may not provide useful information.

[0077] (2) About sequence chunking. Most previous models only use point-by-point input tags, or only handcrafted information in the series data. By aggregating time steps into sub-series level patches, locality can be enhanced and comprehensive semantic information that cannot be obtained at the point level can be captured. However, there are also some potential disadvantages: 1) Information loss. Uniformly splitting the time series into blocks of the same length may lead to important information loss. 2) Context loss: Block-level prediction ignores contextual information in the time series data. 3) Unbalanced distribution: Time series data usually has an uneven distribution and sparsity. 4) Boundary effect: Splitting the time series into blocks of fixed length may introduce additional noise and discontinuity at the boundaries of the blocks.

[0078] (3) About channel independence. Multivariate time series are multi-channel signals, and each Transformer input token can be represented by data from a single channel or multiple channels. Experiments have shown that channel independence has been successful in the PatchTST model. Although channel mixing is less effective in dealing with channel noise and distribution drift problems than channel independence, it has some unique advantages: high information capacity and enhanced feature representation. Since channel independence treats channels as shared models, the model cannot distinguish channels and mainly learns general patterns of multiple channels, resulting in the loss of channel specificity and potential impact on prediction.

[0079] Based on the above reasons, the present invention proposes a multivariate time series prediction method based on Transformer, and the present invention is described in detail below in conjunction with embodiments and drawings.

[0080] As a first embodiment of a multivariate time series prediction method of the present invention, it may include: inputting a target to be predicted into a multivariate time series prediction model to obtain a prediction result.

[0081] The present invention adopts a multivariate time series prediction model to directly obtain prediction results, wherein the specific structure of the multivariate time series prediction model can be partially adjusted to meet the subsequent basic structure.

[0082] like Figure 1 FIG. 1 is a flow chart of an embodiment of a multivariate time series prediction method of the present invention, which may include:

[0083] S101, establishing a multivariate time series prediction data set.

[0084] By summarizing the historical data of the target to be predicted, we collect multidimensional time series data samples containing several attributes. The multidimensional time series data samples must meet the following requirements:

[0085] (1) Evenly spaced. The collected time series data are sorted in chronological order, and the time difference between adjacent data is required to be a constant. If some moments are missing, their timestamps are supplemented and the NULL character is filled in the vacant position.

[0086] (2) The number of features is constant. For the features at each moment in the multivariate time series prediction data set, the feature type and quantity are required to be consistent. If the features at some moments are missing, the character NULL is filled in the empty position.

[0087] (3) Include timestamp information. For each value in the multivariate time series prediction data set, in addition to recording the value itself, the date and time stamp when the value was collected must also be recorded. The timestamp format should be unified and should not contain Chinese characters.

[0088] For the convenience of subsequent description, each data in the multivariate time series prediction data set is recorded as a sequence, and each data in the sequence is recorded as a point.

[0089] S102, cleaning and standardizing the time series data.

[0090] The multivariate time series prediction data set preprocessed in step S101 is completed and standardized:

[0091] (1) Completion: The missing values ​​NULL in the multivariate time series prediction data set are filled with the mean of the two values ​​before and after the missing interval.

[0092] (2) Standardization. The completed multivariate time series prediction data set is standardized using the following formula, so that each attribute column in the entire multivariate time series prediction data set is adjusted to a mean of 0 and a variance of 1:

[0093]

[0094] where x b is the b-th value in the sequence, seqb is the b-th value in the original unstandardized sequence, mean is the mean of the sequence, mean(seq) is the mean of the entire sequence, std is the calculated standard deviation of the sequence, std(seq) is the standard deviation of the entire sequence, and seq is the original unstandardized sequence.

[0095] S103, constructing a multivariate time series prediction model.

[0096] Design a multivariate time series prediction model with dynamic block and channel merging structure, such as Figure 2The figure shows the principle diagram of the multivariate time series prediction model. First, a hybrid decomposition module is established, which analyzes the multivariate time series prediction dataset to separate seasonal and trend features. Then, the dynamic block module is applied to dynamically subdivide the seasonal features and generate subsequences of different scales. Subsequently, the block-based single-channel data is integrated through the similar channel merging module to form the input of the adapted Transformer encoder. These inputs are fed into the Transformer module for feature extraction, and the extracted features are used for seasonal prediction. The predicted seasonal features are combined with the decomposed trend features to form the final prediction output.

[0097] The specific process is as follows:

[0098] Construct hybrid decomposition modules.

[0099] If the seasonal effect is relatively stable to the magnitude of the trend and random fluctuations, and the seasonal amplitude is small, the additive model is usually more suitable. On the contrary, if the magnitude of the seasonal effect is significantly affected by the trend change, and the seasonal fluctuation amplitude is large, the multiplicative model may be more appropriate. Therefore, the present invention proposes a hybrid decomposition algorithm:

[0100] Y t =(T t +C t )*(S t +I t )

[0101] Among them Y t represents the original sequence, T t Indicates trend, S t Indicates seasonality, C t Indicates periodicity, I t Indicates offset.

[0102] Long-term trends and cyclical change factors are added, while seasonal and irregular change factors are multiplied. This combination not only takes trends, seasonality, cycles and randomness into account at the same time, but also has its own advantages while making up for the deficiencies of addition and multiplication.

[0103] Build a dynamic chunking module.

[0104] like Figure 3 The figure shows the principle diagram of the dynamic block module.

[0105] Through experiments on six common data sets, it can be concluded that the block length can be divided into 2, 4, 8, 16, and the time series is represented as T = T1, T2, ..., T n , divided into blocks according to length S1=2, S2=4, S3=8, S4=16, that is, divided into m blocks L=L1, L2, L3, ..., Lm .

[0106] (1) Building a dilated causal convolutional patch model

[0107] First, the dilated causal convolutional patch module consists of an input layer and four partitioning layers. The configuration of each partitioning layer corresponds to the patch size, and the number of interval data is also set based on this. The input layer is the first layer, followed by four partitioning layers. The second layer, with the number of intervals k of 2, is used to partition the sequence into patches of length d = 2; the third layer, k = 4, d = 4; the fourth layer, k = 8, d = 8; and the fifth layer, k = 16, d = 16.

[0108] This module can be simplified into the following formula:

[0109] F(t)=Select(W[j]*X[td*j])

[0110] Where d is the expansion factor, which is also the patch length; W[j] is the block size, that is, the weight; X(td*j) represents the value at time step td*j.

[0111] Therefore, the expansion operation introduces a constant step size between adjacent blocks, forming interval sampling. When d = 2, the data sampling interval is one element. By increasing the number of intervals, the calculation of the top layer can be based on a wider range of input data in the lower layer, which improves the efficiency of block processing and ensures the breadth of information coverage.

[0112] (2) Calculate the fitting score

[0113] To fit the block-level data and calculate the scores, a bottom-up algorithm can be used using the linear regression method.

[0114] First, the length of the time series is set to n, and n / 2 overlapping segments are used to fit the entire series to find the optimal linear fit. Then, the fitting cost when each pair of adjacent segments is merged is calculated, and the corresponding score is generated accordingly. In practice, the present invention uses the least squares method for linear fitting, and uses the mean square error SSE as the evaluation standard for the fitting error.

[0115] The specific formula is as follows:

[0116] SSE(θ0,θ1)=∑[y i -(θ0×x i +θ1)] 2

[0117] Where θ(0, 1) represents the objective function parameter, x i and i Represents the predicted value and the true value.

[0118] In the first layer of blocks, the cost of merging two adjacent data points is evaluated. In the second layer, the cost of combining four data points is evaluated. This pattern continues, and the cost of merging the 8th and 16th data points is calculated. By comparing the scores of each layer, the block solution with the lowest score is selected, which corresponds to the optimal data block.

[0119] To calculate the SSE for each level, use the following formula:

[0120] SSE(i, j) = ∑[y i -(a×x i +b)] 2 +∑[y i -(a×x j +b)] 2

[0121] where x i and i represents the ith predicted value and true value. a and b are the slope and y-intercept of the best fit line for the segment.

[0122] (3) Conduct a comprehensive comparison to select the best block partitioning solution

[0123] To avoid repeatedly selecting the same block configuration, each time an optimal block is selected, the corresponding data location information needs to be removed from the score set to ensure the uniqueness and effectiveness of the subsequent selection process.

[0124] (4) Cycle progressively and continuously search for the best solution

[0125] When the nodes of a certain block layer reach the optimal configuration, the node is included in the selected set, and all the nodes related to it are removed from the subsequent layers. Then, the search optimization process starts from the next layer of the current layer and continues.

[0126] Construct similar channel merging module.

[0127] like Figure 4 As shown, this is a schematic diagram of the principle of the similar channel merging module.

[0128] In the channel-independent backbone, the input T∈R (L×M) Pass through the patch module and generate patch x patch ∈R (M×PN×PL) , predict the data for the next T time points, where N represents the time step, M represents that each time step is a vector of dimension M, PN represents the length of each patch, and PL represents the number of patches in each channel.

[0129] And preprocess the data. Use a learnable positional encoding U∈R (PN×D) The linear projection W∈R (PL×D) Project the patch into the initial token, where D represents the dimension of the Transformer model.

[0130] Channel-independent frameworks use a shared model to process channels, which cannot distinguish channels and mainly learns general patterns of channels, lacking channel specificity. To solve this problem, the present invention introduces a channel identifier, a learnable tensor V∈R (M×D) After linear projection of the patch, labeling adds positional encoding and channel identifiers.

[0131] x token =x patch W+U+V,x token ∈R M×PN×D

[0132] Where W represents linear projection, U represents position encoding, M represents the dimension of each time step vector, PN represents the number of patches, and D represents the model dimension.

[0133] Then, x token It is fed into the Transformer encoder and represented at a high level in a channel-independent manner:

[0134]

[0135] The channel identifier represents the unique characteristics of each channel, which enables the model to distinguish channels and capture unique representations of channels.

[0136] To verify the similarity of two channels, the present invention comprehensively considers the intrinsic correlation (GC) and extrinsic similarity (FDTW). First, the intrinsic correlation is tested, and then the extrinsic similarity test is performed. If the extrinsic similarity test result reaches a preset threshold, it is determined that the two channels can be merged.

[0137] Internal causes: Using Granger Causality Test (GC).

[0138] Build two linear regression models:

[0139]

[0140] Where P is the maximum number of lagged observations included in the model, that is, the autoregressive order; A is the coefficient matrix, that is, the contribution of each lagged observation to X; E is the residual of the time series.

[0141] If X2 is added to the autoregressive model of X1 and the variance of E1 is reduced, it means that X2 is the Granger cause of X1, that is, X2 causes the change of X1.

[0142] The present invention utilizes F test and T test to test the variance.

[0143] F test: compare the variance of the residuals of X1 and X2. If the residual variance is significantly reduced after adding the lag term of X2 to X1, then X2 has Granger causality on X1.

[0144] T test: test the regression coefficient A in X1 separately 12j and the regression coefficient A in X2 22j Is it significantly different from 0? 12j If it is significantly different from 0, it means that X2 can predict X1, that is, X2 has a causal relationship with X1; if A 22j If it is significantly different from 0, it means that X1 can predict X2, that is, X1 has a causal relationship with X2.

[0145] Use the conditions to determine the causal relationship between the two channels: If the F statistic of X2 on X1 is significant, and A 12j If X2 is significantly different from 0, then X2 is the Granger cause of X1. On the contrary, if the F statistic of X1 on X2 is significant, and A 22j is significantly different from 0, then X1 is the Granger cause of X2.

[0146] External factors: Using Fast Dynamic Time Warping (FDTW)

[0147] The FDTW algorithm represents the two channels as vectors of length n and m and performs dynamic programming on an n×m grid. In this grid, the matrix element (i, j) represents q i and c j The distance between two points d(q i , c j ), the similarity between each point of channel Q and each point of C, the smaller the distance, the higher the similarity.

[0148] The distance matrix can be obtained by:

[0149] D(i, j) represents the minimum distance to match the first i points of X with the first j points of Y. The calculation formula is as follows:

[0150]

[0151] Where D(0,0)=0, D(i,0) represents the sequence {x1, x2, ..., x i ) to the i-th element of the sequence {y0). D(0, j) represents the sum of the distances from the unique element of the sequence {x0} to the unique element of the sequence {y1, y2, ..., y i} is the sum of the distances between the ,th elements. d(x i ,y j ) represents the distance between two time points.

[0152] In the matrix grid, a path is searched from the lower left corner to the upper right corner, passing through a number of cells. The cells passed by the path correspond to the points where the two sequences are aligned. This path is defined as a regular path. If the length of the regular path is less than a preset threshold, the two channels are considered to meet the conditions in terms of external similarity and can be merged.

[0153] The threshold formula is as follows:

[0154] θ=α·(μ1+μ2)+β·(σ1+σ2)

[0155] The coefficient is represented by the control coefficient, α is 1, β is 2, μ is the mean, and σ is the standard deviation.

[0156] S104, training a multivariate time series prediction model.

[0157] Based on the multivariate time series prediction data set obtained in step S102 and the multivariate time series prediction model constructed in step S103, the parameters in the multivariate time series prediction model are trained.

[0158] First, we construct the MSE loss function MSE(·), whose formula is:

[0159]

[0160] Where MSE is the loss value obtained by the MSE loss function, output a Represents the a-th prediction result.

[0161] True a Representation and output a The corresponding a-th true value, a=1, 2, ..., L.

[0162] Secondly, the multivariate time series prediction dataset is divided into training set, test set and validation set in a ratio of 7:2:1. The loss is tested on the test set after each round of training. When the loss increases for three consecutive rounds, the training is stopped and the weight file with the minimum loss on the test set is stored.

[0163] S105, prediction output.

[0164] Based on the updated network parameters in step S104, predictions are made on the data.

[0165] In practical applications, the data set used for prediction should be independent and identically distributed with the data set used for training, and should also be processed by step S101 and step S102.

[0166] The invention has been verified in practice and has shown the characteristics of accuracy and speed when put into real tasks. In the algorithm test, the performance of the multivariate time series prediction model in prediction tasks in multiple fields was evaluated, and the results are shown in Table 1.

[0167] Table 1 Prediction results of multivariate time series prediction models in multiple domain prediction tasks

[0168]

[0169] Among them, "Models" represents different prediction models, among which the best prediction results are represented by bold fonts, and the vertical direction represents different tasks. From the results in Table 1, it can be seen that the method proposed in the present invention has achieved comprehensive advantages in 6 data sets in 4 fields, indicating the effectiveness of the method proposed in the present invention.

[0170] The present invention proposes a multivariate time series prediction method and a computer program product for realizing the prediction of multivariate time series. In view of the problems and challenges of traditional models in terms of block segmentation and channel processing, a Transformer variant model for realizing prediction based on channel independence is proposed. The model includes dynamic block segmentation, similar channel merging and hybrid decomposition components. Experimental verification shows that the prediction method proposed in the present invention introduces flexible nonlinear relationships to capture local and global information of the data. The prediction method of the present invention is simple to train and deploy, and can achieve good performance in prediction tasks in various fields, providing a typical example for multivariate time series prediction tasks.

[0171] Compared with the existing time series prediction methods, the advantages of the present invention are mainly reflected in the following aspects:

[0172] 1. The dynamic block strategy is introduced, which breaks through the limitations of the traditional fixed-length scheme and can flexibly divide subsequences of different scales according to the differences in the data, thereby capturing the characteristics of different time scales. In addition, it can also reduce information overlap and save computing resources.

[0173] 2. An effective similar channel merging strategy aims to overcome the limitations of the single channel method. On the internal level, Granger causality is used to determine the internal connection between the two channels; on the external level, distance is used to detect external connections. It cleverly achieves deep mining and comprehensive understanding of data features, thereby constructing a richer and more three-dimensional feature representation.

[0174] 3. Abandoning the traditional method of seasonal periodic decomposition of all data, this paper adopts a divide-and-conquer strategy, performing different operations on different data to better handle data with different change patterns.

[0175] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0176] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multivariate time series forecasting method, It is characterized in that include: Input the target to be predicted into the multivariate time series prediction model to obtain the prediction result; The training method of the multivariate time series prediction model includes: Establishing a multivariate time series prediction data set; the multivariate time series prediction data set includes historical data of the target to be predicted and has several attributes; The multivariate time series prediction data set is used to train the multivariate time series prediction model; the multivariate time series prediction model includes a hybrid decomposition module, a dynamic block module, a similar channel merging module, and a Transformer module; The hybrid decomposition module is used to decompose the multivariate time series prediction data set to obtain data features including seasonal items and trend items; The dynamic block module is used to dynamically block the seasonal item features to obtain subsequences of different scales; The similar channel merging module is used to merge the single channels after block division to obtain the input data required by the Transformer encoder; The Transformer module is used to extract features in synchronization with the upper-level module to obtain extracted features, and perform seasonal item prediction on the extracted features, and then combine the decomposed trend items to output the final prediction results.

2. According to the multivariate time series prediction method of claim 1, It is characterized in that After the multivariate time series prediction data set is established, the following steps are also included: The multivariate time series prediction data set is completed to fill the missing values ​​in the multivariate time series prediction data set with the average of the two values ​​before and after the missing value; The multivariate time series prediction data set after completion is standardized so that the mean of the multivariate time series prediction data set is 0 and the variance is 1.

3. The multivariate time series prediction method according to claim 1 or 2, It is characterized in that The multivariate time series forecasting dataset meets the following requirements: The time difference between adjacent data in the multivariate time series prediction data set is a constant; The feature type and feature quantity at each moment in the multivariate time series prediction data set are consistent; The multivariate time series prediction data set includes the date and time stamps of each data collection, and the date and time stamps corresponding to each data have the same format.

4. According to the multivariate time series prediction method of claim 3, It is characterized in that The specific process of the dynamic block module includes: (1) The optimal number of blocks is obtained through experiments. They are: S 1 =2, S 2 =4, S 3 =8, S 4 =16. The total number of blocks is: PN=n / (S 1 +S 2 +S 3 +S 4 )×4+[n%(S 1 +S 2 +S 3 +S 4 )] Where n represents the lookback window and % represents the remainder. (2) Each input univariate time series is divided into subsequences of different scales through the dilated causal convolutional patch model. This module can be simplified to: F(t)=Select(W[j]*X[td*j]) Where d is the expansion factor, which is also the patch length; W[j] is the block size, that is, the weight; X(td*j) represents the value at time step td*j. (3) Calculate the fitting score. SSE(θ 0 ,the 1 )=∑(yi-(θ 0 *xi+θ 1 )) 2 Where θ(0, 1) represents the objective function parameter, x i and i Represents the predicted value and the true value. (4) Conduct a comprehensive comparison and select the best block partitioning scheme. (5) The loop continues to search for the optimal solution until all data are processed in blocks.

5. According to the multivariate time series prediction method of claim 4, It is characterized in that The specific process of the similar channel merging module includes: Preprocess the data. Introduce a channel identifier, a learnable tensor V∈R (M×D) After linear projection of the patch, the markers add positional encodings and channel identifiers: x token =x patch W+U+V,x token ∈R M×PN×D Where W represents linear projection, U represents position encoding, M represents the dimension of each time step vector, PN represents the number of patches, and D represents the model dimension. Then, x token is fed into the Transformer encoder for a high-level representation in a channel-independent manner: The channel identifier represents the unique characteristics of each channel, which enables the model to distinguish channels and capture unique representations of channels. Consider the internal factors (GC) and external factors (FDTW) of different channels to verify whether the two channels should be merged. Channels can only be merged when both the internal and external factors meet the conditions. The internal cause uses Granger causality detection to establish two linear regression models: Where P is the maximum number of lagged observations included in the model, that is, the autoregressive order; A is the coefficient matrix, that is, the contribution of each lagged observation to X; E is the residual of the time series. The external factor uses fast dynamic time planning to align the two channels through dynamic planning so that the distance between them is minimized.

6. According to the multivariate time series prediction method of claim 5, It is characterized in that The specific process of the hybrid decomposition module includes: Using the mixed decomposition formula: Y t =(T t +C t )*(S t +I t ) where Y t represents the original sequence, T t Indicates trend, S t Indicates seasonality, C t Indicates periodicity, I t Indicates offset. The series is decomposed into seasonal terms and trend terms, and the seasonal terms are used as the main input data part of the model.

7. The multivariate time series prediction method according to claim 6, It is characterized in that When training the multivariate time series prediction model, the loss function used includes: Where MSE is the loss value obtained by the MSE loss function, output a Represents the a-th prediction result. True a Representation and output a The corresponding a-th true value, a=1,2,...,L.

8. A computer program product, It is characterized in that The computer program product comprises a computer program or instructions. When the computer program or instruction is executed by the communication device, the method according to any one of claims 1 to 7 is executed.

Images