Multivariate time series prediction method, device, system and model training method

Abstract

The application discloses a multivariate time series prediction method, device and system and a model training method, relates to the technical field of information, and is used to improve the accuracy of multivariate time series prediction. The application decomposes time series on high-frequency components and low-frequency components, further encodes and decouples low-frequency signal vectors, high-frequency signal vectors and noise signal vectors, adaptively fuses and enhances effective signal parts, respectively predicts effective signals and noise signals, fuses inference results, and obtains prediction results. The application retains effective signals in high-frequency components, isolates signals and noise and utilizes the signals and noise, can stably capture variable dependence, improves the prediction accuracy of effective signals, simultaneously utilizes noise prediction random components, adds the random components to predicted effective signals, and is more matched with real environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a multivariate time series forecasting method, device, system and model training method. BACKGROUND

[0002] In the scenarios of energy management, traffic supervision and Internet of Things systems, multi-dimensional variable index management is involved. After these variable indexes are synchronously collected in time sequence (i.e., a plurality of variable indexes are collected at the same time, and this is repeated and arranged in time sequence), a multivariate time series is obtained. Multivariate time series forecasting (MTSF) is crucial for decision-making in various fields. The so-called forecasting is to predict the values of each variable index at a certain time step in the future according to the multivariate time series collected in the history. For example, in the field of traffic supervision, the variable indexes include traffic flow, passenger flow, vehicle type statistics, precipitation, temperature, wind size, etc. Time series forecasting is to predict the traffic flow, passenger flow, vehicle type statistics, precipitation, temperature, wind size, etc. at each time step (e.g., every 5 minutes) in the future (e.g., the next hour) to provide supporting data for traffic management and decision-making in the future.

[0003] At present, known time series forecasting methods mostly use deep learning architectures such as RNN (Recurrent Neural Network), CNN (Convolutional Neural Network) and Transformer to train time series forecasting models, and then predict the variable indexes at future time steps. However, these methods are often limited by the inherent low signal-to-noise ratio (SNR) in real time series, making it difficult to distinguish between effective signals and noise signals, and the prediction performance is greatly compromised.

[0004] To cope with the challenge of SNR, the traditional approach is to use sliding window filtering or wavelet decomposition in the time domain to filter noise, but the signal processing in the time domain is relatively complex and labor-intensive. Unlike this, after transforming the signal into the frequency domain, the effective signal and the noise signal can be separated through frequency spectrum analysis, and the effective signal can be separated with relatively low computational load, thereby improving the prediction accuracy.

[0005] However, even if the workload of denoising can be reduced by means of frequency domain analysis, the current frequency domain analysis method is to directly select the low-frequency component of the signal, and simply and roughly filter out the high-frequency component as noise, seriously ignoring the valuable information that the high-frequency component may carry. Through research and analysis, the high-frequency component is not only unstructured noise, but also often contains valuable signals reflecting the following contents: short-term changes (such as sudden changes in energy demand) and sudden events (such as device failure or traffic network anomalies in the Internet of Things system). If the high-frequency component is suppressed or eliminated indiscriminately in the current manner, although the noise is discarded, the key information essential for accurate prediction will be lost, resulting in poor prediction accuracy of multivariate time series. SUMMARY

[0006] The application aims to provide a multivariate time series prediction method, device, system and model training method to improve the accuracy of multivariate time series prediction.

[0007] The technical solution adopted by the application is as follows:

[0008] A multivariate time series prediction method, the time series is obtained by synchronously collecting a plurality of variable indicators in time sequence; the method comprises:

[0009] Obtaining a time series under a current scenario;

[0010] Decomposing the obtained time series into high-frequency components and low-frequency components based on a preset frequency threshold;

[0011] Using a parameter-shared embedding layer to respectively vectorize the high-frequency components and the low-frequency components;

[0012] Extracting a low-frequency signal vector, a high-frequency signal vector and a noise signal vector from the vectorized high-frequency components and low-frequency components;

[0013] Adaptively fusing the low-frequency signal vector and the high-frequency signal vector to obtain a fused signal vector;

[0014] Using the low-frequency signal vector to enhance the attention of the fused signal vector to variables to obtain an enhanced variable representation vector;

[0015] Respectively performing synchronous reasoning on the enhanced variable representation vector and the noise signal vector, and fusing the reasoning results to obtain a prediction result for a future time step.

[0016] Further, extracting a low-frequency signal vector, a high-frequency signal vector and a noise signal vector from the vectorized high-frequency components and low-frequency components comprises:

[0017] According to the principle of maximizing the mutual information of the low-frequency signal vector and the high-frequency signal vector and minimizing the mutual information of the low-frequency signal vector and the noise signal vector, the low-frequency signal vector is extracted from the encoding vector of the low-frequency component, and the high-frequency signal vector and the noise signal vector are extracted from the encoding vector of the high-frequency component, respectively.

[0018] Further, the method of maximizing the mutual information of the low-frequency signal vector and the high-frequency signal vector and minimizing the mutual information of the low-frequency signal vector and the noise signal vector comprises:

[0019] The low-frequency signal vector is reconstructed using the high-frequency signal vector, and a first reconstruction loss is calculated.

[0020] The low-frequency signal vector is reconstructed using the noise signal vector, and a second reconstruction loss is calculated.

[0021] The first reconstruction loss and the second reconstruction loss are combined to obtain a total reconstruction loss.

[0022] The total reconstruction loss is minimized.

[0023] Further, the low-frequency signal vector and the high-frequency signal vector are adaptively fused to obtain a fused signal vector, comprising:

[0024] The low-frequency signal vector and the high-frequency signal vector are spliced in the feature dimension to obtain a spliced vector.

[0025] The spliced vector is processed using a multi-layer perception, and the contribution degree of the low-frequency signal vector and the contribution degree of the high-frequency signal vector are calculated by adjusting using a temperature coefficient.

[0026] The low-frequency signal vector and the high-frequency signal vector are weighted and fused using the respective contribution degrees to obtain the fused signal vector.

[0027] Further, the attention of the fused signal vector on variable dependence is improved using the low-frequency signal vector, comprising:

[0028] The Gram matrix of the fused signal vector is calculated using the low-frequency signal vector, and the similarity matrix is adaptively optimized using a trainable parameter.

[0029] The fused signal vector is weighted using the similarity matrix and connected with the fused signal vector in residual connection, and after linear transformation, the feedforward neural network is used for post-processing.

[0030] Further, the Gram matrix of the fused signal vector is calculated using the low-frequency signal vector, and the similarity matrix is adaptively optimized using a trainable parameter, comprising:

[0031] performing layer normalization operation on the low-frequency signal vector, and multiplying the low-frequency signal vector after the layer normalization operation and transposition to obtain a Gram matrix;

[0032] performing element-by-element multiplication on the Gram matrix after activation and a weight matrix and adding a bias vector;

[0033] performing activation operation on the calculation result to obtain the similarity matrix.

[0034] In another aspect, the present application also provides a multivariate time series prediction device, comprising a processor and a storage medium, wherein the storage medium stores a computer program, and the computer program performs the multivariate time series prediction method described above when executed by the processor.

[0035] As an intermediate product of multivariate time series prediction, the present application also provides a multivariate time series prediction model training method, wherein the multivariate time series prediction model is used to predict future time steps of time series from input current time series, and the time series is obtained by synchronously collecting multiple variable indicators in time sequence; the training method comprises:

[0036] obtaining a time series sample set, and dividing the time series sample set into a training set and a test set;

[0037] training the multivariate time series prediction model by using the training set to minimize the training loss, and testing the trained multivariate time series prediction model by using the test set;

[0038] The multivariate time series prediction model is configured to:

[0039] obtain an input time series;

[0040] decompose the obtained time series into high-frequency components and low-frequency components based on a preset frequency threshold;

[0041] perform vector encoding on the high-frequency components and the low-frequency components by using a parameter-shared embedding layer respectively;

[0042] extract a low-frequency signal vector, a high-frequency signal vector and a noise signal vector from the encoding vectors of the high-frequency components and the low-frequency components respectively;

[0043] adaptively fuse the low-frequency signal vector and the high-frequency signal vector to obtain a fused signal vector;

[0044] improve the attention of the fused signal vector in variable dependence by using the low-frequency signal vector to obtain an enhanced variable representation vector;

[0045] Synchronous inference is respectively performed on the enhanced variable representation vector and the noise signal vector, and a prediction result of a future time step is obtained by fusing the inference results;

[0046] The training loss is composed of a mean square error (MSE) between the prediction result and the actual result of all variables at each time step in the time series samples of the training set, and a total reconstruction loss in the process of extracting the low-frequency signal vector, the high-frequency signal vector and the noise signal vector.

[0047] In another aspect, the application also provides another multivariate time series prediction device, which is configured with a multivariate time series prediction model trained by using the above multivariate time series prediction model training method.

[0048] In another aspect, the application also provides a multivariate time series prediction system, which includes an input device, an output device and a multivariate time series prediction device; the input device is connected to the input end of the multivariate time series prediction device, used for receiving a time series to be predicted and inputting into the multivariate time series prediction model; the output device is connected to the output end of the multivariate time series prediction device, used for outputting the time series predicted by the multivariate time series prediction model.

[0049] In summary, due to the adoption of the above technical solutions, the application has the following beneficial effects:

[0050] The application provides a multivariate time series prediction method for the demand of multivariate time series prediction, separates valuable signals and noises from high-frequency components of signals, and adaptively fuses and enhances the valuable signals and low-frequency signals, so that valuable information can be effectively extracted, spatial and temporal dependencies between variables can be stably captured, and signal features can be more accurately extracted. In addition, the future random components are predicted by using the noise signals, and the prediction results of the effective signals are added, so that the final prediction result is more matched with the actual environment. The application not only more effectively extracts effective signals (especially effective signals in high-frequency components), but also utilizes noise signals, and improves the prediction accuracy of multivariate time series in real application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0051] The application will be described by examples and with reference to the accompanying drawings, in which:

[0052] Figure 1 is a flowchart of the multivariate time series prediction method in one embodiment.

[0053] Figure 2is a construction diagram of a multivariate time series prediction model in an embodiment, wherein sub-diagram (a) shows a decomposition method of high-frequency components and low-frequency components, sub-diagram (b) shows a decoupling method of low-frequency signal vectors, high-frequency signal vectors and noise signal vectors, sub-diagram (c) shows a construction process of fusion signal vectors, and sub-diagram (d) shows a construction process of enhanced variable representation vectors.

[0054] Figure 3 is a performance comparison diagram of the present application and a baseline model prediction, Figure 3 sub-diagram (a), sub-diagram (b) and sub-diagram (c) of are test diagrams of the mean square error of the present application method, iTransformer model and FilterNet model respectively on PEMS04 dataset, ETTm1 dataset and Traffic dataset with respect to signal-to-noise ratio.

[0055] Figure 4 is a performance test diagram of the present application on different prediction lengths, Figure 4 sub-diagram (a) and sub-diagram (b) of are test diagrams of the mean square error of the present application method, FilterNet model, iTransformer model, PatchTST model and Crossformer model respectively on Electricity dataset and Traffic dataset with respect to input time series length.

[0056] Figure 5 is a test diagram of the configuration of the regularization coefficient in different embodiments, Figure 5 sub-diagram (a), sub-diagram (b) and sub-diagram (c) of are test diagrams of the mean square error of the present application method respectively on ECL dataset, Traffic dataset and Solar-Energy dataset with respect to regularization coefficient . DETAILED DESCRIPTION

[0057] All features disclosed in this specification, and / or all methods or processes disclosed in this specification may be combined in any combination, except combinations where at least some features and / or steps are mutually exclusive.

[0058] Any of the features disclosed in this specification, unless explicitly stated otherwise, may be replaced by alternative features serving the same, or a similar, purpose.

[0059] In view of the problem that the prediction accuracy of the multivariate time series is poor due to the direct filtering of high-frequency components in the existing frequency domain analysis method, the embodiments of the present application propose a multivariate time series prediction method, device, system and model training method, which utilizes the noise signal while retaining the effective signal of the high-frequency component, thereby improving the accuracy of the multivariate time series in the display application scenario.

[0060] As shown in Figure 1 , in one embodiment, the multivariate time series prediction method proposed by the present application includes the following processes:

[0061] S1, obtaining a time series under a current scenario.

[0062] The application scenarios of the method of the present application are, for example, traffic supervision scenarios, medical detection scenarios, weather prediction scenarios, etc. In the traffic supervision scenario, the variable indicators contained in the time series can be vehicle flow, pedestrian flow, vehicle type (vehicle type, such as truck, car, truck) statistics, rainfall / snowfall, temperature, etc. In the medical detection scenario, the variable indicators contained in the time series can be physiological indicators (such as blood pressure, heart rate, etc.), test indicators (platelet content, blood oxygen saturation, white blood cell content, etc.), detection indicators (electrocardiogram trend, nodule size, etc.), etc. In the weather prediction scenario, the variable indicators contained in the time series can be air temperature, rainfall / snowfall, wind strength, light intensity, humidity, etc.

[0063] According to a predetermined frequency / step size, the value of each variable indicator is recorded at the corresponding time step, and the multivariate time series is obtained according to the recording time sequence. Each variable can obtain a one-dimensional vector, and multiple variables can form a two-dimensional vector matrix.

[0064] According to the input window size of the prediction, the recorded multivariate time series is taken as a sliding value, and a time series of a predetermined length can be obtained.

[0065] S2, decomposing the obtained time series into high-frequency components and low-frequency components based on a preset frequency threshold.

[0066] As an optional implementation, referring to subgraph (a) in Figure 2 , the decomposition method of the high-frequency components and the low-frequency components of the time series includes:

[0067] S21, transforming the time series from time domain to frequency domain.

[0068] The obtained time series is transformed from time domain to frequency domain by using fast Fourier transform (FFT), and is decomposed into ​a set of frequencies and corresponding amplitudes, where R represents a real number field, N represents the number of variable indexes, and L represents the length of the variable time series, represents a transformation to round up.

[0069] The transformation process is represented as:

[0070] Equation (1): ;

[0071] where FFT represents a fast Fourier transform, represents a time series after transformation to the frequency domain.

[0072] S22, by setting a frequency threshold distinguishing high-frequency components and low-frequency components.

[0073] The frequency threshold In some embodiments, it is a proportion threshold. By using the frequency threshold The process of dividing high-frequency components and low-frequency components is:

[0074] The low-frequency components correspond to the first frequency points, and the high-frequency components correspond to the remaining frequency points. In addition, in order to ensure that the distinguished high-frequency components and low-frequency components both contain frequency points, in order to facilitate signal recovery in the time domain, the length of the high-frequency components and the low-frequency components that are insufficient are zero-padded.

[0075] Definition , represents the number (length) of frequency points of the low-frequency components, then the low-frequency components (frequency domain) are represented by the following equation (2):

[0076] Equation (2): .

[0077] High-frequency components (frequency domain) are represented by the following equation (3):

[0078] Equation (3): .

[0079] S23, respectively transform the high-frequency components and the low-frequency components from the frequency domain to the time domain.

[0080] The high-frequency components and the low-frequency components in the frequency domain are restored to the time domain by using the inverse fast Fourier transform IFFT, thereby generating low-frequency components representing the main signal and high-frequency components capturing short-term fluctuations and noise.

[0081] The low-frequency components and the high-frequency components transformed to the time domain are represented as:

[0082] Formula (4): ;

[0083] In the formula, IFFT represents the inverse fast Fourier transform.

[0084] S3, the embedding layer using parameter sharing respectively encodes the high frequency component and the low frequency component into vectors.

[0085] Vector encoding is to project the time domain signal into a vector space. Linear transformation is respectively performed on the low frequency component and the high frequency component to obtain the low frequency component encoding vector and the high frequency component encoding vector :

[0086] Formula (5): ;

[0087] In the formula, represents performing linear transformation, D represents the feature dimension.

[0088] S4, the low frequency signal vector, the high frequency signal vector and the noise signal vector are respectively extracted from the encoding vectors of the high frequency component and the low frequency component.

[0089] As an optional implementation, the method of extracting the low frequency signal vector, the high frequency signal vector and the noise signal vector comprises:

[0090] According to the principle of maximizing the mutual information of the low frequency signal vector and the high frequency signal vector, and minimizing the mutual information of the low frequency signal vector and the noise signal vector, the low frequency signal vector is extracted from the encoding vector of the low frequency component, and the high frequency signal vector and the noise signal vector are respectively extracted from the encoding vector of the high frequency component.

[0091] In some feasible embodiments of the present application, three parallel feature extractors (feature encoders) are respectively used to separate three independent potential representations: the low frequency signal vector , the high frequency signal vector and the noise signal vector . Assuming that the three feature extractors are , the three potential representations are respectively represented as:

[0092] Formula (6): ;

[0093] Formula (7):

[0094] Formula (8): .

[0095] Wherein, .

[0096] It is analyzed that there is also effective signal in the high frequency component, so it can be assumed that and there is a potential time-dependent relationship, while statistically independent of . Therefore, in this application, according to the principle of maximizing the mutual information of the low frequency signal vector and the high frequency signal vector , minimizing the mutual information of the low frequency signal vector and the noise signal vector , the following optimization objective function is designed to minimize:

[0097] Equation (9): ;

[0098] In the formula, J denotes the objective function, denotes the mutual information, for example denotes the mutual information of and , and denotes the mutual information of and . By minimizing J , the mutual information of the low frequency signal vector and the high frequency signal vector is maximized, and the mutual information of the low frequency signal vector and the noise signal vector is minimized.

[0099] In addition, as a feasible optimization method, referring to subgraph (b) in Figure 2 , in some optional embodiments of the present application, the calculation method of information entropy is used to optimize the above objective function. The method of maximizing the mutual information of the low frequency signal vector and the high frequency signal vector, and minimizing the mutual information of the low frequency signal vector and the noise signal vector includes:

[0100] S41, reconstruct the low frequency signal vector using the high frequency signal vector, calculate the first reconstruction loss; reconstruct the low frequency signal vector using the noise signal vector, calculate the second reconstruction loss.

[0101] The present application uses conditional entropy to decompose the mutual information, and the principle is:

[0102] Equation (10): ;

[0103] In the formula, denotes the mutual information of parameter A and parameter B, denotes the information entropy of parameter A, It represents the conditional entropy that parameter A holds true under the condition of parameter B.

[0104] Based on the above principles, the objective function can be obtained. J Conversion form:

[0105] Equation (11): .

[0106] In equation (11), and All are conditional entropy, where:

[0107] Equation (12): ;

[0108] In the formula, g For refactoring functions, E Indicates the expectation. Expressing the request The second norm of the equation is given. Therefore, reducing the reconstruction error directly reduces the conditional entropy. Thus, in this application, the mean squared error is used as the first reconstruction loss. Its design is as follows:

[0109] Equation (13): ;

[0110] In the formula, Indicates the parameter group Perform parameterized MLP (Multilayer Perceptron) model, This indicates that the MLP model is used by The corresponding low-frequency signal vector is reconstructed, in Figure 2 The subgraph (b) is represented as . and They are respectively and The first in i There are several variables. This application minimizes the conditional entropy when optimizing the first reconstruction loss. Conversely, mutual information can be maximized. This mechanism ensures that the high-frequency signal vector retains frequency information that closely matches the low-frequency signal vector, while effectively filtering out irrelevant noise.

[0111] The other part after the objective function transformation is the conditional entropy. The optimization objective is to maximize ,maximize Conversely, minimizing Due to the low-frequency signal vector With high-frequency noise vector They are statistically independent, so it is necessary to estimate the statistical independence of each separately.

[0112] Since minimizing is equivalent to enforcing the orthogonality of both in the vector space, in some optional embodiments of the present application, a cosine similarity based set constraint condition is designed to construct the second reconstruction loss, thus avoiding to the degenerate solution problem of reconstruction. Specifically, a connection loss is designed on the negative cosine similarity to construct the second reconstruction loss :

[0113] Equation (14): ;

[0114] wherein Cosine represents the cosine similarity function, (like ) represents the separation boundary established in the angle domain, represents the i-th variable in , and represents the vector dot product of and .

[0115] S42, combining the first reconstruction loss and the second reconstruction loss to obtain a total reconstruction loss.

[0116] The total reconstruction loss is represented as:

[0117] Equation (15): .

[0118] S43, minimizing the total reconstruction loss. Minimizing the total reconstruction loss means maximizing the mutual information of the low-frequency signal vector and the high-frequency signal vector, and minimizing the mutual information of the low-frequency signal vector and the noise signal vector.

[0119] S5, adaptively fusing the low-frequency signal vector and the high-frequency signal vector to obtain a fused signal vector .

[0120] As an optional embodiment, referring to subgraph (c) in FIG. 8, the method of adaptively fusing the low-frequency signal vector Figure 2 and the high-frequency signal vector includes:

[0121] S51, splicing the low-frequency signal vector and the high-frequency signal vector in the feature dimension to obtain a spliced vector. The spliced vector is represented as .

[0122] ​S52, the splicing vector is processed by using a multi-layer perception (MLP), and a temperature coefficient is used for adjustment, and an activation low-frequency signal vector is obtained . The contribution degree of the high-frequency signal vector is calculated.

[0123] is represented by , the contribution degree of the low-frequency signal vector is represented by , and the contribution degree of the high-frequency signal vector is represented by .

[0124] Formula (16): ;

[0125] In the formula, MLP represents a multi-layer perception, , and sigmoid represents a sigmoid activation function.

[0126] S53, the low-frequency signal vector and the high-frequency signal vector are respectively weighted and fused according to the respective contribution degrees, to obtain a fusion signal vector .

[0127] Formula (17): ;

[0128] In the formula, represents element-wise multiplication. The temperature coefficient helps to adjust the gate behavior in the training process.

[0129] Through the above-mentioned gating mechanism, the application realizes adaptive integration of the low-frequency signal vector and the high-frequency signal vector , so as to dynamically learn more comprehensive time series representation.

[0130] After the adaptive fusion of the low-frequency signal vector and the high-frequency signal vector , the application also models the dependency relationship between variables through a learnable attention mechanism. Specifically, the method further comprises:

[0131] S6, the low-frequency signal vector is used to enhance the fusion signal vector in the variable dependency attention, to obtain an enhanced variable representation vector .

[0132] As an optional implementation, referring to the subgraph (d) in Figure 2 , in step S6, the low-frequency signal vector is used to enhance the fusion signal vector in the variable dependency attention, and the method comprises:

[0133] S61, obtaining a low-frequency signal vector calculating a fusion signal vector gram matrix of the fusion signal vector and adaptively optimizing the gram matrix using trainable parameters to obtain a similarity matrix .

[0134] In some embodiments, step S61 obtains the similarity matrix by:

[0135] performing layer normalization on the low-frequency signal vector and multiplying the layer-normalized low-frequency signal vector by the transposed layer-normalized low-frequency signal vector to obtain the gram matrix . This is represented as:

[0136] Equation (18):

[0137] where denotes the layer normalization operation.

[0138] performing activation on the gram matrix and element-wise multiplying the activated gram matrix by the weight matrix and adding the bias vector .

[0139] performing activation on the result to obtain the similarity matrix .

[0140] The above two operations are represented as:

[0141] Equation (19):

[0142] where denotes the activation function.

[0143] S62, weighting the fusion signal vector using the similarity matrix and performing residual connection between the weighted fusion signal vector and the fusion signal vector , performing linear transformation, and post-processing using at least one level of feedforward neural network (multiple levels of feedforward neural network are represented as FFNs) to obtain the enhanced variable representation vector . This is represented as:

[0144] Equation (20):

[0145] where ​​This represents a feedforward neural network. This indicates linearization. This indicates a residual join operation.

[0146] S7. Represent the enhancement variable vectors respectively. and noise signal vector Perform synchronous reasoning and fuse the reasoning results to obtain the prediction results for future time steps.

[0147] like Figure 2 As shown, in the prediction phase, the variable representation vector is enhanced. and noise signal vector Inference mapping is performed through independent prediction layers (Project), aligning the hidden dimensions of both prediction layers with the prediction time domain (future time steps) T. From low-frequency signal vector and high-frequency signal vector The fusion and enhancement yields the main signal features; while This captures additional high-frequency components, such as potential anomalies or irregularities, characterizing the impact of the real world on the valid signal. The final prediction result is expressed as... :

[0148] Equation (21): ;

[0149] In the formula, These represent the representation vectors for the enhanced variables, respectively. and noise signal vector The prediction layer.

[0150] With the network parameters at each level determined, the above method can be used to predict a (multivariate) time series of a predetermined length T from the obtained time series.

[0151] In this regard, embodiments of this application also provide a method for training a multivariate time series prediction model, which is used to train the network parameters of relevant network layers. This multivariate time series prediction model is used to predict the time series of future time steps from the input current time series.

[0152] Training methods include:

[0153] Step 1: Obtain the time series sample set and divide it into training set and test set.

[0154] Time series samples can be obtained by collecting historical time series data. The ratio of training set to test set can be, for example, 7:3 or 8:2.

[0155] Step 2, training the multivariate time series prediction model with the training set as the target to minimize the training loss, and testing the trained multivariate time series prediction model with the test set.

[0156] The multivariate time series prediction model is configured to:

[0157] S1, obtaining an input time series.

[0158] S2, decomposing the obtained time series into high-frequency components and low-frequency components based on a preset frequency threshold.

[0159] S3, using a parameter-shared embedding layer to respectively vectorize the high-frequency components and the low-frequency components.

[0160] S4, extracting a low-frequency signal vector, a high-frequency signal vector and a noise signal vector from the encoded vectors of the high-frequency components and the low-frequency components.

[0161] S5, adaptively fusing the low-frequency signal vector and the high-frequency signal vector to obtain a fused signal vector.

[0162] S6, using the low-frequency signal vector to enhance the attention of the fused signal vector in variable dependence to obtain an enhanced variable representation vector.

[0163] S7, respectively performing synchronous inference on the enhanced variable representation vector and the noise signal vector, and fusing the inference results to obtain a prediction result for the future time step.

[0164] The training loss described above is composed of the weighted combination of the mean square error (MSE) between the prediction result and the true result of all variables at each time step in the time series samples of the training set, and the total reconstruction loss in the process of extracting the low-frequency signal vector, the high-frequency signal vector and the noise signal vector.

[0165] The features that can be further designed in each of steps S1-S7 in the model training method embodiment can be designed in each of the embodiments of the prediction method described above, which will not be described one by one here.

[0166] As for the training loss involved in the training, assuming that the principle of maximizing the mutual information of the low-frequency signal vector and the high-frequency signal vector and minimizing the mutual information of the low-frequency signal vector and the noise signal vector is still used, the low-frequency signal vector is extracted from the encoded vector of the low-frequency component, and the high-frequency signal vector and the noise signal vector are extracted from the encoded vector of the high-frequency component, then the total reconstruction loss in the process of extracting the low-frequency signal vector, the high-frequency signal vector and the noise signal vector can follow the total reconstruction loss of formula (15) in the previous description .

[0167] The mean square error (MSE) between the prediction result and the true result of all variables in each time step of the time series sample of the training set is calculated by the following method:

[0168] Formula (22): ;

[0169] In the formula, is the mean square error to be solved, respectively represent the prediction value and the true value of the first i variable at the future first t time step.

[0170] Therefore, the training error is represented as:

[0171] Formula (23): ;

[0172] In the formula, is a hyperparameter of the model, and represents a regularization coefficient weighted to the , which is used to adjust the influence of the regularization term, so as to ensure the balance of the training target. It can be determined by a hyperparameter optimization algorithm, or set according to experience (such as =1, 2), but it can be determined that for different application scenarios, the optimal hyperparameter often needs to be adjusted by the historical data of the application field. The performance of the optimal can be selected by setting different coefficient values on the historical data.

[0173] Through the above process, the multivariate time series prediction model can be trained by using the historical time series. The module not only considers the effective components in the high-frequency components of the signal, but also considers the actual influence of the noise signal in the high-frequency components, so that the prediction result is more accurate and more consistent with the real scene.

[0174] According to the idea of the present application, the embodiment of the present application further provides a multivariate time series prediction device, which comprises a processor and a storage medium, and the storage medium stores a computer program. When the computer program is run by the processor, the multivariate time series prediction method of the above-mentioned embodiment is executed.

[0175] In addition, the embodiment of the present application further provides another multivariate time series prediction device, which is configured with the multivariate time series prediction model trained by using the multivariate time series prediction model training method.

[0176] Based on the above prediction device, the embodiment of the present application also provides a multivariate time series prediction system, which comprises an input device, an output device and a multivariate time series prediction device. The input device is connected to the input end of the multivariate time series prediction device, and is used to receive the time series to be predicted and input into the multivariate time series prediction model; the output device is connected to the output end of the multivariate time series prediction device, and is used to output the time series predicted by the multivariate time series prediction model.

[0177] In addition, the embodiment of the present application also conducts systematic experiments on 12 actual multivariate time series data sets. These data sets include ECL (power load), Traffic (traffic), Weather (weather), Solar-Energy (solar energy), 4 PEMS (traffic flow) data sets (PEMS03, PEMS04, PEMS07, PEMS08) and 4 ETT (electric transformer temperature) data sets (ETTh1, ETTh2, ETTm1, ETTm2). These data sets are widely used as benchmark data sets in the field of multivariate time series prediction. The key features of these data sets are shown in Table 1.

[0178] Table 1 Multivariate time series data set feature table

[0179]

[0180] To comprehensively evaluate the effectiveness of the scheme of the present application, three baseline models are selected for performance comparison in the embodiment of the present application. The baseline models include: (i) Transformer-based architecture, including iTransformer model, PatchTST model and FEDformer model; (ii) MLP-based architecture, covering FilterNet model, SOFTS model, TimeMixer model, TSMixer model, FreTS model, TiDE model and DLinear model; (iii) CNN-based model TimesNet model.

[0181] Experimental condition setting: To ensure consistency of the experiment, a history window length of 96 time steps (i.e. the length of the input history time series is 96) is uniformly used in all benchmark tests. The prediction length is configured as follows: the prediction length of the PEMS data set is designed as {12, 24, 48, 96}4, and the test of the remaining benchmark data sets uses {96, 192, 336, 720}4 prediction lengths, respectively.

[0182] To evaluate the prediction performance of each prediction method and facilitate quantitative comparison, the mean square error (MSE) and the mean absolute error (MAE) are used as performance evaluation indicators in the embodiments of the present application. Both indicators are smaller in value, indicating higher prediction accuracy. The definitions are as follows: The predicted value at time step t is denoted as y t, and the true value at time step t is denoted as y t. T represents the total number of prediction steps. The calculation methods of the above two indicators are as follows:

[0183] Equation (24): .

[0184] The experimental results are shown in Table 2. Due to the large amount of data, Table 2 only shows the test indicators under the longest prediction length (i.e., 96 or 720).

[0185] Table 2 Performance indicator test table (excerpts)

[0186]

[0187] Table 2 Performance test table (continued)

[0188]

[0189] Table 2 Performance test table (continued)

[0190]

[0191] According to the experimental results, the method proposed in the present application has obvious advantages over each baseline model in different prediction time domain ranges and different data sets, and has a large improvement in MSE and MAE (indicator decreases, performance improves). The experimental results on 12 data sets show that the present application scheme achieves the lowest MSE in 37 test cases and the lowest MAE in 44 test cases. Although the currently recognized most advanced time domain prediction model SOTA (such as the SOFTS model and the iTransformer model) also focuses on modeling the relationship between variables, the noise sensitivity of these models distorts these relationships, leading to learning of incorrect dependency relationships. However, the present application effectively alleviates this problem by isolating the noise signal and the effective signal and capturing more robust variable dependency relationships from the effective signal only, thereby exhibiting superior performance. On the other hand, the mainstream frequency domain model FilterNet uses a method of selectively filtering time series components. Although this method improves the robustness of the model to some extent, it causes information content to be lost, especially in data sets with a large number of variables such as traffic data and PEMS. The present application innovatively uses a decoupling mechanism to separate the signal from the noise and independently represent each component. This design allows the present application to effectively control noise while retaining key information, thereby exhibiting stronger prediction performance in all benchmark tests.

[0192] Furthermore, to evaluate the robustness of this application under diverse noise conditions, comparative experiments were conducted on the PEMS04, ETTm1, and Traffic datasets. Benchmark tests were performed against two state-of-the-art baseline models: FilterNet and iTransformer. To simulate different noise levels, Gaussian white noise was injected into the input data during training, with target signal-to-noise ratios (SNRs) of {−10, −5, 0, 5, 10, 20} dB. For each SNR value, the signal power was calculated, the corresponding linear noise power was determined, and Gaussian noise was superimposed according to implementation details. This method effectively evaluates the robustness of the model by controlling the gradual degradation of input quality.

[0193] like Figure 3 As shown, this application consistently maintains a low mean squared error across all datasets and signal-to-noise ratio (SNR) levels. Notably, FilterNet's performance significantly degrades on the ETTm1 dataset under low SNR conditions, likely due to its dependence on key frequency components, which are easily masked by noise. Similarly, iTransformer's performance also declines at low SNRs because excessive noise interferes with the attention mechanism's ability to capture effective correlations between variables. In contrast, this application demonstrates more stable performance: compared to FilterNet and iTransformer, its MSE curve maintains a flat trend across different SNR levels, fully demonstrating its superior robustness against noise perturbations.

[0194] In addition, to verify the scientific validity and necessity of the design of each step of this application (such as frequency decoupling), ablation experiments were also conducted in the embodiments of this application.

[0195] This application embodiment removes data from the ECL, Traffic, and Solar-Energy datasets. Figure 2 Ablation experiments were conducted on the frequency decoupling module in the model. The input sequence length was fixed at 96, and the prediction duration was set to 96, 192, 336, and 720 time steps, respectively. As shown in Table 3, the complete model of this application consistently outperformed the ablation model with the frequency decoupling module removed across all benchmarks and prediction durations. Notably, the performance gap gradually widened with increasing prediction duration. This phenomenon was analyzed to be attributed to the fact that the absence of the frequency decoupling module weakened the model's ability to distinguish between signal and noise. High-frequency noise interfered with low-frequency signals, masking potential long-term trends and making it difficult to capture the true correlations between real variables, ultimately affecting overall performance.

[0196] Table 3 Ablation Experiment Test Table

[0197]

[0198] In addition, the embodiments of the present application also aim at the phenomenon that the prediction accuracy of the model based on the Transformer architecture (such as the Informer model, the Autoformer model, and the FEDformer model) decreases when the input time sequence length is extended. The performance of the present application in this aspect is tested to verify the sensitivity of the parameters of the present application.

[0199] The embodiments of the present application compare the present application with the FilterNet model, the iTransformer model, the PatchTST model, and the Crossformer model through experiments. The ECL dataset and the Traffic dataset are used in the experiments, and the input sequence length is {48, 96, 192, 336, 512}, and the fixed prediction duration is 96 time steps. The experimental results are shown in Figure 4 , which show that the present application continuously outperforms all baseline models under all input sequence lengths of the two datasets. It is worth noting that although increasing the input length generally amplifies the signal-to-noise ratio, the performance of the present application decreases more gently than other models. This more smooth performance curve indicates that even in the case of a short input sequence (essentially a low noise level), the present application can effectively distinguish the signal from the noise, thereby significantly improving the overall prediction robustness.

[0200] As mentioned above, the hyperparameter is used to adjust the contribution of the regularization term (i.e. ) in the training loss, thereby ensuring the balance of the training target. The embodiments of the present application also test the influence of the hyperparameter on the training effect.

[0201] To evaluate the influence of the regularization coefficient in the training loss, the present application uses the ECL dataset, the Traffic dataset, and the Solar-Energy dataset to conduct a series of experiments. For each dataset, the input sequence length is set to 96, and the prediction duration is set to 720. By systematically adjusting the regularization coefficient in the range of {0.01, 0.1, 1.0, 2.0, 5.0, 10.0}, the influence of the regularization coefficient on the model performance is evaluated, and the experimental results are shown in Figure 5 .

[0202] As can be seen from Figure 5 , in the ECL dataset, the mean square error continuously decreases with the increase of the value of , which indicates that stronger regularization can effectively improve the effect of the decomposition process. In contrast, the mean square error of the Traffic dataset and the Solar-Energy dataset presents a non-monotonic change trend, and when Performance peaks when the value is in the middle range (around 1.0 or 2.0), but then declines as the value increases. This indicates that excessive regularization may suppress key temporal features, leading to information loss and reduced prediction accuracy. Experimental results show that the regularization coefficient needs to be adjusted according to the characteristics of each dataset. Fine-tuning is essential to achieve optimal performance. For example, in multivariate time series prediction tasks in the transportation field, during the training phase, adjustments can be made to... The design can be set to 1 or 2, but for multivariate time series forecasting tasks in the power sector, the training phase can... Designed for larger values ​​(e.g., 5~10).

[0203] In summary, this application fully extracts the features of the effective signal by decomposing the high-frequency signal and noise signal separately, while preserving the influence of both signal and noise on the prediction results, thus improving prediction accuracy. The mutual information-based decoupling method enhances noise resistance and captures the characteristics of short-term fluctuations and sudden events, further improving the matching degree between the prediction results and the real-world scenario. Compared to the current approach of directly discarding high-frequency components, this application significantly improves the prediction performance of multivariate time series by retaining high-frequency components and decomposing the high-frequency signal and noise for prediction.

[0204] It should be noted that the multivariate time series forecasting method or the multivariate time series forecasting model training method of this application can be applied to various multivariate time series forecasting fields, such as the transportation, weather, energy, power, and medical fields described in the previous embodiments. In specific applications, the acquired multivariate time series is composed of various variables in the corresponding scenario. For example, when applied to the weather field, it is a weather time series forecasting method. Multiple variable indicators include temperature, humidity, rainfall / precipitation, wind speed, and light intensity, etc. By collecting historical time series of these variables, a time series sample is constructed and input into the aforementioned multivariate time series forecasting model for training, thus obtaining the weather time series forecasting model. Using this weather time series forecasting model, future time steps (with pre-set duration) can be predicted for the input weather time series. For example, in the transportation field, this translates to traffic time series forecasting methods. Multiple variables, including traffic flow data, speed data, and density data, are used. Historical time series data of these variables are collected to construct time series samples, which are then input into the aforementioned multivariate time series forecasting model for training. This results in a traffic time series forecasting model, which can then be used to predict future time steps of the input traffic time series. The same principle applies to other fields.

[0205] The application is not restricted to the foregoing specific embodiments. The application extends to any novel one, or any novel combination, of the features disclosed in this specification, and to any novel method or process disclosed in this specification or any novel combination thereof.

Claims

1. A multivariate time series prediction method for weather prediction, the time series being synchronously collected by a plurality of variable indicators in time sequence, the plurality of variable indicators comprising temperature, humidity, rainfall / snowfall, wind strength and light intensity; characterized in that, The method comprises: acquiring a time sequence under a current scene; decomposing the acquired time sequence into high-frequency components and low-frequency components based on a preset frequency threshold; using a parameter-shared embedding layer to respectively vectorize the high-frequency components and the low-frequency components; extracting a low-frequency signal vector, a high-frequency signal vector and a noise signal vector from the vectorized high-frequency components and low-frequency components; adaptively fusing the low-frequency signal vector and the high-frequency signal vector to obtain a fused signal vector; using the low-frequency signal vector to enhance the attention of the fused signal vector on variable dependence to obtain an enhanced variable representation vector; respectively performing synchronous reasoning on the enhanced variable representation vector and the noise signal vector, and fusing the reasoning results to obtain a prediction result for a future time step.

2. The multivariate time series prediction method of claim 1, wherein, The method for extracting a low-frequency signal vector, a high-frequency signal vector and a noise signal vector from the vectorized high-frequency components and low-frequency components comprises: extracting the low-frequency signal vector from the vectorized low-frequency components and the high-frequency signal vector and the noise signal vector from the vectorized high-frequency components according to the principle of maximizing the mutual information of the low-frequency signal vector and the high-frequency signal vector and minimizing the mutual information of the low-frequency signal vector and the noise signal vector.

3. The multivariate time series prediction method of claim 2, wherein, The method for maximizing the mutual information of the low-frequency signal vector and the high-frequency signal vector and minimizing the mutual information of the low-frequency signal vector and the noise signal vector comprises: reconstructing the low-frequency signal vector using the high-frequency signal vector to calculate a first reconstruction loss; reconstructing the low-frequency signal vector using the noise signal vector to calculate a second reconstruction loss; combining the first reconstruction loss and the second reconstruction loss to obtain a total reconstruction loss; minimizing the total reconstruction loss.

4. The multivariate time series prediction method of claim 1, wherein, The method for adaptively fusing the low-frequency signal vector and the high-frequency signal vector to obtain a fused signal vector comprises: splicing the low-frequency signal vector and the high-frequency signal vector in the feature dimension to obtain a spliced vector; processing the spliced vector using a multilayer perception machine and adjusting using a temperature coefficient to activate the contribution degree of the low-frequency signal vector and calculate the contribution degree of the high-frequency signal vector; using the respective contribution degrees to perform weighted fusion on the low-frequency signal vector and the high-frequency signal vector to obtain the fused signal vector.

5. The multivariate time series prediction method of claim 1, wherein, The method for using the low-frequency signal vector to enhance the attention of the fused signal vector on variable dependence comprises: calculating the Gram matrix of the fused signal vector using the low-frequency signal vector and adaptively optimizing using a trainable parameter to obtain a similarity matrix; using the similarity matrix to weight the fused signal vector and perform residual connection with the fused signal vector, and performing post-processing using a feedforward neural network after linear transformation.

6. The multivariate time series prediction method of claim 5, wherein, The method for calculating the Gram matrix of the fused signal vector using the low-frequency signal vector and adaptively optimizing using a trainable parameter comprises: performing layer normalization on the low-frequency signal vector, and then multiplying the low-frequency signal vector subjected to layer normalization and transposition to obtain the Gram matrix; after activating the Gram matrix, performing element-by-element multiplication with a weight matrix and adding a bias vector; performing an activation operation on the calculation result to obtain the similarity matrix.

7. A multivariate time series prediction device for weather prediction, the time series being synchronously collected by a plurality of variable indicators in time sequence, the plurality of variable indicators comprising temperature, humidity, rainfall / precipitation, wind strength and light intensity; the device comprising a processor and a storage medium, the storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, performs the multivariate time series prediction method according to any one of claims 1-6.

8. A multivariate time series prediction device for weather prediction, the time series being synchronously collected by a plurality of variable indicators in time sequence, the plurality of variable indicators comprising temperature, humidity, rainfall / precipitation, wind strength and light intensity; characterized in that, The device is configured with a multivariate time series prediction model for predicting a future time step of a time series from an input current time series; The training method of the multivariate time series prediction model comprises: Obtaining a time series sample set, and dividing a training set and a test set; Training the multivariate time series prediction model by minimizing the training loss using the training set, and testing the trained multivariate time series prediction model using the test set; The multivariate time series prediction model is configured to: Obtain an input time series; Decompose the obtained time series into high-frequency components and low-frequency components based on a preset frequency threshold; Vector encode the high-frequency components and low-frequency components using a parameter-shared embedding layer; Extract low-frequency signal vectors, high-frequency signal vectors, and noise signal vectors from the encoded vectors of the high-frequency components and low-frequency components, respectively; Adaptively fuse the low-frequency signal vectors and high-frequency signal vectors to obtain a fused signal vector; Enhance the attention of the fused signal vector on variable dependence using the low-frequency signal vector to obtain an enhanced variable representation vector; Synchronously infer the enhanced variable representation vector and the noise signal vector, respectively, and fuse the inference results to obtain a prediction result for a future time step; The training loss is composed of the weighted combination of the mean square error (MSE) between the prediction result and the true result of all variables at each time step in the time series samples of the training set and the total reconstruction loss in the process of extracting low-frequency signal vectors, high-frequency signal vectors, and noise signal vectors.

9. A multivariate time series prediction system for weather prediction, the time series being synchronously collected by a plurality of variable indicators in time sequence, the plurality of variable indicators comprising temperature, humidity, rainfall / snowfall, wind strength and light intensity; characterized in that, The system comprises an input device, an output device, and a multivariate time series prediction device according to claim 8; the input device is connected to the input end of the multivariate time series prediction device for receiving the time series to be predicted and inputting to the multivariate time series prediction model; the output device is connected to the output end of the multivariate time series prediction device for outputting the time series predicted by the multivariate time series prediction model.

Images