Multivariate time series prediction method based on timestamp and multi-scale modeling

Abstract

The invention discloses a multivariate time sequence prediction method based on timestamps and multi-scale modeling, and the method comprises the steps: extracting features through multi-scale feature fusion, carrying out the convolution operation of an input time sequence through employing convolution kernels of different scales, extracting and splicing short, medium and long-term features, and carrying out the fusion through employing a multi-head attention mechanism. Therefore, local details and global trends in the time sequence are captured, and the modeling capability of the model on a complex time mode is improved; timestamps are coded into high-dimensional vectors, the high-dimensional vectors are mapped through an encoder, the complex relation between the timestamps is captured, and the prediction capacity of the model for periodic and tendency changes is enhanced; a self-defined loss function is adopted to carry out model training, so that the model robustness is improved, and meanwhile, relatively high prediction precision is kept; and applying the trained model weight to a test set. Through combination of multi-scale feature fusion, timestamp feature extraction and a self-defined loss function, the precision and robustness of multivariate time sequence prediction are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of multivariate time series prediction, and relates to a multivariate time series prediction method based on timestamp and multi-scale modeling. Background Art

[0002] Time series forecasting plays a vital role in numerous fields. In the energy sector, power load forecasting helps power companies rationally plan power generation, ensure stable power supply, and reduce operating costs. In the transportation sector, traffic flow forecasting can optimize traffic signal timing, alleviate congestion, and improve traffic efficiency. In the healthcare sector, predicting disease incidence facilitates the preemptive allocation of medical resources, improving the accessibility and quality of medical services. Multivariate time series forecasting builds on this foundation by comprehensively considering the interrelationships between multiple variables. For example, in meteorological forecasting, simultaneous analysis of the time series of factors such as temperature, air pressure, and humidity can provide more accurate weather forecasts.

[0003] However, current time series forecasting models suffer from several significant shortcomings. For one thing, timestamp information is underutilized. Timestamps contain rich information about seasonality and periodicity, which could provide global guidance for forecasting. However, the reality is that many models treat timestamps as merely a supplementary element, or even completely ignore them, as in models like DLinear. Some models, such as Informer and TimesNet, incorporate timestamps by combining them with location and data embeddings. However, this approach makes it difficult for the network to effectively extract timestamp information. Furthermore, the fusion of multi-scale information suffers from significant drawbacks. Time series data inherently exhibits multi-scale characteristics. For example, urban traffic flow, on short timescales, can experience dramatic hourly and even minute-by-minute fluctuations during peak hours. On longer timescales, traffic patterns differ significantly between weekdays and weekends, and between different seasons. However, existing models struggle to effectively integrate this multi-scale information. Even if some deep learning models can capture certain local dependencies, they lack effective mechanisms for fusing multi-scale features. For example, while recurrent neural networks (RNNs) excel at processing near-term correlations, they struggle to capture long-term information due to vanishing or exploding gradients. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a multivariate time series prediction method based on timestamps and multi-scale modeling to address the deficiencies of traditional methods in feature extraction, temporal information utilization and model robustness.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] The multivariate time series forecasting method based on timestamp and multi-scale modeling includes the following steps:

[0007] Step 1: Construct a multivariate time series prediction model structure based on timestamp and multi-scale modeling;

[0008] Step 1.1: Normalize the multivariate time series data using the mean and standard deviation, and embed the endogenous and exogenous variables in the multivariate time series data.

[0009] Step 1.2, extracting multi-scale features from the endogenous variables after data embedding in step 1.1, and generating endogenous variable features of multi-scale information;

[0010] Step 1.3: Input the endogenous variable features of the multi-scale information and the exogenous variables after data embedding into the encoder for processing to obtain the representation of the endogenous variable features and exogenous variable information that fused the multi-scale information;

[0011] Step 1.4: Map the encoder output of step 1.3 to the prediction length through a linear layer and perform denormalization to obtain the intermediate prediction result;

[0012] In step 1.5, after the historical and future time data of the multivariate time series data are labeled with timestamp attributes by the embedding layer, the dependency between the timestamp attributes is captured by the attention mechanism. The output of the attention mechanism is then mapped to the target space using the projection layer to obtain the initial historical mapping and initial future mapping.

[0013] Step 1.6, based on the quantile deviations of the initial historical mapping and the actual historical observations of the multivariate time series data within the historical window, the initial historical mapping and the initial future mapping are denormalized to generate the final historical mapping and the final future mapping;

[0014] In step 1.7, based on the difference between the final historical mapping and the actual historical observations of the multivariate time series data within the historical window, the combined weights of the final future mapping and the intermediate prediction results of step 1.4 are dynamically adjusted. The final prediction result of the training process, i.e., the value of the next t time steps, is generated through the multivariate time series prediction model structure.

[0015] Step 2: Supervise the model structure of step 1 through a custom loss function to balance the model's robustness to outliers and obtain the model and its weights trained on the dataset.

[0016] In step 3, the time series with a historical window length of T in the test data set is input into the model trained in step 2. The model processes the input multivariate time series and outputs the prediction results of the prediction window with a length of t.

[0017] The present invention also includes the following technical features:

[0018] Specifically, in step 1.1, the multivariate time series data is first normalized, and the original data is converted into a standard normal distribution using the mean and standard deviation of each variable in the data set to eliminate the dimensional differences between different variables; then, data embedding is performed on endogenous variables and exogenous variables respectively.

[0019] Specifically, in step 1.2, a parallel convolution operation is performed on the endogenous variables after the input data is embedded by using multiple convolution kernels of different scales to extract multi-scale features. After the extracted features are spliced, the global dependency between the features is captured through the self-attention mechanism, and finally a feature representation of the endogenous variables that integrates multi-scale information is generated.

[0020] Specifically, step 1.3 includes: the self-attention mechanism acts on the multi-scale features of the endogenous variables to capture the internal dependencies of the endogenous variables, and enhances the understanding of local and global patterns by calculating the correlation between each time step in the endogenous variables and other time steps; the cross-attention mechanism interacts the multi-scale features of the endogenous variables with the exogenous variable embeddings, and optimizes the feature representation using the auxiliary information provided by the exogenous variables; the optimized feature representation is nonlinearly transformed through a feedforward neural network to enhance the expressive power of the model; finally, the encoder layer outputs a representation that combines the multi-scale features and the exogenous variable information.

[0021] Specifically, in step 1.4, the representation of the fused multi-scale features and exogenous variable information obtained in step 1.3 is input into a linear transformation, and the representation in the high-dimensional feature space is projected from the high-dimensional space to a dimension matching the prediction length to generate a preliminary prediction sequence; the preliminary prediction sequence is then denormalized to restore the normalized data to the distribution of the original data, ensuring that the prediction results have the same dimension and distribution characteristics as the actual observed values.

[0022] Specifically, in step 1.5, the historical timestamps and future timestamps of the multivariate time series data are mapped to a high-dimensional space through the embedding layer and converted into a continuous vector representation. Then, the attention mechanism captures the dependencies between these timestamp attributes and dynamically calculates the correlation between different time steps. Finally, the output of the attention mechanism is mapped to the target space through the projection layer to generate an initial historical mapping and an initial future mapping.

[0023] Specifically, in step 1.6, the median and quantile range are used to enhance the robust denormalizer's resistance to outliers.

[0024] Specifically, in step 2, the following custom loss function is used for each variable of the multivariate time series to finally take the average value, and the formula is as follows:

[0025]

[0026] Among them, y represents the true value, represents the predicted value of the model, and δ represents the threshold parameter.

[0027] Compared with the prior art, the present invention has the following technical effects:

[0028] 1. The present invention defines multiple convolution kernels of different scales (sizes of 3, 5, and 7 respectively) to extract time series features in parallel, and combines the self-attention mechanism to capture global dependencies, thereby achieving efficient capture of short-term, medium-term, and long-term patterns in time series, enhancing the model's ability to model diverse time dependencies, and thus improving the accuracy and robustness of predictions.

[0029] 2. The present invention maps timestamp attributes (such as year, month, day, hour, minute, and second) to a high-dimensional space through an embedding layer, and uses the attention mechanism to capture the dependencies between timestamps, thereby effectively capturing temporal patterns such as periodicity and trends in time series, providing the model with rich contextual information, and enhancing the accuracy and interpretability of predictions.

[0030] 3. By customizing the loss function and combining the advantages of mean square error (MSE) and mean absolute error (MAE), the present invention can maintain high fitting accuracy for normal data while enhancing the robustness to outliers, reducing the interference of noise data on model training, and improving the generalization ability and stability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the overall process structure of the multivariate time series prediction method based on timestamp and multi-scale modeling.

[0032] Figure 2 Schematic diagram of the multi-scale feature extraction module.

[0033] Figure 3 This is a schematic diagram of the timestamp module. DETAILED DESCRIPTION

[0034] The present invention provides a multivariate time series prediction method based on timestamp and multi-scale modeling. First, multi-scale feature fusion and timestamp information are used for training. Among them, features are extracted by multi-scale feature fusion, and convolution kernels of different scales (sizes of 3, 5, and 7, respectively) are used to perform convolution operations on the input time series to extract short-term, medium-term, and long-term features. These features are spliced together and fused using a multi-head attention mechanism, thereby capturing local details and global trends in the time series, significantly improving the model's modeling ability for complex time patterns; in order to make full use of timestamp information, timestamps (such as year, month, day, hour, minute, and second, etc.) are encoded into high-dimensional vectors and mapped through an encoder to capture the complex relationship between timestamps and enhance the model's prediction ability for periodic and trend changes. Then, in response to the problem that traditional loss functions are sensitive to outliers, a custom loss function is used for model training. The loss function combines the advantages of mean square error (MSE) and mean absolute error (MAE), thereby improving the model robustness while maintaining high prediction accuracy during training. Finally, the model weights trained under the custom loss function are applied to the test set. By combining multi-scale feature fusion, timestamp feature extraction and custom loss function, the present invention significantly improves the accuracy and robustness of multivariate time series prediction.

[0035] The present invention specifically comprises the following steps:

[0036] Step 1: Build a multivariate time series prediction model structure based on timestamp and multi-scale modeling. Step 1 specifically includes:

[0037] Step 1.1: Normalize the multivariate time series data using the mean and standard deviation, and embed the endogenous and exogenous variables in the multivariate time series data.

[0038] Specifically, in step 1.1, in the multivariate time series prediction model, the multivariate time series data is first normalized. The original data is converted into a standard normal distribution using the mean and standard deviation of each variable in the dataset (such as the power load characteristics in the ETTh2 dataset) to eliminate the dimensional differences between different variables; then, data embedding is performed on the endogenous variables (target variables) and exogenous variables (auxiliary variables).

[0039] The normalized data is mapped to a high-dimensional space through the embedding layer, thereby capturing the potential characteristics and complex relationships of endogenous variables (target variables) and exogenous variables (auxiliary variables), providing high-quality input representation for subsequent multi-scale feature extraction and model processing.

[0040] Step 1.2, extract multi-scale features from the endogenous variables after data embedding in step 1.1, and generate feature representations of the endogenous variables that integrate multi-scale information;

[0041] Specifically, in step 1.2, multi-scale feature extraction is performed on the embedded endogenous variables to capture local and global patterns at different time scales in the time series, thereby enhancing the model's ability to model complex time dependencies. Figure 2 As shown in the figure, this step uses multiple convolution kernels of different scales (sizes of 3, 5, and 7) to perform parallel convolution operations on the input time series, i.e., the endogenous variables after data embedding, to extract multi-scale features. After the extracted features are spliced, the global dependencies between the features are further captured through the self-attention mechanism, and finally a feature representation of the endogenous variables that integrates multi-scale information is generated.

[0042] After concatenating these multi-scale features, the self-attention mechanism is used to capture global dependencies between features, thereby better modeling the complex dynamics of time series. The resulting feature representation not only incorporates local details but also incorporates global context, providing richer and more comprehensive input for subsequent encoding and decoding processes. This approach significantly improves the model's understanding of time series data, enabling it to more accurately predict future trends and changes.

[0043] Step 1.3: Input the endogenous variable features of the multi-scale information and the exogenous variables after data embedding into the encoder for processing to obtain the representation of the endogenous variable features and exogenous variable information that fused the multi-scale information;

[0044] Specifically, in step 1.3, the encoder layer embeds the multi-scale features of the endogenous variables obtained in step 1.2 and the exogenous variables obtained in step 1.1 into the input encoding layer to capture complex dependencies within the time series and generate high-quality representations. The core architecture of the encoder layer includes a self-attention mechanism, a criss-cross attention mechanism, and a feedforward neural network. First, the self-attention mechanism acts on the multi-scale features of the endogenous variables to capture internal dependencies. By calculating the correlation between each time step of the endogenous variables and other time steps, it enhances the understanding of local and global patterns. Next, the criss-cross attention mechanism interacts the multi-scale features of the endogenous variables with the embeddings of the exogenous variables, further optimizing the feature representation using auxiliary information provided by the exogenous variables. Finally, a feedforward neural network performs a nonlinear transformation on the optimized feature representation to enhance the model's expressive power. Throughout this process, normalization and dropout layers are used to stabilize the training process and prevent overfitting. Ultimately, the encoder layer outputs a representation that combines the multi-scale features with information from the exogenous variables, providing high-quality input for subsequent decoders or other modules.

[0045] This design significantly improves the model's ability to understand time series data. By capturing complex dependencies in time series, the encoding layer can more accurately model local and global patterns, thereby enhancing forecasting accuracy. Furthermore, by incorporating auxiliary information from exogenous variables, the encoding layer can better leverage external context, further improving the model's generalization and robustness. Ultimately, the high-quality feature representations generated by the encoding layer provide richer and more reliable input for subsequent modules, enabling the model to more effectively address diverse time series forecasting tasks.

[0046] Step 1.4: Map the encoder output of step 1.3 to the prediction length through a linear layer and perform denormalization to obtain the intermediate prediction result;

[0047] Specifically, in step 1.4, the representation of the fused multi-scale features and exogenous variable information obtained in step 1.3 is input into a linear transformation, directly mapping the representation in the high-dimensional feature space to the target prediction length. This converts the contextual information of the time series into predicted values for future time steps. Specifically, the linear layer learns a set of weight parameters to project the feature dimensions of the encoder output from the high-dimensional space to a dimension that matches the prediction length, thereby generating a preliminary prediction sequence. Next, the preliminary prediction sequence is denormalized to restore the normalized data to the distribution of the original data, ensuring that the prediction results have the same dimensionality and distribution characteristics as the actual observed values.

[0048] After linear mapping, intermediate prediction values for future time steps can be directly generated, simplifying the prediction process; at the same time, the denormalization process ensures that the prediction results have the same dimension and distribution characteristics as the actual observation values, making the output easier to understand and apply.

[0049] In step 1.5, after the historical and future time data of the multivariate time series data are labeled with timestamp attributes by the embedding layer, the dependency between the timestamp attributes is captured by the attention mechanism. The output of the attention mechanism is then mapped to the target space using the projection layer to obtain the initial historical mapping and initial future mapping.

[0050] Specifically, in step 1.5, as shown in the attached Figure 3 As shown, the embedding layer maps the historical and future timestamps of multivariate time series data (e.g., year, month, day, hour, minute, and second) into a high-dimensional space, converting them into continuous vector representations that better express the semantic information of temporal features. Next, an attention mechanism is used to capture the dependencies between these timestamp attributes, dynamically calculating correlations between different time steps and focusing on the temporal context that is most important for the current prediction. Finally, the projection layer maps the output of the attention mechanism into the target space, generating initial historical and future maps.

[0051] The combination of the embedding layer and the attention mechanism effectively captures key temporal patterns in time series, such as cyclical changes, long-term trends, and dynamic correlations between time points. This enables the model to more accurately understand temporal context and extract the temporal features most relevant to the prediction task. By generating high-quality initial representations, this module provides rich temporal information for subsequent prediction tasks, significantly enhancing the model's ability to model time-related features. This not only improves prediction accuracy but also enables the model to better adapt to complex temporal dependencies, resulting in greater robustness and generalization in multivariate time series prediction tasks.

[0052] Step 1.6, based on the initial historical mapping and the quantile deviation of the actual historical observations of the multivariate time series data within the historical window, the initial historical mapping and the initial future mapping are denormalized to generate the final mappings, namely the final historical mapping and the final future mapping;

[0053] Specifically, traditional denormalization processes typically consider distribution biases in the mean and standard deviation, but this approach is susceptible to extreme values and lacks robustness when the observations contain outliers. In step 1.6, denormalization does not rely on the mean and standard deviation, but instead uses the median and quantile range to enhance the robust denormalizer's resistance to outliers.

[0054] Step 1.6 of this invention is a crucial step in correcting model prediction bias and improving prediction accuracy. The initial historical mapping is the model's preliminary prediction of the data within the historical time window, while the actual historical observations provide information about the true data distribution. By calculating the quantile deviation between the two within the historical window, the model can quantify the systematic deviation between the prediction and the true value. Using this deviation information, the initial mapping is denormalized to adjust the prediction to a distribution closer to the true value, thus generating the final mapping.

[0055] By performing denormalization based on the quantile deviations of the initial historical mapping and actual historical observations, the model can effectively correct for systematic errors in the forecast results and reduce the model's forecast bias within the historical window. This correction mechanism not only enhances the stability and reliability of the forecast results, but also brings the model output closer to the distribution range of actual observations.

[0056] In step 1.7, based on the difference between the final historical mapping and the time observations within the historical window, i.e., the actual historical observations of the multivariate time series data, the combined weights of the final future mapping and the intermediate prediction results of step 1.4 are dynamically adjusted, and the final prediction result of the training process, i.e., the value of the next t time steps, is generated through the multivariate time series prediction model structure.

[0057] The final mapping reflects the corrected forecast distribution, while the time observations within the historical window provide real reference information. By calculating the difference between the final historical mapping and the historical true value, the model can dynamically evaluate the accuracy of the forecast and adjust the combined weight of the final future mapping and intermediate forecast results accordingly. Figure 3 As shown in Figure 3, the dynamic adjustment mechanism enables the model to flexibly combine global context information and local prediction results to produce the final result predicted during the training process, that is, the value of the next t time steps, generating a more accurate final prediction.

[0058] In step 2, the model structure constructed in step 1 is supervised and trained through a custom loss function to balance the robustness of the model to outliers, and obtain the model and its weights after training with the dataset.

[0059] Specifically, in step 2, the following custom loss function is used to finally take the average value for each variable in the multivariate time series. The formula is as follows:

[0060]

[0061] Among them, y represents the true value, Represents the predicted value of the model, δ represents the threshold parameter, and the loss combines the advantages of mean square error (MSE) and mean absolute error (MAE).

[0062] This property makes this loss function excellent when processing time series data, especially when there is noise or outliers in the data. By using a custom loss for supervised training, the model can maintain high prediction accuracy while enhancing robustness to outliers, thereby improving overall generalization and stability.

[0063] In step 3, the time series with a historical window length of T in the test dataset is input into the model trained in step 2. The model will process the input multivariate time series and output the prediction results of the prediction window with a length of t.

[0064] The present invention also provides a multivariate time series prediction device based on timestamp and multi-scale modeling, comprising:

[0065] The model building module is used to build a multivariate time series prediction model structure based on timestamp and multi-scale modeling; this module is specifically used to achieve: normalizing the multivariate time series data using the mean and standard deviation, and embedding the endogenous variables and exogenous variables in the multivariate time series data respectively; extracting multi-scale features from the endogenous variables after data embedding to generate endogenous variable features of multi-scale information; inputting the endogenous variable features of multi-scale information and the exogenous variables after data embedding into the encoder for processing to obtain the representation of the endogenous variable features and exogenous variable information that fuses the multi-scale information; mapping the encoder output to the prediction length through a linear layer, and performing denormalization to obtain the intermediate prediction result; historical time data and future After the incoming time data is marked with timestamp attributes by the embedding layer, the dependency between timestamp attributes is captured through the attention mechanism, and the output of the attention mechanism is mapped to the target space using the projection layer to obtain the initial historical mapping and the initial future mapping; based on the quantile deviation of the initial historical mapping and the actual historical observation values of the multivariate time series data within the historical window, the initial historical mapping and the initial future mapping are denormalized to produce the final historical mapping and the final future mapping; according to the difference between the final historical mapping and the actual historical observation values of the multivariate time series data within the historical window, the combined weights of the final future mapping and the intermediate prediction results are dynamically adjusted, and the final result predicted during the training process, that is, the value of the next t time steps, is generated through the multivariate time series prediction model structure.

[0066] The model training module is used to supervise the constructed model structure through a custom loss function, balance the model's robustness to outliers, and obtain the model and its weights after training with the data set;

[0067] The prediction module is used to input the time series with a historical window length of T in the test data set into the trained model. The model processes the input multivariate time series and outputs the prediction results of the prediction window with a length of t.

[0068] The present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the multivariate time series prediction method based on timestamp and multi-scale modeling are implemented.

[0069] The present invention also provides a computer-readable storage medium for storing program instructions, which can be executed by a processor to implement the steps of the above-mentioned multivariate time series prediction method based on timestamp and multi-scale modeling.

[0070] The computer program product can be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product.

[0071] Each functional unit can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a volatile or non-volatile computer-readable storage medium that can be executed by a processor. Based on this understanding, the essence of this solution or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0072] Specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent modifications made on the basis of the technical solution of this application fall within the protection scope of the present invention.

[0073] Example:

[0074] This embodiment provides a multivariate time series prediction method based on timestamp and multi-scale modeling. In the model construction process of this method: supervise the model training through a custom loss function. Through multiple iterative training, the model can gradually optimize and advance its prediction ability so that its prediction results are closer to the true value. Input a multivariate time series with a historical window length of T into the model. The model will process the input multivariate time series and output the prediction result of the prediction window with a length of t for the input multivariate time series. Figure 1 As shown, the method includes:

[0075] (1) The ETTh2 dataset is used to verify the accuracy of the model. The ETTh2 dataset contains sensor data from different power transformers, including seven features such as the date the data point was recorded, temperature, current, and voltage. The sampling rate is 1 hour. We divide the training set, validation set, and test set into a 6:2:2 ratio. The model is trained using the training set and validation set.

[0076] (2) Use the data from the training set to train the model. The training steps are as follows:

[0077] The multivariate time series data are normalized using mean and standard deviation, and data embedding is performed on endogenous and exogenous variables respectively;

[0078] Perform multi-scale feature extraction on endogenous variables;

[0079] Embed multi-scale features and exogenous variables into the input encoder for processing;

[0080] Mapping to the prediction length through a linear layer and performing denormalization to obtain the intermediate prediction result;

[0081] The timestamp attributes marked by the embedding layer are used to capture dependencies through the attention mechanism, and the projection layer is used to obtain the initial mapping;

[0082] Based on the quantile deviations between the initial historical mapping and the actual historical observations within the historical window, the initial mapping is denormalized to produce the final mapping;

[0083] Based on the difference between the final mapping and the time observations within the historical window, the combined weights of the global mapping and the intermediate prediction results are dynamically adjusted to produce the final prediction result;

[0084] (3) The model is validated using the validation set data and supervised training is performed using a custom loss function. The error value is recorded in real time and fed back to the model. Through multiple iterative training, the model can gradually optimize and improve its prediction ability so that its prediction results are closer to the true value.

[0085] (4) The data of the test set is used as the input of the trained model to obtain the training results of the multivariate time series.

[0086] The present invention uses mean square error (MSE) and mean absolute error (MAE) to judge the results of the multivariate time series prediction model based on timestamp and multi-scale modeling.

[0087] The mean squared error (MSE) is the average of the sum of the squares of the differences between the regression model's predicted values and the true values, and is calculated as:

[0088]

[0089] Where D represents the number of multivariate time series features, T represents the previous time of the multivariate time series to be predicted, t represents the length of the prediction window, and y represents the true value of the time step from T+1 to T+t. Represents the predicted value of the model at time steps T+1 to T+t.

[0090] The mean absolute error (MAE) is the average of the absolute values of the differences between the predicted values and the true values of the regression model. The calculation formula is:

[0091]

[0092] Where D represents the number of multivariate time series features, T represents the previous time of the multivariate time series to be predicted, t represents the length of the prediction window, and y represents the true value of the time step from T+1 to T+t. Represents the predicted value of the model at time steps T+1 to T+t.

[0093] The experimental results on the ETTh2 dataset are shown in Table 1, where 96, 192, 336, and 720 represent the size of the experimental prediction window. The experimental history window length is 96, and the number of experimental rounds is 10. The entire experiment was carried out on a server running the Ubuntu 20.04.6LTS operating system, with a processor of Intel(R)Xeon(R)Gold 6326CPU@2.90GHz. According to Table 1, the method of the present invention has good performance in multivariate time series analysis of time series data. The baseline selected TimeXer and PatchTST based on the Transformer model, TimesNet based on CNN, TIDE model based on RNN, and linear model Dlinear; the experimental results are shown in Table 1:

[0094] Table 1 Experimental results on the ETTh2 dataset

[0095]

[0096] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the scope of protection of the present invention.

[0097] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. To avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0098] In addition, the various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A multivariate time series prediction method based on timestamp and multi-scale modeling, characterized by: The following steps are involved: Step 1: Construct a multivariate time series prediction model structure based on timestamp and multi-scale modeling; Step 1.1: Normalize the multivariate time series data using the mean and standard deviation, and embed the endogenous and exogenous variables in the multivariate time series data. Step 1.2, extract multi-scale features from the endogenous variables after data embedding in step 1.1 to generate endogenous variable features of multi-scale information; Step 1.3: Input the endogenous variable features of the multi-scale information and the exogenous variables after data embedding into the encoder for processing to obtain the representation of the endogenous variable features and exogenous variable information that fused the multi-scale information; Step 1.4: Map the encoder output of step 1.3 to the prediction length through a linear layer and perform denormalization to obtain the intermediate prediction result; In step 1.5, after the historical and future time data of the multivariate time series data are labeled with timestamp attributes by the embedding layer, the dependency between the timestamp attributes is captured by the attention mechanism. The output of the attention mechanism is then mapped to the target space using the projection layer to obtain the initial historical mapping and initial future mapping. Step 1.6, based on the quantile deviations of the initial historical mapping and the actual historical observations of the multivariate time series data within the historical window, the initial historical mapping and the initial future mapping are denormalized to generate the final historical mapping and the final future mapping; In step 1.7, based on the difference between the final historical mapping and the actual historical observations of the multivariate time series data within the historical window, the combined weights of the final future mapping and the intermediate prediction results of step 1.4 are dynamically adjusted. The final prediction result of the training process, i.e., the value of the next t time steps, is generated through the multivariate time series prediction model structure. Step 2: Supervise the model structure of step 1 through a custom loss function to balance the model's robustness to outliers and obtain the model and its weights trained on the dataset. In step 3, the time series with a historical window length of T in the test data set is input into the model trained in step 2. The model processes the input multivariate time series and outputs the prediction results of the prediction window with a length of t.

2. The multivariate time series prediction method based on timestamp and multi-scale modeling according to claim 1, characterized in that: In step 1.1, the multivariate time series data is first normalized, and the original data is converted into a standard normal distribution using the mean and standard deviation of each variable in the data set to eliminate the dimensional differences between different variables; then, data embedding is performed on endogenous variables and exogenous variables respectively.

3. The multivariate time series prediction method based on timestamp and multi-scale modeling according to claim 1, characterized in that: In step 1.2, a parallel convolution operation is performed on the endogenous variables after the input data is embedded by using multiple convolution kernels of different scales to extract multi-scale features. After the extracted features are spliced, the global dependency between the features is captured through the self-attention mechanism, and finally a feature representation of the endogenous variables that integrates multi-scale information is generated.

4. The multivariate time series prediction method based on timestamp and multi-scale modeling according to claim 1, characterized in that: The step 1.3 includes: a self-attention mechanism acts on the multi-scale features of the endogenous variables to capture the internal dependencies of the endogenous variables, and enhances the understanding of local and global patterns by calculating the correlation between each time step in the endogenous variables and other time steps; a cross-attention mechanism interacts the multi-scale features of the endogenous variables with the exogenous variable embedding, and optimizes the feature representation using the auxiliary information provided by the exogenous variables; a feedforward neural network is used to perform a nonlinear transformation on the optimized feature representation to enhance the expressive power of the model; finally, the encoder layer outputs a representation that combines the multi-scale features and the exogenous variable information.

5. The multivariate time series prediction method based on timestamp and multi-scale modeling according to claim 1, characterized in that: In step 1.4, the representation of the fused multi-scale features and exogenous variable information obtained in step 1.3 is input into a linear transformation, and the representation in the high-dimensional feature space is projected from the high-dimensional space to a dimension matching the prediction length to generate a preliminary prediction sequence; the preliminary prediction sequence is then denormalized to restore the normalized data to the distribution of the original data, ensuring that the prediction results have the same dimension and distribution characteristics as the actual observed values.

6. The multivariate time series prediction method based on timestamp and multi-scale modeling according to claim 1, characterized in that: In step 1.5, the historical timestamps and future timestamps of the multivariate time series data are mapped to a high-dimensional space through an embedding layer and converted into a continuous vector representation. Then, the attention mechanism captures the dependencies between these timestamp attributes and dynamically calculates the correlation between different time steps. Finally, the output of the attention mechanism is mapped to the target space through a projection layer to generate an initial historical mapping and an initial future mapping.

7. The multivariate time series prediction method based on timestamp and multi-scale modeling according to claim 1, characterized in that: In step 1.6, the median and quantile range are used to enhance the robust denormalizer's resistance to outliers.

8. The multivariate time series prediction method based on timestamp and multi-scale modeling according to claim 1, characterized in that: In step 2, the following custom loss function is used to finally take the average value for each variable in the multivariate time series. The formula is as follows: Among them, y represents the true value, represents the predicted value of the model, and δ represents the threshold parameter.

Images