A time series prediction method based on adaptive segmentation and cross-variable attention
By constructing a Transformer model with adaptive segmentation embedding and cross-variable attention, the problems of periodic feature matching and variable dependency modeling in time series forecasting are solved, and more efficient time series forecasting performance is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG NORMAL UNIV
- Filing Date
- 2026-03-11
- Publication Date
- 2026-06-26
AI Technical Summary
Existing Transformer-based time series forecasting models struggle to adaptively match the periodic characteristics of time series and effectively model the dependencies between variables and time steps, leading to a decline in forecasting performance.
We construct a Transformer model that integrates adaptive segmentation embedding and cross-variable attention mechanisms. Through the adaptive segmentation embedding module and the cross-variable attention layer, we achieve effective fusion of local and global information of time series and accurate modeling of dependencies between variables and between time steps.
It significantly improves the accuracy and generalization ability of time series forecasting, and is applicable to time series forecasting tasks in multiple fields such as power consumption and meteorological monitoring.
Smart Images
Figure CN121808295B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of time series forecasting and data mining technology, and more specifically to a time series forecasting method based on adaptive segmentation and cross-variable attention. Background Technology
[0002] Time series forecasting, as one of the core tasks in the field of data mining, aims to predict future trends by modeling historical time series data. In recent years, the Transformer model, with its powerful attention mechanism, has made breakthroughs in fields such as natural language processing and computer vision, and has subsequently been widely applied to time series forecasting tasks.
[0003] However, existing Transformer-based time series forecasting models have many shortcomings: some models use a fixed-length segmentation strategy to divide the time series, which can reduce computational complexity, but it is easy to destroy the periodic characteristics of the time series and it is difficult to balance the capture of local fine-grained information and global overall information; some models use a channel-independent strategy, ignoring the potential dependencies between variables, which often have practical physical or business significance, such as the correlation between temperature and humidity in meteorological data, or the correlation between different users' consumption in electricity data; other models attempt to model the dependencies between variables through complex architectures, but introduce a lot of interactive noise, leading to a decrease in forecast performance.
[0004] Therefore, how to design a segmentation method that can adaptively match the periodic characteristics of time series, while efficiently modeling the dependencies between variables and time steps, and improving the performance of time series prediction, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of the above problems, this invention is proposed to provide a time series prediction method based on adaptive segmentation and cross-variable attention to overcome or at least partially solve the above problems. It constructs a Transformer model that integrates adaptive segmentation embedding and cross-variable attention mechanisms. By designing an adaptive segmentation embedding module and a cross-variable attention layer, it achieves effective fusion of local and global information of the time series and accurate modeling of dependencies between variables and between time steps, thereby significantly improving prediction accuracy.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] In a first aspect, embodiments of the present invention provide a time-series prediction method based on adaptive segmentation and cross-variable attention, comprising the following steps:
[0008] Step 1: Obtain the raw data consisting of time series data and perform normalization processing to construct the training dataset;
[0009] Step 2: Construct a Transformer model that integrates adaptive segmentation embedding and cross-variable attention mechanisms, and train it using the training dataset to obtain a time series prediction model. The time series prediction model adopts an encoder architecture, including an adaptive segmentation embedding module, L sets of cross blocks, and a prediction head stacked sequentially, where L is a positive integer greater than or equal to 1. The adaptive segmentation embedding module performs adaptive segmentation and embedding labeling on the data to generate an embedding label set. The L sets of cross blocks perform L layers of temporal attention and cross-variable attention iterative reinforcement processing on the embedding label set to generate the final feature representation. The prediction head predicts and generates future time series based on the final feature representation. Each set of cross blocks includes a cross attention module, a feedforward network layer, and two sets of normalization layers.
[0010] Step 3: Normalize the time series data to be predicted and input it into the time series prediction model to obtain the future time series sequence;
[0011] Step 4: Perform inverse normalization on the future time series to restore it to the scale of the original data, obtain the prediction results, and complete the time series prediction task.
[0012] Preferably, the original data contains S time steps and N variables at each time step, and the data format is as follows: Where S and N are positive integers greater than or equal to 1. This represents the time series of all variables at the S-th time step; the original data is normalized to eliminate differences in units, and then divided into training, validation, and test sets according to a preset ratio to construct the training dataset.
[0013] Preferably, the specific processing procedure of the adaptive segmentation embedding module is as follows:
[0014] Step 211: Perform a Fourier transform on the time series of each variable in the training dataset to obtain the frequency domain vector;
[0015] Step 212: Select the frequencies with the highest and second highest amplitudes in the frequency domain vector, and calculate the greatest common divisor. Use the reciprocal of the greatest common divisor as the segment period.
[0016] Step 213: Based on the segmented period, use tensor reshaping operations to adaptively segment the time series of each variable at different time steps to obtain several subsequences; ensure that complete periodic information is retained within each segment;
[0017] Step 214: Apply a fully connected layer to perform a linear transformation embedding operation on all subsequences to generate embedding tags of dimension D, thus obtaining the embedding tag set. ;
[0018] Step 215: Use the Dropout layer to apply random deactivation to the embedded tags in the embedded tag set to achieve regularization and reduce the risk of overfitting.
[0019] Preferably, the cross-attention module includes a cross-time attention layer and a cross-variable attention layer stacked sequentially;
[0020] A cross-temporal attention layer performs multi-head self-attention computation on the embedded tags in the embedded tag set to capture the dependencies between time steps. The result is then processed sequentially through the first residual connection, the first layer normalization, a multi-layer feedforward network, the second residual connection, and the second layer normalization to output the temporal attention weight matrix. The expression is:
[0021] ;
[0022]
[0023] A cross-variable attention layer introduces a learnable relay vector. c represents the fixed length of the relay vector. Indicates the embedding dimension; Represents the time attention weight matrix. This represents the attention weight value of the corresponding variable. This represents the embedding tag of the i-th variable in the embedding tag set, using the relay vector as the query vector, and the weight value corresponding to the subsequence of the j-th segment in the temporal attention weight matrix. Attention is computed using key and value vectors, aggregating the dependency information of all variables to obtain an aggregated representation. ;by As a query vector, the aggregation representation Attention is computed using key and value vectors, and the aggregated representation is assigned to each variable to obtain the aggregated vector. The cross-attention weight matrix is then processed sequentially through the first residual connection, the first layer normalization, the multi-layer feedforward network, the second residual connection, and the second layer normalization. The multilayer feedforward network employs a multilayer perceptron (MLP); the expression is:
[0024] ;
[0025] ;
[0026] ;
[0027] ;
[0028] in, This represents the weight value corresponding to the subsequence of the j-th segment in the temporal attention weight matrix; This represents the first attention calculation in the cross-variable attention phase; This indicates the second attention calculation in the intervariate attention phase; This indicates multi-head self-attention computation; Representation layer normalization; Indicates residual connection; This represents a multilayer feedforward network.
[0029] Preferably, the time complexity of attention computation in the cross-attention module is optimized to... Where D is the embedding label dimension and c represents the fixed length of the relay vector. Time complexity is used to measure the time spent by the module in computation. The cross-variable attention module structure set in this invention effectively reduces the time complexity.
[0030] Preferably, the prediction head represents the final feature. Perform a projection operation, mapping to a length of Future time series The expression for the prediction head is:
[0031] ;
[0032] in, Represents the final feature representation; Projection represents the projection operation; This represents the future time series of the i-th variable.
[0033] Preferably, the loss function L2 is used as the optimization objective during model training; the ADAM optimizer is used to train the model parameters, and an early stopping strategy is employed to prevent overfitting. The loss function L2 is the mean squared error (MSE) loss function.
[0034] Preferably, the model's hyperparameters include the learning rate, embedding dimension, and number of cross blocks, with the initial learning rate set to a range of [value missing]. Embedded Dimensions Number of cross blocks .
[0035] Preferably, the hyperparameters are adjusted using a validation set until the model's performance on the validation set stabilizes, thus obtaining a trained time-series prediction model.
[0036] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a time series prediction method based on adaptive segmentation and cross-variable attention. It constructs a Transformer model that integrates adaptive segmentation embedding and cross-variable attention mechanism. Through the synergistic effect of adaptive segmentation embedding and cross-variable attention mechanism, it achieves accurate time series prediction, effectively balances the capture of local and global information of time series, accurately models the dependencies between variables and time steps, optimizes the performance of time series prediction, and significantly improves the accuracy and generalization ability of time series prediction in multiple scenarios. It is applicable to time series prediction tasks in multiple fields such as power consumption, meteorological monitoring, and exchange rate fluctuations. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0038] Figure 1 This is a schematic diagram of the ASformer model structure provided in an embodiment of the present invention;
[0039] Figure 2 This is a comparison chart of the visualization results of the ASformer model and the iTransformer model provided in the embodiments of the present invention;
[0040] Figure 3 This is a parameter sensitivity analysis diagram provided in an embodiment of the present invention. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] This invention discloses a time-series prediction method based on adaptive segmentation and cross-variable attention, comprising the following steps:
[0043] S1: Obtain the raw data consisting of time series data and perform normalization processing to construct the training dataset;
[0044] S2: Construct a Transformer model that integrates adaptive segmentation embedding and cross-variable attention mechanisms, and train it using the training dataset to obtain a time-series prediction model; such as Figure 1 As shown, the time series prediction model adopts an encoder architecture, which includes an adaptive segmentation embedding module, L sets of cross blocks, and a prediction head stacked sequentially, where L is a positive integer greater than or equal to 1. Figure 1 In (a), the adaptive segmentation and embedding module performs adaptive segmentation and embedding label generation on the data to generate an embedding label set. The L groups of cross-blocks perform L layers of temporal attention and cross-variable attention iterative reinforcement processing on the embedding label set to generate the final feature representation. The prediction head predicts and generates future time series based on the final feature representation. Each cross-block includes a cross-attention module, a feedforward network layer and two sets of normalization layers.
[0045] S3: Normalize the time series data to be predicted and input it into the time series prediction model to obtain the future time series sequence;
[0046] S4: Perform inverse normalization on the future time series to restore it to the scale of the original data, obtain the prediction results, and complete the time series prediction task.
[0047] Furthermore, the original data contains S time steps and N variables at each time step, with the data format as follows: Where S and N are positive integers greater than or equal to 1. This represents the time series of all variables at the S-th time step. This represents the time series of different variables at time step s. Let i represent the time series of the i-th variable at different time steps. The original data is normalized to eliminate differences in units, and the dataset is divided into training set, validation set and test set according to a preset ratio to construct the training dataset.
[0048] Furthermore, during model training, the loss function L2 is used as the optimization objective; the ADAM optimizer is employed to train the model parameters, and an early stopping strategy is used to prevent overfitting. The loss function L2 is the mean squared error (MSE) loss function.
[0049] Furthermore, the model's hyperparameters include the learning rate, embedding dimension, and number of cross blocks, with the initial learning rate set to a range of [value missing]. Embedded Dimensions Number of cross blocks .
[0050] Furthermore, the hyperparameters are adjusted using the validation set until the model's performance on the validation set stabilizes, resulting in a successfully trained time-series prediction model.
[0051] The model constructed in this invention adopts an encoder-only architecture, which is lightweight and avoids the interactive noise introduced by complex architectures. Its prediction accuracy and generalization ability are significantly better than existing Transformer, linear and TCN models. In addition, the model has good stability to changes in input length. Even if the input sequence continues to lengthen, it can still maintain stable prediction performance. It is suitable for long time series prediction scenarios. Moreover, the model has low hyperparameter sensitivity and can maintain stable performance under different prediction lengths, and has high deployment flexibility.
[0052] In one specific embodiment, the adaptive segmentation embedding module performs the following process:
[0053] S211: Time series of each variable in the training dataset Perform a Fourier transform to obtain the frequency domain vector;
[0054] S212: Select the frequencies with the highest and second highest amplitudes in the frequency domain vector, calculate the greatest common divisor, and use the reciprocal of the greatest common divisor as the segment period;
[0055] S213: Based on the segmented period, tensor reshaping operation is used to adaptively segment the time series of each variable at different time steps to obtain several subsequences; ensuring that complete periodic information is retained in each segment;
[0056] S214: A fully connected layer is used to perform a linear transformation embedding operation on all subsequences to generate embedding tags of dimension D, thus obtaining a set of embedding tags. ;
[0057] S215: A Dropout layer is used to apply random deactivation to the embedded tags in the embedded tag set to achieve regularization and reduce the risk of overfitting.
[0058] The adaptive segmentation and embedding module adaptively obtains the time series period and segments it through Fourier transform, avoiding the destruction of periodic information by fixed-length segments, effectively balancing the capture of local fine-grained information and global information, and improving the model's ability to extract time series features.
[0059] In one specific embodiment, the iterative processing of L sets of cross blocks achieves deep feature extraction and gradual strengthening of dependencies. The cross attention module in the cross block includes a cross-time attention layer and a cross-variable attention layer stacked sequentially. Figure 1 In section (b), the temporal attention layer performs multi-head self-attention computation on the embedded tags in the embedded tag set to capture the dependencies between time steps. The layer then sequentially passes through the first residual connection, the first layer normalization, a multi-layer feedforward network, the second residual connection, and the second layer normalization to output the temporal attention weight matrix. The expression is:
[0060] ;
[0061]
[0062] in, This represents the embedding tag of the i-th variable in the embedding tag set; This indicates multi-head self-attention computation; Representation layer normalization; Indicates residual connection; This represents a multilayer feedforward network.
[0063] Figure 1 In section (c), a cross-variable attention layer is introduced, which incorporates a learnable relay vector. c represents the fixed length of the relay vector. Indicates the embedding dimension; Represents the time attention weight matrix. This represents the attention weight value of the corresponding variable. This represents the embedding tag of the i-th variable in the embedding tag set, using the relay vector as the query vector, and the weight value corresponding to the subsequence of the j-th segment in the temporal attention weight matrix. Attention is computed using key and value vectors, aggregating the dependency information of all variables to obtain an aggregated representation. ;by As a query vector, the aggregation representation Attention is computed using key and value vectors, and the aggregated representation is assigned to each variable to obtain the aggregated vector. The process involves sequentially passing through the first residual connection, the first layer normalization, a multi-layer feedforward network, the second residual connection, and the second layer normalization to output the intervariate attention weight matrix. The multilayer feedforward network uses a multilayer perceptron (MLP); the expression is:
[0064] ;
[0065] ;
[0066] ;
[0067] ;
[0068] in, This represents the weight value corresponding to the subsequence of the j-th segment in the temporal attention weight matrix; This represents the first attention calculation in the cross-variable attention phase; This indicates the second attention calculation in the intervariate attention phase; Representation layer normalization; Indicates residual connection; This represents a multilayer feedforward network.
[0069] Furthermore, the time complexity of attention computation in the cross-attention module is optimized to... Where D is the embedding label dimension and c represents the fixed length of the relay vector. Time complexity is used to measure the computation time of the module. The cross-variable attention module structure set in this invention effectively reduces the time complexity. The cross-attention module adopts a two-stage attention mechanism, which not only accurately captures the dependencies between time steps, but also efficiently models the dependencies between variables through the relay vector, while reducing the computational complexity from Reduce to This improved model efficiency while ensuring modeling effectiveness.
[0070] In one specific embodiment, the prediction head pairs the final feature representation. Perform a projection operation, mapping to a length of Future time series The expression for the prediction head is:
[0071] ;
[0072] in, Represents the final feature representation; Projection represents the projection operation; This represents the future time series of the i-th variable.
[0073] On the other hand, using the electricity consumption dataset ECL as the prediction object, the time-series prediction method based on adaptive segmentation and cross-variable attention of this invention is used for prediction. In this embodiment, the dataset contains hourly electricity consumption data of 321 users. The specific implementation steps are as follows:
[0074] S1: Data preprocessing; Obtain the raw data of the ECL dataset, which contains S time steps and N variables, S=96, N=321, and the data format is as follows. The data is mapped to the [0, 1] interval using the Min-Max normalization method; it is divided into training set, validation set and test set in a ratio of 8:1:1, and the prediction length τ is set to {96, 192, 336, 720}.
[0075] S2: Input the training set into the constructed Transformer model (ASformer model) that integrates adaptive segmentation embedding and cross-variable attention mechanism for training. Set the loss function to L2, using the mean squared error loss function, and the initial learning rate of the ADAM optimizer is... The training batch size is 32, and the number of training rounds is 100.
[0076] S21: The Adaptive Segmentation and Embedding Module (ASE module) performs adaptive segmentation and embedding labeling on the time series data in the training set; for the time series... Perform a Fourier transform to convert the sequence into a frequency domain vector. Calculate the greatest common divisor of the two most significant frequencies in the frequency domain to obtain the period of each sequence. Segment the sequence according to the period. Embed each segment using a fully connected layer, setting the embedding dimension D=256, and generate an embedding tag set. ;
[0077] S22: The L-group cross blocks perform L-layer iterative enhancement processing on the embedded label set to generate the final feature representation. Each cross block includes a cross attention module (CAL module), a feedforward network, and two sets of normalization layers.
[0078] The cross-attention module performs iterative reinforcement processing on the embedded tag set using temporal and cross-variable attention; the cross-temporal attention layer across time stages performs multi-head self-attention computation on the embedded tags in the embedded tag set H, and after layer normalization and processing by a multi-layer feedforward network using MLP, outputs... The transvariate attention layer in the transvariate stage first performs attention computation. Aggregate variable dependencies, then compute through attention. The allocation dependency information is processed by layer normalization and a multi-layer feedforward network using MLP before being output. ;
[0079] After processing through all the cross blocks in sequence, the final feature representation is output. ;
[0080] S23: The prediction head uses a fully connected layer to represent the final features. The projection is a sequence of the corresponding predicted length τ. For example, when τ=96, the output is... ;
[0081] S3: After every 10 rounds of training, the model's performance on the validation set is verified. An early stopping strategy is used to prevent overfitting, resulting in a completed time-series prediction model.
[0082] S4: Input the preprocessed test set data into the model, output the prediction sequence, perform inverse Min-Max normalization on the prediction sequence, and obtain the original scale power consumption prediction results.
[0083] This embodiment was tested on an A100 graphics card. The model was implemented using the PyTorch framework and compared with a benchmark model to verify the effectiveness of the invention. The benchmark models included seven mainstream time series prediction models: FEDformer, Crossformer, PatchTST, iTransformer, Dlinear, TiDE, and TimesNet.
[0084] The experimental results are shown in Table 1 below, where ETT(Avg) is the power transformer temperature dataset and ECL is the power load dataset. The ASformer model proposed in this invention has a mean squared error (MSE) of 0.173 and a mean absolute error (MAE) of 0.265 on the ECL dataset, both of which are better than all the benchmark models. On four datasets, including the ETT series (ETTh1, ETTh2, ETTm1, ETTm2), exchange rate, and weather datasets, the model has excellent average performance and achieves the best results in 6 out of 8 comparison scenarios.
[0085] Table 1. Indicator data for each model on different datasets.
[0086]
[0087] Ablation experiments were conducted on the dataset with input lengths of 96, 192, 336, and 720. The results are shown in Table 2 below. After removing the ASE module, the average MSE of the model (ASformer-ASE) increased to 0.179. After removing the CAL module, the average MSE of the model (ASformer-CAL) increased to 0.184. After removing both modules, the average MSE of the model (ASformer-ASE-CAL) reached as high as 0.187, demonstrating that the ASE and CAL modules play a key role in improving model performance. Compared with the fixed segmentation strategy, the adaptive segmentation strategy reduced the model MSE by about 2.3%, effectively preserving the periodic characteristics of the time series.
[0088] Table 2 Ablation Experiment Results
[0089]
[0090] like Figure 2 The image shows a comparison of the visualization results of the ASformer model and the iTransformer model. The horizontal axis represents the time step, and the vertical axis represents the numerical value. Figure 2 In the middle, (a) and (b) represent the visualization results of ASformer and iTransformer on the ECL dataset with an input length of 96 and a prediction length of 192, respectively; Figure 2In the middle, (c) and (d) represent the visualization results of ASformer and iTransformer on the Exchange dataset with an input length of 192 and a prediction length of 96, respectively; Figure 2 In the middle, (e) and (f) represent the visualization results of ASformer and iTransformer predicting a length of 96 from an input of 96 on the Weather dataset, respectively; Figure 2 In the diagram, (g) and (h) represent the visualization results of ASformer and iTransformer predicting a length of 96 from an input of length 96 on the Weather dataset, respectively. Adaptive segmentation embedding significantly improves the ability to extract fine-grained information, such as... Figure 2 As shown in (a) and (b), focusing on the areas marked with red boxes in ASformer, these local areas are more detailed and better match the actual results. The adaptive segmentation embedding also exhibits strong robustness. Figure 2 Figures (c) and (d) further demonstrate that ASformer still achieves good results on datasets with weak periodicity, and its accuracy is higher than that of iTransformer. Figure 2 As shown in (e)-(h), while iTransformer can reflect trends, it falls short in handling local details. This is because iTransformer's embedding method is relatively coarse and cannot achieve a balance between capturing local and global information, while ASformer represents certain finer information, such as fluctuations and changes in local regions.
[0091] Figure 3 The sensitivity analysis diagram of the model parameters is shown below. Figure 3 In the middle (a), the results of the sensitivity analysis of the relay vector are shown. The horizontal axis represents the length of the relay vector, and the vertical axis represents the magnitude of the MSE. Figure 3 In the middle (b), the results of the sensitivity analysis of the learning rate are shown. The horizontal axis represents the learning rate and the vertical axis represents the MSE. Figure 3 In the middle (c), the sensitivity analysis results for input length are shown, with the horizontal axis representing the input length and the vertical axis representing the MSE magnitude. Visualization analysis indicates that the ASformer model performs better in capturing local fine-grained features, and the predicted results show higher consistency with the local fluctuations of the true values. The model exhibits good stability to changes in input length; when the input length increases from 48 to 720, the MSE fluctuation is less than 5%, significantly outperforming models such as Crossformer and FEDformer.
[0092] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0093] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A time-series prediction method based on adaptive segmentation and cross-variable attention, characterized in that, Includes the following steps: Step 1: Obtain the raw data of the electricity consumption ECL dataset and perform normalization processing to construct the training dataset; Step 2: Construct a Transformer model that integrates adaptive segmentation embedding and cross-variable attention mechanisms, and train it using the training dataset to obtain a time series prediction model. The time series prediction model adopts an encoder architecture, including an adaptive segmentation embedding module, L sets of cross blocks, and a prediction head stacked sequentially, where L is a positive integer greater than or equal to 1. The adaptive segmentation embedding module performs adaptive segmentation and embedding labeling on the data to generate an embedding label set. The L sets of cross blocks perform L layers of temporal attention and cross-variable attention iterative reinforcement processing on the embedding label set to generate the final feature representation. The prediction head predicts and generates future time series based on the final feature representation. Each set of cross blocks includes a cross attention module, a feedforward network layer, and two sets of normalization layers. Step 3: Normalize the time series data to be predicted and input it into the time series prediction model to obtain the future time series sequence; Step 4: Perform inverse normalization on the future time series to restore it to the scale of the original data, obtain the power consumption prediction results, and complete the time series prediction task; The specific processing procedure of the adaptive segmentation embedding module is as follows: Step 211: Perform a Fourier transform on the time series of each variable in the training dataset to obtain the frequency domain vector; Step 212: Select the frequencies with the highest and second highest amplitudes in the frequency domain vector, and calculate the greatest common divisor. Use the reciprocal of the greatest common divisor as the segment period. Step 213: Based on the segmentation period, use tensor reshaping operation to adaptively segment the time series of each variable at different time steps to obtain several subsequences; Step 214: Use a fully connected layer to perform embedding operations on all subsequences to generate embedding tags, and obtain an embedding tag set; Step 215: Apply random deactivation to the embedded tags in the embedded tag set using the Dropout layer to achieve regularization; The cross-attention module consists of a time-series attention layer and a cross-variable attention layer stacked sequentially; The cross-temporal attention layer performs multi-head self-attention computation on the embedded tags in the embedded tag set, captures the dependencies between time steps, and outputs the temporal attention weight matrix after layer normalization and multi-layer feedforward network processing. The cross-variable attention layer uses the relay vector as the query vector and the temporal attention weight matrix as the key and value vectors for attention calculation, aggregating the dependency information of all variables to obtain the aggregated representation; it uses the temporal attention weight matrix as the query vector and the aggregated representation as the key and value vectors for attention calculation, assigns the aggregated representation to each variable, and outputs the cross-attention weight matrix after layer normalization and multi-layer feedforward network processing.
2. The time series prediction method based on adaptive segmentation and cross-variable attention as described in claim 1, characterized in that, The original data contains S time steps and N variables at each time step, and the data format is as follows: Where S and N are positive integers greater than or equal to 1. This represents the time series of all variables at the S-th time step.
3. The time series prediction method based on adaptive segmentation and cross-variable attention as described in claim 1, characterized in that, The prediction head projects the final feature representation, mapping it to a length of... Future time series The expression for the prediction head is: ； in, Represents the final feature representation; Projection represents the projection operation; This represents the future time series of the i-th variable.
4. The time series prediction method based on adaptive segmentation and cross-variable attention as described in claim 1, characterized in that, During model training, the loss function L2 is used as the optimization objective; the ADAM optimizer is used to train the model parameters.