Time sequence enhanced Transform time sequence prediction method and system
By introducing a sequence decomposition module and a loss constraint module in the Transformer time series prediction model, the time series data is decomposed into the trend term and periodic term, and modeled through linear model and Transformer, the problem of difficult to take into account in the existing technology is solved, and more efficient time series prediction is achieved.
Patent Information
- Application Number
- CN202510256757.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-05
- Publication Date
- 2025-06-24
AI Technical Summary
In the prior art, when dealing with complex patterns and long-term dependencies, prediction accuracy is difficult to meet actual needs, and when processing large-scale industrial data, it is difficult to take into account both prediction accuracy and computing efficiency.
A time-series prediction method for transformer with temporal enhancement is proposed. The time series data is decomposed into trend terms and periodic terms through the sequence decomposition module, the trend terms are modeled using a linear model and the temporal information is retained. The periodic term is modeled using Transformer, and the period-trend prediction pair is generated in combination with the loss constraint module, and the prediction results are dynamically adjusted through an adaptive mechanism.
It effectively enhances the chronological information retention ability of Transformer, improves the model's prediction performance for future sequences, improves prediction accuracy and takes into account the computing efficiency.
Smart Images
Figure CN120197649A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical fields of machine learning, deep learning, and time series prediction, and particularly relates to a time-order enhanced Transformer time series prediction method and system. Background Art
[0002] Time series prediction is an important analysis and prediction method, widely used in fields such as finance, economy, healthcare, and environmental monitoring. Traditional time series prediction methods mainly include autoregressive model (AR), moving average model (MA), autoregressive moving average model (ARMA), autoregressive integrated moving average model (ARIMA), and seasonal autoregressive integrated moving average model (SARIMA). These methods perform well in dealing with linear and seasonal data, can handle non-stationary time series through differencing operations, and can capture the periodic characteristics of time series to a certain extent. With the development of deep learning technology, new methods such as recurrent neural network (RNN), long short-term memory network (LSTM), gated recurrent unit (GRU), convolutional neural network (CNN), and Transformer have gradually been applied to time series prediction. Among them, the Transformer model, through its architecture based on the attention mechanism, gets rid of the limitations of traditional RNNs, can handle longer time dependencies, and has higher parallel computing efficiency, and has gradually been widely used in time series prediction.
[0003] However, there are still some limitations in the existing technology when dealing with complex patterns and long-time dependencies. Traditional time series prediction methods often have difficulty meeting the actual requirements of prediction accuracy when facing non-linear relationships and complex dynamic systems. For example, autoregressive models and moving average models are only applicable to linear relationships, while ARIMA and SARIMA models have limited effects when dealing with long-time dependencies and high-order seasonality. Although deep learning methods such as RNN and LSTM can capture the dynamic characteristics of time series, they are easily affected by the problems of gradient vanishing or gradient explosion, resulting in difficult model training. Although the Transformer has advantages in dealing with long sequences, in time series prediction, its ability to retain time-order information is insufficient, which may lead to a decrease in prediction accuracy. In addition, when dealing with large-scale industrial data, existing methods often have difficulty balancing prediction accuracy and computational efficiency at the same time, restricting their application in actual scenarios. Summary of the Invention
[0004] To solve the above technical problems, the present invention proposes a time-order enhanced Transformer time series prediction method and system to solve the problems existing in the above prior art.
[0005] To achieve the above object, in a first aspect, the present invention provides a Transformer time series prediction method with enhanced time sequence, including:
[0006] Obtain various time series data in industrial equipment and store them in a database;
[0007] Preprocess the time series data to obtain the original sequence;
[0008] According to the Transformer framework, construct a time series prediction model, which includes a sequence decomposition module, a linear model, a cross-attention mechanism, and a loss constraint module;
[0009] Decompose the original sequence through the sequence decomposition module to obtain a trend term part and a periodic term part;
[0010] Model the trend term part through the linear model to obtain a first prediction result and retain the time sequence information in the original sequence;
[0011] Model the periodic term part through the cross-attention mechanism to obtain a second prediction result;
[0012] Generate a period-trend prediction pair through the loss constraint module; dynamically adjust the first prediction result and the second prediction result through an adaptive mechanism to obtain an optimal combination of the period-trend prediction pair;
[0013] According to the optimal combination of the period-trend prediction pair, merge the first prediction result and the second prediction result to obtain a final prediction result.
[0014] Preferably, the formula for decomposing the original sequence through the sequence decomposition module is:
[0015] X t = AvgPool(Padding(X))
[0016] X S = X - X t
[0017] where X s , X t ∈R L×F respectively represent the periodic part and the trend part obtained by decomposition, L is the input length of the sequence, and F represents the number of features of the input sequence.
[0018] Preferably, the calculation formula of the linear model is:
[0019] output = w * input + b;
[0020] Among them, input is the input sequence, output is the output sequence, w is the linear weight matrix, and b is the bias matrix.
[0021] Preferably, the calculation formula of the cross-attention mechanism is:
[0022]
[0023] Among them, K and V are respectively obtained by linear mapping of the same periodic tail through different random parameters, and Q is obtained by linear mapping of the periodic term through different random parameters.
[0024] Preferably, the loss function of the loss constraint module is:
[0025]
[0026] Among them, SeasonalError is the relative error of the periodic term prediction, TrendError is the relative error of the trend term prediction, L is the length of the sequence, S i is the true value of the periodic term, is the predicted value of the periodic term, T i is the true value of the trend term, is the predicted value of the trend term, and α is a hyperparameter.
[0027] Preferably, the loss function of the time series prediction model is:
[0028]
[0029] Among them, y i is the true value, is the predicted value; MSE measures the model performance by calculating the average of the squared differences between the predicted value and the true value; MAE evaluates the model performance by calculating the average of the absolute differences between the predicted value and the true value.
[0030] In a second aspect, the present invention provides a time-order enhanced Transformer time series prediction system, including:
[0031] A data acquisition unit for acquiring various time series data in industrial equipment and storing them in a database;
[0032] A preprocessing unit for preprocessing the time series data to obtain the original sequence;
[0033] A period-trend modeling unit for constructing a time series prediction model based on a Transformer backbone, where the time series prediction model includes a sequence decomposition module, a linear model, a cross-attention mechanism, and a loss constraint module; decomposing the original sequence through the sequence decomposition module to obtain a trend term part and a periodic term part; modeling the trend term part through the linear model to obtain a first prediction result and retaining the time order information in the original sequence; modeling the periodic term part through the cross-attention mechanism to obtain a second prediction result;
[0034] A period-trend prediction loss constraint unit for generating period-trend prediction pairs through the loss constraint module; dynamically adjusting the first prediction result and the second prediction result through an adaptive mechanism to obtain an optimal combination of the period-trend prediction pairs;
[0035] A result prediction unit for merging the first prediction result and the second prediction result according to the optimal combination of the period-trend prediction pairs to obtain a final prediction result.
[0036] In a third aspect, the present invention also discloses a computer device, including a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the steps of the method described in the first aspect.
[0037] In a fourth aspect, the present invention also discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect.
[0038] In a fifth aspect, the present invention also discloses a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect.
[0039] Compared with the prior art, the present invention has the following advantages and technical effects:
[0040] The present invention provides a method for predicting time series of Transformer enhanced by time sequence, which includes: first, obtaining various time series data in industrial equipment and storing them in a database; second, preprocessing the time series data to obtain an original sequence; then, constructing a time series prediction model according to the Transformer framework, where the time series prediction model includes a sequence decomposition module, a linear model, a cross-attention mechanism, and a loss constraint module; decomposing the original sequence through the sequence decomposition module to obtain a trend term part and a periodic term part; modeling the trend term part through the linear model to obtain a first prediction result and retaining the time sequence information in the original sequence; modeling the periodic term part through the cross-attention mechanism to obtain a second prediction result; generating a periodic-trend prediction pair through the loss constraint module; dynamically adjusting the first prediction result and the second prediction result through an adaptive mechanism to obtain an optimal combination of the periodic-trend prediction pair; finally, merging the first prediction result and the second prediction result according to the optimal combination of the periodic-trend prediction pair to obtain a final prediction result.
[0041] The present invention makes full use of the characteristic of the linear model to retain time sequence information. By decomposing the original sequence into a periodic term and a trend term, using the linear model to model the trend term to retain the time sequence information in the original time series, and using Transformer to model the periodic term, the Transformer-based model can obtain the ability to retain time sequence information.
[0042] The present invention introduces a general Transformer time series prediction network based on sequence decomposition modeling, which can effectively enhance the ability of Transformer to retain time sequence information and improve the prediction performance of the model for future sequences.
[0043] The present invention introduces a periodic-trend pair constraint loss function for the sequence decomposition modeling network. By enabling the model to generate as many periodic-trend prediction pairs as possible at different time steps to cancel out errors with each other, the prediction accuracy is improved. Description of the Drawings
[0044] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application. In the drawings:
[0045] Figure 1 is a flowchart of the method according to the embodiment of the present invention;
[0046] Figure 2 is a schematic diagram of the time series prediction model according to the embodiment of the present invention;
[0047] Figure 3 Schematic diagram of the system according to an embodiment of the present invention. Detailed implementation manners
[0048] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.
[0049] It should be noted that the steps shown in the flowchart of the drawings may be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.
[0050] Embodiment 1
[0051] As Figure 1 shown, this embodiment provides a Transformer time series prediction method with enhanced time order, including:
[0052] S1. Obtain the original data from industrial equipment sensors;
[0053] Specifically, collect data from various sensors in the industrial field. These sensors may include temperature sensors, pressure sensors, humidity sensors, flow sensors, etc. The data collected by the sensors in real time provides rich original materials for time series prediction. Ensuring the integrity and accuracy of this data is the basis for high-quality prediction.
[0054] S2. Perform data preprocessing on the original data obtained in S1, mainly including data cleaning, data standardization, outlier processing, and data partitioning. In the process of data partitioning, this embodiment divides the preprocessed data set into a training set, a test set, and a validation set according to a ratio of 6:2:2;
[0055] Data preprocessing is a crucial step in time series prediction, aiming to convert the original data into a format suitable for model training. The main steps are as follows: data cleaning, standardization, outlier processing, and data partitioning. Specifically, it includes:
[0056] S2-1. Data cleaning: Ensure the continuity and integrity of the data by removing invalid data and noise and filling in missing values. For example, use the interpolation method to fill in missing values and remove obvious incorrect data points.
[0057] S2-2. Standardization: Convert the data to a unified scale, usually using z-score standardization or min-max scaling to eliminate the influence brought by different dimensions. This helps to improve the training efficiency and prediction accuracy of the model.
[0058] S2-3. Outlier Handling: Identify and handle outliers in the data. For example, use the box plot method or the z-score method to detect and handle outliers, and avoid the negative impact of these outliers on model training.
[0059] S2-4. Data Partitioning: Divide the preprocessed dataset into a training set, a test set, and a validation set in a ratio of 6:2:2. The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate the generalization ability of the model.
[0060] S3. Decompose the original sequence into a periodic term and a trend term through sequence decomposition technology, and send them into the periodic and trend prediction networks respectively;
[0061] Specifically, for the sequence decomposition, the input time series is decomposed into a periodic term and a trend term. Periodic term: Capture the periodic features in the time series (such as daily fluctuations, seasonal changes). Trend term: Capture the long-term change trend in the time series. Use the moving average method to extract the trend term, and subtract the trend term from the original sequence to obtain the periodic term sequence.
[0062] For a sequence X ∈ R of length L as input L×F , the decomposition process is as follows:
[0063] X t = AvgPool(Padding(X))
[0064] X S = X - X t
[0065] where X s , X t ∈ R L×F represent the periodic part and the trend part obtained by the decomposition respectively. F represents the number of features of the input sequence. In this embodiment, AvgPool(·) is used for moving average and padding operations are performed to keep the sequence length unchanged.
[0066] S4. Model the trend term through a linear model and retain the time order information, and at the same time achieve the prediction of the trend term;
[0067] Specifically, for trend term prediction, first normalize the trend term and then use a linear model to model and predict the trend term. Then perform inverse normalization to obtain the prediction result, and use the prediction result obtained by the linear model as a part before the final result merger to retain the time order features in the original sequence and enhance the time order information in the prediction result.
[0068] The calculation process of the linear model is as follows: output = w * input + b, where input is the input sequence, output is the output sequence, w is the linear weight matrix, and b is the bias matrix.
[0069] As an innovative implementation, in the framework of separately modeling the cycle and trend, this embodiment uses a linear model to model the trend term, alleviating the problem that the Transformer cannot effectively retain the time sequence information.
[0070] S5. Model the periodic features of the cycle term and the periodic tail through the cross-attention mechanism, and at the same time realize the prediction of the cycle term;
[0071] As an innovative implementation, for the modeling and prediction of the cycle term sequence, this embodiment first separately extracts the information within a small time window from the cycle term that is close to the prediction window and uses it as the cycle tail; then passes the cycle tail information through local convolution and a linear layer to obtain a high-dimensional representation of the cycle tail, which is used as the value (Value) and key (Key) of the cross-attention. Then, the cycle term passes through global convolution and a linear layer to obtain a high-dimensional representation of the cycle term, which is used as the query (Query) of the cross-attention. Finally, use the cross-attention mechanism to mix-model the cycle term and the cycle tail information, and reduce the dimension through a linear layer to obtain the prediction result.
[0072] The calculation process of the cross-attention score is as follows: Among them, the key (Key, K) and value (Value, V) are respectively obtained by linear mapping of the same periodic tail through different random parameters, while the query (Query, Q) is also obtained by linear mapping of the cycle term through different random parameters.
[0073] S6. The cycle-trend pair loss function constrains the model to generate more cycle-trend prediction pairs that can cancel each other's errors during prediction, dynamically adjusts the prediction values of the cycle term and the trend term through an adaptive mechanism, finds the optimal combination of cycle-trend prediction pairs, combines the prediction results of the trend term and the cycle term, and improves the prediction accuracy;
[0074] Specifically, the cycle-trend pair loss function constrains the model to generate more cycle-trend prediction pairs that can cancel each other's errors during prediction, dynamically adjusts the prediction values of the cycle term and the trend term through an adaptive mechanism, finds the optimal combination of cycle-trend prediction pairs, alleviates the problem of error growth after combining the cycle and trend terms, and improves the prediction accuracy. This part uses the loss function and the adaptive adjustment algorithm to enable the model to generate as many cycle-trend prediction pairs that can cancel each other's errors as possible at different time steps, improving the prediction accuracy.
[0075] The loss function used by the described loss constraint module is:
[0076]
[0077] Among them, SeasonalError is the relative error of the periodic term prediction, TrendError is the relative error of the trend term prediction, L is the length of the sequence, S i is the true value of the periodic term, is the predicted value of the periodic term, T i is the true value of the trend term, is the predicted value of the trend term, and α is the hyperparameter.
[0078] As an innovative implementation method, in this embodiment, for the model structure of separately modeling the period - trend, a period - trend pair constraint loss function is proposed. By constraining the model to generate more period - trend pairs that can cancel out errors with each other, the prediction accuracy is improved.
[0079] S7. Use the loss function module to calculate the loss function value of the predicted value F and the true value Y of the time - series prediction model, calculate the gradient of the time - series prediction model parameters, and use the optimization algorithm to perform backpropagation of the time - series prediction model to update the time - series prediction model parameters;
[0080] Specifically, the loss functions MSE and MAE of the time - series prediction model are respectively:
[0081]
[0082] Among them, y i is the true value, is the predicted value. MSE measures the model performance by calculating the average of the squared differences between the predicted value and the true value; MAE evaluates the model performance by calculating the average of the absolute differences between the predicted value and the true value.
[0083] S8. When the time - series prediction model has converged, save the parameters of the time - series prediction model, complete the training, and obtain the trained time - series prediction model;
[0084] Specifically, after the model has converged, save the parameters of the model.
[0085] S9. Predict the data in the test dataset according to the trained time - series prediction model to obtain the prediction result.
[0086] Specifically, predict the test dataset according to the model saved in S8 to obtain the prediction result of the test set. By evaluating the accuracy of the prediction result (such as calculating metrics such as prediction error, prediction accuracy, recall rate, etc.), the prediction performance and generalization ability of the model are evaluated.
[0087] To verify the effectiveness of this embodiment, instance verification is carried out on the datasets collected in four real scenarios, including the ETTh1 dataset, the ETTh2 dataset, the ETTm1 dataset, and the ETTm2 dataset. These four real datasets are the power transformer data provided by a power company, including the power transformer load, oil temperature, etc. data from 2016 to 2018. Among them, the sampling granularity of the ETTm1 dataset and the ETTm2 dataset is 15 minutes, and the sampling granularity of the ETThl dataset and the ETTh2 dataset is 1 hour.
[0088] Using the above datasets, sequence prediction and evaluation are carried out according to the following steps:
[0089] S1. Read the data from the power transformer sensor database in chronological order;
[0090] S2. Conduct data analysis and preprocessing on the original training dataset obtained in S1, and divide the preprocessed dataset into a training set, a test set, and a validation set according to the ratio of 6:2:2;
[0091] S3. Decompose the original sequence into a periodic term and a trend term through sequence decomposition technology, and send them into the periodic and trend prediction networks respectively;
[0092] S4. Model the trend term through a linear model and retain the chronological information, while realizing the prediction of the trend term;
[0093] S5. Model the periodic features of the periodic term and the periodic tail through a cross-attention mechanism, while realizing the prediction of the periodic term;
[0094] S6. Generate more cycle-trend prediction pairs that can cancel each other's errors through the cycle-trend pair constraint loss function for the prediction results of the trend term and the periodic term, dynamically adjust the prediction values of the periodic term and the trend term through an adaptive mechanism, find the optimal combination of cycle-trend prediction pairs, merge the prediction results of the trend term and the periodic term, and improve the prediction accuracy;
[0095] S7. Use the hybrid loss function module to calculate the loss function value of the predicted value F and the true value Y of the time series prediction model, calculate the gradient of the time series prediction model parameters, and use the optimization algorithm to perform the backpropagation of the time series prediction model to update the time series prediction model parameters;
[0096] S8. When the time series prediction model has converged, save the parameters of the time series prediction model, complete the training, and obtain the trained time series prediction model;
[0097] S9. Predict the data in the test dataset according to the trained time series prediction model to obtain the prediction results.
[0098] Table 1
[0099]
[0100] Table 1 presents the test evaluation results when the method proposed in this embodiment is verified on four power transformer datasets (ETTh1 dataset, ETTh2 dataset, ETTm1 dataset, and ETTm2 dataset). At the same time, the results of other existing methods are listed in the figure. By comparison, the effectiveness of this embodiment is better demonstrated, comparing the mean square error and the mean absolute error respectively. Among them, the model prediction results in bold are the optimal results, and the model prediction results underlined are the sub-optimal results. The comparison methods cover models based on Transformer, models based on graph neural networks, and models based on multi-layer perceptrons, including (TimeMixer, MSGNet, iTransformer, PatchTST, DLinear). The test results show that this embodiment is mostly superior to the compared models in terms of the mean square error and the mean absolute error of the prediction results, proving the feasibility and effectiveness of this embodiment in using the sequence decomposition network to model the trend and periodic terms separately and using the linear model to retain the time order information.
[0101] Figure 1 Shows the Transformer time series prediction process with time order enhancement in this embodiment. It includes the whole process from data collection to prediction implementation: industrial sensor data collection, data preprocessing, dataset division (the ratio of training set: validation set: test set is 6:2:2), periodic-trend decomposition, periodic term modeling, trend term modeling, combination of periodic-trend term prediction results, loss function constraint on the combined results, parameter adjustment, model verification, saving the best model, using the best model for prediction, and evaluating the prediction results.
[0102] Figure 2It shows a schematic diagram of the Transformer time series prediction model with enhanced chronological order in this embodiment. The model structure of this embodiment is divided into two main parts: the trend term linear modeling branch and the periodic term cross-attention modeling branch. Among them, the trend term linear modeling branch is used to model the trend features in the original sequence and extract chronological order information; the periodic term cross-attention modeling branch obtains a high-dimensional representation of the periodic tail through local convolution and linear layers and uses it as the value (Value) and key (Key) of the cross-attention. At the same time, this branch obtains a high-dimensional representation of the periodic term through global convolution and linear layers and uses it as the query (Query) of the cross-attention. Finally, this branch uses the cross-attention mechanism to mix and model the periodic term and the periodic tail information and reduces the dimension through a linear layer to obtain the prediction result. The model proposed in this embodiment enhances the ability of the Transformer to retain chronological order information by integrating the linear branch and the Transformer branch in parallel.
[0103] Embodiment 2
[0104] Based on the same inventive concept, as Figure 3 shown, this embodiment also provides a Transformer time series prediction system with enhanced chronological order, including:
[0105] A data acquisition unit, which is used to collect various time series data in industrial equipment and securely store them in a database;
[0106] A preprocessing unit, which performs preprocessing steps such as data cleaning, standardization, and outlier processing on the collected content, and divides the data into a training set and a test set;
[0107] A period-trend modeling unit, which uses a linear model to model the trend term part obtained by decomposing the original sequence and retains the chronological order information in the original sequence to achieve enhanced chronological order features, and uses the cross-attention mechanism to model the periodic term part obtained by decomposing the original sequence;
[0108] A period-trend prediction loss constraint unit, which generates more period-trend prediction pairs that can cancel each other's errors by loss constraint on the prediction results of the trend term and the periodic term, and dynamically adjusts the prediction values of the periodic term and the trend term through an adaptive mechanism to find the optimal combination of period-trend prediction pairs;
[0109] A result prediction unit, which combines the prediction results obtained from the periodic term and the trend term to obtain the final prediction result.
[0110] A Transformer time series prediction system with enhanced chronological order provided in this embodiment has all the advantages of a Transformer time series prediction method with enhanced chronological order provided in Embodiment 1.
[0111] Embodiment III
[0112] This embodiment also discloses a computer device, including a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the steps of the method described in Embodiment I.
[0113] Embodiment IV
[0114] This embodiment also discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in Embodiment I are implemented.
[0115] Embodiment V
[0116] This embodiment also discloses a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in Embodiment I are implemented.
[0117] The above is only a preferred specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A time-order enhanced Transformer time series prediction method, characterized in that: The following steps are involved: Obtain various time series data from industrial equipment and store them in the database; Preprocessing the time series data to obtain an original sequence; According to the Transformer skeleton, a time series prediction model is constructed, wherein the time series prediction model includes a sequence decomposition module, a linear model, a cross attention mechanism, and a loss constraint module; Decomposing the original sequence by the sequence decomposition module to obtain a trend item part and a period item part; Modeling the trend item part by using the linear model to obtain a first prediction result, and retaining the time sequence information in the original sequence; Modeling the period term part by the cross attention mechanism to obtain a second prediction result; Generate a cycle-trend forecast pair through the loss constraint module; dynamically adjust the first forecast result and the second forecast result through an adaptive mechanism to obtain an optimal combination of the cycle-trend forecast pair; According to the optimal combination of the cycle-trend prediction pair, the first prediction result and the second prediction result are combined to obtain a final prediction result.
2. The method according to claim 1, characterized in that The formula for decomposing the original sequence by the sequence decomposition module is: X t =AvgPool(Padding(X)) X S =X-X t Among them, X s , X t ∈R L×F They represent the periodic part and trend part obtained by decomposition respectively, L is the input length of the sequence, and F represents the number of features of the input sequence.
3. The method according to claim 1, characterized in that The calculation formula of the linear model is: output = w * input + b; Among them, input is the input sequence, output is the output sequence, w is the linear weight matrix, and b is the bias matrix.
4. The method according to claim 1, characterized in that: The calculation formula of the cross attention mechanism is: Among them, K and V are obtained by linear mapping of the same periodic tail with different random parameters, and Q is obtained by linear mapping of the periodic term with different random parameters.
5. The method according to claim 1, characterized in that The loss function of the loss constraint module is: Among them, SeasonalError is the relative error of the periodic item prediction, TrendError is the relative error of the trend item prediction, L is the length of the sequence, S i is the true value of the periodic term, is the predicted value of the periodic term, T i is the true value of the trend term, is the predicted value of the trend term, and α is a hyperparameter.
6. The method according to claim 1, characterized in that The loss function of the time series prediction model is: Among them, y i is the true value, is the predicted value; MSE measures the model performance by calculating the average of the squared differences between the predicted value and the true value; MAE evaluates the model performance by calculating the average of the absolute differences between the predicted value and the true value.
7. A time-order enhanced Transformer time series prediction system, characterized in that: include: A data acquisition unit, used to acquire various time series data from industrial equipment and store them in a database; A preprocessing unit, used for preprocessing the time series data to obtain an original sequence; A cycle-trend modeling unit is used to construct a time series prediction model according to the Transformer skeleton, wherein the time series prediction model includes a sequence decomposition module, a linear model, a cross attention mechanism, and a loss constraint module; the original sequence is decomposed by the sequence decomposition module to obtain a trend term part and a cycle term part; Modeling the trend item part by using the linear model to obtain a first prediction result, and retaining the time sequence information in the original sequence; Modeling the period term part by the cross attention mechanism to obtain a second prediction result; A cycle-trend prediction loss constraint unit, configured to generate a cycle-trend prediction pair through the loss constraint module; dynamically adjust the first prediction result and the second prediction result through an adaptive mechanism to obtain an optimal combination of the cycle-trend prediction pair; The result prediction unit is used to combine the first prediction result and the second prediction result according to the optimal combination of the period-trend prediction pair to obtain a final prediction result.
8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.