Adaptive empirical mode decomposition prediction method based on multi-scale Transform

By introducing adaptive empirical modal decomposition technology in the multi-scale Transformer model, the shortcomings of existing models in dealing with non-stationary time series are solved, and high accuracy and robust prediction of complex time series are achieved, and adaptive Ability to effectively handle noise interference.

CN118964842APending Publication Date: 2024-11-15UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410986999.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing machine learning models perform poorly when processing non-stationary time series data, lack adaptability, and are difficult to provide accurate predictions of complex time series behaviors.

Method used

A multi-scale Transformer-based adaptive empirical modal decomposition prediction method (EMDformer) is proposed. By combining adaptive empirical modal decomposition technology and multi-scale Transformer model, complex non-stationary time series data are decomposed and time dependence of data is captured. , improve the accuracy and robustness of predictions, and have adaptability to effectively avoid noise interference in the data.

Benefits of technology

Accurate prediction of non-stationary time series is achieved, adaptive ability, can effectively extract time series details information, retain useful information to the greatest extent, reduce noise impact, and significantly improve prediction accuracy and robustness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964842A_ABST
    Figure CN118964842A_ABST
Patent Text Reader

Abstract

The invention provides an adaptive empirical mode decomposition prediction method based on a multi-scale Transform, and the method comprises the following steps: S1, carrying out the preprocessing of data based on a time sequence; s2, inputting a result of the S1 into an adaptive empirical mode decomposition prediction algorithm structure, and performing adaptive empirical mode decomposition on an input data set to obtain a limited number of eigenmode functions; s3, the model inputs the result of the S2 into a multi-scale encoder to obtain encoding feature vectors of all scales; s4, inputting the result of S3 into a multi-scale decoder to obtain decoding feature vectors of all scales; and S5, performing splicing and linear transformation on the result of the S4 to obtain a time sequence prediction result. According to the algorithm provided by the invention, an adaptive empirical mode decomposition technology and a multi-scale Transform model are combined. Therefore, the method not only improves the accuracy and robustness of prediction, but also has adaptive capability, and can effectively avoid the interference of noise in the data to adapt to the non-stationary time sequence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and is applied to time series prediction tasks, and specifically relates to an adaptive empirical mode decomposition prediction method based on a multi-scale Transformer.

[0002] Background Technology - Time Series Forecasting

[0003] Time series prediction plays a vital role in analysis and decision-making in various fields, directly affecting resource allocation, production planning and operation management. In order to help organizations and enterprises make better decisions and management, accurate prediction of time series is extremely important. At present, machine learning algorithms have made significant progress in time series prediction. However, due to the poor performance of currently used machine learning models in processing non-stationary time series data and the lack of adaptive ability, it is difficult to provide accurate predictions for complex time series behaviors, which limits its wide application in various fields. The Transformer model is a neural network proposed in 2017, which learns sequence context semantics and data meaning by capturing relationships in sequence data. By combining adaptive empirical mode decomposition technology and multi-scale architecture design to decompose complex non-stationary time series data, the Transformer model can well capture the time dependency of data from different scales, thereby improving the accuracy and robustness of prediction. Therefore, in response to the above problems, the present invention proposes an adaptive empirical mode decomposition prediction method (EMDformer) based on multi-scale Transformer. By combining adaptive empirical mode decomposition technology and a multi-scale Transformer model, the present invention not only improves the accuracy and robustness of prediction, but also has adaptive capabilities, which can effectively avoid the interference of noise in the data to adapt to non-stationary time series. Summary of the invention

[0004] This paper proposes an adaptive empirical mode decomposition prediction method (EMDformer) based on multi-scale Transformer, aiming to solve the shortcomings of existing machine learning models in processing complex non-stationary time series data, especially non-stationary time series prediction tasks.

[0005] The technical solution of the present invention is: an adaptive empirical mode decomposition prediction method (EMDformer) based on a multi-scale Transformer, comprising the following steps:

[0006] S1: Preprocess the data based on time series related data;

[0007] S2: Input the preprocessed data into the adaptive empirical mode decomposition prediction algorithm (EMDformer) structure, perform adaptive empirical mode decomposition on the input data set, and obtain a finite number of intrinsic mode functions (IMF);

[0008] S3: The model inputs the obtained IMFs into the encoder of the designed multi-scale Transformer module in parallel in a certain order to obtain the encoded feature vectors at each scale;

[0009] S4: Input the encoded feature vector of the encoder into the decoder of the designed multi-scale Transformer module to obtain the decoded feature vectors at each scale;

[0010] S5: Concatenate and linearly transform the decoded feature vectors obtained at each scale to obtain the time series prediction result.

[0011] Furthermore, step S1 includes the following sub-steps:

[0012] S11. Clean the existing time series data set and remove outliers;

[0013] S12. Process the cleaned data, use time length (such as the first day, the second day, etc.) to represent the time dimension and normalize it to obtain the input time series T.

[0014] Furthermore, step S2 includes the following sub-steps:

[0015] S21, finding the maximum value and the minimum value of the input time series T, and obtaining the maximum value point and the minimum value point corresponding to the maximum value and the minimum value respectively;

[0016] S22, mirror-copy the data, connect the front and back of the data together to obtain a mirror data ring, then perform cubic spline difference on the data, and then select the part of the original data;

[0017] S23. In the selected part, draw envelopes based on the maximum and minimum points, and calculate the average value K of the two envelopes. 1 and K 2 ;

[0018] S24. Subtract the average value K of the two envelopes from the time series T 1 and K 2 , obtain the intermediate time series T′ and judge it according to the limiting conditions. If T′ meets the conditions, it is saved as IMF; otherwise, the intermediate time series T′ is used as the input time series T, and sub-steps S21 to S24 are repeated;

[0019] The limiting conditions are as follows. If (1) and (2) are satisfied at the same time, stop:

[0020] (1) The difference between the number of maximum and minimum values ​​of the time series over the entire time length is no more than one;

[0021] (2) At any time, the average values ​​of the envelope defined by the local maximum point and the envelope defined by the local minimum point are close to zero.

[0022] S25, subtract the saved IMF from the input time series T to obtain a new time series T″, and determine whether T″ satisfies the stopping condition. If so, stop the decomposition; if not, take T″ as the input time series T, and repeat steps S21 to S25;

[0023] The stopping conditions are as follows: if (1) or (2) is met, the decomposition stops:

[0024] (1) The number of zero points and the number of extreme points of the sequence differ by only one;

[0025] (2) The number of extreme points in the sequence is less than three.

[0026] S26. The obtained IMF is passed through a minimum mean square error filter (LMS filter) to adjust the parameters and structure of the filter in real time to achieve dynamic removal of noise in the time series and reduce the impact of noise.

[0027] Among them, the LMS filter usually adopts a finite impulse response (FIR) structure with an adjustable coefficient ω 0 [n],ω 1 [n],…,ω M-1 [n], where M is the filter order. Let x[n] be the filter input, d[n] be the desired signal, e[n] be the error, y[n] be the filter output, where n is the time index. The filter output formula, error calculation formula, and coefficient update formula are as follows:

[0028]

[0029] e[n]=d[n]-y[n]

[0030] ω k [n+1]=ω k [n]+μ·e[n]·x[nk]

[0031] The filter coefficients are adjusted based on the error e[n] and the input signal x[n]. The LMS filter uses the gradient descent method to update the coefficients to minimize the mean square error, and μ is the step size factor (learning rate) that controls the update amplitude.

[0032] Furthermore, step S3 includes the following sub-steps:

[0033] S31, sorting the obtained IMFs, performing linear transformation and normalization to obtain a representation sequence of the IMFs;

[0034] S32. For the IMF characterization sequence obtained in S31, use sine and cosine functions of different frequencies to calculate its relative position information. The corresponding calculation formula is:

[0035]

[0036]

[0037] Among them, pos represents the position of the encoding time step, s represents the dimension, and d model Represents the network output dimension.

[0038] S33, adding the IMF representation sequence and its relative position information to obtain its high-dimensional representation sequence;

[0039] S34, concatenating the high-dimensional representation sequence obtained in S33 layer by layer to obtain a multi-scale coding sequence. The multi-scale coding sequence is linearly transformed and input into a multi-layer feedforward neural network to complete the learning of the data set and generate a data sequence after the encoder learning;

[0040] The formula of the multi-layer feedforward neural network FFN(x) is:

[0041] FFN(x)=max(0,xW 1 +b 1 )W 2 +b 2

[0042] Among them, the matrix where d ff is the inner dimension value. 1 、b 2 is the parameter that needs to be updated through network learning, x is the multi-scale coding sequence, and max() means taking the maximum value.

[0043] S35, normalize the data sequence generated by the encoder learning in step S34, and add it to the high-dimensional representation sequence generated in step S33 and the multi-scale encoding sequence in step S34 to obtain the encoding feature vectors at each scale.

[0044] Further, step S4 includes the following sub-steps:

[0045] S41, based on the scale feature coding vectors outputted from step S3, scale splicing is performed layer by layer to obtain a multi-scale decoding sequence. The multi-scale decoding sequence is linearly transformed and inputted into a multi-layer feedforward neural network to generate a decoding sequence after being learned by the decoder;

[0046] The formula of the multi-layer feedforward neural network FFN(x) is:

[0047] FFN(x)=max(0,xW 3 +b 3 )W 4 +b 4

[0048] Among them, the matrix where d ff is the inner dimension value. 3 、b 4 is the parameter that needs to be updated through network learning, x is the multi-scale decoding sequence, and max() means taking the maximum value.

[0049] S42, normalizing the decoded sequence generated by the decoder in step S41, and adding it to the multi-scale decoded sequence in S41 to obtain a decoded feature vector at each scale.

[0050] Furthermore, step S5 includes the following sub-steps:

[0051] S51, concatenating the decoded feature vectors at each scale obtained in step S42 in the channel dimension to obtain a feature vector containing global information;

[0052] S52: Perform a linear transformation on the feature vector containing global information obtained in step S51 to obtain a time series prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The present invention is a flow chart of the method.

[0054] Figure 2 This is the performance result of the algorithm model proposed in the present invention on the ETTh dataset.

[0055] Figure 3 This is the performance result of the algorithm model proposed in this invention on the ETTm dataset.

[0056] Figure 4 This is the performance result of the algorithm model proposed in this invention on the Weather dataset. DETAILED DESCRIPTION

[0057] The embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0058] Before describing the specific embodiments of the present invention, in order to make the scheme of the present invention clearer and more complete, the abbreviations and key terms used in the present invention are first defined as follows:

[0059] Multi-layer feedforward neural network: a structure in which each layer of neurons is fully interconnected with the neurons in the next layer, and there are no same-layer links or cross-layer links between neurons.

[0060] like Figure 1 As shown, the present invention provides an adaptive empirical mode decomposition prediction method (EMDformer) based on a multi-scale Transformer, comprising the following steps:

[0061] S1: Preprocess the data based on time series related data;

[0062] S2: Input the preprocessed data into the adaptive empirical mode decomposition prediction algorithm (EMDformer) structure, perform adaptive empirical mode decomposition on the input data set, and obtain a finite number of intrinsic mode functions (IMF);

[0063] S3: The model inputs the obtained IMFs into the encoder of the designed multi-scale Transformer module in parallel in a certain order to obtain the encoded feature vectors at each scale;

[0064] S4: Input the encoded feature vector of the encoder into the decoder of the designed multi-scale Transformer module to obtain the decoded feature vectors at each scale;

[0065] S5: Concatenate and linearly transform the decoded feature vectors obtained at each scale to obtain the time series prediction result.

[0066] In the embodiment of the present invention, there are default values ​​in the data set, which will interfere with the model's learning of the data. Therefore, in order to predict the time series and improve the accuracy of the prediction, the present invention uses the time length (such as the first day, the second day, etc.) to represent it in the time dimension to enhance the learning ability of the model. Based on the above ideas, the following data set preprocessing method is proposed.

[0067] In the example of the present invention, step S1 includes the following sub-steps:

[0068] S11. Clean the existing time series data set and remove outliers;

[0069] S12. Process the cleaned data, use time length (such as the first day, the second day, etc.) to represent the time dimension and normalize it to obtain the input time series T.

[0070] Previous time series prediction models often use traditional statistical methods such as exponential smoothing to decompose non-stationary time series. However, non-stationary time series have high complexity. These methods are likely to ignore the detailed information in the non-stationary time series while being affected by non-linear noise, thereby affecting the model's analysis of its intrinsic characteristics. In an embodiment of the present invention, the present invention designs an adaptive empirical mode decomposition method to decompose the time series, which can eliminate the influence of non-linear noise and retain the detailed information in the non-stationary time series.

[0071] Specifically, in the field of time series prediction, the present invention designs an adaptive empirical mode decomposition method, which combines the mirror copy method and the minimum mean square error filter to eliminate the noise problem and endpoint effect problem introduced in the empirical mode decomposition. In this way, the present invention can effectively cope with the high complexity of non-stationary time series and remove the nonlinear noise in the non-stationary time series. By improving the decomposition algorithm, the present invention can effectively combat nonlinear noise while extracting detailed information of the non-stationary time series, retaining the useful information in the non-stationary time series to the greatest extent, and facilitating the subsequent extraction of multi-scale information and improving prediction accuracy. Based on the above ideas, the present invention proposes an adaptive empirical mode decomposition method to make the prediction results of the model more accurate.

[0072] In the present example, step S2 includes the following sub-steps:

[0073] S21, finding the maximum value and the minimum value of the input time series T, and obtaining the maximum value point and the minimum value point corresponding to the maximum value and the minimum value respectively;

[0074] S22, mirror-copy the data, connect the front and back of the data together to obtain a mirror data ring, then perform cubic spline difference on the data, and then select the part of the original data;

[0075] S23. In the selected part, draw envelopes based on the maximum and minimum points, and calculate the average value K of the two envelopes. 1 and K 2 ;

[0076] S24. Subtract the average value K of the two envelopes from the time series T 1 and K 2 , obtain the intermediate time series T′ and judge it according to the limiting conditions. If T′ meets the conditions, it is saved as IMF; otherwise, the intermediate time series T′ is used as the input time series T, and sub-steps S21 to S24 are repeated;

[0077] The limiting conditions are as follows. If (1) and (2) are satisfied at the same time, stop:

[0078] (1) The difference between the number of maximum and minimum values ​​of the time series over the entire time length is no more than one;

[0079] (2) At any time, the average values ​​of the envelope defined by the local maximum point and the envelope defined by the local minimum point are close to zero.

[0080] S25, subtract the saved IMF from the input time series T to obtain a new time series T″, and determine whether T″ satisfies the stopping condition. If so, stop the decomposition; if not, take T″ as the input time series T, and repeat steps S21 to S25;

[0081] The stopping conditions are as follows: if (1) or (2) is met, the decomposition stops:

[0082] (1) The number of zero points and the number of extreme points of the sequence differ by only one;

[0083] (2) The number of extreme points in the sequence is less than three.

[0084] S26. The obtained IMF is passed through a minimum mean square error filter (LMS filter) to adjust the parameters and structure of the filter in real time to achieve dynamic removal of noise in the time series and reduce the impact of noise.

[0085] Among them, the LMS filter usually adopts a finite impulse response (FIR) structure with an adjustable coefficient ω 0 [n],ω 1 [n],…,ω M-1 [n], where M is the filter order. Let x[n] be the filter input, d[n] be the desired signal, e[n] be the error, y[n] be the filter output, where n is the time index. The filter output formula, error calculation formula, and coefficient update formula are as follows:

[0086]

[0087] e[n]=d[n]-y[n]

[0088] ω k [n+1]=ω k [n]+μ·e[n]·x[nk]

[0089] The filter coefficients are adjusted based on the error e[n] and the input signal x[n]. The LMS filter uses the gradient descent method to update the coefficients to minimize the mean square error, and μ is the step size factor (learning rate) that controls the update amplitude.

[0090] In the present example, step S3 includes the following sub-steps:

[0091] S31, sorting the obtained IMFs, performing linear transformation and normalization to obtain a representation sequence of the IMFs;

[0092] S32. For the IMF characterization sequence obtained in S31, use sine and cosine functions of different frequencies to calculate its relative position information. The corresponding calculation formula is:

[0093]

[0094]

[0095] Among them, pos represents the position of the encoding time step, s represents the dimension, and d model Represents the network output dimension.

[0096] S33, adding the IMF representation sequence and its relative position information to obtain its high-dimensional representation sequence;

[0097] S34, concatenating the high-dimensional representation sequence obtained in S33 layer by layer to obtain a multi-scale coding sequence. The multi-scale coding sequence is linearly transformed and input into a multi-layer feedforward neural network to complete the learning of the data set and generate a data sequence after the encoder learning;

[0098] The formula of the multi-layer feedforward neural network FFN(x) is:

[0099] FFN(x)=max(0,xW 1 +b 1 )W 2 +b 2

[0100] Among them, the matrix where d ff is the inner dimension value. 1 、b 2 is the parameter that needs to be updated through network learning, x is the multi-scale coding sequence, and max() means taking the maximum value.

[0101] S35, normalize the data sequence generated by the encoder learning in step S34, and add it to the high-dimensional representation sequence generated in step S33 and the multi-scale encoding sequence in step S34 to obtain the encoding feature vectors at each scale.

[0102] In the present example, step S4 includes the following sub-steps:

[0103] S41, based on the scale feature coding vectors outputted from step S3, scale splicing is performed layer by layer to obtain a multi-scale decoding sequence. The multi-scale decoding sequence is linearly transformed and inputted into a multi-layer feedforward neural network to generate a decoding sequence after being learned by the decoder;

[0104] The formula of the multi-layer feedforward neural network FFN(x) is:

[0105] FFN(x)=max(0,xW 3 +b 3 )W 4 +b 4

[0106] Among them, the matrix where d ff is the inner dimension value. 3 、b 4 is the parameter that needs to be updated through network learning, x is the multi-scale decoding sequence, and max() means taking the maximum value.

[0107] S42, normalizing the decoded sequence generated by the decoder in step S41, and adding it to the multi-scale decoded sequence in S41 to obtain a decoded feature vector at each scale.

[0108] In the present example, step S5 includes the following sub-steps:

[0109] S51, concatenating the decoded feature vectors at each scale obtained in step S42 in the channel dimension to obtain a feature vector containing global information;

[0110] S52: Perform a linear transformation on the feature vector containing global information obtained in step S51 to obtain a time series prediction result.

[0111] In the implementation example of the present invention, an adaptive empirical mode decomposition prediction method (EMDformer) based on a multi-scale Transformer created by the present invention realizes accurate prediction of non-stationary time series. Through the designed adaptive empirical mode decomposition method and the improved multi-scale Transformer model, the present invention well realizes the prediction of non-stationary time series. In the implementation example, some experimental scenarios are designed and compared with other deep learning prediction methods to verify that the method proposed by the invention improves the accuracy of non-stationary time series prediction.

[0112] In order to verify the effectiveness of the method proposed in the present invention, three advanced deep learning models (PatchTST, FEDformer, Autoformer) were selected in the actual example to compare with the algorithm model EMDformer proposed in the present invention and a prediction experiment on the data set was carried out. To ensure the fairness of the comparison, all models use the same initialization settings for hyperparameters. That is, the seven models all use the same initial learning rate, batch size and early stopping mechanism. Among them, the learning rate is initialized to 0.00001, and the hyperparameters are tuned by the Adam optimizer. The experimental parameters are shown in Table 1.

[0113] Table 1:

[0114] Hyperparameters Numeric Learning Rate 0.00001 Optimizer Adam Optimizer Iterations 100 times Batch size 32 batch sizes

[0115] In the experiment, we selected five real-world time series datasets, including ETTh1, ETTh2, ETTm1, ETTm2, and Weather, to verify the versatility of the model. The introduction of these datasets is as follows:

[0116] The Electricity Transformer Temperature (ETT) dataset is an important public dataset for non-stationary time series prediction. It records the temperature of electric transformers from July 2016 to July 2018 and multiple features related to power load and energy consumption, such as power demand, historical load, etc. The ETT dataset has two versions with different sampling intervals: ETTh and ETTm. The sampling interval of the ETTh dataset is 1 hour, so the dataset capacity is small. The longer sampling interval helps the model capture the long-term trend and macro pattern of the time series without being disturbed by short-term fluctuations, which can effectively help decision makers formulate medium- and long-term operation strategies and maintenance plans. The ETTm dataset records data every 15 minutes, so the dataset capacity is large. This high-frequency sampling helps the model analyze and predict changes in a shorter time scale, which has an important impact on real-time monitoring, short-term load forecasting, and the formulation of emergency response strategies. By comparing the prediction performance of the model on the ETTh and ETTm datasets, the robustness of the model can be verified. The ETTh dataset with low-frequency time intervals is more suitable for verifying the model's ability to capture long-term trends and seasonal patterns, while the ETTm dataset with high-frequency time intervals can verify the model's ability to handle short-term fluctuations and noise. If the model maintains consistent high accuracy on the above two datasets of different time scales and shows robust prediction performance, it means that the model has strong robustness. The Weather dataset is a time series dataset containing multiple meteorological variables, covering climate information collected every 10 minutes in 2020, and recording multiple meteorological indicators including temperature, air pressure, humidity, and wind speed. These data are very important for studying weather patterns, developing climate prediction models, and meteorological-related decision support systems. By comparing the performance of the model on the ETT and Weather datasets, we can understand the generalization ability of the model. The ETT dataset focuses on transformer temperature prediction in the energy field, while the Weather dataset involves a variety of environmental variables in the meteorological field. The two are different in terms of data nature, seasonal patterns, and data noise levels. If the model performs well on both datasets, it means that the model has cross-domain accuracy, stability, and adaptability. The stronger the generalization ability of the model, the more reliable and effective the model will be in real-world applications.

[0117] The data set is divided into a training set, a validation set, and a test set in chronological order. The training set is used for model training and parameter adjustment; the validation set is used for model performance verification and training adjustment, and is not used to update parameters; the main function of the test set is to evaluate model performance.

[0118] Before the experiment, the data set needs to be preprocessed, and the existing non-stationary time series data set needs to be cleaned to remove outliers. Then, the cleaned data is processed, and the time dimension is represented by the time length (such as the first day, the second day, etc.) and normalized to obtain the input time series T.

[0119] This experiment uses three indicators: Mean Squared Error (MSE), Mean Absolute Error (MAE) and Mean Directional Accuracy (MDA) to evaluate the performance of the algorithm proposed in the present invention.

[0120] The mean square error (MSE) reflects the average value of the square of the difference between the predicted result of the model proposed in the present invention and the true value. Since the value of MSE is non-negative, the closer its value is to 0, the smaller the error between the predicted value and the true value, and the better the fitting effect of the model. The calculation formula of the mean square error (MSE) is as follows:

[0121]

[0122] Where n is the number of samples, y i is the true value of the i-th day, is the predicted value for day i.

[0123] The mean absolute error (MAE) measures the accuracy of the prediction results. The mean absolute error refers to the average of the absolute values ​​of the difference between the predicted value and the true value. The smaller the mean absolute error (MAE), the higher the accuracy of the algorithm model proposed by the present invention. The calculation formula of the mean absolute error (MAE) is as follows:

[0124]

[0125] Where n is the number of samples, y i is the true value of the i-th day, is the predicted value for day i.

[0126] The average directional accuracy MDA is used to measure the directional accuracy of the prediction results of the algorithm proposed in the present invention. MDA is an important indicator for evaluating the directional prediction accuracy of the prediction model, and is particularly suitable for time series prediction. Unlike MSE and MAE, it focuses on whether the direction of the predicted value is consistent with the actual value (i.e., the rising and falling trends). The larger the average directional accuracy MDA, the greater the directional prediction accuracy. The calculation formula of the average directional accuracy MDA is as follows:

[0127]

[0128]

[0129] Where n is the number of samples, y i is the true value of the i-th day, y i-1 is the true value on day i-1 is the predicted value for day i.

[0130] In the calculation formulas of the three indicators, mean square error (MSE), mean absolute error (MAE), and mean direction accuracy (MDA), n is the total number of samples of the evaluation data. is the predicted value of each sample by the algorithm model proposed in this invention, y i is the true value of the data corresponding to the predicted value.

[0131] The comparative data of the time series prediction results of the algorithm model proposed in this invention and other three advanced deep learning prediction models on ETTh1, ETTh2, ETTm1, ETTm2 and Weather are shown in Table 2.

[0132] Table 2:

[0133]

[0134] As shown in Table 2, the MAE, MSE and MDA of EMDformer's prediction results on ETTh1, ETTh2, ETTm1, ETTm2 and Weather are mostly the best results on each data set, and EMDformer is significantly better than other models. This shows that EMDformer has better prediction results for time series than other models.

[0135] Figure 2 , Figure 3 and Figure 4 The prediction results of EMDformer on ETTh, ETTm and Weather are shown respectively. Among them, the orange line (GroundTruth) represents the true value of the time series in the corresponding data set, and the blue line (Prediction) represents the true prediction value of EMDformer in the corresponding data set. The graphical results show that EMDformer has good prediction performance on different data sets. Combined with the experimental results in Table 2 above, the EMDformer model performs well, has stronger adaptability and higher prediction performance, can significantly reduce prediction errors, and helps organizations and enterprises make better decisions and management.

[0136] The working principle and process of the present invention are as follows: In the present invention, firstly, the data is preprocessed based on the time series related data. The preprocessed data is input into the adaptive empirical mode decomposition prediction algorithm structure, and the input data set is adaptively empirical mode decomposition to obtain a finite number of intrinsic mode functions. The obtained intrinsic mode functions are input into the encoder of the designed multi-scale Transformer module in parallel in a certain order to obtain the encoded feature vectors at each scale. The encoded feature vectors of the encoder are input into the decoder of the designed multi-scale Transformer module to obtain the decoded feature vectors at each scale. The decoded feature vectors obtained at each scale are spliced ​​and linearly transformed to obtain the time series prediction result.

[0137] The beneficial effects of the present invention are as follows: the adaptive empirical mode decomposition prediction method (EMDformer) based on multi-scale Transformer proposed by the present invention realizes the effective decomposition of non-stationary time series. The method effectively combats nonlinear noise while extracting detailed information of non-stationary time series, and retains useful information in non-stationary time series to the greatest extent. At the same time, the present invention realizes accurate prediction of time series, has adaptive ability, can effectively avoid the interference of noise in data, and adapt to non-stationary time series.

[0138] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. An adaptive empirical mode decomposition prediction method based on a multi-scale Transformer (a deep neural network architecture, which is mainly used for classification and prediction tasks), characterized in that: The following steps are involved: S1: Preprocess the data based on time series related data; S2: Input the preprocessed data into the adaptive empirical mode decomposition prediction algorithm structure, perform adaptive empirical mode decomposition on the input data set, and obtain a finite number of intrinsic mode functions (IMF); S3: The model inputs the obtained IMFs into the encoder of the designed multi-scale Transformer module in parallel in a certain order to obtain the encoded feature vectors at each scale; S4: Input the encoded feature vector of the encoder into the decoder of the designed multi-scale Transformer module to obtain the decoded feature vectors at each scale; S5: Concatenate and linearly transform the decoded feature vectors obtained at each scale to obtain the time series prediction result.

2. The adaptive empirical mode decomposition prediction method based on multi-scale Transformer according to claim 1 is characterized in that: The step S1 comprises the following sub-steps: S11. Clean the existing time series data set and remove outliers; S12. Process the cleaned data, use time length (such as the first day, the second day, etc.) to represent the time dimension and normalize it to obtain the input time series T.

3. The adaptive empirical mode decomposition prediction method based on multi-scale Transformer according to claim 1 is characterized in that: The step S2 comprises the following sub-steps: S21, finding the maximum value and the minimum value of the input time series T, and obtaining the maximum value point and the minimum value point corresponding to the maximum value and the minimum value respectively; S22, mirror-copy the data, connect the front and back of the data together to obtain a mirror data ring, then perform cubic spline difference on the data, and then select the part of the original data; S23, in the selected part, draw envelopes based on the maximum value point and the minimum value point respectively, and calculate the average values ​​K1 and K2 of the two envelopes; S24, subtract the average values ​​K1 and K2 of the two envelopes from the time series T to obtain the intermediate time series T′ and make a judgment based on the limiting conditions. If T′ meets the conditions, it is saved as IMF; otherwise, the intermediate time series T′ is used as the input time series T, and substeps S21 to S24 are repeated; The limiting conditions are as follows. If (1) and (2) are satisfied at the same time, stop: (1) The difference between the number of maximum and minimum values ​​of the time series over the entire time length is no more than one; (2) At any time, the average values ​​of the envelope defined by the local maximum point and the envelope defined by the local minimum point are close to zero. S25, subtract the saved IMF from the input time series T to obtain a new time series T″, and determine whether T″ satisfies the stopping condition. If so, stop the decomposition; if not, take T″ as the input time series T, and repeat steps S21 to S25; The stopping conditions are as follows: if (1) or (2) is met, the decomposition stops: (1) The number of zero points and the number of extreme points of the sequence differ by only one; (2) The number of extreme points in the sequence is less than three. S26. The obtained IMF is passed through a minimum mean square error filter (LMS filter) to adjust the parameters and structure of the filter in real time to achieve dynamic removal of noise in the time series and reduce the impact of noise. Among them, the LMS filter usually adopts a finite impulse response (FIR) structure with adjustable coefficients ω0[n], ω1[n],…,ω M-1 [n], where M is the filter order. Let x[n] be the filter input, d[n] be the desired signal, e[n] be the error, y[n] be the filter output, where n is the time index. The filter output formula, error calculation formula, and coefficient update formula are as follows: e[n]=d[n]-y[n] oh k [n+1]=ω k [n]+μ·e[n]·x[nk] The filter coefficients are adjusted based on the error e[n] and the input signal x[n]. The LMS filter uses the gradient descent method to update the coefficients to minimize the mean square error, and μ is the step size factor (learning rate) that controls the update amplitude.

4. The adaptive empirical mode decomposition prediction method based on multi-scale Transformer according to claim 1, characterized in that: The step S3 comprises the following sub-steps: S31, sorting the obtained IMFs, performing linear transformation and normalization to obtain a representation sequence of the IMFs; S32. For the IMF characterization sequence obtained in S31, use sine and cosine functions of different frequencies to calculate its relative position information. The corresponding calculation formula is: Among them, pos represents the position of the encoding time step, s represents the dimension, and d model Represents the network output dimension. S33, adding the IMF representation sequence and its relative position information to obtain its high-dimensional representation sequence; S34, concatenating the high-dimensional representation sequence obtained in S33 layer by layer to obtain a multi-scale coding sequence. The multi-scale coding sequence is linearly transformed and input into a multi-layer feedforward neural network to complete the learning of the data set and generate a data sequence after the encoder learning; The formula of the multi-layer feedforward neural network FFN(x) is: FFN(x)=max(0,xW1+b1)W2+b2 Among them, the matrix where d ff is the inner dimension value. b1 and b2 are parameters that need to be updated through network learning, x is a multi-scale encoding sequence, and max() means taking the maximum value. S35, normalize the data sequence generated by the encoder learning in step S34, and add it to the high-dimensional representation sequence generated in step S33 and the multi-scale encoding sequence in step S34 to obtain the encoding feature vectors at each scale.

5. The adaptive empirical mode decomposition prediction method based on multi-scale Transformer according to claim 1, characterized in that: The step S4 comprises the following sub-steps: S41, based on the scale feature coding vectors outputted from step S3, scale splicing is performed layer by layer to obtain a multi-scale decoding sequence. The multi-scale decoding sequence is linearly transformed and inputted into a multi-layer feedforward neural network to generate a decoding sequence after being learned by the decoder; The formula of the multi-layer feedforward neural network FFN(x) is: FFN(x)=max(0,xW3+b3)W4+b4 Among them, the matrix where d ff is the inner dimension value. b3 and b4 are parameters that need to be updated through network learning, x is the multi-scale decoding sequence, and max() means taking the maximum value. S42, normalizing the decoded sequence generated by the decoder in step S41, and adding it to the multi-scale decoded sequence in S41 to obtain a decoded feature vector at each scale.

6. The adaptive empirical mode decomposition prediction method based on multi-scale Transformer according to claim 1, characterized in that: The step S5 comprises the following sub-steps: S51, concatenating the decoded feature vectors at each scale obtained in step S42 in the channel dimension to obtain a feature vector containing global information; S52: Perform a linear transformation on the feature vector containing global information obtained in step S51 to obtain a time series prediction result.