Time series prediction method and system based on Fourier transform and agent self-attention

Abstract

A time series prediction method and system based on Fourier transform and proxy self-attention is constructed, a time series prediction model including a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module is constructed, and a training set obtained based on the historical data of the prediction target is input into the time series prediction model for training. The present invention designs an algorithm model based on the intrinsic characteristics of the time series, which can not only train the prediction model on a large scale, but also has faster convergence performance than the model of the same period, solves the problem of large running memory consumption, has the characteristics of fast prediction speed and strong generalization ability, and its network structure can be applied to weather forecasting, financial analysis, and electric power energy forecasting. It gives full play to the advantages of deep neural networks and has the advantages of simple design, good robustness, high detection accuracy, and high prediction accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction, and specifically relates to a time series prediction method and system based on Fourier transform and proxy self-attention. Background Art

[0002] Time series forecasting technology is widely used in daily production and life. In the retail industry, predicting the expected sales of a certain product can better control inventory; in the advertising field, predicting user preferences and needs through user historical browsing data, and then actively recommending corresponding items to users; in the weather field, predicting weather changes in the next few days based on historical weather data; in the economic field, predicting stock or exchange rate trends through changes in historical economic indices, etc.

[0003] At present, prediction algorithms based on deep learning have achieved remarkable achievements in prediction accuracy. The model architecture of the algorithm mainly includes two categories: recurrent neural network and transformer architecture. Among them, the algorithm based on the recurrent neural network model architecture cannot use parallel computing during training, and when the goal is long-term prediction, errors will accumulate, resulting in a decrease in prediction accuracy; the prediction algorithm based on the transformer architecture avoids the shortcomings of recurrent neural networks, but its memory consumption is too large and the inference cost is high. Therefore, the existing technology has the problems of recurrent neural networks being unable to calculate in parallel, large errors in long-term calculations, and large memory consumption and cost of transformer architecture, which affects the efficiency and accuracy of prediction. Summary of the invention

[0004] The purpose of the present invention is to provide a time series prediction method and system based on Fourier transform and proxy self-attention with higher prediction accuracy in order to solve the above problems existing in the prior art.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, the present invention proposes a time series prediction method based on Fourier transform and proxy self-attention, comprising:

[0007] S1. Constructing a time series prediction model, the model includes a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module;

[0008] S2, inputting the training set obtained based on the historical data of the prediction target into the time series prediction model for training;

[0009] S3, performing time series prediction based on the trained time series prediction model;

[0010] The S2 includes:

[0011] S21, initializing model parameters so that the mean of the model parameters is distributed within a fixed interval;

[0012] S22, through the Fourier convolution processing module, the time series sample signal Convert into frequency domain characteristic signal;

[0013] S23. Add a different position feature matrix w to each time series variable pos ∈R D×L , get the complete data sequence information, D is the length of the convolutional neural network, and L is the length of the time series sample;

[0014] S24, performing proxy self-attention processing on the data sequence information based on the proxy self-attention module to obtain a mapping result;

[0015] S25. Based on the nonlinear gating module, the mapping result is processed nonlinearly using the following formula to obtain the output O of a layer of gated attention:

[0016] OutPut=(OW v )⊙(OW u )

[0017] O = O + RMSnorm (Output);

[0018]

[0019] In the above formula, W v ∈R D×D , W u ∈R D×D are two learnable transformation matrices, O∈R L×D , ⊙ is the Hadamard product, is the i-th variable in the initial time series segment fused with position information;

[0020] S26, regard S23, S24, and S25 as a layer of attention units, repeat the iteration 3 times to form a 3-layer gated attention unit;

[0021] S27, based on the feature fusion and linear prediction module of the time series prediction model, the output feature O∈R of the 3-layer gated attention unit L×D Tiling, the following formula is used to perform feature fusion and linear prediction on the output features:

[0022]

[0023] In the above formula, W p ∈R (L*D)×H is a matrix mapping, H is the predicted sequence length;

[0024] S28, resample the training data set to obtain new time series samples, and return to S22 to loop until the error of the verification set no longer decreases, and the training is completed;

[0025] The S22 includes:

[0026] S221, will Perform discrete Fourier transform and convert it into the following complex domain feature sequence:

[0027]

[0028] In the above formula, is the i-th dimension scalar in time step L, is the real value of the complex imaginary part after Fourier transform;

[0029] S222. Use a convolutional neural network to transform the complex domain feature sequence into D one-dimensional sequences, and merge them to obtain the following sequence:

[0030]

[0031] In the above formula, D is the length of the convolutional neural network, is a one-dimensional vector of length D,

[0032] The S24 includes:

[0033] S241, construct three different mapping matrices W Q , W K ∈R D×D , W V ∈R D×D , processed by matrix multiplication get in, is the i-th scalar of the agent self-attention module dimension;

[0034] S242, use the following formula to calculate Q∈R L×D , K∈R L×D Find the attention score:

[0035]

[0036]

[0037] In the above formula, are two attention score matrices, A∈R n×D is the agent matrix, D is the dimension of the agent self-attention module, and n is a fixed value;

[0038] S243, swap the order of self-attention calculation, so that linear attention becomes

[0039] S244. Use the following formula to add an identity mapping and normalization layer to the residual structure:

[0040]

[0041] The method further comprises:

[0042] S01. Fill in missing values ​​in historical data information through interpolation;

[0043] S02. Normalize the complete historical data;

[0044] S03. Divide the normalized historical data information into the following fixed-length time series sample pairs:

[0045] ((x1,…,x L ), (x L+1 , …, x L+H ));

[0046] In the above formula, (x1, ..., x L ) is the input time series segment, (x L+1 , ..., x L+H ) is a prediction time series segment;

[0047] S04. Perform cluster analysis on the input time series segments and remove outlier time series sample pairs.

[0048] In a second aspect, the present invention proposes a time series prediction system based on Fourier transform and proxy self-attention with high prediction accuracy, including a time series prediction model construction module, a model training module, and a time series prediction module;

[0049] The time series prediction model construction module is used to construct a time series prediction model, and the model includes a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module;

[0050] The model training module is used to input a training set obtained based on historical data of the prediction target into the time series prediction model for training;

[0051] The time series prediction module performs time series prediction based on the trained time series prediction model;

[0052] The model training module includes an initialization parameter unit, a Fourier convolution processing unit, a position information insertion unit, a proxy self-attention unit, a nonlinear gating unit, a feature fusion and a linear prediction unit;

[0053] The initialization parameter unit is used to initialize the model parameters so that the mean values ​​of the model parameters are distributed within a fixed interval;

[0054] The Fourier convolution processing unit is used to convert the time series samples into Convert into frequency domain characteristic signal;

[0055] The position information insertion unit is used to add a different position feature matrix w to each time series variable. pos ∈R D ×L , get the complete data sequence information, D is the length of the convolutional neural network, and L is the length of the time series sample;

[0056] The proxy self-attention unit is used to perform proxy self-attention processing on the data sequence information based on the proxy self-attention module to obtain a mapping result;

[0057] The nonlinear gating unit is used to perform nonlinear processing on the mapping result based on the nonlinear gating module using the following formula to obtain the output O of a layer of gated attention:

[0058] OutPut=(OW v )⊙(OW u )

[0059] O = O + RMSnorm (Output);

[0060]

[0061] In the above formula, W v ∈R D×D , W u ∈R D×D are two learnable transformation matrices, O∈R L×D , ⊙ is the Hadamard product, is the i-th variable in the initial time series segment fused with position information;

[0062] The feature fusion and linear prediction unit is used to combine the feature fusion and linear prediction module based on the time series prediction model with the output feature O∈R of the 3-layer gated attention unit. L×D Tiling, the following formula is used to perform feature fusion and linear prediction on the output features:

[0063]

[0064] In the above formula, W p ∈R (L*D)×H is a matrix mapping, H is the predicted sequence length;

[0065] The Fourier convolution processing unit includes a Fourier transform unit and a convolution unit:

[0066] The Fourier transform unit is used to Perform discrete Fourier transform and convert it into the following complex domain feature sequence:

[0067]

[0068] In the above formula, is the i-th dimension scalar in time step L, is the real value of the complex imaginary part after Fourier transform;

[0069] The convolution unit is used to transform the complex domain feature sequence into D one-dimensional sequences using a convolutional neural network, and merge them to obtain the following sequence:

[0070]

[0071] In the above formula, D is the length of the convolutional neural network, is a one-dimensional vector of length D,

[0072] The proxy self-attention unit includes a mapping matrix unit, an attention score solving unit, an exchange self-attention calculation order unit, and a structure adding unit:

[0073] The mapping matrix unit is used to construct three different mapping matrices W Q , W K ∈R D×D , W V ∈R D×D , processed by matrix multiplication in, is the i-th scalar of the agent self-attention module dimension;

[0074] The attention score solving unit is used to solve Q∈R using the following formula L×D , K∈R L×D Find the attention score:

[0075]

[0076]

[0077] In the above formula, are two attention score matrices, A∈R n×D is the agent matrix, D is the dimension of the agent self-attention module, and n is a fixed value;

[0078] The self-attention calculation order exchange unit is used to exchange the self-attention calculation order so that the linear attention becomes

[0079] The structure adding unit is used to add an identity mapping and a normalization layer to the residual structure using the following formula:

[0080]

[0081] The system also includes a missing value filling unit, a normalization unit, a time series sample pair division unit, and a cluster elimination unit;

[0082] The missing value filling unit is used to fill in the missing values ​​in the historical data information by interpolation;

[0083] The normalization unit is used to perform normalization processing on the complete historical data;

[0084] The time series sample pair division unit is used to divide the normalized historical data information into the following fixed-length time series sample pairs:

[0085] ((x1,…,x L ), (x L+1 , …, x L+H ));

[0086] In the above formula, (x1, ..., x L ) is the input time series segment, (x L+1 , ..., x L+H ) is a prediction time series segment;

[0087] The cluster elimination unit is used to perform cluster analysis on the input time series segments and eliminate outlier time series sample pairs.

[0088] Compared with the prior art, the present invention has the following beneficial effects:

[0089] The present invention proposes a time series prediction method and system based on Fourier transform and proxy self-attention. The method first constructs a time series prediction model including a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module, and then inputs a training set obtained based on historical data of the prediction target into the time series prediction model for training, and then performs time series prediction based on the trained time series prediction model. On the one hand, the method uses Fourier transform to extract frequency domain information, and a learnable convolution layer to process frequency domain information, extract deep time series information, emphasize the periodic signal of the sequence, and reasonably expand the processing dimension. On the other hand, the method The method uses the proxy attention mechanism to reduce the time complexity to linear complexity while maintaining the rank reduction function of the softMax function, greatly improving the scalability of the model. The algorithm model designed based on the intrinsic characteristics of the time series can not only train the prediction model on a large scale, but also has faster convergence performance than the model of the same period, solving the problem of large running memory consumption. It has the characteristics of fast prediction speed and strong generalization ability. Its network structure is not limited to weather forecasting, financial analysis, power energy forecasting, etc. It gives full play to the advantages of deep neural networks and has the advantages of simple design, good robustness, high detection accuracy and high prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0091] Figure 2 The structure diagram of the system of the present invention is shown in FIG. DETAILED DESCRIPTION

[0092] The present invention is further described in detail below in conjunction with specific implementations and drawings.

[0093] The present invention discloses a time series prediction method and system based on Fourier transform and proxy self-attention. The method performs time series prediction based on historical data of the prediction target by training a time series prediction model including a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module, so as to solve the problems of insufficient algorithm accuracy, insufficient training speed and excessive resource consumption in the current time series prediction algorithm.

[0094] Embodiment 1:

[0095] This embodiment takes the user's electricity load as the research object and performs time series prediction on the commonly used user electricity load in the field. The training set and the test set are divided in a ratio of 8:2, the time granularity is 60 minutes, and the total length of the continuous time step is 26,304. The experimental training and test data features are composed of the historical 10-day electricity consumption data of 321 target users and the electricity consumption data of the next 3 days. After the training model, the prediction of the user's electricity consumption data for the next day is obtained.

[0096] like Figure 1 As shown, the time series prediction method based on Fourier transform and agent self-attention is carried out in the following steps:

[0097] 1. Historical data cleaning:

[0098] Obtain the historical data information of the prediction target, use the nearest neighbor interpolation method and the single linear interpolation method to fill in the missing values ​​in the historical data; decompose the historical data into pairs of fixed-length time series samples ((x1,...,x L ), (x L+1 , ..., x L+H )),(x1,...,x L ) is the model input time series segment, (x L+1 , ..., x L+H ) is the prediction segment, that is, the output data of the model; the k-mean method is used to compare (x1, ..., x L ) fragments for cluster analysis, and remove the data points whose cluster distance exceeds a certain threshold and whose proportion is less than 5%, that is, screen out the outliers ((x1,...,x L ), (x L+1 , ..., x L+H )) sample pairs, in order to increase the generalization of the trained model, and then obtain multiple time series sample pairs ((x1,...,x L ), (x L+1 , ..., x L+H )), divided into training set and validation set in the ratio of 8:2.

[0099] 2. Construct a time series prediction model that includes Fourier convolution processing module, agent self-attention module, nonlinear gating module, feature fusion and linear prediction module.

[0100] 3. Initialize the model parameters so that the mean of the model parameters is distributed within a fixed interval.

[0101] 4. Batch sample time series sample pairs ((x1,...,x L ), (x L+1 , ..., x L+H ))Do normalization:

[0102]

[0103] In the above formula, x mean For x i The mean value of x var For x i The variance of x i is the i-th value in the timing window length L, The data is of length L after normalization. The normalization layer is used to stabilize the input distribution of the model. Record x i The mean and variance of the predicted time series segment (x L+1 , ..., x L+H ) to mitigate the distribution shift of non-stationary time series data and enhance prediction accuracy.

[0104] 5. Through the Fourier convolution processing module, the time series sample signal Convert it into frequency domain feature signals to enhance the periodic feature representation of the sequence and expand the feature dimension, including:

[0105] Will Perform discrete Fourier transform and convert it into the following complex domain feature sequence:

[0106]

[0107] In the above formula, is the i-th dimension scalar in time step L, is the real value of the complex imaginary part after Fourier transform;

[0108] The complex domain feature sequence is transformed into D one-dimensional sequences using D convolutional neural networks with a length of 2 and a sliding distance of 2, and the following sequence is obtained by merging:

[0109]

[0110] In the above formula, D is the length of the convolutional neural network, is a one-dimensional vector of length D,

[0111] 6. Add a different position feature matrix w to each time series variable pos ∈R D×L , get the complete data sequence information, D is the length of the convolutional neural network, and L is the length of the time series sample.

[0112] 7. Based on the proxy self-attention module, the data sequence information is processed by proxy self-attention to obtain the mapping results, including:

[0113] Construct three different mapping matrices W Q , W K ∈R D×D , W v ∈R D×D , processed by matrix multiplication get In the middle of Ju is the i-th scalar of the agent self-attention module dimension;

[0114] Q∈R is calculated by the following formula:L×D , K∈R L×D Find the attention score:

[0115]

[0116]

[0117] In the above formula, are two attention score matrices, A∈R n×D is the proxy matrix, D is the dimension of the proxy self-attention module, n is a fixed value, and the proxy attention mechanism is used to reduce the original O(L 2 )The time complexity is reduced to O(L) linear complexity;

[0118] Swap the order of self-attention calculation to make linear attention become Reduced attention complexity;

[0119] The following formula is used to add identity mapping and normalization layers to the residual structure:

[0120]

[0121] Using the RMSNorm normalization method can alleviate the problems of gradient explosion, gradient disappearance and training instability in deep learning models.

[0122] 8. Based on the nonlinear gating module, the following formula is used to perform nonlinear processing on the mapping result to obtain the output O of one layer of gated attention:

[0123] OutPut=(OW v )⊙(OW u )

[0124] O = O + RMSnorm (Output);

[0125]

[0126] In the above formula, W v ∈R D×D , W u ∈R D×D are two learnable transformation matrices, O∈R L×D , ⊙ is the Hadamard product, This is the i-th variable in the initial time sequence segment that incorporates the position information; this operation omits all activation operations and uses gating operations to obtain nonlinear transformations, achieving the function of the original transformer feedforward layer. The use of full gating operations ensures efficient parallel capabilities and GPU acceleration capabilities.

[0127] 9. Consider steps 6, 7, and 8 as one layer of attention unit and repeat the iteration three times to form a three-layer gated attention unit.

[0128] 10. Based on the feature fusion and linear prediction module of the time series prediction model, the output feature O∈R of the 3-layer gated attention unit is converted into L×D Tiling, using a matrix mapping W p ∈R (L*D)×H , mapping the prediction results The following formula is used to perform feature fusion and linear prediction on the output features:

[0129]

[0130] In the above formula, W p ∈R (L*D)×H is a matrix mapping, and H is the predicted sequence length.

[0131] 11. Map out the prediction results and restore the statistical variables obtained in step 4:

[0132]

[0133] 12. Calculate the model parameter gradient according to the loss function, and update the model parameters in reverse order:

[0134]

[0135] By calculating the error and using a small batch sample gradient descent algorithm, the optimization target is approached iteratively, and a special optimization algorithm is used to speed up the training.

[0136] 13. Resample the training data set to obtain new time series samples and return to step 4 to loop until the error of the validation set no longer decreases and the training is completed.

[0137] Embodiment 2:

[0138] The process is the same as in Example 1, except that:

[0139] This example uses weather temperature as the research object, and performs time series forecasting on the commonly used weather temperature in the field. The training set and the test set are divided in a ratio of 8:2, and the time granularity is 10 minutes. The total length of the continuous time step is 52,696. The experimental training and test data features are composed of historical 3-day temperature data and future 1-day temperature data of 21 regions. After training the model, the temperature forecast for the next 1 day is obtained.

[0140] Embodiment 3:

[0141] The process is the same as in Example 1, except that:

[0142] This example uses traffic flow as the research object, and performs time series prediction on the commonly used traffic flow in the field. The training set and the test set are divided in a ratio of 8:2, and the time granularity is 60 minutes. The total length of the continuous time step is 17544. The experimental training and test data features are composed of 10-day historical traffic flow data of 862 intersections and traffic flow data for the next 3 days. After training the model, the traffic flow prediction for the next 3 days is obtained.

[0143] Embodiment 4:

[0144] The process is the same as in Example 1, except that:

[0145] This example uses the national daily exchange rate as the research object, and performs time series forecasting on the daily exchange rates of eight countries commonly used in the field. The time length is from 1990 to 2016, and the training set and the test set are divided in a ratio of 8:2, and the time granularity is 24 hours. The total length of the continuous time step is 7586. The experimental training and test data features are composed of 336-day exchange rate data of eight countries and exchange rate data for the next 96 days. After training the model, the exchange rate forecast for the next 96 days is obtained.

[0146] Embodiment 5:

[0147] like Figure 2 As shown, the time series prediction system based on Fourier transform and agent self-attention includes a data cleaning module, a time series prediction model building module, a model training module, and a time series prediction module.

[0148] The data cleaning module includes a missing value filling unit, a normalization unit, a time series sample pair division unit, and a cluster elimination unit. The missing value filling unit is used to obtain the historical data information of the prediction target and use the nearest neighbor interpolation method and the single linear interpolation method to fill the missing values ​​in the historical data; the normalization unit is used to normalize the complete historical data; the time series sample pair division unit is used to divide the normalized historical data information into the following fixed-length time series sample pairs:

[0149] ((x1, ..., x L ), (x L+1 , …, x L+H ));

[0150] In the above formula, (x1, ..., x L ) is the input time series segment, (x L+1 , ..., x L+H ) is the predicted time series segment, i.e., the output data of the model;

[0151] The cluster elimination unit is used to use the k-mean method to remove (x1, ..., x L) fragments for cluster analysis, and remove the data points whose cluster distance exceeds a certain threshold and whose proportion is less than 5%, that is, screen out the outliers ((x1,...,x L ), (x L+1 , ..., x L+H )) sample pairs, in order to increase the generalization of the trained model, and then obtain multiple time series sample pairs ((x1,...,x L ), (x L+1 ..., x L+H )), divided into training set and validation set in the ratio of 8:2.

[0152] The time series prediction model construction module is used to construct a time series prediction model, and the model includes a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module.

[0153] The model training module is used to input the training set obtained based on the historical data of the prediction target into the time series prediction model for training, including an initialization parameter unit, a Fourier convolution processing unit, a position information insertion unit, a proxy self-attention unit, a nonlinear gating unit, a feature fusion and a linear prediction unit.

[0154] The initialization parameter unit is used to initialize the model parameters so that the mean values ​​of the model parameters are distributed within a fixed interval.

[0155] The Fourier convolution processing unit is used to convert the time series samples into Converts the frequency domain characteristic signal into a Fourier transform unit and a convolution unit. The Fourier transform unit is used to convert Perform discrete Fourier transform and convert it into the following complex domain feature sequence:

[0156]

[0157] In the above formula, is the i-th dimension scalar in time step L, is the real value of the complex imaginary part after Fourier transform;

[0158] The convolution unit is used to transform the complex domain feature sequence into D one-dimensional sequences using a convolutional neural network, and merge them to obtain the following sequence:

[0159]

[0160] In the above formula, D is the length of the convolutional neural network, is a one-dimensional vector of length D,

[0161] The position information insertion unit is used to add a different position feature matrix w to each time series variable. pos ∈R D ×L , get the complete data sequence information, D is the length of the convolutional neural network, and L is the length of the time series sample.

[0162] The proxy self-attention unit is used to perform proxy self-attention processing on the data sequence information based on the proxy self-attention module to obtain a mapping result, including a mapping matrix unit, an attention score solving unit, an exchange self-attention calculation order unit, and a structure adding unit. The mapping matrix unit is used to construct three different mapping matrices W Q , W K ∈R D×D , W V ER D ×D , processed by matrix multiplication get in, is the i-th scalar of the agent self-attention module dimension;

[0163] The attention score solving unit is used to solve Q∈R using the following formula L×D , K∈R L×D Find the attention score:

[0164]

[0165]

[0166] In the above formula, are two attention score matrices, A∈R n×D is the agent matrix, D is the dimension of the agent self-attention module, and n is a fixed value;

[0167] The self-attention calculation order exchange unit is used to exchange the self-attention calculation order so that the linear attention becomes

[0168] The structure adding unit is used to add an identity mapping and a normalization layer to the residual structure using the following formula:

[0169]

[0170] The nonlinear gating unit is used to perform nonlinear processing on the mapping result based on the nonlinear gating module using the following formula to obtain the output O of a layer of gated attention:

[0171] OutPut=(OW v )⊙(OW u )

[0172] O = O + RMSnorm (Output);

[0173]

[0174] In the above formula, W v ∈R D×D , W u ∈R D×D are two learnable transformation matrices, O∈R L×D , ⊙ is the Hadamard product, is the i-th variable in the initial time series segment that incorporates the position information.

[0175] The feature fusion and linear prediction unit is used to combine the feature fusion and linear prediction module based on the time series prediction model with the output feature O∈R of the 3-layer gated attention unit. L×D Tiling, the following formula is used to perform feature fusion and linear prediction on the output features:

[0176]

[0177] In the above formula, W p ∈R (L*D)×H is a matrix mapping, and H is the predicted sequence length.

[0178] The time series prediction module performs time series prediction based on the trained time series prediction model.

Claims

1. A time series prediction method based on Fourier transform and proxy self-attention, characterized in that: The method comprises: S1. Constructing a time series prediction model, the model includes a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module; S2, inputting the training set obtained based on the historical data of the prediction target into the time series prediction model for training; S3, performing time series prediction based on the trained time series prediction model; The S2 includes: S21, initializing model parameters so that the mean of the model parameters is distributed within a fixed interval; S22, through the Fourier convolution processing module, the time series sample signal Convert into frequency domain characteristic signal; S23. Add a different position feature matrix w to each time series variable pos ∈R D×L , get the complete data sequence information, D is the length of the convolutional neural network, and L is the length of the time series sample; S24, performing proxy self-attention processing on the data sequence information based on the proxy self-attention module to obtain a mapping result; S25. Based on the nonlinear gating module, the mapping result is processed nonlinearly using the following formula to obtain the output O of a layer of gated attention: OutPut=(OW v )⊙(OW u ); O = O + RMSnorm (Output); In the above formula, OutPut is the output of the agent self-attention module, W v ∈R D×D , W u ∈R D×D are two learnable transformation matrices, O∈R L×D , ⊙ is the Hadamard product, is the i-th variable in the initial time series segment that incorporates the position information; S26, regard S23, S24, and S25 as a layer of attention units, repeat the iteration 3 times to form a 3-layer gated attention unit; S27, based on the feature fusion and linear prediction module of the time series prediction model, the output feature O∈R of the 3-layer gated attention unit L×D Tiling, the following formula is used to perform feature fusion and linear prediction on the output features: In the above formula, W p ∈R (L*D)×H is a matrix mapping, H is the predicted sequence length; S28, resample the training data set to obtain new time series samples, and return to S22 to loop until the error of the verification set no longer decreases, and the training is completed; The S22 includes: S221, will Perform discrete Fourier transform and convert it into the following complex domain feature sequence: In the above formula, is the i-th dimension scalar in time step L, is the real value of the complex imaginary part after Fourier transform; S222. Use a convolutional neural network to transform the complex domain feature sequence into D one-dimensional sequences, and merge them to obtain the following sequence: In the above formula, D is the length of the convolutional neural network, is a one-dimensional vector of length D, The S24 includes: S241, construct three different mapping matrices W Q , W K ∈R D×D , W V ∈R D×D , processed by matrix multiplication get in, is the i-th scalar of the agent self-attention module dimension; S242, use the following formula to calculate Q∈R L×D , K∈R L×D Find the attention score: In the above formula, are two attention score matrices, A∈R n×D is the agent matrix, D is the dimension of the agent self-attention module, and n is a fixed value; S243, swap the order of self-attention calculation, so that linear attention becomes S244. Use the following formula to add an identity mapping and normalization layer to the residual structure:

2. The time series prediction method based on Fourier transform and proxy self-attention according to claim 1, characterized in that: The method further comprises: S01. Fill in missing values ​​in historical data information through interpolation; S02. Normalize the complete historical data; S03. Divide the normalized historical data information into the following fixed-length time series sample pairs: ((x1,...,x L ),(x L+1 ,...,x L+H )); In the above formula, (x1, ..., x L ) is the input time series segment, (x L+1 , ..., x L+H ) is a prediction time series segment; S04. Perform cluster analysis on the input time series segments and remove outlier time series sample pairs.

3. A time series prediction system based on Fourier transform and proxy self-attention, characterized in that: The system includes a time series prediction model building module, a model training module, and a time series prediction module; The time series prediction model construction module is used to construct a time series prediction model, and the model includes a Fourier convolution processing module, a proxy self-attention module, a nonlinear gating module, a feature fusion and a linear prediction module; The model training module is used to input a training set obtained based on historical data of the prediction target into the time series prediction model for training; The time series prediction module performs time series prediction based on the trained time series prediction model; The model training module includes an initialization parameter unit, a Fourier convolution processing unit, a position information insertion unit, a proxy self-attention unit, a nonlinear gating unit, a feature fusion and a linear prediction unit; The initialization parameter unit is used to initialize the model parameters so that the mean values ​​of the model parameters are distributed within a fixed interval; The Fourier convolution processing unit is used to convert the time series samples into Convert into frequency domain characteristic signal; The position information insertion unit is used to add a different position feature matrix w to each time series variable. pos ∈R D×L , get the complete data sequence information, D is the length of the convolutional neural network, and L is the length of the time series sample; The proxy self-attention unit is used to perform proxy self-attention processing on the data sequence information based on the proxy self-attention module to obtain a mapping result; The nonlinear gating unit is used to perform nonlinear processing on the mapping result based on the nonlinear gating module using the following formula to obtain the output O of a layer of gated attention: OutPut=(OW v )⊙(OW u ); O = O + RMSnorm (Output); In the above formula, OutPut is the output of the agent self-attention module, W v ∈R D×D , W u ∈R D×D are two learnable transformation matrices, O∈R L×D , ⊙ is the Hadamard product, is the i-th variable in the initial time series segment that incorporates the position information; The feature fusion and linear prediction unit is used to combine the feature fusion and linear prediction module based on the time series prediction model with the output feature O∈R of the 3-layer gated attention unit. L×D Tiling, the following formula is used to perform feature fusion and linear prediction on the output features: In the above formula, W p ∈R (L×D)×H is a matrix mapping, H is the predicted sequence length; The Fourier convolution processing unit includes a Fourier transform unit and a convolution unit: The Fourier transform unit is used to Perform discrete Fourier transform and convert it into the following complex domain feature sequence: In the above formula, is the i-th dimension scalar in time step L, is the real value of the complex imaginary part after Fourier transform; The convolution unit is used to transform the complex domain feature sequence into D one-dimensional sequences using a convolutional neural network, and merge them to obtain the following sequence: In the above formula, D is the length of the convolutional neural network, is a one-dimensional vector of length D, The proxy self-attention unit includes a mapping matrix unit, an attention score solving unit, an exchange self-attention calculation order unit, and a structure adding unit: The mapping matrix unit is used to construct three different mapping matrices W Q , W K ∈R D×D , W v ∈R D×D , processed by matrix multiplication in, is the i-th scalar of the agent self-attention module dimension; The attention score solving unit is used to solve Q∈R using the following formula L×D , K∈R L×D Find the attention score: In the above formula, are two attention score matrices, A∈R n×D is the agent matrix, D is the dimension of the agent self-attention module, and n is a fixed value; The self-attention calculation order exchange unit is used to exchange the self-attention calculation order so that the linear attention becomes The structure adding unit is used to add an identity mapping and a normalization layer to the residual structure using the following formula:

4. The time series prediction system based on Fourier transform and proxy self-attention according to claim 3, characterized in that: The system also includes a missing value filling unit, a normalization unit, a time series sample pair division unit, and a cluster elimination unit; The missing value filling unit is used to fill in the missing values ​​in the historical data information by interpolation; The normalization unit is used to perform normalization processing on the complete historical data; The time series sample pair division unit is used to divide the normalized historical data information into the following fixed-length time series sample pairs: ((x1,...,x L ),(x L+1 ,...,x l+H )); In the above formula, (x1, ..., x L ) is the input time series segment, (x L+1 , ..., x L+H ) is a prediction time series segment; The cluster elimination unit is used to perform cluster analysis on the input time series segments and eliminate outlier time series sample pairs.

Images