Multi-scale double-branch long-time sequence prediction method based on frequency domain

Abstract

The invention discloses a multi-scale double-branch long-time sequence prediction method based on a frequency domain. Comprising the following steps: extracting frequency domain characteristics of rich scales by realizing a multi-scale segmentation mechanism in a frequency domain, capturing global and local characteristics of each scale at the same time by adopting a double-branch architecture strategy, providing a new loss function insensitive to abnormal values, training a neural network through the loss function, and obtaining an abnormal value of the neural network. And a more real prediction result can be obtained. According to the method, a multi-scale fragment segmentation module, a double-branch information extractor and a loss function insensitive to abnormal values are combined, the compactness of a time sequence expressed in a frequency domain is fully considered, and the limitation that an existing prediction method is single in frequency domain resolution and generally only considers global or local single information is overcome; the method has important theoretical and practical significance in the field of long-time sequence prediction.

Description

Technical Field

[0001] The invention belongs to time series prediction, in particular to a multi-scale dual-branch long time series prediction method based on frequency domain. Background Art

[0002] Time series forecasting is a long-standing and vital task. Its core lies in extracting useful patterns from historical observation data to predict future events and trends. Accurate time series forecasting can provide important forward-looking information for organizations and individuals in many fields, thereby supporting more effective decision-making. In particular, long-term series forecasting can reveal data change trends over a long period of time and has broad application prospects. For example, in the financial market, long-term series forecasting can help investors identify long-term trends and optimize investment strategies; in meteorology, accurate long-term weather forecasts can provide valuable support for agricultural production and disaster prevention and mitigation; in energy management, predicting future energy demand can help power companies optimize grid operations and resource allocation. Therefore, accurate forecasting results are particularly critical for decision support systems.

[0003] Extracting implicit information from historical time series is quite complex, and existing methods mainly focus on extracting time domain features. Due to the complex dependency characteristics of time series and the sparse representation of information in the time domain, traditional methods often use complex network structures and modules such as multi-head self-attention to capture and utilize this information. These methods often involve a large number of parameters and calculations, resulting in long training and inference times. Relatively speaking. In the frequency domain, time series have a more compact representation, and frequency domain methods directly facilitate the identification and analysis of basic frequency domain components, and can effectively capture long-term trends, seasonal patterns, and cyclical fluctuations in time series.

[0004] However, existing frequency-domain-based methods all use limited frequency-domain resolution, which may not be sufficient to simultaneously capture multiple important frequency components in a time series, especially when the series contains multiple dynamics ranging from low-frequency long-term trends to high-frequency short-term fluctuations. On the other hand, many models either focus on long-term predictions of global trends and ignore the importance of local fluctuations; or pay too much attention to short-term data changes and fail to capture the key periodic features that drive the long-term behavior of time series. This leads to an incomplete understanding of the dynamics of time series. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention discloses a multi-scale dual-branch long time series prediction method based on frequency domain, which is a multi-resolution frequency domain model that simultaneously models global and local features, thereby improving the accuracy of long time series prediction. First, the time series data is divided into time segments of different lengths according to different time scales, and the frequency domain representation of each time segment is obtained by Fourier transform. Then, the long-term periodic information and short-term local dependency of the time series are respectively obtained by using a dual-branch information extractor. Then, the extracted feature segments are spliced ​​back to the length of the original sequence, and the frequency domain representation of the historical window is converted into the frequency domain representation of the future window through a complex fully connected layer, and finally the time series prediction result is obtained by inverse Fourier transform. In addition, the present invention proposes a new loss function that is insensitive to outliers. By training a neural network with this loss function, more realistic prediction results can be obtained. The proposed method fully considers the compactness of the time series representation in the frequency domain, overcomes the limitation that the existing prediction methods usually only consider global or local single information, and has important theoretical and practical significance for the field of long time series prediction.

[0006] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0007] The multi-scale dual-branch long time series prediction method based on the frequency domain performs the following steps to build and optimize the frequency domain prediction model so that it can be used to predict the historical long-term observation sequence data. The method includes:

[0008] Step 1: Collect long-term historical observation sequence data and normalize them in batches;

[0009] Step 2: Divide the observed sequence data of each attribute variable into data combinations of multiple time segments of different scales;

[0010] Step 3: Convert the observed sequence data from the time domain to the frequency domain;

[0011] Step 4: Construct a dual-branch information extractor to process and predict the frequency domain representation of each scale;

[0012] Step 5: Weighted average the prediction results of all scales to generate a complete prediction result;

[0013] Step 6: Denormalize the prediction results;

[0014] Step 7: Repeat steps 2-6 above, select the Adam optimizer, use the integral function of the arctan function as the loss function, adjust the model parameters through back propagation, and optimize the frequency domain prediction model.

[0015] The time series data is long time series data of multiple attribute variables related to the same industrial process.

[0016] The data normalization adopts the z standardization method, and the processed data distribution is a standard normal distribution.

[0017] The scale inequality is to set the time scale set D = {D1, D2, ..., D i ,…,D M}, where each time scale corresponds to a time segment of different length.

[0018] The frequency domain transformation is to use Fast Fourier Transform (FFT) for each time segment.

[0019] The dual-branch information extractor includes: extracting temporal information within and between historical time segments by iteratively using an intra-segment fully connected layer, an inter-segment information extractor, and an intra-segment fully connected layer to generate a deep frequency domain feature representation.

[0020] The processing of the frequency domain representation of each scale by using a dual-branch information extractor includes:

[0021] a. Use the intra-fragment complex fully connected layer to map the frequency domain representation of each scale segment to the implicit feature space, and obtain in represents the output result of the first complex fully connected layer in the i-th scale segment;

[0022] b. Use the inter-fragment complex fully connected layer to mine the global period information and obtain in represents the processing result of the i-th scale global information extractor;

[0023] c. Use the inter-fragment complex convolutional layer to mine short-term temporal dynamic changes and obtain in represents the processing result of the i-th scale local information extractor;

[0024] d. Use the intra-fragment complex fully connected layer again to refine and strengthen the frequency domain feature expression of each fragment, and obtain the global information respectively and local information in represents the output result of the i-th scale global information extractor, Represents the output result of the i-th scale local information extractor.

[0025] The predicting of the frequency domain representation of each scale includes:

[0026] a. Convert the frequency domain representation of each segment into the time domain;

[0027] b. Splice the temporal deep features of each segment, and the length after splicing becomes the length of the original input sequence;

[0028] c. Convert the processed time series to the frequency domain through FFT to obtain the splicing result of the global information time domain features The splicing result of local information time domain features

[0029] d. Through the complex fully connected layer MLP, the global information and local information deep frequency domain representation of the historical time window are predicted to obtain the frequency domain representation of the future time window;

[0030] e. Convert the frequency domain representation of the global information and local information prediction results to the time domain to obtain the prediction results based on global information Prediction results based on local information

[0031] f. Average the global information and local information prediction results to obtain the prediction result at this scale.

[0032] The denormalization process performs z-denormalization using the mean and variance of the z-normalization used in step 1.

[0033] The model parameter update process uses ArcTanLoss: the integral function and of the arctan function, plus the mean absolute error MAE and a regularization constraint term. The specific function form is:

[0034]

[0035] where Y and Represent the true value and predicted value respectively. a, k, α, β, and γ are hyperparameters, which are obtained through grid search method.

[0036] The present invention has the following beneficial effects and advantages:

[0037] 1. Compared with the existing optimal method in long time series prediction tasks, the method of the present invention uses a multi-scale segment division method to make frequency domain features of different scales complement each other.

[0038] 2. The method of the present invention adopts a global-local dual-branch extractor to simultaneously model global periodic information and local dynamic relationships to obtain prediction results with more sufficient information.

[0039] 3. In the process of model training, the method of the present invention uses the ArcTanLoss loss function to guide parameter update, which improves the effect of neural network training. In addition, the method has the advantages of small number of parameters and fast prediction in a short time. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention is a flow chart of the method.

[0041] Figure 2 This is a flow chart of the dual-branch global-local information extractor of the present invention.

[0042] Figure 3 It is a function curve diagram of the loss function of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation method of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0045] like Figure 1 As shown, it is a flow chart of the multi-scale dual-branch long time series prediction method based on frequency domain of the present invention. The programming language used in the program execution step of the present invention is not limited to MATLAB, Python, etc. The time series data can be industrial historical data of transformer working scenes (such as 336 hours of data of 7 variables such as transformer oil temperature and effective load), weather historical data (air pressure, wind speed, wind direction, precipitation, air pressure, saturated water vapor pressure difference, steam pressure, temperature, humidity 720 hours of data) long-term sampling historical data, processed, predicted mining to obtain various attribute data at a certain moment or a certain time period in the future.

[0046] The specific steps of the present invention are as follows:

[0047] Step 1: Normalize the time series data in batches using the z-standardization method. The formula for z-standardization is:

[0048]

[0049] Where x is the original data of the current batch, μ is the mean of the data of the current batch, and σ is the standard deviation of the data of the current batch. The processed data should obey the standard normal distribution.

[0050] Step 2: For each variable, divide it into multiple time segments of different lengths according to different time scales.

[0051] Step 2-1: Set and select an appropriate time scale set D = {D1, D2, ..., D M}, each time scale D i Corresponding to time segments of different lengths. For example, the length of the time segment corresponding to D1 is 10 time sampling units, the length of the time segment corresponding to D2 is 20 time sampling units, the length of the time segment corresponding to D3 is 50 time sampling units, and so on.

[0052] Step 2-2: For each variable, calculate the time scale D i For example, if D1 = 10, the time series is divided into segments of length 10 (corresponding to 10 sampling points); if D2 = 20, it is divided into segments of length 20 (corresponding to 20 sampling points), and so on, to generate segments of different time lengths.

[0053] Step 3: Convert the time series data from the time domain to the frequency domain to obtain the frequency domain representation of each time segment, which includes amplitude and phase information.

[0054]

[0055] where x i [n] is the i-th time series segment, N is the length of the time series segment, X is the frequency domain transform result of the time series segment, j is an imaginary unit, e is a natural constant, and k is a frequency index, which represents the k-th frequency component in the frequency domain.

[0056] Step 4: Use a dual-branch information extractor to process the frequency domain representation of each scale.

[0057] Step 4-1, a. For multiple segments of input data at each scale, use an intra-segment complex fully connected layer to map the frequency domain representation of each segment to an implicit feature space.

[0058]

[0059] in represents the output of the first complex fully connected layer in the i-th scale segment, X i represents the combination of time segments of the i-th scale. FLinear(·) represents a complex fully connected layer.

[0060] b. Apply a complex fully connected layer to the implicit feature representation of all clips to mine global periodic information. The complex fully connected layer used here shares the weights of each frequency component.

[0061]

[0062] in represents the processing result of the i-th scale global information extractor, FLinear(·) represents the complex fully connected layer, and Transpose(·) is the tensor transposition operation.

[0063] c. Use complex convolutional layers to process the implicit feature representation of the clip to mine short-term temporal dynamic changes. The complex convolutional layers used here share weights for each frequency component.

[0064]

[0065] in represents the processing result of the i-th scale local information extractor, FLinear(·) represents the complex fully connected layer, and Transpose(·) is the tensor transposition operation.

[0066] d. Use the intra-fragment complex fully connected layer again to refine and strengthen the frequency domain feature expression of each fragment to obtain global information and local information respectively.

[0067]

[0068] in represents the output result of the i-th scale global information extractor, Represents the output result of the i-th scale local information extractor.

[0069] Step 4-2: a. Convert the frequency domain representation of each segment into the time domain.

[0070] b. Concatenate the time-domain deep features of each segment, and the concatenated length becomes the length of the original input sequence.

[0071] c. Convert the processed time series to the frequency domain through FFT;

[0072]

[0073] in represents the concatenation result of the temporal features of the global information at the i-th scale, represents the concatenation result of the temporal features of the local information at the i-th scale, and Concat(·) represents the vector concatenation operation.

[0074] d. Through the complex fully connected layer, the deep frequency domain representation of the global information and local information of the historical time window is converted into the frequency domain representation of the future time window.

[0075] e. Convert the frequency domain representation of global information and local information prediction results to the time domain

[0076]

[0077] in represents the prediction result of the i-th scale based on global information, represents the prediction result of the i-th scale based on local information, and IFFT(·) represents the inverse Fourier transform.

[0078] f. Average the global information and local information prediction results to get the prediction result at this scale

[0079]

[0080] where Y i Represents the prediction result of the i-th scale.

[0081] Step 5: Weighted average the prediction results of all scales to form a complete prediction result.

[0082]

[0083] Step 6: Denormalize the prediction results;

[0084] =Y′·σ+μ

[0085] Where σ and μ are the standard deviation and mean calculated in step 1, respectively.

[0086] Step 7: Repeat steps 2-6 and use the Adam optimizer to optimize the model parameters. The parameter update process uses ArcTanLoss, which is based on the integral function of the arctan function, plus the mean absolute error (MAE) and a regularization constraint term. The specific function form is:

[0087]

[0088] where Y and Represent the true value and predicted value respectively. a, k, α, β, and γ are hyperparameters, which are obtained through grid search method.

[0089] The embodiments described in the above description will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

Claims

1. A multi-scale dual-branch long time series prediction method based on frequency domain, characterized in that: Perform the following steps to build and optimize the frequency domain prediction model so that it can be used to predict historical long-term observation sequence data. The method includes: Step 1: Collect long-term historical observation sequence data and normalize them in batches; Step 2: Divide the observed sequence data of each attribute variable into data combinations of multiple time segments of different scales; Step 3: Convert the observed sequence data from the time domain to the frequency domain; Step 4: Construct a dual-branch information extractor to process and predict the frequency domain representation of each scale; Step 5: Weighted average the prediction results of all scales to generate a complete prediction result; Step 6: Denormalize the prediction results; Step 7: Repeat steps 2-6 above, select the Adam optimizer, use the integral function of the arctan function as the loss function, adjust the model parameters through back propagation, and optimize the frequency domain prediction model.

2. The multi-scale dual-branch long time series prediction method based on frequency domain according to claim 1 is characterized in that: The time series data is long time series data of multiple attribute variables related to the same industrial process.

3. The multi-scale dual-branch long time series prediction method based on frequency domain according to claim 1 is characterized in that: The data normalization adopts the z standardization method, and the processed data distribution is a standard normal distribution.

4. The multi-scale dual-branch long time series prediction method based on frequency domain according to claim 1 is characterized in that: The scale inequality is to set the time scale set D = {D1, D2, ..., D i ,…,D M }, where each time scale corresponds to a time segment of different length.

5. The multi-scale dual-branch long time series prediction method based on frequency domain according to claim 1 is characterized in that: The frequency domain transformation is to use Fast Fourier Transform (FFT) for each time segment.

6. The multi-scale dual-branch long time series prediction method based on frequency domain according to claim 1 is characterized in that: The dual-branch information extractor includes: extracting temporal information within and between historical time segments by iteratively using an intra-segment fully connected layer, an inter-segment information extractor, and an intra-segment fully connected layer to generate a deep frequency domain feature representation.

7. The frequency-domain-based multi-scale dual-branch long time series prediction method according to claim 6, characterized in that: The processing of the frequency domain representation of each scale by using a dual-branch information extractor includes: a. Use the intra-fragment complex fully connected layer to map the frequency domain representation of each scale segment to the implicit feature space, and obtain in represents the output result of the first complex fully connected layer in the i-th scale segment; b. Use the inter-fragment complex fully connected layer to mine the global period information and obtain in represents the processing result of the i-th scale global information extractor; c. Use the inter-fragment complex convolutional layer to mine short-term temporal dynamic changes and obtain in represents the processing result of the i-th scale local information extractor; d. Use the intra-fragment complex fully connected layer again to refine and strengthen the frequency domain feature expression of each fragment, and obtain the global information respectively and local information in represents the output result of the i-th scale global information extractor, Represents the output result of the i-th scale local information extractor.

8. The frequency-domain-based multi-scale dual-branch long time series prediction method according to claim 6, characterized in that: The predicting of the frequency domain representation of each scale includes: a. Convert the frequency domain representation of each segment into the time domain; b. Splice the temporal deep features of each segment, and the length after splicing becomes the length of the original input sequence; c. Convert the processed time series to the frequency domain through FFT to obtain the splicing result of the global information time domain features The splicing result of local information time domain features d. Through the complex fully connected layer MLP, the global information and local information deep frequency domain representation of the historical time window are predicted to obtain the frequency domain representation of the future time window; e. Convert the frequency domain representation of the global information and local information prediction results to the time domain to obtain the prediction results based on global information Prediction results based on local information f. Average the global information and local information prediction results to obtain the prediction result at this scale.

9. The multi-scale dual-branch long time series prediction method based on frequency domain according to claim 1, characterized in that: The denormalization process performs z-denormalization using the mean and variance of the z-normalization used in step 1.

10. The multi-scale dual-branch long time series prediction method based on frequency domain according to claim 1, characterized in that: The model parameter update process uses ArcTanLoss: the integral function and of the arctan function, plus the mean absolute error MAE and a regularization constraint term. The specific function form is: where Y and Represent the true value and predicted value respectively. a, k, α, β, and γ are hyperparameters, which are obtained through grid search method.

Images