Frequency domain based multi-scale dual-branch long time series prediction method

Abstract

The application discloses a multi-scale double-branch long time sequence prediction method based on a frequency domain. The method comprises the following steps: extracting rich scale frequency domain features by realizing a multi-scale segmentation mechanism in the frequency domain; adopting a double-branch architecture strategy to simultaneously capture global and local characteristics of each scale; in addition, a new loss function insensitive to abnormal values is proposed, and a neural network is trained through the loss function, so that a more real prediction result can be obtained. The application combines a multi-scale segment division module, a double-branch information extractor and the loss function insensitive to abnormal values, fully considers the compactness of the time sequence in the frequency domain, overcomes the limitations of the single frequency domain resolution and the limitation of usually only considering global or local single information of the existing prediction method, and has important theoretical and practical significance for the long time sequence prediction field.

Description

Technical Field

[0001] This invention pertains to time series prediction, specifically a multi-scale, two-branch long-time series prediction method based on the frequency domain. Background Technology

[0002] Time series forecasting is a long-standing and crucial task. Its core lies in extracting useful patterns from historical observational data to predict future events and trends. Accurate time series forecasting can provide organizations and individuals with important forward-looking information across multiple fields, thereby supporting more effective decision-making. Long-term series forecasting, in particular, reveals data trends over extended periods and has broad application prospects. For example, in financial markets, long-term series forecasting can help investors identify long-term trends and optimize investment strategies; in meteorology, accurate long-term weather forecasts can provide valuable support for agricultural production and disaster prevention and mitigation; and in energy management, predicting future energy demand can help power companies optimize grid operations and resource allocation. Therefore, accurate forecasting results are especially critical for decision support systems.

[0003] Extracting the implicit information from historical time series is quite complex, and existing methods mainly focus on extracting time-domain features. Due to the complex dependencies of time series and the sparse representation of information in the time domain, traditional methods often employ complex network structures and modules, such as multi-head self-attention, to capture and utilize this information. These methods often involve a large number of parameters and computations, resulting in long training and inference times. In contrast, time series have a more compact representation in the frequency domain, and frequency-domain methods directly facilitate the identification and analysis of basic frequency domain components, effectively capturing long-term trends, seasonal patterns, and periodic fluctuations in time series.

[0004] However, existing frequency-domain-based methods all use limited frequency resolution. This fixed resolution may be insufficient to simultaneously capture multiple important frequency components present in a time series, especially when the series contains various dynamics ranging from low-frequency long-term trends to high-frequency short-term fluctuations. On the other hand, many models either focus on long-term predictions of global trends, neglecting the importance of local fluctuations, or overemphasize short-term data changes, failing to capture the key periodic features driving the long-term behavior of time series. This leads to an incomplete understanding of time series dynamics. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention discloses a multi-scale, bi-branch long-term series prediction method based on the frequency domain. This method is a multi-resolution frequency domain model that simultaneously models global and local features, thus improving the accuracy of long-term series prediction. First, the time-series data is divided into time segments of varying lengths according to different time scales, and the frequency domain representation of each time segment is obtained through Fourier transform. Next, a bi-branch information extractor is used to extract the long-term periodic information and short-term local dependencies of the time series. Then, the extracted feature segments are concatenated back to the length of the original sequence, and the frequency domain representation of the historical window is converted to the frequency domain representation of the future window through a complex fully connected layer. Finally, the time-series prediction result is obtained through inverse Fourier transform. Furthermore, this invention proposes a novel loss function that is insensitive to outliers. Training a neural network using this loss function can yield more realistic prediction results. The proposed method fully considers the compactness of the time series representation in the frequency domain, overcoming the limitations of existing prediction methods that typically only consider single global or local information, and has significant theoretical and practical implications for the field of long-term series prediction.

[0006] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0007] The frequency domain-based multi-scale bi-branch long-term series prediction method constructs and optimizes a frequency domain prediction model to predict historical long-term observation series data, including the following steps:

[0008] Step 1: Collect long-term historical observation sequence data and normalize them in batches;

[0009] Step 2: Divide the observed sequence data of each attribute variable into a combination of data from multiple time segments of different scales;

[0010] Step 3: Convert the observed sequence data from the time domain to the frequency domain;

[0011] Step 4: Construct a dual-branch information extractor to process and predict the frequency domain representation at each scale;

[0012] Step 5: Take a weighted average of the prediction results at all scales to generate the complete prediction result;

[0013] Step 6: Perform inverse normalization on the prediction results;

[0014] Step 7: Iterate through steps 2-6 above, select the Adam optimizer, use the integral function of the arctan function as the loss function, backpropagate to adjust the model parameters, and optimize the frequency domain prediction model.

[0015] The time series data refers to long-term series data of multiple attribute variables related to the same industrial process.

[0016] The data normalization is performed using the z-standardization method, and the processed data distribution is a standard normal distribution.

[0017] The unequal scales refer to the set of time scales. Each time scale corresponds to a time segment of different lengths.

[0018] The frequency domain transformation is performed by applying a Fast Fourier Transform (FFT) to each time segment.

[0019] The dual-branch information extractor includes: extracting temporal information within and between historical time segments by iteratively using an intra-segment fully connected layer, an inter-segment information extractor, and an intra-segment fully connected layer, thereby generating a deep frequency domain feature representation.

[0020] The process of using a dual-branch information extractor to process the frequency domain representation at each scale includes:

[0021] a. Using an intra-fragment complex fully connected layer, the frequency domain representation of each scale fragment is mapped to the latent feature space, resulting in... ;in This represents the output of the first complex fully connected layer within the i-th scale segment;

[0022] b. Use inter-segment complex fully connected layers to mine global periodic information, and obtain ;in This represents the processing result of the global information extractor at the i-th scale;

[0023] c. Use inter-segment complex convolutional layers to mine short-term temporal dynamic changes, and obtain ;in This represents the processing result of the local information extractor at the i-th scale;

[0024] d. The frequency domain feature representation of each segment is refined and enhanced again using intra-segment complex fully connected layers to obtain global information. and local information ;in This represents the output of the global information extractor at the i-th scale. This represents the output of the local information extractor at the i-th scale.

[0025] The prediction of the frequency domain representation for each scale includes:

[0026] a. Convert the frequency domain representation of each segment to the time domain;

[0027] b. Concatenate the temporal deep features of each segment, and the length of the concatenated sequence becomes the length of the original input sequence;

[0028] c. Transform the processed time series to the frequency domain using FFT to obtain the concatenated result of the global information time-domain features. The splicing result of local information temporal features ;

[0029] d. By using a complex fully connected layer MLP, the global and local information of the historical time window is deeply represented in the frequency domain to predict the frequency domain representation of the future time window;

[0030] e. Transform the frequency domain representation of the prediction results based on global and local information to the time domain to obtain the prediction result based on global information. Prediction results based on local information ;

[0031] f. Averaging the prediction results of global and local information yields the prediction result at this scale.

[0032] The inverse normalization process uses the z-standardized mean and variance used in step 1 for z-inverse normalization.

[0033] The model parameter update process uses ArcTanLoss: an integral function of the arctan function, plus the mean absolute error (MAE) and a regularization constraint term, with the specific function form as follows:

[0034]

[0035] in and These represent the actual value and the predicted value, respectively. , , , , These are hyperparameters, which are obtained through a grid search method.

[0036] The present invention has the following beneficial effects and advantages:

[0037] 1. Compared with the existing best methods, the method of this invention makes the frequency domain features of different scales complement each other by using a multi-scale segmentation method in long-term series prediction tasks.

[0038] 2. The method of the present invention employs a global-local dual-branch extractor, which simultaneously models global periodic information and local dynamic relationships to obtain prediction results with more comprehensive information.

[0039] 3. The method of this invention uses the ArcTanLoss loss function to guide parameter updates during model training, thereby improving the training effect of the neural network. Furthermore, this method has the advantage of a small number of parameters and rapid prediction within a short time. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method of the present invention.

[0041] Figure 2 This is a flowchart of the dual-branch global-local information extractor of the present invention.

[0042] Figure 3 This is a graph of the loss function of the present invention. Detailed Implementation

[0043] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0045] like Figure 1 The diagram shows a flowchart of the frequency domain-based multi-scale bi-branch long-term series prediction method of the present invention. The programming language used in the program execution steps of the present invention is not limited to MATLAB, Python, etc. Long-term sampling historical data, such as industrial historical data of transformer operating scenarios (e.g., 336 hours of data on 7 variables including transformer oil temperature and effective load) and weather historical data (720 hours of data on air pressure, wind speed, wind direction, precipitation, air pressure, saturated vapor pressure difference, vapor pressure, temperature, and humidity), can be processed to predict and mine various attribute data for a future moment or time period.

[0046] The specific steps of this invention are as follows:

[0047] Step 1: Normalize the time series data in batches using the z-standardization method. The z-standardization formula is:

[0048]

[0049] in This is the raw data for the current batch. This represents the average value of the data in the current batch. This represents the standard deviation of the data in the current batch. The processed data should follow a standard normal distribution.

[0050] Step 2: For each variable, divide it into multiple combinations of time segments of different lengths according to different time scales.

[0051] Step 2-1: Select an appropriate time scale set Each time scale Corresponding to time segments of different lengths. For example: The corresponding time segment length is 10 time sampling units. The corresponding time segment length is 20 time sampling units. The corresponding time segment length is 50 time sampling units, and so on.

[0052] Step 2-2: For each variable, according to the time scale Divide it. For example, if Then the time series will be divided into segments of length 10 (corresponding to 10 sampling points); if Then it is divided into segments of length 20 (corresponding to 20 sampling points), and so on, to generate combinations of segments of different time lengths.

[0053] Step 3: Convert the time series data from the time domain to the frequency domain to obtain the frequency domain representation of each time segment. The frequency domain representation includes amplitude and phase information.

[0054]

[0055] in It is the i-th time series segment. It is the length of the time series segment. This is the frequency domain transformation result of this time series segment. It is the imaginary unit. is a natural constant, and k is a frequency index, representing the k-th frequency component in the frequency domain.

[0056] Step 4: Process the frequency domain representation of each scale using a dual-branch information extractor.

[0057] Step 4-1, a. For the multi-segment input data at each scale, use an intra-segment complex fully connected layer to map the frequency domain representation of each segment to a hidden feature space.

[0058]

[0059] in This represents the output of the first complex fully connected layer within the i-th scale segment. This represents the combination of time segments at the i-th scale. This indicates a complex fully connected layer.

[0060] b. Complex fully connected layers are applied to the latent feature representations of all segments to mine global periodic information. The complex fully connected layers used here share weights for each frequency component.

[0061]

[0062] in This represents the processing result of the global information extractor at the i-th scale. Represents a complex fully connected layer. It is a tensor transpose operation.

[0063] c. Use complex convolutional layers to process the latent feature representations of the segments to uncover short-term temporal dynamics. The complex convolutional layers used here share weights for each frequency component.

[0064]

[0065] in This represents the processing result of the local information extractor at the i-th scale. Represents a complex fully connected layer. It is a tensor transpose operation.

[0066] d. Use intra-fragment fully connected layers again to refine and enhance the frequency domain feature representation of each fragment, obtaining global and local information respectively.

[0067]

[0068]

[0069] in This represents the output of the global information extractor at the i-th scale. This represents the output of the local information extractor at the i-th scale.

[0070] Step 4-2, a. Convert the frequency domain representation of each segment to the time domain.

[0071] b. Concatenate the temporal deep features of each segment, and the length of the concatenated sequence becomes the length of the original input sequence.

[0072] c. Transform the processed time series to the frequency domain using FFT;

[0073]

[0074]

[0075] in This represents the concatenation result of the temporal features of the global information at the i-th scale. This represents the splicing result of the temporal features of the local information at the i-th scale. This indicates a vector concatenation operation.

[0076] d. Through complex fully connected layers, the deep frequency domain representation of global and local information of historical time windows is transformed into the frequency domain representation of future time windows.

[0077] e. Transform the frequency domain representation of the prediction results based on global and local information to the time domain.

[0078]

[0079]

[0080] in This represents the prediction result at the i-th scale based on global information. This represents the prediction result based on local information at the i-th scale. This represents the inverse Fourier transform.

[0081] f. Averaging the prediction results from global and local information yields the prediction result at this scale.

[0082]

[0083] in This represents the prediction result at the i-th scale.

[0084] Step 5: Take a weighted average of the prediction results at all scales to form a complete prediction result.

[0085]

[0086] Step 6: Perform inverse normalization on the prediction results;

[0087]

[0088] in and These are the standard deviation and mean calculated in step 1, respectively.

[0089] Step 7: Iterate through steps 2-6 repeatedly, using the Adam optimizer to optimize the model parameters. The parameter update process uses ArcTanLoss, which is based on the integral function of the arctan function, plus the mean absolute error (MAE) and a regularization constraint term. The specific function form is as follows:

[0090]

[0091] in and These represent the actual value and the predicted value, respectively. , , , , These are hyperparameters, which are obtained through a grid search method.

[0092] The embodiments described above will help those skilled in the art to further understand the present invention, but do not limit the present invention in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

Claims

1. A multi-scale, two-branch long-time series prediction method based on the frequency domain, characterized in that, The following steps are used to construct and optimize a frequency domain prediction model for predicting long-term historical observation sequences: Step 1: Collect long-term historical observation sequence data and normalize it in batches; the observation sequence data includes transformer oil temperature and effective load data in transformer working scenarios, or weather data such as air pressure, wind speed, wind direction, precipitation, air pressure, saturated water vapor pressure difference, vapor pressure, temperature, and humidity. Step 2: Divide the observed sequence data of each attribute variable into a combination of data from multiple time segments of different scales; Step 3: Convert the observed sequence data from the time domain to the frequency domain; Step 4: Construct a dual-branch information extractor to process and predict the frequency domain representation at each scale; the dual-branch information extractor includes: extracting temporal information within and between historical time segments by iteratively using intra-segment complex fully connected layers, inter-segment complex fully connected layers, inter-segment complex convolutional layers, and intra-segment complex fully connected layers to generate deep frequency domain feature representations; Step 5: Take a weighted average of the prediction results at all scales to generate the complete prediction result; Step 6: Perform inverse normalization on the prediction results; Step 7: Iterate through steps 2-6, selecting the Adam optimizer and using the integral function of the arctan function as the loss function. Backpropagate is used to adjust the model parameters and optimize the frequency domain prediction model. The model parameter update process uses ArcTanLoss: the integral function of the arctan function plus the mean absolute error (MAE) and a regularization constraint term. The specific function form is as follows: ； in and These represent the actual value and the predicted value, respectively. , , , , These are hyperparameters, which are obtained through a grid search method.

2. The frequency-domain-based multi-scale dual-branch long-time series prediction method according to claim 1, characterized in that, The data normalization is performed using the z-standardization method, and the processed data distribution is a standard normal distribution.

3. The frequency-domain-based multi-scale dual-branch long-time series prediction method according to claim 1, characterized in that, The unequal scales refer to the set of time scales. Each time scale corresponds to a time segment of different lengths.

4. The multi-scale dual-branch long-time series prediction method based on the frequency domain according to claim 1, characterized in that, The frequency domain transformation is performed by applying a Fast Fourier Transform (FFT) to each time segment.

5. The multi-scale dual-branch long-time series prediction method based on the frequency domain according to claim 1, characterized in that, The dual-branch information extractor processes the frequency domain representation for each scale by including: a. Using an intra-fragment complex fully connected layer, the frequency domain representation of each scale fragment is mapped to the latent feature space, resulting in... ;in This represents the output of the first complex fully connected layer within the i-th scale segment; b. Use inter-segment complex fully connected layers to mine global periodic information, and obtain ;in This represents the processing result of the global information extractor at the i-th scale; c. Use inter-segment complex convolutional layers to mine short-term temporal dynamic changes, and obtain ;in This represents the processing result of the local information extractor at the i-th scale; d. The frequency domain feature representation of each segment is refined and enhanced again using intra-segment complex fully connected layers to obtain global information. and local information ;in This represents the output of the global information extractor at the i-th scale. This represents the output of the local information extractor at the i-th scale.

6. The multi-scale dual-branch long-time series prediction method based on the frequency domain according to claim 1, characterized in that, The prediction of the frequency domain representation for each scale includes: a. Convert the frequency domain representation of each segment to the time domain; b. Concatenate the temporal deep features of each segment, and the length of the concatenated sequence becomes the length of the original input sequence; c. Transform the processed time series to the frequency domain using FFT to obtain the concatenated result of the global information time-domain features. The splicing result of local information temporal features ; d. By using a complex fully connected layer MLP, the global and local information of the historical time window is deeply represented in the frequency domain to predict the frequency domain representation of the future time window; e. Transform the frequency domain representation of the prediction results based on global and local information to the time domain to obtain the prediction result based on global information. Prediction results based on local information ; f. Averaging the prediction results of global and local information yields the prediction result at this scale.

7. The frequency-domain-based multi-scale dual-branch long-time series prediction method according to claim 1, characterized in that, The inverse normalization process uses the z-standardized mean and variance used in step 1 for z-inverse normalization.

Images