Time Series Prediction Method Based on Variable Decomposition and Convolutional Attention Modeling

Abstract

The present invention belongs to the technical field of time series prediction, and particularly relates to a time series prediction method based on variable decomposition and convolutional attention modeling. First, variable decomposition is performed on the historical time series data of the electrical transformer temperature. The learnable one-dimensional convolutional operation is used to decompose the historical time series data into trend and seasonal components, overcoming the problem of insufficient flexibility of traditional decomposition methods. In the modeling of seasonal components, a simple and efficient linear transformation is adopted to extract features by learning the periodic patterns of seasonal changes. In the modeling of trend components, a convolutional attention structure is adopted to capture the non-linear relationships in the long-term trends while ensuring the computational efficiency of the model. Finally, the features of the seasonal components and trend components are fused and passed through a residual learning module to further improve the prediction accuracy of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction, and particularly relates to a time series prediction method based on variable decomposition and convolutional attention modeling. Background Art

[0002] Time series prediction is a process of using historical time series data to predict the values at future time points. Time series data is data arranged in chronological order and has time dependence. The prediction of the temperature time series data of an electrical transformer is an important issue, which predicts future time changes based on past observations of the time series.

[0003] With the development of deep learning technology, the models for time series prediction have gradually shifted from traditional statistical methods to neural network-based methods. Typical deep learning models include the Recurrent Neural Network (RNN), which can capture short-term and long-term dependencies in time series; the Long Short-Term Memory Network (LSTM) and the Gated Recurrent Unit (GRU), which are two variants of RNN that solve the gradient vanishing problem of traditional RNN by introducing a gating mechanism and can effectively model long-term dependencies; the Convolutional Neural Network (CNN), which is mainly used to extract local features in time series and performs well in dealing with local time dependencies and feature learning; and the Transformer model, which captures long-range dependencies through the self-attention mechanism and supports parallel computing, and has been widely used in time series prediction in recent years. However, they usually ignore the seasonal and trend components in the data, or lack sufficient flexibility in modeling these components. Especially when there are complex non-linear patterns in the data, traditional neural network models are difficult to effectively distinguish and model different components, thus affecting the prediction effect. Summary of the Invention

[0004] In order to overcome the problems in the prior art, the present invention proposes a time series prediction method based on variable decomposition and convolutional attention modeling.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] The present invention provides a time series prediction method based on variable decomposition and convolutional attention modeling, including the following steps:

[0007] Obtain the historical time series data of the electrical transformer temperature;

[0008] Construct an electrical voltage temperature time series prediction model. The processing steps in the constructed electrical voltage temperature time series prediction model include: preprocessing the historical time series data; performing variable Gaussian distribution decomposition on the preprocessed time series data to obtain a trend component and a seasonal component; modeling the trend component to capture the long-term non-linear relationship in the time series; performing linear mapping processing on the seasonal component to retain its periodic characteristics and avoid introducing excessive complex relationships; adjusting the weights according to the contribution degrees of the trend component and the seasonal component to achieve effective integration of information;

[0009] Train the electrical voltage temperature time series prediction model to obtain a trained electrical voltage temperature time series prediction model;

[0010] Input the electrical voltage temperature time series data to be predicted into the trained electrical voltage temperature time series prediction model to obtain a prediction result.

[0011] Furthermore, the preprocessing of the historical time series data includes: normalizing the historical time series data and performing affine transformation on the normalized historical time series data.

[0012] Furthermore, the performing variable Gaussian distribution decomposition on the preprocessed time series data to obtain a trend component and a seasonal component includes:

[0013] Initialize the convolutional kernel weights using a Gaussian distribution;

[0014] Pad the preprocessed historical time series data to keep the length of the time series data consistent before and after convolution, and smooth the time series data with the initialized convolutional kernel to extract the trend component:

[0015] ;

[0016] In the above formula, represents the trend component; represents the preprocessed historical time series data; represents the initialized convolutional kernel weights; represents padding; represents convolution;

[0017] Subtract the trend component from the preprocessed time series data to obtain the seasonal component .

[0018] Furthermore, the modeling of the trend component to capture the long-term non-linear relationship in the time series includes:

[0019] Construct multiple channels, input the trend component into each channel, perform convolution operations on each channel using convolution kernels of different sizes, and achieve feature integration between different channels through pointwise convolution to form a multi-scale fusion feature representation;

[0020] Process the fused features using a 1×1 convolution, and then multiply the result element-wise with the input trend component.

[0021] Furthermore, all convolution operations adopt the form of depthwise separable convolution.

[0022] Furthermore, adjusting the weights according to the contribution degrees of the trend component and the seasonal component to achieve effective integration of information includes:

[0023] Perform weighted fusion on the prediction results of the seasonal component and the trend component to obtain the weighted fusion features:

[0024] ;

[0025] Among them, represents the weighted fusion features; represents the result of feature extraction of the seasonal component; represents the result of feature extraction of the trend component; represents the weighting coefficient, which is used to adjust the contributions of the seasonal component and the trend component.

[0026] Furthermore, in the constructed electrical transformer temperature time series prediction model, it further includes:

[0027] Through residual connection, add the weighted fusion features and the preprocessed time series data to obtain the residual fusion features:

[0028] ;

[0029] Among them, represents the weighted fusion features; is the weight matrix; is the bias vector; is the intermediate feature processed by the fully connected layer and the ReLU activation function; represents the residual fusion features.

[0030] Compared with the prior art, the present invention has the following technical effects:

[0031] The present invention first performs variable decomposition on the historical time series data of the electrical transformer temperature, and uses learnable one-dimensional convolution operations to accurately decompose the time series data into trend components and seasonal components, overcoming the problem of insufficient flexibility in traditional decomposition methods. In the modeling of seasonal components, a simple and efficient linear transformation is adopted to extract features by learning the periodic patterns of seasonal changes. In the modeling of trend components, a convolutional attention structure is adopted to capture the non-linear relationships in the long-term trends while ensuring the computational efficiency of the model. Finally, the features of the seasonal components and trend components are fused and passed through a residual learning module to further improve the prediction accuracy of the model. Description of the Drawings

[0032] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0033] Figure 1 It is a flowchart of the time series prediction method based on variable decomposition and convolutional attention modeling of the present invention;

[0034] Figure 2 It is a fitting diagram of the prediction results obtained on the ETTm transformer temperature dataset;

[0035] Figure 3 It is a schematic structural diagram of the time series prediction method based on variable decomposition and convolutional attention modeling of the present invention. Detailed Embodiments

[0036] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific embodiments, structures, features, and effects of the technical solutions proposed according to the present invention. The specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0037] In an embodiment of the present invention, referring to Figures 1-3 , a time series prediction method based on variable decomposition and convolutional attention modeling is provided, including the following steps:

[0038] Obtain historical time series data of the electrical transformer temperature; construct an electrical transformer temperature time series prediction model, which includes: preprocessing the historical time series data; performing variable decomposition of the Gaussian distribution on the preprocessed time series data to obtain a trend component and a seasonal component; modeling the trend component to capture the long-term non-linear relationship in the time series; performing linear mapping processing on the seasonal component to retain its periodic characteristics and avoid introducing excessive complex relationships; adjusting the weights according to the contribution degrees of the trend component and the seasonal component to achieve effective integration of information;

[0039] Train the electrical transformer temperature time series prediction model to obtain a trained electrical transformer temperature time series prediction model;

[0040] Input the time series data of the electrical transformer temperature to be predicted into the trained electrical transformer temperature time series prediction model to obtain a prediction result.

[0041] The following details each of the above steps:

[0042] Step 100: Obtain historical time series data of the electrical transformer temperature and construct an electrical transformer temperature time series prediction model.

[0043] To construct an electrical transformer temperature time series prediction model, refer to Figure 3 , the electrical transformer temperature time series prediction model includes a reversible normalization module, a variable decomposition module, a linear modeling module, a convolutional attention module, a feature fusion module, a residual learning module, and a prediction layer; the input of the reversible normalization module receives the historical time series data of the electrical transformer temperature, the output of the reversible normalization module is connected to the input of the variable decomposition module, the output of the variable decomposition module is respectively connected to the input of the linear modeling module and the input of the convolutional attention module, the output of the linear modeling module and the output of the convolutional attention module are respectively connected to the input of the feature fusion module, the output of the feature fusion module is connected to the input of the residual learning module, the input of the residual learning module is connected to the output of the reversible normalization module, and the output of the residual learning module is connected to the prediction layer, and the prediction data is output through the prediction layer.

[0044] Among them, the reversible normalization module is used to preprocess the historical time series data of the electrical transformer temperature; the variable decomposition module is used to perform variable decomposition of the Gaussian distribution on the preprocessed data to obtain a trend component and a seasonal component; the convolutional attention module is used to model the trend component to capture the long-term non-linear relationship in the time series; the linear modeling module is used to perform linear mapping processing on the seasonal component to retain its periodic characteristics and avoid introducing excessive complex relationships; the feature fusion module is used to adjust the weights according to the contribution degrees of the trend and seasonal components to achieve effective integration of information; the residual learning module is used to alleviate the problem of gradient disappearance in the deep network.

[0045] As a specific example, this step 200 may include the following sub-steps:

[0046] Step 210: Preprocess the historical time series data of the electrical transformer temperature.

[0047] In this embodiment, a specific implementation manner of step 210 may be:

[0048] Step 2101: The historical time series data of the electrical transformer temperature is represented as , and its normalization calculation formula is as follows:

[0049] ;

[0050] where N is the batch size, D is the number of variables, L is the time step; μ is the mean, is the standard deviation; represents the normalized historical time series data of the electrical transformer temperature; represents the historical time series data of the electrical transformer temperature.

[0051] Step 2102: The normalized value is subjected to an affine transformation through the following formula:

[0052] ;

[0053] where, represents the historical time series data of the electrical transformer temperature after the affine transformation; γ represents the linear transformation factor of the affine transformation; β is the translation factor of the affine transformation.

[0054] Step 220: Perform variable decomposition of the Gaussian distribution on the preprocessed time series data to obtain a trend component and a seasonal component.

[0055] Extracting the long-term trend component of time series data is of great significance for capturing the overall change pattern. To achieve this goal, a convolutional-based variable decomposition module is adopted, which can smooth the time series data through convolutional operations and extract the trend component. The extraction of the trend component helps to better understand the global change characteristics of the time series, while separating the short-term periodic fluctuations and extracting the trend component.

[0056] This initialization method ensures that the central position of the convolutional kernel has the largest weight, while the weights of the positions far from the center gradually decrease. This convolutional operation will pay more attention to the local central region of the time series, providing a weighted moving average to extract smooth trend information. The convolutional operation is equivalent to performing a weighted moving average on the time axis, and the weight distribution of the Gaussian kernel makes the adjacent time steps have a greater impact on the trend decomposition. Compared with the ordinary moving average, the Gaussian kernel can smoother the fluctuations in the time series, especially having better robustness for sequences with more noise.

[0057] Step 2201: Initialize the weights of the convolutional kernel using a Gaussian distribution, so that the weight of the central position is the largest and the weight of the edge position is smaller.

[0058] Specifically, define a convolutional kernel, and each element of the convolutional kernel is initialized by the following formula:

[0059] ;

[0060] where K is the size of the convolutional kernel, is the standard deviation that controls the weight distribution of the convolutional kernel.

[0061] Step 2202: Pad the preprocessed time series data of the input to keep the sequence length consistent before and after convolution, and use the initialized convolutional kernel to smooth the time series data to extract the trend component. The formula is as follows:

[0062] ;

[0063] In the above formula, represents the trend component; represents the preprocessed historical time series data; represents the initialized weights of the convolutional kernel; represents the padding; represents the convolution.

[0064] Step 2203: Subtract the trend component from the preprocessed time series data of the input to obtain the seasonal component , and the formula is expressed as follows:

[0065] ;

[0066] In the above formula, represents the seasonal component; represents the preprocessed time series data of the input.

[0067] Step 230: Model the trend component using a convolutional attention mechanism to capture the long-term non-linear relationships in the time series.

[0068] The main function of the convolutional attention network is to extract features at different levels through multi-scale convolutions. It combines convolutional kernels of different sizes and captures patterns and features at different scales through multiple convolutional operations. Finally, it fuses the features at different scales and multiplies them element-wise with the original input to achieve the effect of the attention mechanism.

[0069] In this embodiment, a specific implementation of step 230 can be:

[0070] Step 2301: To effectively capture features at different scales in the time series data, construct multiple channels, input the trend component into each channel, perform convolutional operations on each channel using convolutional kernels of different sizes, and achieve feature integration between different channels through pointwise convolution to form a multi-scale fused feature representation.

[0071] The sizes of the convolutional kernels include 7×7, 11×11, and 21×21, etc. For the sake of efficiency, referring to Figure 3 , they are actually 1×7, 7×1, 1×11, 11×1, 1×21, and 21×1. The combination of these convolutional kernels can ensure the simultaneous capture of short-term local features and long-term trends.

[0072] Among them, all convolutional operations adopt the form of depthwise separable convolution. This convolution method first independently performs convolution on each channel, and then realizes feature integration between different channels through pointwise convolution (1×1 convolution).

[0073] The features extracted by different convolutional kernels are fused together to form a multi-scale feature representation. These different convolutional kernels extract multi-level features, including local details and global trends. After fusion, these features can more comprehensively describe the trend patterns in the input time series. The fusion process is achieved through simple element-wise addition, enabling the model to retain the feature information of each scale.

[0074] Step 2302: Process the fused features using a 1×1 convolution, and then multiply the result element-wise with the input trend component.

[0075] This operation can be regarded as an attention mechanism, which dynamically adjusts the importance of features at each position by weighting the original features, enabling the model to adaptively selectively amplify or suppress the trend part in the features.

[0076] Step 240: Perform a linear mapping process on the seasonal component to retain its periodic characteristics and avoid introducing excessive complex relationships.

[0077] The characteristics of the seasonal component usually show relatively regular periodic changes. Therefore, for the feature extraction of this part, the present invention adopts a linear transformation. This simple and effective method can well retain the seasonal signal without introducing additional modeling complexity. Since the characteristics of the seasonal component are relatively stable and periodic, complex convolution or attention mechanisms may bring the risk of overfitting. Through linear transformation, the periodic signal can be directly identified in the input features and these features can be maintained in the output, ensuring that the seasonal changes are fully reflected in the prediction results.

[0078] Step 250: Adopt a feature fusion strategy to adjust the weights according to the contribution degrees of the trend component and the seasonal component to achieve effective integration of information.

[0079] After completing the modeling of the seasonal component and the trend component, the prediction results of the two are weighted and fused to finally obtain the weighted fusion feature. The process of weighted fusion can be expressed as:

[0080] ;

[0081] where, represents the weighted fusion feature; represents the result of the feature extraction of the seasonal component; represents the result of the feature extraction of the trend component; represents the weighting coefficient used to adjust the contributions of the seasonal component and the trend component. Preferably, take 0.7.

[0082] Step 260: Mitigate the problem of gradient vanishing in the deep network through a residual learning module.

[0083] Through residual connection, the weighted fusion feature and the preprocessed time series data are added together to obtain the residual fusion feature:

[0084] ;

[0085] where, represents the weighted fusion feature; is the weight matrix; is the bias vector; is the intermediate feature processed by the fully connected layer and the ReLU activation function; represents the residual fusion feature.

[0086] Step 270: Use the residual fusion feature for predicting the time series.

[0087] Map the residual fusion feature to the output space through a fully connected layer . The lengths of the output prediction sequences are 96, 192, 336, and 720. This prediction layer realizes the mapping from high-dimensional features to predicted values through a simple and effective linear transformation. This design not only simplifies the computational complexity but also improves the prediction accuracy and generalization ability of the model.

[0088] Step 300: Train the electric transformer temperature time series prediction model to obtain a trained electric transformer temperature time series prediction model.

[0089] Step 400: Input the electric transformer temperature time series data to be predicted into the trained electric transformer temperature time series prediction model to obtain a prediction result.

[0090] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A time series prediction method based on variable decomposition and convolutional attention modeling, characterized in that Including the following steps: Obtain the historical time series data of the electrical transformer temperature; Construct an electrical transformer temperature time series prediction model. The processing steps in the constructed electrical transformer temperature time series prediction model include: preprocessing the historical time series data; performing Gaussian distribution variable decomposition on the preprocessed time series data to obtain a trend component and a seasonal component; using a convolutional attention mechanism to model the trend component to capture the long-term non-linear relationship in the time series; performing linear mapping processing on the seasonal component to retain its periodic characteristics and avoid introducing excessive complex relationships; adjusting the weights according to the contribution degrees of the trend component and the seasonal component to achieve effective integration of information; Train the electrical transformer temperature time series prediction model to obtain a trained electrical transformer temperature time series prediction model; Input the time series data of the electrical transformer temperature to be predicted into the trained electrical transformer temperature time series prediction model to obtain a prediction result; The modeling of the trend component to capture the long-term non-linear relationship in the time series includes: constructing multiple channels, inputting the trend component into each channel, performing convolution operations on each channel using convolutional kernels of different sizes, and realizing feature integration between different channels through pointwise convolution to form a multi-scale fusion feature representation; using a 1×1 convolution to process the fused features, and then multiplying the result element-wise with the input trend component; All convolution operations adopt the form of depthwise separable convolution.

2. The time series prediction method based on variable decomposition and convolutional attention modeling according to claim 1, wherein, The preprocessing of the historical time series data includes: normalizing the historical time series data and performing affine transformation on the normalized historical time series data.

3. A time series prediction method based on variable decomposition and convolutional attention modeling according to claim 2, characterized in that, The performing of Gaussian distribution variable decomposition on the preprocessed time series data to obtain a trend component and a seasonal component includes: Initializing the convolutional kernel weights using a Gaussian distribution; Padding the preprocessed historical time series data to keep the length of the time series data consistent before and after convolution, and using the initialized convolutional kernel to smooth the time series data to extract the trend component: ； In the above formula, represents the trend component; represents the historical time series data after preprocessing; represents the initialization of the convolutional kernel weights; represents padding; represents convolution; Subtract the trend component from the preprocessed time series data to obtain the seasonal component .

4. The time series prediction method based on variable decomposition and convolutional attention modeling according to claim 3, characterized in that The adjusting of the weights according to the contribution degrees of the trend component and the seasonal component to achieve effective integration of information includes: Performing weighted fusion on the prediction results of the seasonal component and the trend component to obtain a weighted fusion feature: ； Among them, represents the feature after weighted fusion; represents the result of feature extraction of the seasonal component; represents the result of feature extraction of the trend component; represents the weighting coefficient, which is used to adjust the contributions of the seasonal component and the trend component.

5. A time series prediction method based on variable decomposition and convolutional attention modeling according to claim 4, characterized in that In the constructed electrical transformer temperature time series prediction model, it further includes: Adding the weighted fusion feature and the preprocessed time series data through residual connection to obtain a residual fusion feature: ； Among them, represents the feature after weighted fusion; is the weight matrix; is the bias vector; is the intermediate feature processed by the fully connected layer and the ReLU activation function; represents the residual fusion feature.

Images