Potential data component extraction time series prediction method based on multi-scale features

Abstract

The invention belongs to the technical field of time series prediction, and particularly relates to a time series prediction method for potential data component extraction based on multi-scale features, and the method comprises the steps: obtaining the temperature time series data of a current electric transformer, integrating the historical electric transformer temperature time sequence as a feature into the current electric transformer temperature time sequence data to form fused feature data; performing feature extraction of different scales on the fused feature data to obtain multi-scale features, and performing dimension reduction to obtain dimension-reduced multi-scale features; performing multi-level data normalization on the fused feature data, calculating residual data, and performing dimension reduction to obtain residual data features after dimension reduction; and on the basis of the multi-scale features after dimension reduction and the residual data features after dimension reduction, constructing and training an electric transformer temperature and time prediction model, and performing prediction by using the trained electric transformer temperature and time prediction model.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction, and in particular relates to a time series prediction method based on extraction of potential data components based on multi-scale features. Background Art

[0002] Multivariate time series forecasting is a classic learning problem that involves analyzing time series to predict future trends based on historical information. In particular, long-term forecasting is a well-known challenge due to feature correlation and long- and short-term temporal dependencies in time series. This problem is common in real-world applications that require observations. The prediction of time series data of electrical transformer temperature is an important problem, which predicts future temporal changes based on past observations of the time series. However, in the real world, due to the complexity and non-stationarity of temporal data, series data often presents a more complex temporal variation pattern, in which many variations such as ups and downs of noise and fluctuations are mixed together, which brings severe challenges to the forecasting work.

[0003] Traditional multi-scale feature extraction often directly processes and predicts features, focusing on all sample features, and past dimensionality reduction techniques such as the Spatial Predictor Envelope regression dimensionality reduction method and linear regression-based methods usually focus on simplifying high-dimensional data while retaining key features. At the same time, ordinary normalization is suitable for simple, small-scale data scenarios with relatively uniform feature distribution, but when faced with complex data with multiple dimensions, multiple scales, non-uniform distribution or more noise, its performance may be significantly limited, and it is particularly important to implement different processing for different features. The above methods often face two major limitations in application: first, they usually cannot fully capture the complex interactions between features; second, it is difficult to effectively deal with the noise and uneven distribution problems that are widely present in actual data. Summary of the invention

[0004] In order to overcome the problems in the prior art, the present invention proposes a time series prediction method based on potential data component extraction of multi-scale features.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: The present invention provides a time series prediction method based on potential data component extraction of multi-scale features, comprising the following steps: Acquire the current electric transformer temperature time series data, and integrate the historical electric transformer temperature time series data as features into the current electric transformer temperature time series data to form fused feature data; Extracting features of different scales from the fused feature data to obtain multi-scale features; reducing the dimension of the multi-scale features using regularized latent component regression to obtain multi-scale features after dimensionality reduction; Perform multi-level data normalization on the fused feature data, calculate residual data, apply regularized latent component regression to reduce the dimension of the residual data, and obtain residual data features after dimension reduction; Based on the multi-scale features after dimensionality reduction and the residual data features after dimensionality reduction, an electric transformer temperature-time prediction model is constructed and trained, and prediction is performed using the trained electric transformer temperature-time prediction model.

[0006] Furthermore, the acquisition of the current electric transformer temperature time series data and the integration of the historical electric transformer temperature time series as features into the current electric transformer temperature time series data to form fused feature data include: Starting from the starting point of the current transformer temperature time series data, set a length of The look-back window will move along the time series data, one time step at a time; for each time step , extracted from the current transformer temperature time series data arrive In continuous data points as input data features; For the time step , capture the time before the current time step Historical data to form a retrospective feature; The backtracking features are combined with input data features to form fused feature data.

[0007] Furthermore, the fused feature data is subjected to feature extraction at different scales to obtain multi-scale features, including: Apply sliding windows of different scales to the fused feature data to obtain data features of different scales; The data features of different scales are spliced ​​and weighted summed through the weight matrix to obtain the fused multi-scale features.

[0008] Furthermore, regularized latent component regression is used to reduce the dimension of multi-scale features to obtain the reduced-dimensional multi-scale features, including: After multi-scale feature extraction, the data of each sliding window in the channel is used as the input matrix to set the dimension of the target matrix; and the input matrix and the target matrix are preprocessed; Calculating a covariance matrix between a preprocessed input matrix and a preprocessed target matrix; and decomposing the covariance matrix to determine the main directions of the preprocessed input matrix and the preprocessed target matrix, thereby obtaining a projection direction weight vector of the input matrix and a projection direction weight vector of the target matrix; The projection direction weight vector of the input matrix is ​​used to generate the principal component and fit it to form fitting data; Regularized latent component regression is used to reduce the dimension of the fitting data and obtain multi-scale features after dimension reduction.

[0009] Furthermore, the preprocessing of the input matrix and the target matrix includes: centralizing and standardizing the input matrix and the target matrix.

[0010] Further, the covariance matrix is ​​decomposed to determine the main directions of the preprocessed input matrix and the preprocessed target matrix, thereby obtaining the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix, including: Through singular value decomposition, the covariance matrix break down: ; In the above formula, is a left singular vector, representing the preprocessed input matrix The main direction of is a right singular vector, representing the target matrix after preprocessing The main direction of The projection direction weight vector of the input matrix is ​​the left singular vector The first column in corresponds to the maximum singular value, and the projection direction weight vector of the target matrix is ​​the right singular vector The first column in corresponds to the largest singular value.

[0011] Furthermore, the projection direction weight vector based on the input matrix is ​​used to generate the principal component: ; In the above formula, the weight vector is the optimal value determined in the iteration, Indicates k principal components, X Represents the input matrix.

[0012] Furthermore, it also includes normalizing the multi-scale features after dimensionality reduction.

[0013] Compared with the prior art, the present invention has the following technical effects: The present invention first adds some important historical data to the original data as new features to capture the information of specific historical time steps; divides the predicted review window into different size scales, captures the corresponding time features from different long-term and short-term review windows, applies regularized latent component regression to reduce the dimension and regularize the data of different scales, and captures the most important feature parts through principal component extraction, so that the long-term trend and short-term fluctuations in the data can be processed separately, while also avoiding the problem that the model cannot focus on the most important part and too much noise when the multi-scale feature dimension is high. At the same time, the reversibility of multi-level data normalization is used to adapt to the distribution differences of data, and normalization is performed based on the data mean and standard deviation as the effective range of the feature. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A flowchart of a time series prediction method based on multi-scale feature potential data component extraction according to the present invention; Figure 2 Fitting diagram of the prediction results on the ETTh transformer temperature data set; Figure 3 Residual plot of the prediction results on the ETTh transformer temperature dataset. DETAILED DESCRIPTION

[0016] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. The specific features, structures or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention.

[0017] In one embodiment of the present invention, referring to Figure 1-Figure 3 , provides a time series prediction method based on potential data component extraction based on multi-scale features, including the following steps: Acquire the current electric transformer temperature time series data, and integrate the historical electric transformer temperature time series data as features into the current electric transformer temperature time series data to form fused feature data; Extracting features of different scales from the fused feature data to obtain multi-scale features; reducing the dimension of the multi-scale features using regularized latent component regression to obtain multi-scale features after dimensionality reduction; Perform multi-level data normalization on the fused feature data, calculate residual data, apply regularized latent component regression to reduce the dimension of the residual data, and obtain residual data features after dimension reduction; Based on the multi-scale features after dimensionality reduction and the residual data features after dimensionality reduction, an electric transformer temperature-time prediction model is constructed and trained, and prediction is performed using the trained electric transformer temperature-time prediction model.

[0018] The following is a detailed explanation of each of the above steps: Step 100: Acquire current electric transformer temperature time series data, and integrate historical electric transformer temperature time series as features into current electric transformer temperature time series data to form fused feature data.

[0019] As an example, this step may include: Step 1001: Obtain current transformer temperature time series data, and generate input data features and prediction targets through a sliding window method.

[0020] The current transformer temperature time series data As input, represents the step size of the lookback window, Represents the feature dimension, for each time step Generating input data features through sliding windows , prediction target , the prediction target is the future observation value.

[0021] In this implementation, a specific implementation of step 1001 may be: Starting from the starting point of the current transformer temperature time series data, set a length of The look-back window will move along the time series data, one time step at a time; for each time step , extracted from the current transformer temperature time series data arrive of continuous data points as input data features , input data features , Indicates the current time point Input data features Contains from arrive of Historical data points.

[0022] Will arrive In continuous data points as prediction targets , the prediction target is ; Among them, the initial lookback window The length is set to 96 historical steps, and the prediction range It changes. Available options are 96, 192, 336 and 720.

[0023] Step 1002: Design the local historical step length, and integrate the historical information of the time step as a feature into the input data feature of the current time step to form a fusion feature.

[0024] The historical information of the time step is integrated into the input data features of the current time step as a feature to capture the information of a specific historical time step or the important part of the historical time step. By looking back at the step length, it is possible to obtain historical information farther away than the traditional sliding window method, enhancing the model's ability to capture long-term dependencies; at the same time, it is also possible to focus on key information in the short term and improve the model's ability to obtain recent dependencies.

[0025] In this implementation, a specific implementation of step 1002 may be: Step 10021: For the time step , capture the time before the current time step The historical data is added as features to the input data features of the current time step to form a retrospective feature: ; In the above formula, Represents the backtracking feature, Indicates the time step before the current time step data.

[0026] Step 10022: Combine the backtracking features and input data features to form fused feature data.

[0027] The combination of the backtracking features and the original features is represented by the feature vector It is expressed as: , the fusion feature dimension is .

[0028] Linear regression model combining sliding window and backtracking features: ; In the above formula, represents the estimated value of the predicted target, represents the weight of the input data feature, represents the weight of the backtracking feature, Represents the bias term.

[0029] Step 200: extracting features of different scales from the fused feature data to obtain multi-scale features; and reducing the dimension of the multi-scale features by using regularized latent component regression to obtain multi-scale features after dimensionality reduction.

[0030] As an example, this step may include: Step 2001: The fused feature data enters multiple channels simultaneously, and each channel performs feature extraction at different scales.

[0031] The sliding windows in each channel are generated at different scales to form a multi-resolution representation of the time series. The input fusion feature dimension is F. For each scale S, the review window and the target window are downsampled, that is, the size of the review window is gradually reduced to capture more detailed time fluctuations.

[0032] Step 20011: Obtain data features at different scales.

[0033] Get different scales The number of sliding windows under , the matrix feature dimension is : ; The scale used is , obtaining sliding windows of different scales, which enables the model to extract features from multiple review windows, enabling it to capture feature information at different time scales: ; For each scale , a series of sliding windows of different sizes will be generated. For example, at scale 1, the window size may be the same as the time step of the original data; at scale 2, the window size may be half of the original data; at scale 4, the window size may be one-quarter of the original data; these windows will slide in time with different step sizes or overlapping degrees to capture features at different scales.

[0034] Step 20012: Concatenate data features of different scales, perform weighted summation, and obtain fused multi-scale features.

[0035] The data features in channels of different scales are spliced ​​and summed according to certain weights, even if the features of different scales are passed through a trainable weight matrix Corresponding elements Multiply and add to get the fused multi-scale feature X. The fused multi-scale feature X is the feature distribution after the weighted sum of the data features of channels of different scales: .

[0036] Step 2002: Using the data of each sliding window in different channels as the input matrix, set the target matrix dimension.

[0037] After multi-scale feature extraction, the data of each sliding window in different channels is used as the input matrix , and the target matrix Based on this, a set of new low-dimensional latent components, namely principal components, are found, so that the principal components can both explain the variance of the input matrix and have the maximum correlation with the target matrix; and through iteration, the optimal linear projection of the input matrix and the target matrix is ​​found.

[0038] Step 2003: Preprocess the input matrix and the target matrix.

[0039] The preprocessing includes centering and standardization. Centering is to subtract the mean of each feature so that the mean of each feature is 0, eliminate the offset of different features, and improve the convergence speed of the model. Standardization adjusts the data to zero mean and unit variance (i.e., the standard deviation is 1), scales the features to a similar numerical range, and eliminates scale problems.

[0040] For the input matrix and the target matrix Perform centering and standardization to maximize the preprocessed input matrix And the preprocessed target matrix The covariance between them is used to find the maximum correlation component between them in their respective projection directions. ,in, yes X The projection direction weight vector, yes Y The projection direction weight vector.

[0041] By linear algebra methods, the objective function can be rewritten as , which is a bilinear optimization problem, where and Need to optimize simultaneously.

[0042] Step 2004: Calculate the covariance matrix between the preprocessed input matrix and the preprocessed target matrix, decompose the covariance matrix, determine the main directions of the preprocessed input matrix and the preprocessed target matrix, and obtain the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix.

[0043] In this implementation, a specific implementation of step 2004 may be: Step 20041: Calculate the preprocessed input matrix And the preprocessed target matrix The covariance matrix between , the covariance matrix Describes the linear relationship between input features and target variables; Step 20042: Through singular value decomposition (SVD), the covariance matrix break down: ; In the above formula, is a left singular vector, representing the preprocessed input matrix The main direction of is a right singular vector, representing the target matrix after preprocessing The main direction of Step 20043: Input Matrix X The projection direction weight vector is the left singular vector The first column in corresponds to the largest singular value, and the target matrix Y The projection direction weight vector is the right singular vector The first column in corresponds to the largest singular value.

[0044] Step 2005: Based on the input matrix X The projection direction weight vector To generate the principal components and perform fitting.

[0045] Each principal component is an integrated feature obtained by combining multiple features with different weights. k principal components Calculated by the following formula: ; In the above formula, the weight vector is the optimal value determined in the iteration, Represents a single feature that combines the sum of multiple features, through Represents the maximum possible explanation of the input matrix X and the target matrix Y The covariance of .

[0046] By capturing the most important features through principal component extraction, it is possible to process long-term trends and short-term fluctuations in the data separately. At the same time, it also avoids the problem of the model being unable to focus on the most important parts and having too much noise when the multi-scale feature dimensions are high.

[0047] Step 2006: Use the Regularized Latent Component Regression (RLCR) model to reduce the dimension of the multi-scale features to obtain the multi-scale features after dimension reduction; RLCR is a dimensionality reduction method that combines regularization and latent variable models.

[0048] Step 2007: Perform multi-level data normalization on the multi-scale features after dimensionality reduction, so that the mean and standard deviation of each data point are standardized, and restore the data to the original distribution through reverse operations.

[0049] Step 300: Perform multi-level data normalization on the fused feature data, calculate residual data, apply regularized latent component regression to reduce the dimension of the residual data, and obtain residual data features after dimensionality reduction.

[0050] As an example, this step may include: Step 3001: Perform multi-level data normalization on the fused feature data.

[0051] Input data fused with specific historical data is a A three-dimensional array, where is the number of time steps, is the feature dimension, is the feature dimension of each time step, and each instance is normalized so that the mean and standard deviation of each data point T are standardized.

[0052] Step 30011: Use the following formula to combine the input data with specific historical data Standardization is performed to map all features to the same interval, which is determined by the mean and variance of the data, effectively eliminating the scale differences between features.

[0053] ; in, represents the standardized data features, represents the mean of each feature, represents the standard deviation of each feature, is a small smoothing value used to avoid division by zero problems.

[0054] Step 30012: Scale the normalized data features to between the specified minimum and maximum values: ; In the above formula, Represents a value between the specified minimum and maximum values; Indicates the specified minimum value, Indicates the specified maximum value.

[0055] In this way, all feature values ​​are constrained to the same scale, thus avoiding the influence of some features being too large or too small during model training, and its reversibility allows the data to be restored to the scale of the original data after normalization.

[0056] Step 3002: Fit the normalized feature data, calculate the difference between the fused feature data and the fitting value as the residual, and calculate the residual data.

[0057] Specifically, linear regression can be used to fit the normalized data, and then the difference between the fused feature data and the fitted value is calculated as the residual data.

[0058] Step 3003: Apply the regularized latent component regression model for dimensionality reduction.

[0059] Step 3004: Restore the model output value to the standardized scale : ; Based on the standardized scale, restore to the original data space : ; This is very important for tasks that require explanation and verification of prediction results. The prediction results can be more conveniently denormalized to the original scale, ensuring the interpretability and verifiability of the prediction results.

[0060] Step 400: Based on the multi-scale features after dimensionality reduction and the residual data features after dimensionality reduction, a time series prediction model for the temperature of an electric transformer is constructed. The temperature time series prediction model for the electric transformer performs preliminary predictions through the above steps 100, 200, and 300, and obtains a difference between a predicted value and an actual value, which is the residual. Training is performed repeatedly according to the size of this difference. Repeating the above steps, as time goes by, new data continues to enter, and the prediction ability of the initially constructed model may decrease. Therefore, it is necessary to regularly update and retrain the model to ensure that it can continue to provide accurate predictions. After the training is completed, the trained temperature time series prediction model for the electric transformer is used for actual predictions.

[0061] Specifically, the multi-scale features after dimensionality reduction and the residual data features after dimensionality reduction are combined into a new feature set. This set will be used as the input of the prediction model. The prediction model, such as the linear regression model, is trained based on the mean square error MSE as the loss function, and finally the target data is predicted and output.

[0062] Use regularization to further fit the details of the trend and make the model more generalizable through the loss function. The following formula is used to make the trend model more generalizable: ; In the above formula, represents the true value; Represents the regression coefficient or model parameter vector, which represents the weight that the model needs to learn; A hyperparameter representing the strength of regularization.

[0063] Experimental results analysis: Experiments are conducted on the publicly available power transformer temperature dataset ETTh1. All time series are split into look-back windows L = 96, forecast horizons H ∈ {96, 192, 336, 720}, and a step size of 1, which means that each subsequent window is shifted by one step, and the obtained error results are between 0.08 and 0.3.

[0064] By predicting the ETTh transformer temperature time series data set, this lightweight time series prediction method based on multi-scale feature-based latent data component extraction produces good and accurate predictions. Figure 2 , the trend of the predicted value is roughly consistent with the actual value, especially in some smooth areas, the model captures the trend well.

[0065] In order to prove the feasibility and authenticity of the present invention, refer to Figure 3 , the ETT transformer temperature characteristic diagram is compared. The residual is the difference between the model prediction value and the true value. The residual diagram can show the distribution of these differences. Ideally, the residual should be close to a normal distribution with a mean of zero, indicating that the model evenly distributes errors at each prediction point and has no systematic deviation. The residual diagram helps detect whether there are systematic errors in the model, and test the model's fit quality, whether there are deviations, overfitting, or uncaptured trends.

[0066] There are fewer restrictions on the equipment used, and the equipment relied on by the invention for time series prediction to achieve the same effect is much higher than that of the present invention. The present invention involves fewer parameters and has a fast running speed, and can achieve the same prediction accuracy at a faster speed.

[0067] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A time series prediction method based on potential data component extraction based on multi-scale features, characterized in that: The following steps are involved: Acquire the current electric transformer temperature time series data, and integrate the historical electric transformer temperature time series data as features into the current electric transformer temperature time series data to form fused feature data; Extracting features of different scales from the fused feature data to obtain multi-scale features; Regularized latent component regression is used to reduce the dimension of multi-scale features to obtain the reduced-dimensional multi-scale features; Perform multi-level data normalization on the fused feature data, calculate residual data, apply regularized latent component regression to reduce the dimension of the residual data, and obtain residual data features after dimension reduction; Based on the multi-scale features after dimensionality reduction and the residual data features after dimensionality reduction, an electric transformer temperature-time prediction model is constructed and trained, and prediction is performed using the trained electric transformer temperature-time prediction model.

2. According to claim 1, a time series prediction method based on potential data component extraction based on multi-scale features is characterized in that: The method of obtaining the current electric transformer temperature time series data and integrating the historical electric transformer temperature time series data as features into the current electric transformer temperature time series data to form fused feature data includes: Starting from the starting point of the current transformer temperature time series data, set a length of The look-back window will move along the time series data, one time step at a time; for each time step , extracted from the current transformer temperature time series data arrive In continuous data points as input data features; For the time step , capture the time before the current time step Historical data to form a retrospective feature; The backtracking features are combined with input data features to form fused feature data.

3. The time series prediction method based on multi-scale feature potential data component extraction according to claim 2 is characterized in that: The fused feature data is subjected to feature extraction at different scales to obtain multi-scale features, including: Apply sliding windows of different scales to the fused feature data to obtain data features of different scales; The data features of different scales are spliced ​​and weighted summed through the weight matrix to obtain the fused multi-scale features.

4. The time series prediction method based on multi-scale feature potential data component extraction according to claim 3 is characterized in that: Regularized latent component regression is used to reduce the dimension of multi-scale features to obtain the reduced-dimensional multi-scale features, including: After multi-scale feature extraction, the data of each sliding window in the channel is used as the input matrix to set the dimension of the target matrix; and the input matrix and the target matrix are preprocessed; Calculating a covariance matrix between a preprocessed input matrix and a preprocessed target matrix; and decomposing the covariance matrix to determine the main directions of the preprocessed input matrix and the preprocessed target matrix, thereby obtaining a projection direction weight vector of the input matrix and a projection direction weight vector of the target matrix; The projection direction weight vector of the input matrix is ​​used to generate the principal component and fit it to form fitting data; Regularized latent component regression is used to reduce the dimension of the fitting data and obtain multi-scale features after dimension reduction.

5. The time series prediction method based on multi-scale feature potential data component extraction according to claim 4 is characterized in that: The preprocessing of the input matrix and the target matrix includes: centralizing and standardizing the input matrix and the target matrix.

6. The time series prediction method based on multi-scale feature potential data component extraction according to claim 4 is characterized in that: Decomposing the covariance matrix, determining the main directions of the preprocessed input matrix and the preprocessed target matrix, thereby obtaining the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix, including: Through singular value decomposition, the covariance matrix break down: ； In the above formula, is a left singular vector, representing the preprocessed input matrix The main direction of is a right singular vector, representing the target matrix after preprocessing The main direction of The projection direction weight vector of the input matrix is ​​the left singular vector The first column in corresponds to the maximum singular value, and the projection direction weight vector of the target matrix is ​​the right singular vector The first column in corresponds to the largest singular value.

7. The time series prediction method based on multi-scale feature potential data component extraction according to claim 6 is characterized in that: The projection direction weight vector based on the input matrix to generate the principal component is: ； In the above formula, the weight vector is the optimal value determined in the iteration, Indicates k principal components, X Represents the input matrix.

8. The time series prediction method based on multi-scale feature potential data component extraction according to claim 7 is characterized in that: It also includes normalizing the multi-scale features after dimensionality reduction.

Images