Time Series Prediction Method for Extracting Latent Data Components Based on Multi-Scale Features

Abstract

The present invention belongs to the technical field of time series prediction, and particularly relates to a time series prediction method for extracting potential data components based on multi-scale features. The method includes: obtaining the current temperature time series data of an electric transformer, integrating the historical temperature time series of the electric transformer as features into the current temperature time series data of the electric transformer to form fusion feature data; performing feature extraction on the fusion feature data at different scales to obtain multi-scale features, and reducing the dimension to obtain the multi-scale features after dimension reduction; performing multi-level data normalization on the fusion feature data, calculating residual data, and reducing the dimension to obtain the residual data features after dimension reduction; constructing a temperature time prediction model for the electric transformer based on the multi-scale features after dimension reduction and the residual data features after dimension reduction, training the model, and using the trained temperature time prediction model for the electric transformer to make predictions.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction, and particularly relates to a time series prediction method for extracting potential data components based on multi-scale features. Background Art

[0002] Multivariate time series prediction is a classical learning problem, which includes analyzing time series to predict future trends based on historical information. In particular, due to the feature correlation and long-term and short-term time dependencies in time series, long-term prediction is a well-known challenge. Such problems are common in those real-world applications that require observation. The prediction of the temperature time series data of an electrical transformer is an important issue, which predicts future time changes based on past observations of the time series. However, in the real world, due to the complexity and non-stationarity of time data, the sequence data often presents relatively complex time variation laws, and many variation forms such as the floating noise and fluctuations are mixed together, bringing severe challenges to the prediction work.

[0003] Traditional multi-scale feature extraction often directly processes and predicts features, focusing on all sample features. And past dimensionality reduction techniques such as the Spatial Predictor Envelope regression dimensionality reduction method, and linear regression-based methods usually focus on retaining key features while simplifying high-dimensional data. At the same time, ordinary normalization is applicable to simple, small-scale, and relatively uniform data scenarios, but its performance may be significantly limited when facing complex data with multiple dimensions, multiple scales, non-uniform distributions, or more noise. It is particularly important to implement different processing for different features. The above methods often face two major limitations in application: First, they usually cannot fully capture the complex interaction relationships between features; second, it is difficult to effectively cope with the noise and uneven distributions widely existing in actual data. Summary of the Invention

[0004] In order to overcome the problems in the prior art, the present invention proposes a time series prediction method for extracting potential data components based on multi-scale features.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] The present invention provides a time series prediction method for extracting potential data components based on multi-scale features, including the following steps:

[0007] Obtain the current temperature time series data of the electrical transformer, and integrate the historical temperature time series of the electrical transformer as features into the current temperature time series data of the electrical transformer to form fusion feature data;

[0008] Extract features of the fusion feature data at different scales to obtain multi-scale features; use regularized latent component regression to reduce the dimension of the multi-scale features to obtain the multi-scale features after dimension reduction;

[0009] Perform multi-level data normalization on the fusion feature data, calculate the residual data, and apply regularized latent component regression to reduce the dimension of the residual data to obtain the residual data features after dimension reduction;

[0010] Based on the multi-scale features after dimension reduction and the residual data features after dimension reduction, construct an electrical transformer temperature-time prediction model and train it, and use the trained electrical transformer temperature-time prediction model for prediction.

[0011] Furthermore, the obtaining of the current electrical transformer temperature-time series data and the incorporation of the historical electrical transformer temperature-time series as features into the current electrical transformer temperature-time series data to form fusion feature data include:

[0012] Starting from the starting point of the current electrical transformer temperature-time series data, set a retrospective window with a length of , and the retrospective window will move along the time series data, moving one time step each time; for each time step , extract from the current electrical transformer temperature-time series data from to among the consecutive data points as input data features;

[0013] For the time step , capture the historical data before the current time step to form a retrospective feature;

[0014] Combine the retrospective feature and the input data features to form fusion feature data.

[0015] Furthermore, extracting features of the fusion feature data at different scales to obtain multi-scale features includes:

[0016] Apply sliding windows of different scales to the fusion feature data to obtain data features of different scales;

[0017] Stitch together the data features of different scales and perform weighted summation through a weight matrix to obtain the fused multi-scale features.

[0018] Furthermore, using regularized latent component regression to reduce the dimension of the multi-scale features to obtain the multi-scale features after dimension reduction includes:

[0019] After multi-scale feature extraction, the data of each sliding window in the channel is used as the input matrix, and the dimension of the target matrix is set; and the input matrix and the target matrix are preprocessed;

[0020] Calculate the covariance matrix between the preprocessed input matrix and the preprocessed target matrix; and decompose the covariance matrix to determine the main directions of the preprocessed input matrix and the preprocessed target matrix, so as to obtain the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix;

[0021] Generate principal components based on the projection direction weight vector of the input matrix and perform fitting to form fitting data;

[0022] Use regularized latent component regression to reduce the dimension of the fitting data to obtain the multi-scale features after dimension reduction.

[0023] Further, the preprocessing of the input matrix and the target matrix includes: centering and normalizing the input matrix and the target matrix.

[0024] Further, decomposing the covariance matrix to determine the main directions of the preprocessed input matrix and the preprocessed target matrix, so as to obtain the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix, includes:

[0025] Through singular value decomposition, decompose the covariance matrix Decompose:

[0026] ;

[0027] In the above formula, is the left singular vector, representing the main direction of the preprocessed input matrix ; is the right singular vector, representing the main direction of the preprocessed target matrix ;

[0028] The projection direction weight vector of the input matrix is the first column in the left singular vector corresponding to the largest singular value, and the projection direction weight vector of the target matrix is the first column in the right singular vector corresponding to the largest singular value.

[0029] Further, the generating of the principal components based on the projection direction weight vector of the input matrix is:

[0030] ;

[0031] In the above formula, the weight vector is the optimal value determined in the iteration, represents thek a main component, X representing the input matrix.

[0032] Furthermore, it also includes normalizing the multi-scale features after dimensionality reduction.

[0033] Compared with the prior art, the present invention has the following technical effects:

[0034] The present invention first adds some important historical data to the original data as new features to capture the information at a specific historical time step; divides the prediction review window into different size scales, captures the corresponding time features from the long-term and short-term review windows, applies regularized latent component regression to reduce the dimension and regularize the data at different scales, and captures the most important feature part through principal component extraction, enabling the long-term trend and short-term fluctuations in the data to be processed separately, and at the same time avoiding the problems that the model cannot focus on the most important part and there is too much noise when the multi-scale feature dimension is high. At the same time, the reversibility of multi-level data normalization is used to adapt to the distribution differences of the data, and normalization is performed based on the data mean and standard deviation as the effective range of the feature. Description of the Drawings

[0035] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1 is a flowchart of the time series prediction method for potential data component extraction based on multi-scale features of the present invention;

[0037] Figure 2 is a fitting graph of the prediction results in the ETTh transformer temperature dataset;

[0038] Figure 3 is a residual graph of the prediction results in the ETTh transformer temperature dataset. Detailed Embodiments

[0039] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific embodiments, structures, features and effects of the technical solutions proposed according to the present invention. The specific features, structures or characteristics in one or more embodiments can be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0040] In one embodiment of the present invention, referring to Figures 1-3 , a time series prediction method for extracting potential data components based on multi-scale features is provided, including the following steps:

[0041] Obtain the current electrical transformer temperature time series data, and incorporate the historical electrical transformer temperature time series as features into the current electrical transformer temperature time series data to form fused feature data;

[0042] Extract features of different scales from the fused feature data to obtain multi-scale features; use regularized latent component regression to reduce the dimension of the multi-scale features to obtain the dimension-reduced multi-scale features;

[0043] Perform multi-level data normalization on the fused feature data, calculate the residual data, and apply regularized latent component regression to reduce the dimension of the residual data to obtain the dimension-reduced residual data features;

[0044] Based on the dimension-reduced multi-scale features and the dimension-reduced residual data features, construct an electrical transformer temperature time prediction model and train it, and use the trained electrical transformer temperature time prediction model for prediction.

[0045] The following details each of the above steps:

[0046] Step 100: Obtain the current electrical transformer temperature time series data, and incorporate the historical electrical transformer temperature time series as features into the current electrical transformer temperature time series data to form fused feature data.

[0047] As an example, this step may include:

[0048] Step 1001: Obtain the current electrical transformer temperature time series data, and generate input data features and prediction targets through a sliding window method.

[0049] Take the current electrical transformer temperature time series data as input, where represents the step size of the retrospective window, represents the feature dimension, and for each time step generate input data features , prediction targets through a sliding window, and the prediction target is the future observed value.

[0050] In this implementation, a specific implementation manner of step 1001 may be:

[0051] Starting from the starting point of the current electrical transformer temperature time series data, set a length of The review window will move along the time series data, one time step at a time; for each time step , extract from the current electrical transformer temperature time series data from to of consecutive data points as input data features , the input data features , represents the data value at the current time point . The input data features include the data from to of historical data points.

[0052] will to in the consecutive data points as the prediction target , the prediction target is ; among them, the initial review window length is set to 96 historical steps, and the prediction range is variable, can be 96, 192, 336, and 720.

[0053] Step 1002: Design the local historical steps, integrate the historical information of the time step as features into the input data features of the current time step to form the integrated features.

[0054] Integrate the historical information of the time step as features into the input data features of the current time step to capture the information of a specific historical time step or the important part of the historical time step. Through the backtracking steps, historical information farther than the traditional sliding window method can be obtained, enhancing the model's ability to capture long-term dependence relationships; at the same time, it can also focus on the key information in the short term and improve the model's ability to obtain short-term dependence relationships.

[0055] In this implementation, a specific implementation method of step 1002 can be:

[0056] Step 10021: For the time step , capture the first historical data before the current time step and add it as features to the input data features of the current time step to form the backtracking feature:

[0057] ;

[0058] In the above formula, represents the backtracking feature, represents the One data.

[0059] Step 10022: Combine the backtracking feature and the input data feature to form fused feature data.

[0060] The combination of the backtracking feature and the original feature is represented by the feature vector as: , and the dimension of the fused feature is .

[0061] Linear regression model combining a sliding window and a backtracking feature:

[0062] ;

[0063] In the above formula, represents the estimated value of the prediction target, represents the weight of the input data feature, represents the weight of the backtracking feature, represents the bias term.

[0064] Step 200: Extract features of different scales from the fused feature data to obtain multi-scale features; use regularized latent component regression to reduce the dimension of the multi-scale features to obtain the dimension-reduced multi-scale features.

[0065] As an example, this step may include:

[0066] Step 2001: The fused feature data enters multiple channels simultaneously, and each channel performs feature extraction of different scales.

[0067] In each channel, the sliding window is generated at different ratios to form a multi-resolution representation of the time series. The dimension of the input fused feature is F. For each scale S, both the retrospective window and the target window are downsampled, that is, the size of the retrospective window is gradually reduced to capture more detailed time fluctuations.

[0068] Step 20011: Obtain data features of different scales.

[0069] Obtain the features of the number of sliding windows at different scales , and the dimension of the matrix feature at this time is : :

[0070] ;

[0071] The scales used are , and sliding windows of different scales are obtained, which enables the model to extract features from multiple retrospective windows and enables it to capture feature information at different time scales:

[0072] ;

[0073] For each scale , a series of sliding windows of different sizes are generated. For example, at scale 1, the window size may be the same as the time step of the original data; at scale 2, the window size may be half of the original data; at scale 4, the window size may be a quarter of the original data; these windows will slide in time with different strides or overlapping degrees to capture features at different scales.

[0074] Step 20012: Concatenate the data features at different scales, perform weighted summation, and obtain the fused multi-scale features.

[0075] Concatenate the data features in different scale channels, sum them according to certain weights, that is, multiply and add the corresponding elements of different scale features through a trainable weight matrix corresponding elements to obtain the fused multi-scale feature X. The fused multi-scale feature X is the feature distribution after comprehensively weighting and summing the data features in different scale channels:

[0076] .

[0077] Step 2002: Use the data of each sliding window in different channels as the input matrix, and set the dimension of the target matrix.

[0078] After multi-scale feature extraction, use the data of each sliding window in different channels as the input matrix , and based on the target matrix , find a new set of low-dimensional latent components, that is, principal components, so that the principal components can not only explain the variance of the input matrix, but also be maximally correlated with the target matrix; and through iteration, find the optimal linear projection of the input matrix and the target matrix.

[0079] Step 2003: Preprocess the input matrix and the target matrix.

[0080] Wherein the preprocessing includes centering and standardization; centering is to subtract the mean of each feature so that the mean of each feature is 0, eliminate the offset of different features, and improve the convergence speed of the model; while standardization adjusts the data to zero mean and unit variance (i.e., standard deviation is 1), scales the features to a similar numerical range, and eliminates the scale problem.

[0081] Preprocess the input matrix and the target matrix , maximize the preprocessed input matrix and the preprocessed target matrix The covariance between them is found, and the maximum correlation components of the two in their respective projection directions are obtained. , where is X the projection direction weight vector of is Y the projection direction weight vector of

[0082] Through linear algebra methods, the objective function can be rewritten as , which is a bilinear optimization problem, where and need to be optimized simultaneously.

[0083] Step 2004: Calculate the covariance matrix between the preprocessed input matrix and the preprocessed target matrix, and decompose the covariance matrix to determine the principal directions of the preprocessed input matrix and the preprocessed target matrix, thereby obtaining the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix.

[0084] In this implementation, a specific implementation manner of step 2004 can be:

[0085] Step 20041: Calculate the covariance matrix between the preprocessed input matrix and the preprocessed target matrix , and the covariance matrix describes the linear relationship between the input features and the target variables;

[0086] Step 20042: Through singular value decomposition (SVD), decompose the covariance matrix :

[0087] ;

[0088] In the above formula, is the left singular vector, representing the principal direction of the preprocessed input matrix ; is the right singular vector, representing the principal direction of the preprocessed target matrix ;

[0089] Step 20043: The projection direction weight vector X of the input matrix is the first column corresponding to the largest singular value in the left singular vector , and the projection direction weight vector Y of the target matrix is the first column corresponding to the largest singular value in the right singular vector .

[0090] Step 2005: Based on the input matrixX Projection direction weight vector to generate principal components and perform fitting.

[0091] Each principal component is an integrated feature obtained from numerous features according to different weights. The k th principal component is calculated by the following formula:

[0092] ;

[0093] In the above formula, the weight vector is the optimal value determined in the iteration, represents a single feature that combines the sum of multiple features, and by represents maximizing the interpretation of the input matrix X and the target matrix Y covariance.

[0094] By extracting the most important feature parts through principal component extraction, it is possible to separately process the long-term trends and short-term fluctuations in the data, and at the same time avoid the problems that the model cannot focus on the most important parts and there is too much noise in the case of a high multi-scale feature dimension.

[0095] Step 2006: Use the Regularized Latent Component Regression (RLCR) model to reduce the dimension of the multi-scale features to obtain the reduced multi-scale features;

[0096] RLCR is a dimensionality reduction method that combines regularization and latent variable models.

[0097] Step 2007: Perform multi-level data normalization on the reduced multi-scale features so that the mean and standard deviation of each data point are standardized, and restore the data to the original distribution through reverse operations.

[0098] Step 300: Perform multi-level data normalization on the fused feature data, calculate the residual data, and apply regularized latent component regression to reduce the dimension of the residual data to obtain the reduced residual data features.

[0099] As an example, this step may include:

[0100] Step 3001: Perform multi-level data normalization on the fused feature data.

[0101] The input data fused with specific historical data is a three-dimensional array, where is the number of time steps, is the feature dimension, is the feature dimension at each time step. By normalizing each instance, the mean and standard deviation of each data point T are standardized.

[0102] Step 30011: Normalize the input data fused with specific historical data through the following formula to map all features to the same interval, which is jointly determined by the mean and variance of the data, effectively eliminating the scale differences between features.

[0103] ;

[0104] where represents the data features after normalization, represents the mean of each feature, represents the standard deviation of each feature, is a small smoothing value used to avoid division by zero problems.

[0105] Step 30012: Scale the data features after normalization between the specified minimum and maximum values:

[0106] ;

[0107] In the above formula, represents the value between the specified minimum and maximum values; represents the specified minimum value, represents the specified maximum value.

[0108] In this way, all feature values are restricted within the same scale, thus avoiding the influence of some features being too large or too small during model training, and its reversibility allows the data to still be restored to the original data scale after data normalization.

[0109] Step 3002: Fit the normalized feature data, calculate the difference between the fused feature data and the fitted value as the residual, and calculate the residual data.

[0110] Specifically, linear regression can be used to fit the normalized data, and then calculate the difference between the fused feature data and the fitted value as the residual data.

[0111] Step 3003: Apply the regularized latent component regression model for dimensionality reduction.

[0112] Step 3004: Restore the model output value to the scale after normalization :

[0113] ;

[0114] Restore to the original data space based on the standardized scale :

[0115] ;

[0116] This is very important for tasks that require interpreting and validating prediction results. The prediction results can be more conveniently inverse-normalized to the original scale, ensuring the interpretability and verifiability of the prediction results.

[0117] Step 400: Based on the multi-scale features after dimensionality reduction and the residual data features after dimensionality reduction, construct an electric transformer temperature time series prediction model. The electric transformer temperature time series prediction model is initially predicted through the above steps 100, 200, and 300 to obtain a difference between the predicted value and the actual value, which is the residual. According to the magnitude of this difference, training is repeatedly performed. Repeating the above multiple steps, as time goes by, new data continuously enters, and the prediction ability of the initially constructed model may decline. Therefore, the model needs to be updated and retrained regularly to ensure that it can continuously provide accurate predictions. After the training is completed, the trained electric transformer temperature time series prediction model is used for actual prediction.

[0118] Specifically, the multi-scale features after dimensionality reduction and the residual data features after dimensionality reduction are combined into a new feature set. This set will be used as the input of the prediction model. The prediction model, such as a linear regression model, is trained according to the mean square error MSE as the loss function, and finally the target data is predicted and output.

[0119] Using regularization, further fit the details of the trend, and make the generalization ability of the model better through the loss function. The generalization ability of the trend model is made better through the following formula:

[0120] ;

[0121] In the above formula, represents the true value; represents the regression coefficient or the model parameter vector, indicating the weights that the model needs to learn; represents the hyperparameter of the regularization strength.

[0122] Experimental result analysis:

[0123] Experiments are carried out on the publicly available power transformer temperature dataset ETTh1. All time series are segmented into a retrospective window L = 96, the prediction range H ∈ {96, 192, 336, 720}, and the step size is 1, which means that each subsequent window is moved one step, and the resulting error results are between 0.08 and 0.3.

[0124] By predicting the ETTh transformer temperature time series dataset, this lightweight time series prediction method based on multi-scale feature extraction of potential data components has achieved good accurate predictions. Refer to Figure 2 , the trend of the predicted values is roughly consistent with the actual values. Especially in some smooth regions, the model has well captured the trend.

[0125] To prove the feasibility and authenticity of the present invention, refer to Figure 3 . The temperature feature map of the ETT transformer was compared. The residual is the difference between the predicted value and the true value of the model. The residual plot can show the distribution of these differences. Ideally, the residuals should be approximately normally distributed with a mean of zero, indicating that the model evenly distributes errors at each prediction point without systematic bias. And the residual plot helps to detect whether there are systematic errors in the model, test the fitting quality of the model, whether there are biases, overfitting, or uncaught trends.

[0126] There are fewer restrictions on the devices used. The devices relied on by the time series prediction inventions that achieve the same effect are much higher than those of the present invention. The present invention involves fewer parameters and has a fast operation speed, and can achieve the same prediction accuracy at a faster speed.

[0127] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A time series prediction method for extracting potential data components based on multi-scale features, characterized in that Including the following steps: Obtain the current power transformer temperature time series data, and incorporate the historical power transformer temperature time series as features into the current power transformer temperature time series data to form fused feature data, including: Starting from the starting point of the current electrical transformer temperature time series data, set a retrospective window with a length of . The retrospective window will move along the time series data, moving one time step each time; for each time step , extract from the current electrical transformer temperature time series data from to in consecutive data points as input data features; for time step , capture the historical data before the current time step to form a retrospective feature; combine the retrospective feature and the input data feature to form a fused feature data; Extract features of different scales from the fused feature data to obtain multi-scale features; use regularized latent component regression to reduce the dimension of the multi-scale features to obtain the dimension-reduced multi-scale features; Perform multi-level data normalization on the fused feature data, calculate the residual data, and apply regularized latent component regression to reduce the dimension of the residual data to obtain the dimension-reduced residual data features; Based on the dimension-reduced multi-scale features and the dimension-reduced residual data features, construct a power transformer temperature time prediction model and train it, and use the trained power transformer temperature time prediction model for prediction.

2. The time series prediction method for extracting potential data components based on multi-scale features according to claim 1, characterized in that, Extract features of different scales from the fused feature data to obtain multi-scale features, including: Apply sliding windows of different scales to the fused feature data to obtain data features of different scales; Stitch together the data features of different scales and perform weighted summation through a weight matrix to obtain the fused multi-scale features.

3. The time series prediction method for extracting potential data components based on multi-scale features according to claim 2, characterized in that, Use regularized latent component regression to reduce the dimension of the multi-scale features to obtain the dimension-reduced multi-scale features, including: After multi-scale feature extraction, use the data of each sliding window in the channel as the input matrix, set the target matrix dimension; and preprocess the input matrix and the target matrix; Calculate the covariance matrix between the preprocessed input matrix and the preprocessed target matrix; and decompose the covariance matrix to determine the main directions of the preprocessed input matrix and the preprocessed target matrix, thereby obtaining the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix; Generate principal components based on the projection direction weight vector of the input matrix and perform fitting to form fitting data; Use regularized latent component regression to reduce the dimension of the fitting data to obtain the dimension-reduced multi-scale features.

4. A time series prediction method for extracting potential data components based on multi-scale features according to claim 3, characterized in that, The preprocessing of the input matrix and the target matrix includes: centering and normalizing the input matrix and the target matrix.

5. The time series prediction method for extracting potential data components based on multi-scale features according to claim 3, characterized in that, Decompose the covariance matrix to determine the main directions of the preprocessed input matrix and the preprocessed target matrix, thereby obtaining the projection direction weight vector of the input matrix and the projection direction weight vector of the target matrix, including: Through singular value decomposition, the covariance matrix is decomposed: ； In the above formula, is the left singular vector, representing the main direction of the preprocessed input matrix ; is the right singular vector, representing the main direction of the preprocessed target matrix ; The projection direction weight vector of the input matrix is the left singular vector The first column in corresponds to the largest singular value, and the projection direction weight vector of the target matrix is the right singular vector. The first column in corresponds to the largest singular value.

6. The time series prediction method for extracting potential data components based on multi-scale features according to claim 5, wherein The generation of the principal components based on the projection direction weight vector of the input matrix is: ； In the above formula, the weight vector is the optimal value determined in the iteration, represents the k th principal component, X represents the input matrix.

7. A time series prediction method for extracting potential data components based on multi-scale features according to claim 6, characterized in that, It also includes normalizing the dimension-reduced multi-scale features.

Images