Typical industry electricity consumption prediction method based on wavelet decomposition and Transform neural network

Abstract

The invention discloses a typical industry power consumption prediction method based on wavelet decomposition and a Transform neural network, and belongs to the technical field of power load prediction, and the method comprises the following steps: carrying out the normalization preprocessing of original current data, and eliminating the dimensional difference; performing soft threshold denoising on the high-frequency subsequences, and retaining effective data features; the processed low-frequency sub-sequence and the denoised high-frequency sub-sequence are used as independent input channels of a Transform model respectively; the fusion features trained by the Transform model are input into a full connection layer, a current prediction value of a specified time step in the future is output, a prediction variance is added to provide a confidence interval, and the prediction time step is 24 hours. According to the method, the problems of multi-scale feature capture deficiency, high-frequency noise sensitivity and long-period dependence on modeling in a typical industry power consumption prediction process are solved.

Description

Technical Field

[0001] This invention belongs to the field of power load forecasting technology, specifically relating to a typical industry power consumption forecasting method based on wavelet decomposition and Transformer neural network. Background Technology

[0002] Currently, driven by the wave of industrial intelligence and energy digitalization, accurate time-series forecasting has become a key technology for ensuring the stable operation of power systems and improving energy efficiency management. However, facing complex industrial current data, existing mainstream time-series forecasting methods still have significant limitations, restricting further improvements in forecast accuracy. First, traditional statistical models (such as ARIMA) mainly rely on a single time scale for analysis, making it difficult to effectively capture the intertwined multi-scale features in current data—including instantaneous fluctuations at the second or minute level, as well as long-term evolution trends at the hour, day, and even seasonal levels. This lack of multi-scale feature extraction capability directly limits forecast accuracy. Second, current data collected in industrial fields inevitably contains high-frequency non-stationary noise caused by electromagnetic interference, sensor measurement errors, etc. Traditional models are generally highly sensitive to this type of noise interference, lack sufficient robustness, and are prone to forecast distortion, deviating significantly from the actual value. Furthermore, although sequence models based on recurrent neural networks (such as RNN / LSTM) have made progress in handling sequence dependencies, their inherent recursive calculation mechanism is inefficient and inadequate in capturing long-term dependencies spanning weeks or even months. At the same time, the gradient vanishing or exploding problem that is difficult to avoid during the long-term dependency propagation process further constitutes a fundamental bottleneck for stable and efficient modeling of long-term temporal correlations.

[0003] The three shortcomings of the existing technology are:

[0004] I. Deficiencies in capturing multi-scale features: Traditional statistical models (such as ARIMA) cannot simultaneously capture the instantaneous fluctuations at the second level and the long-term trends at the hour / season level in current data.

[0005] Second, the model is sensitive to high-frequency noise. Existing models lack robustness to high-frequency non-stationary noise caused by electromagnetic interference, sensor noise, etc., which leads to prediction distortion.

[0006] Third, the defects in long-term dependency modeling: Recurrent neural networks such as RNN / LSTM suffer from gradient vanishing / exploding problems and are inefficient at capturing long-term dependencies across weeks / months.

[0007] These shortcomings together constitute the core challenge that urgently needs to be overcome in the field of industrial current timing prediction. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a typical industry power consumption forecasting method based on wavelet decomposition and Transformer neural network, thus solving the problems mentioned in the background art.

[0009] The objective of this invention is achieved as follows: a typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network, comprising the following steps: normalizing the original current data to eliminate dimensional differences; applying soft thresholding to the high-frequency subsequences to retain effective data features; using the processed low-frequency subsequences and the denoised high-frequency subsequences as independent input channels of the Transformer model; inputting the fused features trained by the Transformer model into a fully connected layer, outputting the predicted current value at a specified future time step, and adding a prediction variance to provide a confidence interval, wherein the prediction time step is 24 hours. By normalizing the original current data to eliminate dimensional differences, using wavelet decomposition with a db4 wavelet basis, and a 3-layer decomposition, the low-frequency subsequences are decomposed into long-term trends, and the high-frequency subsequences into short-term fluctuations. Discrete wavelet transform is used to separate the original sequence into multiple scales, covering dynamic features from the second level to the seasonal level, thus solving the problem of missing multi-scale feature capture. By applying soft thresholding to the high-frequency subsequences, effective data features are preserved. Soft thresholding denoising reduces noise in the high-frequency subsequences, filtering out non-stationary noise such as electromagnetic interference, while retaining effective signal features, thus addressing the problem of high-frequency noise sensitivity. The low-frequency subsequences and the denoised high-frequency subsequences are used as independent input channels for the Transformer model, directly replacing RNN / LSTM. The attention mechanism is computationally non-recursive, avoiding gradient problems, and parallel computing efficiently models cross-month dependencies, solving the problem of modeling long-period dependencies.

[0010] Furthermore, the normalization preprocessing specifically includes: performing a discrete wavelet transform on the preprocessed current sequence using the db4 wavelet basis, decomposing it into low-frequency and high-frequency subsequences. The low-frequency subsequence reflects the long-term trend of the current, while the high-frequency subsequence captures short-term fluctuations and noise. The specific choice of the db4 wavelet basis, specifying its use for discrete wavelet transform, addresses the "defect in capturing multi-scale features" because the db4 basis has been experimentally proven to be superior to other bases (such as Haar or Sym) in capturing the time-varying characteristics of current data (e.g., second-level fluctuations), resulting in higher decomposition accuracy and computational efficiency. Addressing the problem that traditional models (such as ARIMA) cannot handle multi-scale data in the background technology, this feature enhances creativity and reduces prediction errors by optimizing the wavelet basis type.

[0011] Furthermore, in the Transformer model, a multi-head attention mechanism is used for training. Multi-head attention learns the local and global dependencies of features at different scales in parallel through multiple attention heads. The model loss function uses mean squared error loss, and the model optimization uses the Adam optimization function. The discrete wavelet transform has three decomposition layers. The low-frequency subsequences correspond to approximation coefficients, and the high-frequency subsequences correspond to detail coefficients. The optimized setting of three decomposition layers: Specifying three wavelet decomposition layers balances feature granularity and computational cost, solving the "multi-scale feature capture defect," because three layers are experimentally optimal, effectively separating second-level, minute-level, and hour-level dynamics simultaneously, avoiding over-decomposition or under-decomposition. Through the combination of multi-head attention mechanism, mean squared error loss, and Adam optimizer: the multi-head attention mechanism learns feature dependencies in parallel, combined with mean squared error loss (MSE) and the Adam optimization function, solving the "long-period dependency modeling defect" (RNN / LSTM gradient vanishing), improving training efficiency and global dependency capture capability (such as cross-week / monthly trends). It overcomes the inefficiency of RNN / LSTM in the background technology, creatively optimizes model training, and reduces convergence time.

[0012] Furthermore, the multi-head attention mechanism of the Transformer model includes four attention heads. The parameters of the Transformer model include: an input dimension of 3, a mapping feature dimension of 64, an encoder layer of 4, a training batch size of 32, a total training epochs of 50, and a learning rate of 0.001. The specific configuration of four attention heads: limiting the multi-head attention to four heads optimizes the parallel learning capability of features and solves the "long-cycle dependency modeling defect," as four heads are optimal in experiments and can efficiently capture local and global dependencies at different scales (such as daily / weekly fluctuations in industrial electricity consumption). Parameter combinations (input dimension 3, feature dimension 64, encoder layer 4, etc.): specific parameters such as an input dimension of 3 (corresponding to the time step), a feature dimension of 64, and a layer size of 4, solve the "long-cycle dependency modeling defect." These parameters, after experimental tuning, optimize training efficiency and prediction accuracy (e.g., batch size of 32 and epochs of 50 reduce training time by 30%, and a learning rate of 0.001 avoids gradient oscillations).

[0013] Furthermore, the Transformer model, after training, includes an output layer, which uses a fully connected layer to generate predicted values ​​and confidence intervals. The fully connected layer outputs predicted values ​​and confidence intervals: integrating the fully connected layer in the final output layer directly generates predicted values ​​and confidence intervals, resolving the derivative problem (prediction uncertainty) of the "high-frequency noise sensitivity defect." This provides decision support in typical industry applications (such as power grid dispatching).

[0014] The beneficial effects of this invention are as follows: By normalizing the original current data to eliminate dimensional differences, wavelet decomposition using the db4 wavelet basis and a 3-level decomposition, the low-frequency subsequence is decomposed into long-term trends, and the high-frequency subsequence into short-term fluctuations. Discrete wavelet transform is used to separate the original sequence across multiple scales, covering dynamic features from the second level to the seasonal level, thus solving the problem of missing multi-scale feature capture. Soft thresholding denoising is applied to the high-frequency subsequence to retain effective data features, filtering out non-stationary noise such as electromagnetic interference while preserving effective signal features, thus solving the problem of high-frequency noise sensitivity. By using the low-frequency subsequence and the denoised high-frequency subsequence as independent input channels for the Transformer model, it directly replaces RNN / LSTM. The attention mechanism is non-recursive, avoiding gradient problems, and parallel computing efficiently models cross-month dependencies, solving the problem of modeling long-period dependencies. Attached Figure Description

[0015] Figure 1 This is a flowchart of the wavelet and Transformer of this invention;

[0016] Figure 2 This is a flowchart of the wavelet processing of the present invention;

[0017] Figure 3 This is a diagram of the Transformer multi-channel input structure of the present invention;

[0018] Figure 4 This is a comparison chart of the actual and predicted values ​​of the current time-series data of this invention;

[0019] Figure 5 This is a diagram of the wavelet decomposition coefficients of this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. It should be noted that this is only for the purpose of more clearly illustrating and explaining the present invention.

[0021] Example 1

[0022] like Figure 1-5As shown, this embodiment discloses a typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network. The method includes the following steps: normalizing the original current data to eliminate dimensional differences; applying soft thresholding to the high-frequency subsequences to retain effective data features; using the processed low-frequency subsequences and the denoised high-frequency subsequences as independent input channels of the Transformer model; inputting the fused features trained by the Transformer model into a fully connected layer to output the predicted current value at a specified future time step, with an added prediction variance to provide a confidence interval, wherein the prediction time step is 24 hours. By normalizing the original current data to eliminate dimensional differences, using wavelet decomposition with a db4 wavelet basis, and a 3-layer decomposition, the low-frequency subsequences are decomposed into long-term trends, and the high-frequency subsequences into short-term fluctuations. Discrete wavelet transform is used to separate the original sequence into multiple scales, covering dynamic features from the second level to the seasonal level, thus solving the problem of missing multi-scale feature capture. By applying soft thresholding to the high-frequency subsequences, effective data features are preserved. Soft thresholding denoising reduces noise in the high-frequency subsequences, filtering out non-stationary noise such as electromagnetic interference, while retaining effective signal features, thus addressing the problem of high-frequency noise sensitivity. The low-frequency subsequences and the denoised high-frequency subsequences are used as independent input channels for the Transformer model, directly replacing RNN / LSTM. The attention mechanism is computationally non-recursive, avoiding gradient problems, and parallel computing efficiently models cross-month dependencies, solving the problem of modeling long-period dependencies.

[0023] Example 2

[0024] like Figure 1-5As shown, this embodiment discloses a typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network. The method includes the following steps: normalizing the original current data to eliminate dimensional differences; applying soft thresholding to the high-frequency subsequences to retain effective data features; using the processed low-frequency subsequences and the denoised high-frequency subsequences as independent input channels of the Transformer model; inputting the fused features trained by the Transformer model into a fully connected layer to output the predicted current value at a specified future time step, with an added prediction variance to provide a confidence interval, wherein the prediction time step is 24 hours. By normalizing the original current data to eliminate dimensional differences, using wavelet decomposition with a db4 wavelet basis, and a 3-layer decomposition, the low-frequency subsequences are decomposed into long-term trends, and the high-frequency subsequences into short-term fluctuations. Discrete wavelet transform is used to separate the original sequence into multiple scales, covering dynamic features from the second level to the seasonal level, thus solving the problem of missing multi-scale feature capture. By applying soft thresholding to the high-frequency subsequences, effective data features are preserved. Soft thresholding denoising reduces noise in the high-frequency subsequences, filtering out non-stationary noise such as electromagnetic interference, while retaining effective signal features, thus addressing the problem of high-frequency noise sensitivity. The low-frequency subsequences and the denoised high-frequency subsequences are used as independent input channels for the Transformer model, directly replacing RNN / LSTM. The attention mechanism is computationally non-recursive, avoiding gradient problems, and parallel computing efficiently models cross-month dependencies, solving the problem of modeling long-period dependencies.

[0025] To achieve better results, the normalization preprocessing specifically includes: performing a discrete wavelet transform on the preprocessed current sequence using the db4 wavelet basis, decomposing it into low-frequency and high-frequency subsequences. The low-frequency subsequence reflects the long-term trend of the current, while the high-frequency subsequence captures short-term fluctuations and noise. The specific choice of the db4 wavelet basis, specifying its use for discrete wavelet transform, addresses the "defect in capturing multi-scale features" because the db4 basis has been experimentally proven to be superior to other bases (such as Haar or Sym) in capturing the time-varying characteristics of current data (e.g., second-level fluctuations), resulting in higher decomposition accuracy and computational efficiency. Addressing the problem that traditional models (such as ARIMA) cannot handle multi-scale data in the background technology, this feature enhances creativity and reduces prediction errors by optimizing the wavelet basis type.

[0026] To achieve better results, a multi-head attention mechanism is used for training in the Transformer model. Multi-head attention learns local and global dependencies of features at different scales in parallel through multiple attention heads. The model loss function uses mean squared error loss, and the model optimization uses the Adam optimization function. The discrete wavelet transform has three decomposition layers. The low-frequency subsequences correspond to approximation coefficients, and the high-frequency subsequences correspond to detail coefficients. The optimized setting of three decomposition layers: Specifying three wavelet decomposition layers balances feature granularity and computational cost, solving the "multi-scale feature capture defect," as three layers are optimal in experiments and can effectively separate second-level, minute-level, and hour-level dynamics simultaneously, avoiding over-decomposition or under-decomposition. Through the combination of multi-head attention mechanism, mean squared error loss, and Adam optimizer: the multi-head attention mechanism learns feature dependencies in parallel, combined with mean squared error loss (MSE) and the Adam optimization function, solving the "long-period dependency modeling defect" (RNN / LSTM gradient vanishing), improving training efficiency and global dependency capture capability (such as cross-week / monthly trends). It overcomes the inefficiency of RNN / LSTM in the background technology, creatively optimizes model training, and reduces convergence time.

[0027] For better performance, the Transformer model's multi-head attention mechanism includes four attention heads. The Transformer model parameters include: an input dimension of 3, a mapping feature dimension of 64, an encoder layer of 4, a training batch size of 32, a total training epochs of 50, and a learning rate of 0.001. The specific configuration of four attention heads optimizes the parallel learning capability of features and addresses the "long-cycle dependency modeling defect," as four heads are optimal in experiments, efficiently capturing local and global dependencies at different scales (such as daily / weekly fluctuations in industrial electricity consumption). The parameter combinations (input dimension 3, feature dimension 64, encoder layer 4, etc.) address the "long-cycle dependency modeling defect." These parameters, after experimental tuning, optimize training efficiency and prediction accuracy (e.g., batch size 32 and epochs 50 reduce training time by 30%, and a learning rate of 0.001 avoids gradient oscillations).

[0028] To achieve better results, the Transformer model includes an output layer after training. This output layer uses a fully connected layer to generate predicted values ​​and confidence intervals. The fully connected layer outputs predicted values ​​and confidence intervals: Integrating the fully connected layer in the final output layer directly generates predicted values ​​and confidence intervals, addressing the derivative problem of "high-frequency noise sensitivity" (prediction uncertainty). This provides decision support in typical industry applications (such as power grid dispatch).

[0029] In this embodiment, experiments are conducted based on waveform data. Taking a science park as an example, five days of continuously collected waveform data are used as the sample set. After Fourier transform, the data from the first four days are used as the training set, and the last day is used as the test set. Wavelet decomposition and Transformer prediction and ARIMA / LSTM prediction experiments are compared.

[0030] The two models are as follows:

[0031] The basic structure of a Transformer: Assuming the input time series is: x = [x1, x2, ..., x...] T The key to Transformer is:

[0032] Position encoding: Since Transformer does not have a concept of time sequence, position encoding is used to supplement timing information.

[0033] Commonly used sine and cosine encoding:

[0034]

[0035] Where pos represents the time position and d is the embedding dimension.

[0036] Self-attention mechanism: Perform a linear transformation on the input to obtain the query Q, key K, and value V, and then calculate the attention.

[0037]

[0038] Multi-head attention refers to using multiple attention heads in parallel, concatenating the results, and then linearly mapping them.

[0039] Discrete wavelet transform, given an input sequence x[n], passes it through a low-pass filter h[n] and a high-pass filter g[n] to obtain:

[0040]

[0041] We can continue to recursively decompose a1[n] to obtain the multi-scale wavelet coefficients: x[n] → a L ,d L The specific steps for d1 are as follows:

[0042] Step 1: Data processing for multi-scale wavelet decomposition;

[0043] 1.1 Data preprocessing: The raw current data is normalized to eliminate dimensional differences.

[0044] 1.2 Wavelet basis function selection: The current sequence is subjected to a 3-level discrete wavelet transform (DWT) using the db4 wavelet basis, which decomposes it into a low-frequency subsequence (approximation coefficients) and a high-frequency subsequence (detail coefficients).

[0045] 1.3 Subsequence Reconstruction: Low-frequency subsequence: reflects the long-term trend of current (e.g., daily / weekly periodicity). High-frequency subsequence: captures short-term fluctuations and noise (e.g., equipment transient response).

[0046] 1.4 Noise Suppression: Apply soft thresholding to the high-frequency subsequences to denoise them while preserving effective data features.

[0047] Step 2: Transformer multi-scale training;

[0048] 2.1 Input Feature Construction: The low-frequency subsequence after wavelet decomposition and the high-frequency subsequence after denoising are used as independent input channels of the Transformer.

[0049] 2.2 Self-attention mechanism optimization: Multi-Head Attention: Learns the local and global dependencies of features at different scales in parallel through 4 attention heads.

[0050] 2.3 Model Loss: The mean squared error of the MSE is defined as the loss function.

[0051] 2.4 Model Optimization: The Adam optimization function was used for optimization.

[0052] Step 3: Prediction results are generated;

[0053] Output layer design: The fused features are input into a fully connected layer, which outputs the predicted current value for the next T steps (e.g., 24 hours). A prediction variance is added to the output layer to provide a prediction confidence interval, enhancing the reliability of the results.

[0054] By effectively combining multi-scale signal decomposition and global temporal modeling, prediction accuracy and scene adaptability are significantly improved.

[0055] Its core effect is reflected in:

[0056] 1. Deep multi-scale feature fusion: By using wavelet decomposition to accurately separate and reconstruct the low-frequency trend and high-frequency fluctuation of current data, and combining the powerful global self-attention mechanism of Transformer, a unified modeling and deep mining of long and short-term time series features is achieved, which effectively overcomes the shortcomings of traditional methods in multi-scale feature extraction.

[0057] 2. High robustness noise suppression: By performing soft threshold denoising on the high-frequency subsequence, high-frequency non-stationary noise interference caused by electromagnetic interference, sensor errors, etc. is significantly suppressed, which enhances the robustness of the model to complex industrial data and avoids distortion of prediction results.

[0058] 3. Scenario-based Dynamic Association Modeling: The model can adapt to the nonlinear coupling laws of dynamic elements such as current fluctuation characteristics and harmonic content under different typical industrial scenarios (e.g., office building users, catering users, lighting users), forming a multi-scale-scenario-based coupling mapping network, which significantly improves the predictive model's adaptability to specific users. The final output of high-precision current prediction sequences and their confidence intervals provides a reliable basis for the dynamic control of power equipment, harmonic mitigation, and the formulation of demand response strategies. It realizes refined and scenario-adaptive power grid energy efficiency management, effectively solving the industry pain points of traditional methods such as noise sensitivity, weak long-cycle dependency modeling, and poor scenario adaptability.

[0059] The experimental results are shown in Table 1:

[0060] Table 1: Comparison of Current Prediction Timing Chart

[0061]

[0062] According to Table 1, it can be seen that when wavelet and Transformer are combined for time series prediction, the MAPE and RMSE indices are significantly lower than those of ARIMA and LSTM, resulting in the best prediction performance.

[0063] Table 2. Time-series comparison of wavelet denoising

[0064]

[0065] According to Table 2, it can be seen that the model prediction accuracy is improved after wavelet high-frequency denoising.

[0066] The experimental results are shown in Table 1, the algorithm parameters are shown in Table 3, and the experimental result images are shown in Table 4. Figure 4 As shown. According to Figure 4 As shown, the red part is the true curve, and the blue part is the predicted curve. Based on the image and experimental comparison, the wavelet + Transformer algorithm predicts the curve better than ARIMA and LSTM images. Figure 5 It can be seen that without wavelet decomposition, the low-frequency part can show the trend of the time series, while the high-frequency part corresponds to faster fluctuations and noise.

[0067] Table 3. Wavelet + Transformer Parameter Table

[0068]

[0069]

[0070] By setting the parameters in Table 3 for wavelet decomposition and Transformer combined prediction, the accuracy of power prediction indicators is improved compared to ARIMA and LSTM models. The experimental results are shown in Table 1.

[0071] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network, characterized in that, Includes the following steps: The raw current data is preprocessed by normalization to eliminate dimensional differences; Apply soft thresholding to high-frequency subsequences to denoise them while preserving effective data features; The processed low-frequency subsequence and the denoised high-frequency subsequence are used as independent input channels of the Transformer model. The fused features from the Transformer model training are input into the fully connected layer, which outputs the current prediction value at a specified future time step, and adds the prediction variance to provide a confidence interval, where the prediction time step is 24 hours.

2. The typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network according to claim 1, characterized in that, The normalization preprocessing specifically includes: using the db4 wavelet basis to perform discrete wavelet transform on the preprocessed current sequence, decomposing it into low-frequency subsequences and high-frequency subsequences, wherein the low-frequency subsequences reflect the long-term trend of the current, and the high-frequency subsequences capture short-term fluctuations and noise.

3. The typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network according to claim 1, characterized in that, In the Transformer model, a multi-head attention mechanism is used for training. Multi-head attention learns the local and global dependencies of features at different scales in parallel through multiple attention heads. The model loss function adopts mean squared error loss, and the model optimization adopts Adam optimization function.

4. The typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network according to claim 2, characterized in that, The discrete wavelet transform has a decomposition layer of 3.

5. The typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network according to claim 2, characterized in that, The low-frequency subsequence corresponds to the approximation coefficients, and the high-frequency subsequence corresponds to the detail coefficients.

6. The typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network according to claim 3, characterized in that, The multi-head attention of the Transformer model includes four attention heads.

7. The typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network according to claim 6, characterized in that, The parameters of the Transformer model include: input dimension of 3, mapping feature dimension of 64, encoder layer of 4, training batch of 32, total training epochs of 50, and learning rate of 0.

001.

8. The typical industry electricity consumption forecasting method based on wavelet decomposition and Transformer neural network according to claim 1, characterized in that, The Transformer model, after training, includes an output layer, which uses a fully connected layer to generate predicted values ​​and confidence intervals.

Images