A photovoltaic power prediction method based on inverted Transformer and weighted decomposition

Abstract

A photovoltaic (PV) power prediction method based on an inverted Transformer combined with weighted decomposition consists of an inverted Transformer architecture, a weighted series decomposition module, and a probabilistic sparse attention mechanism. Step 1: The inverted Transformer architecture captures the complex nonlinear and volatile characteristics of PV power data, thereby improving the model's ability to learn multivariate correlations. Step 2: The weighted series decomposition module extracts the temporal characteristics and periodicity of PV data and introduces weights consistent with the actual PV power output variation pattern to constrain the model's output. Step 3: The probabilistic sparse attention mechanism combined with the inverted architecture enables the prediction method to maintain high computational efficiency when processing long-sequence data, enhancing model performance. Step 4: A two-stage structure is constructed to process different characteristics of PV data separately to improve prediction accuracy, robustness, and computational efficiency. Step 5: PV power prediction is performed through the prediction layer.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power generation prediction, specifically to a method for photovoltaic power prediction using an inverted Transformer combined with weighted decomposition. Background Technology

[0002] The explosive growth and large-scale grid connection of new energy sources in my country is an inevitable trend. Photovoltaic complementary systems are being used more and more widely in agriculture. Agricultural users are both energy producers and consumers. Compared with traditional energy power generation technologies, photovoltaic power generation output is affected by meteorological factors such as temperature and actual irradiance, resulting in greater randomness and fluctuations. This brings difficulties to load dispatching and consumption. Due to the time-varying nature of uncontrollable natural conditions (such as sunlight, wind, and temperature), photovoltaic power generation output also poses difficulties for the power grid in load dispatching and consumption. Therefore, by predicting photovoltaic power generation, the power generation plan of the photovoltaic power generation system can be adjusted, or a plan for local photovoltaic consumption can be formulated in advance, thereby achieving balanced dispatching of the power grid and greatly improving energy utilization efficiency.

[0003] Photovoltaic power generation data exhibits highly nonlinear, volatile, and periodic characteristics, posing numerous challenges to traditional forecasting methods. First, the nonlinear and dynamic nature of photovoltaic power generation data makes capturing long-term dependencies difficult, especially when forecasting power generation over long time intervals; forecasting later time periods is often more challenging than earlier ones. Second, the periodic characteristics of photovoltaic power generation, such as the diurnal and nighttime variations in power generation, further complicate forecasting.

[0004] Currently, artificial intelligence technology has developed rapidly in recent years, providing convenience for the development of various industries. Among them, deep learning technology has shone brightly in various industries, and the technology of using deep learning for prediction has become increasingly mature. Some researchers have also begun to use neural networks for photovoltaic power generation prediction. However, existing photovoltaic power prediction methods based on neural networks have two main drawbacks: firstly, photovoltaic power generation data has many characteristics, containing low-quality information and exhibiting significant fluctuations; secondly, traditional neural networks are prone to getting trapped in local extrema, resulting in inaccurate predictions. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a photovoltaic power prediction method that combines an inverted Transformer with weighted decomposition.

[0006] This invention is a photovoltaic power prediction method combining an inverted Transformer and weighted decomposition. The method utilizes an inverted Transformer architecture, a weighted series decomposition module, and a probabilistic sparse attention mechanism to fully capture the complex nonlinear and volatile characteristics of photovoltaic power data time series. The weighted series decomposition divides the original data into trend, seasonal, and random components, and then processes the decomposed data accordingly. The probabilistic sparse attention mechanism adjusts the attention coefficients so that a few key dot products in the photovoltaic data provide the main attention, thereby reducing computational complexity and improving the method's computational efficiency.

[0007] The present invention has the following advantages:

[0008] 1. A method for photovoltaic power prediction combining inverted Transformer and weighted decomposition is proposed. This method divides the model into two stages to address different features in the data. Complex nonlinear and volatile features are extracted through the inverted Transformer architecture and probabilistic sparse attention mechanism; the periodic features of the data are captured through weighted series decomposition.

[0009] 2. An inverted Transformer architecture is constructed and a sparse attention mechanism is integrated to capture the complex nonlinear and volatile characteristics of data and reduce computational complexity. This network can capture the correlation between multiple variables more naturally through the attention mechanism and the inversion of responsibility of the feedforward neural network.

[0010] 3. Construct a weighted series decomposition module, which decomposes the first-stage output data into detrended, seasonal, and random components using the moving average method. This addresses periodic characteristics and uses a weighting mechanism to standardize the characteristics of photovoltaic power, thereby improving the model's accuracy.

[0011] 4. Extensive comparative experiments were conducted on two sets of traffic photovoltaic datasets. The experiments show that the method of this invention achieves the best prediction performance compared with existing baseline methods. Attached Figure Description

[0012] Figure 1 It is a photovoltaic power prediction method that combines inverted Transformer with weighted decomposition: TSformer structure diagram;

[0013] Figure 2 This is a diagram comparing the inverted Transformer and the original Transformer;

[0014] Figure 3 This is a periodic analysis graph of the dataset used;

[0015] Figure 4This is a comparison chart of the performance of our method model with several commonly used baseline models;

[0016] Figure 5 It is a visualization of the model ablation experiment; Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the embodiments.

[0018] Implementation steps

[0019] This invention is a photovoltaic power prediction method based on inverted Transformer combined with weighted decomposition, comprising the following steps:

[0020] Step S1: Obtain the historical operation dataset of the photovoltaic power station and preprocess the historical data;

[0021] The specific method for step S1 is as follows:

[0022] Acquire photovoltaic processing data from the photovoltaic power station and check for outliers in the data. Mark outliers. For outlier detection, since the historical data of the photovoltaic power station conforms to a normal distribution and the data does not change much in a short period of time, the 3σ principle can be used for outlier detection.

[0023] Step S2: Embed the processed data using a multilayer perceptron to transform it into a high-dimensional feature representation;

[0024] The specific method for step S2 is as follows:

[0025] The input data sequence is embedded using a multilayer perceptron to generate a high-dimensional feature representation of the sequence. If the input sequence X = [x1, x2, ..., x...], ... n Each data point in the sequence is embedded using a multilayer perceptron to form a new sequence E. E is then aggregated to obtain variable labels V. These variable labels are used as model input.

[0026] Step S3: Model construction. The first stage uses the inverted Transformer architecture; the second stage introduces a weighted series decomposition method.

[0027] The specific method for step S3 is as follows:

[0028] In PyTorch, an inverted Transformer architecture model and a weighted series decomposition model are built. The processed data sequentially passes through sparse attention, layer normalization, a feedforward neural network, and another layer normalization to form the inverted architecture. Then, a mapping layer outputs the first-stage model result. The inverted Transformer architecture is as follows: Figure 2As shown, the entire photovoltaic power sequence is independently embedded into variant tokens, where the embedded tokens aggregate the global representation of the sequence, and the entire sequence of the same variable is mapped to a high-dimensional feature representation. Secondly, an attention mechanism is applied to the embedded variant tokens to reveal multivariate correlations between variables. Next, a feedforward neural network (FFN) is used to extract the sequence representation of each token. Then, through a series decomposition module based on moving averages and a weighting mechanism module based on trigonometric functions, the final model result is output through a mapping layer. The overall architecture is as follows. Figure 1 As shown, the data undergoes processing in the first and second stages before the final output result is obtained.

[0029] Step S4: Combine the two-stage models and train them using the historical dataset.

[0030] The specific method for step S4 is as follows:

[0031] The first-stage model and the second-stage model are sequentially concatenated to form a complete model. The processed data is then divided into training, validation, and test sets in a 7:2:1 ratio. The photovoltaic power output for the next 96-960 time steps is predicted using the first 96 time steps of the dataset.

[0032] The dataset used in step S4 has a time step of 15 minutes, and the two datasets contain 17761 and 20000 time steps respectively. Figure 3 As shown, a visualization analysis of the dataset revealed that the actual photovoltaic power output has a fixed zero value and exhibits a clear diurnal periodicity, with the variation pattern closely related to the Sin function.

[0033] Step S5: Use the trained model to make a prediction, and inversely normalize the output to get the required photovoltaic power value.

[0034] The specific method for step S5 is as follows:

[0035] The data is input into the trained model. The model uses an early stopping strategy during training to record the training epoch when the model performs best, and then retrains the model using that training epoch.

[0036] Step S5 uses MSE, MAE, RMSE, and SMAPE as objective functions.

[0037] experiment

[0038] Step S1: Two publicly available photovoltaic (PV) power generation datasets from the Alice Springs region of the DKASC PV system in Australia, namely the Sanyo and Solibro power plant datasets, are used. Both datasets contain five features: temperature, humidity, total horizontal radiation, solar thermal index, and power. The datasets are split into training, testing, and validation sets in a 7:2:1 ratio according to time sequence for predicting PV power data for the next 96-960 time steps. Detailed information about the experimental datasets is shown in Table 1.

[0039] Table 1. Description of Experimental Datasets

[0040]

[0041] Step S2: Since some data is missing in both datasets, the K-nearest neighbor method is used to fill in the missing values ​​in the experiment. Before inputting the data into the prediction method, the data is subjected to min-max normalization to restrict the data to [0, 1]. The normalization formula is:

[0042]

[0043] Where X is the original data, X 标准化 These are the standardized data, where μ is the mean and σ is the standard deviation.

[0044] Step S2, Evaluation Metrics and Baseline Methods: The mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) are used to assess the error between actual photovoltaic power and the predicted results.

[0045] 1) Mean Absolute Error (MAE):

[0046]

[0047] 2) Mean Square Error (MSE):

[0048]

[0049] 3) Root Mean Square Error (RMSE):

[0050]

[0051] 4) Mean Absolute Percentage Error (MAPE):

[0052]

[0053] Where N is the number of observations, y i and y iThese represent the actual photovoltaic power and the predicted results, respectively; the smaller the predicted values ​​of MAE, MSE, RMSE, and MAPE, the better the prediction effect.

[0054] The main comparisons are with deep learning methods and other baseline methods, including:

[0055] 1) Naive: Based on simple historical averaging or persistent forecasting methods;

[0056] 2) GBRT (Gradient Boosting Regression Tree): Gradient boosting regression tree improves prediction performance by integrating multiple decision trees;

[0057] 3) LSTM (Long Short-Term Memory): A type of recurrent neural network that excels at capturing long-term dependencies in time series data;

[0058] 4) GRU (Gated Recurrent Unit): A variant of recurrent neural network (RNN) that is simpler in design than LSTM; it controls the flow of information through update gates and reset gates, effectively handling long-term dependencies in sequential data;

[0059] 5) Autoformer: Designed specifically for long-term series forecasting, it employs a hierarchical self-attention mechanism; it improves forecast accuracy by automatically selecting important temporal features.

[0060] 6) Transformer: Based on self-attention mechanism, it can process sequential data in parallel; it uses multi-head attention mechanism to capture dependencies between different positions;

[0061] 7) NSformer: An improved version of Transformer, focusing on capturing subtle features in time series; by enhancing the attention mechanism, it can better identify subtle changes in the sequence;

[0062] 8) DLinear: A linear model combined with deep learning technology;

[0063] 9) FEDformer: Combines frequency domain and time domain information to enhance time series forecasting capabilities;

[0064] 10) Informer: Optimized for long sequence prediction, it adopts a sparse self-attention mechanism; by reducing unnecessary computation, it significantly improves computational efficiency and memory usage;

[0065] Step S3, Experimental Results and Analysis

[0066] The model's performance was compared with 10 common baseline methods for 1-day and 10-day predictions; on both datasets, the method achieved the best prediction results across all evaluation metrics.

[0067] The experimental results in Tables 2 and 3 show that the Naive model assumes future power values ​​equal to current values, while the GBRT model is a gradient-enhanced regression tree model. According to the experimental results, both the Naive and GBRT models perform poorly in predicting future photovoltaic power. On the Sanyo dataset, the Naive model has MSEs of 1.819 (1 day) and 1.828 (10 days), which are significantly worse than other models because it cannot extract the complex nonlinearities of the sequence. GBRT also lacks the ability to extract the temporal series nature of photovoltaic data, resulting in poor prediction performance. LSTM and GRU are able to extract the temporal dependencies in the photovoltaic sequence and therefore perform well. On the Sanyo dataset, LSTM has an MSE of 0.171 when predicting the next day. Compared to LSTM, the method of this invention reduces the MSE by 13.56%. This indicates that the model of this invention can learn more explicit multivariate correlations and volatility than LSTM.

[0068] The model of this invention can capture the multivariate correlations of photovoltaic data and extract long-term dependencies. On the Sanyo dataset, compared with Transformer, the method of this invention reduces MSE, MAE, and RMSE by approximately 52.92%, 31.69%, and 31.23% (1 day), respectively; compared with Pyraformer, MSE is reduced by 19.90% (1 day) and 8.20% (10 days). The DLinear and Autoformer models consider the periodicity and seasonality of time series. The RMSE of DLinear on the Sanyo dataset is 0.418 (1 day) and 0.456 (10 days), while the model of this invention improves MSE by 12.57% (1 day) and 19.23% (10 days), respectively, compared to DLinear. These results demonstrate that the model of this invention can not only extract the multivariate correlations of photovoltaic data but also capture the periodicity of the sequence.

[0069] On the Solibro power plant dataset, the model of this invention also achieved significant performance improvements. When predicting the next day, compared to the Transformer, the method of this invention reduced MSE, MAE, and RMSE by 5.58%, 9.06%, and 31.23%, respectively. When predicting the next ten days, the MSE value was improved by 56.29%, 67.36%, 47.48%, and 43.44% compared to FEDformer, DLinear, Informer, and Pyraformer, respectively. The performance on these two datasets demonstrates the excellent predictive performance of the model of this invention.

[0070] Figure 4 The paper presents a comparison of four models across two datasets using four metrics: (a) Mean Absolute Error (MAE), (b) Mean Squared Error (MSE), (c) Root Mean Squared Error (RMSE), and (d) Symmetric Mean Absolute Percentage Error (SMAPE). The results demonstrate that the proposed model achieves significant performance advantages and is significantly superior to other models in terms of prediction accuracy. This indicates that the model of this invention can extract the complex nonlinear characteristics, multivariate correlations, and periodicity of photovoltaic data, thus exhibiting better predictive performance and generalization ability.

[0071] Table 2. Performance comparison of different traffic flow prediction methods on the Solibro dataset

[0072]

[0073] Table 3. Performance comparison of different traffic flow prediction methods on the Sanyo dataset

[0074]

[0075]

[0076] Step S3-1, Ablation Experiment

[0077] To further investigate the performance of different modules of the proposed TSformer method, five variants of the method were designed to verify the impact of each module, and these five variants were compared with the average values ​​of MAE, RMSE, and MAPE of the TSformer method on the Solibro and Sanyo datasets.

[0078] The differences between these five variant methods and the original method are as follows:

[0079] 1) No Inverted Transformer Architecture: Based on the original model method, the inverted Transformer backbone module is removed;

[0080] 2) No probabilistic sparse attention mechanism: The probabilistic sparse attention mechanism is removed from the original model method;

[0081] 3) Unweighted mechanism: Based on the original model method, the weighting mechanism based on the sine function is removed;

[0082] 4) No decomposition module: Based on the original model method, the sequence decomposition module is removed;

[0083] 5) No components: Remove all improved modules from the original model;

[0084] To better explain the performance of the original model, ablation experiments were visualized and analyzed. For example... Figure 5 As shown, the original model consistently demonstrates significant advantages in MSE, MAE, RMSE, and SMAPE metrics. Compared to the model without the inverted Transformer architecture, the original model shows a relatively significant performance improvement, indicating that the inverted Transformer backbone module plays a crucial role in the model. The inverted Transformer backbone can better model the characteristics between different variables in photovoltaic data through back attention and feedforward neural networks, thereby improving the accuracy of photovoltaic predictions. Meanwhile, the model without the inverted Transformer architecture is slightly inferior to the model without any components in terms of MAE and SMAPE metrics, while the single ablation model enhancement after introducing the inverted Transformer backbone is more stable, indicating that the inverted Transformer backbone not only improves prediction accuracy but also enhances model stability.

[0085] Under the MSE metric, the original model improved performance by approximately 10.12%, 3.82%, and 2.58% compared to the model without the probable sparse attention mechanism, the model without the weighting mechanism, and the model without the decomposition module, respectively. This indicates that the ProbSparse attention mechanism can enhance the model's ability to extract long-range dependencies, while the weighting mechanism and weighted sequence decomposition unit constructed based on photovoltaic physical constraints and sine functions can extract the periodic features of photovoltaic data, further improving the accuracy of photovoltaic power prediction.

[0086] The original model significantly outperformed the five variant models in photovoltaic power prediction, highlighting the effectiveness of the inverted Transformer backbone, ProbSparse attention, and the weighted sequence decomposition module that takes into account physical constraints in capturing the complex nonlinearity, multivariate correlation, and periodicity of photovoltaic data.

[0087] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A photovoltaic power prediction method combining inverted Transformer and weighted decomposition, characterized in that, Includes the following steps: Step S1: Obtain the historical operation dataset of the photovoltaic power station and preprocess the historical data; Step S2: Embed the processed data using a multilayer perceptron to transform it into a high-dimensional feature representation; Step S3: Model construction. The first stage uses the inverted Transformer architecture; the second stage introduces a weighted series decomposition method. Step S3 specifically involves: The first-stage model consists of an inverted Transformer and a sparse attention mechanism. The embedded sequence first passes through the sparse attention mechanism, then through layer normalization and a feedforward neural network, and finally after normalization, it is input into the first-stage prediction output layer to output the first-stage prediction result. The second-stage model mainly consists of weighted series decomposition. The output of the first stage first passes through the sequence decomposition unit and is decomposed into trend, seasonal and random components. Then, it passes through the weighting mechanism and is assigned weight values ​​by a sine function. Finally, it passes through the prediction layer composed of a feedforward neural network to output the prediction results. The steps of the weighted series decomposition method in step S3 are as follows: The sequence decomposition unit decomposes photovoltaic data into seasonal components, trend components, and random components; The weighting mechanism maps the input value to the interval [0, π / 2], calculates the weight using a sine function, sets the relevant range, and retains reasonable weight values. The reconstruction unit receives the output from the weighting mechanism and combines the weighted features with other features through addition operations. The moving average method is used for sequence decomposition, and the specific steps are as follows: Choose a window size and calculate the average of all points within that window for each point in the time series. This average will replace the original data points, generating a new smoothed series. After the moving average processing, the instantaneous fluctuations in the series are suppressed, and what remains is mainly a representation of the long-term trajectory of the data. The smoothed series essentially reflects the trend component of the original series. By subtracting the trend component from the original series, the seasonal component and random fluctuations are represented. The calculation formula is as follows: x t =AvgPool(Padding(x)) x s =x--x t Where, x t With x s These represent the trend term and the seasonal term, respectively. AvgPool(·) is used to perform a moving average operation, and Padding(x) represents a padding operation to keep the sequence length constant. The specific steps of the weighting mechanism and refactoring unit execution are as follows: First, the photovoltaic power data is mapped to a suitable range for comparison with a sine function. Then, the variation patterns of the photovoltaic power data and the sine function are analyzed. A weight matrix is ​​generated using the variation patterns of the sine function. The weights are multiplied by the decomposed tensor of the photovoltaic data sequence to obtain the prediction result. The calculation steps are as follows: Among them, T e With T e These are the trainable parameters in the model, representing the start and end hours of the time range, X represents the current input time data, θ represents the angle, and W represents the weight matrix; Finally, through additive reconstruction, the weighted features are combined with other relevant features, and the calculation is as follows: Among them W i F is the i-th weight value in the weight matrix. i It is the i-th eigenvalue in the decomposed feature tensor; Step S4: Combine the two-stage models and train them using the historical dataset. Step S5: Use the trained model to make a prediction, and then inversely normalize the output to get the required photovoltaic power value.

2. The photovoltaic power prediction method based on inverted Transformer combined with weighted decomposition according to claim 1, characterized in that: Step S1 specifically involves: Step S1-1: Perform a continuity check on the data and mark missing and duplicate data; Check the data for outliers and mark them. For outlier detection, since the historical data of photovoltaic power plants follows a normal distribution and the data does not change much in a short period of time, the 3σ principle is used for outlier detection. Step S1-2: Imput the marked outliers and missing values. Photovoltaic power generation data and meteorological data have strong continuity over time, with relatively small fluctuations within consecutive time periods. Therefore, the K-nearest neighbor method is used to impute missing values. This method calculates the average of the K nearest neighbors of each outlier and then fills this average value into the outlier or missing value. The calculation formula is as follows: Among them, X i This is data that needs to be filled in. Step S1-3: Standardize the filled data. Data normalization is the process of transforming the original data into standardized data with a specific range and mean. The StandScaler normalization method is selected, and the calculation formula is as follows: Where X is the original data, X 标准化 These are the standardized data, where μ is the mean and σ is the standard deviation. The 3σ rule states that in a normal distribution, 68.27% of data fall within one standard deviation of the mean, 95.45% fall within two standard deviations, and 99.73% fall within three standard deviations. Therefore, when the difference between an observed value and the mean exceeds three standard deviations, the value is considered an outlier. In step S2, the data is embedded using a multilayer perceptron, and the specific operation is as follows: A data processing method in which an input data sequence is embedded using a multilayer perceptron to generate a high-dimensional feature representation of the sequence; if the input sequence X = [x1, x2, ..., x...] n For each data point in the sequence, a multilayer perceptron embedding is performed to combine them into a new sequence E. Then, E is aggregated to obtain variable labels V. The variable labels are used as model input.

3. The photovoltaic power prediction method based on inverted Transformer combined with weighted decomposition according to claim 1, characterized in that: Each normalization layer normalizes the input data, giving it a distribution with a mean of 0 and a variance of 1; the calculation formula is: Where H = {h1, h2, ..., h n } represents N embedded tags, each tag having dimension D, Mean(h n () represents the mean of the markers. Indicates the standard deviation of the mark; In the sparse attention layer, the input data is transformed into a query matrix Q, a key matrix K, and a value matrix V, and these matrices are then processed; M(q) is used. t K) represents the sparsity of the i-th query. KL divergence is used to measure the sparsity of queries, and the calculation formula is: Where, q i Let k represent the i-th query vector. j Let L represent the j-th key vector, d represent the scaling factor, and L represent the value of L. K M(q) represents the sequence length of the keys; i K) represents the log-weighted sum of sparse attention, M(q) i K) represents the maximum weighted value of sparse attention; the formula for calculating sparse attention is as follows: in This represents a sparse matrix with the same dimension as q, consisting only of the first u queries under the sparsity evaluated by M(q,K), where u = c·lnL Q Controlled by a constant sampling factor c; the probabilistic sparse self-attention mechanism has a query complexity of only O(lnL) per query. Q It employs a probabilistic sparse self-attention mechanism, which can generate different sparse query key pairs to avoid severe information loss.

4. The photovoltaic power prediction method based on inverted Transformer combined with weighted decomposition according to claim 1, characterized in that: Step S4 specifically involves connecting the first stage and the second stage in series, with the output of the first stage becoming the input of the second stage, and the output of the second stage being the final result.

5. The photovoltaic power prediction method based on inverted Transformer combined with weighted decomposition according to claim 1, characterized in that: Step S5 is as follows: Historical power generation data of photovoltaic power plants is obtained, and the data is preprocessed and then input into the photovoltaic power prediction model; The first-stage model, consisting of an inverted Transformer and sparse attention, captures the complex nonlinear and volatile characteristics of the data and inputs them into the second-stage model. The periodic characteristics of the data are captured by decomposing the data using the moving average method, and then the data is normalized by weighting and combining it with a sine function. The output results are then inversely normalized to obtain the predicted value of photovoltaic power generation.

6. The method according to claim 5, characterized in that: The parameters for evaluating the model's performance are as follows: During training, MAE (Mean Absolute Error), MSE (Mean Squared Error), RMSE (Root Mean Squared Error), and SMAPE (Symmetric Mean Absolute Percentage Error) were selected as evaluation metrics; the calculation formulas are as follows: Where n represents the number of samples, y i Representing the true value, y′ i This represents the predicted value.

Images