A photovoltaic power generation prediction method and related devices

Abstract

This application belongs to a prediction method. Aiming at the technical problems that when using the Transformer model for time series prediction currently, the coupled embedding of synchronous multi-dimensional variables restricts the expression of asynchronous spatio-temporal relationships, and the coupled decoding of synchronous multi-dimensional variables restricts the learning of asynchronous spatio-temporal features, a photovoltaic power prediction method and related device are provided. The historical photovoltaic power time series is obtained, value embedding, position embedding, and time embedding are respectively performed, the value embedding result, the position embedding result, and the time embedding result are added to obtain the total embedding result, and then vector enhanced attention processing is performed on the total embedding result to obtain the attention enhanced result. Finally, spatio-temporal cross aggregation is performed on the attention enhanced result to obtain the prediction result.

Description

Technical Field

[0001] This application belongs to a prediction method, and specifically relates to a photovoltaic power prediction method and related devices. Background Art

[0002] As a sustainable and clean energy technology, photovoltaic power generation has been widely used in the global energy supply. However, due to its dependence on meteorological conditions (such as solar radiation, temperature, wind speed, etc.), photovoltaic power generation has significant randomness and volatility, making the accurate prediction of photovoltaic power a complex and important research issue. In practical applications, grid managers and photovoltaic power station operators require high-precision power prediction models to achieve balanced power dispatching and stable power supply, reduce the demand for reserve capacity, and operating costs.

[0003] Currently, time series prediction methods are widely used for photovoltaic power prediction. Especially models based on Transformer, due to their superiority in processing long sequence data and capturing spatio-temporal features, have become a mainstream choice. However, in the photovoltaic power generation scenario, such models still have the following two limitations in capturing complex asynchronous spatio-temporal relationships and mitigating accidental fluctuations:

[0004] (1) The coupled embedding of synchronous multi-dimensional variables limits the expression of asynchronous spatio-temporal relationships.

[0005] (2) The coupled decoding of synchronous multi-dimensional variables limits the learning of asynchronous spatio-temporal features. Summary of the Invention

[0006] This application addresses the technical problems that when using the Transformer model for time series prediction, the coupled embedding of synchronous multi-dimensional variables limits the expression of asynchronous spatio-temporal relationships, and the coupled decoding of synchronous multi-dimensional variables limits the learning of asynchronous spatio-temporal features, and provides a photovoltaic power prediction method and related devices.

[0007] To achieve the above objectives, this application is implemented by adopting the following technical solutions:

[0008] In the first aspect, this application proposes a photovoltaic power prediction method, including:

[0009] Obtain the historical photovoltaic power time series;

[0010] Decompose the historical photovoltaic power time series into multiple two-dimensional time series;

[0011] Perform Fourier transform on each two-dimensional time series to obtain the frequency components of the two-dimensional time series;

[0012] Normalize the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series;

[0013] Extract features from the frequency components of the historical photovoltaic power generation time series to obtain a value embedding result;

[0014] Perform positional encoding on the historical photovoltaic power generation time series on the time axis and then expand it on the time axis to obtain a positional embedding result;

[0015] Perform time embedding on the historical photovoltaic power generation time series to obtain a time embedding result;

[0016] Add the value embedding result, the positional embedding result, and the time embedding result to obtain a total embedding result;

[0017] Perform vector-enhanced attention processing on the total embedding result to obtain an attention-enhanced result; the attention aggregation operation in the vector-enhanced attention processing is: multiply the attention weights by the total embedding result;

[0018] Perform spatio-temporal cross-aggregation on the attention-enhanced result to obtain a prediction result.

[0019] Further, the disassembling of the historical photovoltaic power generation time series into multiple two-dimensional time series includes:

[0020] Regard each variable at the same moment in the historical photovoltaic power generation time series as an independent individual, keep the order relationship of different variables at different moments, and disassemble the historical photovoltaic power generation time series.

[0021] Further, the extracting of features from the frequency components of the historical photovoltaic power generation time series includes:

[0022] Perform a two-dimensional convolution operation on the frequency components of the historical photovoltaic power generation time series to extract features.

[0023] Further, before performing Fourier transform on each two-dimensional time series, it also includes padding each two-dimensional time series:

[0024] S1-1, perform Fourier transform on the value of each variable in the two-dimensional time series to obtain the frequency component corresponding to the value of each variable;

[0025] S1-2, obtain the frequency component with the largest amplitude among the frequency components corresponding to the values of each variable, and denote it as the frequency component with the largest amplitude;

[0026] S1-3, divide the length of the two-dimensional time series corresponding to the frequency component with the largest amplitude by the peak index of the non-zero index to obtain the minimum positive period as the optimal window length;

[0027] S1 - 4, perform mirror padding of half of the optimal window length at both ends of each two - dimensional time series.

[0028] Furthermore, the normalization processing of the frequency components of the two - dimensional time series includes:

[0029]

[0030] where, represents the result of the normalization processing of the frequency components of the two - dimensional time series, represents the total number of time instances of the two - dimensional time series, represents a specific time point in the two - dimensional time series, and the range is from 1 to , represents the time dimension the amplitude of the frequency component;

[0031]

[0032] where, represents frequency amplitude standardization, represents the first half of the Fourier transform, represents the window function, represents the two - dimensional time series data of the time dimension , represents the two - dimensional time series data of the time dimension , represents the two - dimensional time series data of the time dimension , represents the optimal window length.

[0033] Furthermore, the position encoding of the historical photovoltaic power generation time series on the time axis includes:

[0034] Perform position encoding of the historical photovoltaic power generation time series on the time axis using a two - dimensional convolution with a kernel size of 1×3.

[0035] Furthermore, the vector enhanced attention processing of the total embedding result includes:

[0036] S2 - 1, generate query vectors , key vectors and value vectors respectively through a linear layer:

[0037]

[0038]

[0039]

[0040] Among them, represents the linear transformation function for generating query vectors, represents the prediction target, represents the linear transformation function for generating key vectors, represents the linear transformation function for generating value vectors, represents the time dimension for the embedding result;

[0041] S2-2, use the query vector to query the component vectors of the key vector and obtain the attention weights:

[0042]

[0043] Among them, represents the group attention weights;

[0044]

[0045] Among them, represents the attention weight at time represents the query vector at time

[0046] S2-3, add and normalize the group attention weights to obtain a group of two-dimensional attention weights;

[0047] S2-4, obtain the attention aggregation feature through the following formula, that is, the attention enhancement result:

[0048]

[0049] Among them, represents a group of two-dimensional attention weights obtained in step S2-3.

[0050] Furthermore, perform spatio-temporal cross-aggregation on the attention enhancement result, including:

[0051] S3-1, generate a group of key vectors through two fully connected layers respectively:

[0052]

[0053] Among them, represents the result of performing a downsampling convolution operation on ;

[0054] S3-2, initialize a zero tensor with the same length as the prediction target ;

[0055] S3-3, perform value embedding, positional embedding, and temporal embedding on the zero tensor to obtain a sequence containing positional information and temporal information ;

[0056] S3-4, according to the sequence , generate a series of query vectors through a fully connected layer ;

[0057] S3-5, through a series of query vectors , query the spatio-temporal features generated in the spatio-temporal cross-aggregation encoding stage to obtain the corresponding attention scores:

[0058]

[0059] Among them, represents the weight learned by the th query vector , represents the feature dimension, represents the th query vector , represents the query vector ;

[0060] S3-6, use the attention scores for aggregation:

[0061]

[0062] Among them, represents the prediction result.

[0063] In a second aspect, the present application proposes a photovoltaic power prediction system, including:

[0064] An acquisition module for acquiring a historical photovoltaic power time series;

[0065] A disassembling module for disassembling the historical photovoltaic power time series into a plurality of two-dimensional time series according to time;

[0066] A transformation module for performing Fourier transform on each two-dimensional time series to obtain the frequency components of the two-dimensional time series;

[0067] A normalization module for normalizing the frequency components of a two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series;

[0068] A value embedding module for extracting features from the set of tensors to obtain a value embedding result;

[0069] A position embedding module for performing position encoding on the historical photovoltaic power generation time series on the time axis and then expanding it on the time axis to obtain a position embedding result;

[0070] A time embedding module for performing time embedding on the historical photovoltaic power generation time series to obtain a time embedding result;

[0071] An addition module for adding the value embedding result, the position embedding result, and the time embedding result to obtain a total embedding result;

[0072] An enhancement module for performing vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result;

[0073] An aggregation module for performing spatio-temporal cross aggregation on the attention enhanced result to obtain a prediction result.

[0074] In a third aspect, the present application proposes a computer program product, the computer program product includes instructions, and when the instructions are executed by a processor, the above-mentioned photovoltaic power generation prediction method is implemented.

[0075] Compared with the prior art, the present application has the following beneficial effects:

[0076] This application proposes a photovoltaic power prediction method. The historical photovoltaic power time series is decomposed into multiple two-dimensional time series, and Fourier transform is performed on each two-dimensional time series. Then, the frequency components of the two-dimensional time series are normalized. Next, features are extracted from the set of tensors of the frequency components of the historical photovoltaic power time series to obtain a value embedding result. The historical photovoltaic power time series is position-encoded on the time axis and then extended on the time axis to obtain a position embedding result. This application encodes each variable independently in the same time and space to form a decoupled form of embedding vector. At the same time, the ability of the frequency domain information to suppress unexpected fluctuations is introduced. Then, the total embedding result obtained by adding the value embedding result, the position embedding result, and the time embedding result is processed by vector-enhanced attention to obtain an attention-enhanced result. A two-dimensional vector-enhanced attention mechanism is used to decouple and process useful synchronous and asynchronous spatio-temporal information for prediction, emphasizing the information useful for prediction and reducing redundant calculations in the aggregation operation. Finally, a spatio-temporal cross-aggregation mechanism is proposed to aggregate features from two dimensions of time and space, and finally obtain a prediction result. This application makes full use of time and space information to ensure the accuracy of prediction.

[0077] This application also proposes a photovoltaic power prediction system, an electronic device, and a computer storage medium, which have all the advantages of the above photovoltaic power prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of this application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0079] Figure 1 It is the first flow schematic diagram of the photovoltaic power prediction method of this application;

[0080] FIG. 2 (a) to FIG. 2 (c) are schematic diagrams of the visualization results of the ISDFormerr model, the Transformerr model, and the Autoformer model on the Solar dataset in the multi-variable to single-variable prediction task in the embodiments of this application; among them, FIG. 2 (a) corresponds to the ISDFormer model, FIG. 2 (b) corresponds to the Autoformer model, and FIG. 2 (c) corresponds to the Transformer model;

[0081] FIG. 3(a) and FIG. 3(b) are schematic diagrams of feature attention allocated to two groups of experiments during the verification of the embedding layer in the embodiments of the present application; among them, FIG. 3(a) is the experiment of group E, and FIG. 3(b) is the experiment of group F.

[0082] FIG. 4(a) and FIG. 4(b) are schematic diagrams of the prediction results of two groups of experiments during the verification of the embedding layer in the embodiments of the present application; among them, FIG. 4(a) is group E, and FIG. 4(b) is group F.

[0083] Figure 5 It is a schematic diagram of a photovoltaic power prediction system of the present application. Detailed implementation manners

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Components of the embodiments of the present application described and illustrated herein generally may be arranged and designed in a variety of different configurations.

[0085] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts fall within the scope of protection of the present application.

[0086] It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0087] In the description of the embodiments of the present application, it should be noted that if terms such as "upper", "lower", "horizontal", "inner", etc. are used to indicate the orientation or positional relationship, it is based on the orientation or positional relationship shown in the drawings or the orientation or positional relationship in which the product of the invention is usually placed during use. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation to the present application. In addition, terms such as "first", "second", etc. are only used for descriptive distinction and cannot be construed as indicating or implying relative importance.

[0088] In addition, if the term "horizontal" appears, it does not mean that the component is required to be absolutely horizontal, but it can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but it can be slightly inclined.

[0089] In the description of the embodiments of the present application, it should also be noted that unless otherwise clearly specified and limited, if the terms "set", "install", "connected", "connected" are used, they should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations.

[0090] The accuracy of photovoltaic power prediction is crucial for the stable operation and efficient management of the power grid. Although the Transformer-based time series prediction model has shown powerful capabilities in processing sequence data and spatio-temporal features, in the specific application scenario of photovoltaic power generation, it still faces some specific challenges and limitations:

[0091] (1) The coupled embedding of synchronous multi-dimensional variables limits the expression of asynchronous spatio-temporal relationships.

[0092] In photovoltaic power prediction, there are many factors affecting power output, including but not limited to light intensity, temperature, wind speed, humidity, cloud cover, etc. In the embedding stage, traditional Transformer-based models map all variables affecting power output into a coupled vector at the same time. This method of coupled embedding hinders the model from effectively learning the complex asynchronous spatio-temporal relationships between variables. For example, different meteorological factors have different effects on power generation at different time points, and the method of coupled embedding cannot fully express these asynchronous effects, which will limit the accurate prediction of photovoltaic power. In addition, the direct point-by-point coding method ignores the randomness in time series data, making it impossible for the attention mechanism to effectively detect and process the negative impact of abnormal meteorological conditions on power prediction.

[0093] (2) The coupled decoding of synchronous multi-dimensional variables limits the learning of asynchronous spatio-temporal features.

[0094] In the encoding stage, the self-attention mechanism in the Transformer model aims to capture the context relationships between variables. However, in the scenario of photovoltaic power prediction, the learning of such context relationships is often too complex and may not have practical application value. In addition, in the decoding stage, the commonly used one-dimensional aggregation operation cannot effectively capture the differences in time and space of each meteorological factor when dealing with the two-dimensional asynchronous spatio-temporal relationship of photovoltaic data, thus affecting the prediction results.

[0095] Based on the above situation, the present application proposes a photovoltaic power prediction method and related device, and the present application will be described in detail below in combination with embodiments and drawings.

[0096] AsFigure 1 As shown in the figure, the following is the first schematic flowchart of the photovoltaic power generation prediction method of the present application, which may include:

[0097] S101, Obtain the historical photovoltaic power generation power time series.

[0098] In practical applications, historical data can be obtained through the monitoring system of the photovoltaic power station, the intelligent power grid database or meteorological data combined with the performance of the photovoltaic panels to form the historical photovoltaic power generation power time series. In order to improve the accuracy of subsequent predictions, the obtained data can also be cleaned to remove outliers and missing values, or smoothed to reduce noise.

[0099] S102, Decompose the historical photovoltaic power generation power time series into multiple two-dimensional time series.

[0100] S103, Perform Fourier transform on each two-dimensional time series to obtain the frequency components of the two-dimensional time series.

[0101] Through Fourier transform, the two-dimensional time series can be transformed from the time domain to the frequency domain to reveal hidden periodic characteristics. The frequency components of the two-dimensional time series obtained after transformation can reflect the signal strength at different frequencies.

[0102] S104, Normalize the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation power time series.

[0103] In the present application, the dimensions of different frequency components are unified for subsequent processing. In practical applications, methods such as min-max normalization can be used to convert the frequency components into values within a standard range and organize them into tensor form.

[0104] S105, Extract features from the set of tensors of the frequency components of the historical photovoltaic power generation power time series to obtain the value embedding result.

[0105] Extracting key features in the tensor can provide valuable information. In practical applications, a convolutional neural network, a feature selection algorithm or an embedding layer in deep learning can be used to convert the high-dimensional tensor into a low-dimensional, dense vector representation.

[0106] S106, Perform position encoding on the historical photovoltaic power generation power time series on the time axis, and then expand it on the time axis to obtain the position embedding result.

[0107] Performing position encoding can assign unique position information to each time point in the historical photovoltaic power generation power time series to help understand the time sequence.

[0108] S107, Perform time embedding on the historical photovoltaic power generation power time series to obtain the time embedding result.

[0109] Performing time embedding and considering the periodic characteristics of time itself (such as days, weeks, seasons, etc.) can provide additional time context. In practical applications, time embedding vectors can be generated through periodic functions (such as sine or cosine functions) or time feature engineering (such as dates, weekdays, months, etc.).

[0110] S108, Add the value embedding result, the position embedding result, and the time embedding result to obtain the total embedding result.

[0111] Integrating all embedding information can form a comprehensive vector representation for subsequent prediction tasks.

[0112] S109, Perform vector enhanced attention processing on the total embedding result to obtain the attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiply the attention weight by the total embedding result.

[0113] Through the attention mechanism, it is possible to dynamically focus on the information most important for prediction. When specifically performing the attention aggregation operation, the attention weight can be calculated and multiplied by the attention enhanced result to enhance the key information.

[0114] S110, Perform spatio-temporal cross-aggregation on the attention enhanced result to obtain the prediction result.

[0115] This application combines information in the time dimension and the space dimension for final prediction. Through spatio-temporal cross-aggregation, further feature extraction and sequence modeling are performed on the attention enhanced result, and finally, the prediction result of the photovoltaic power generation can be output.

[0116] The following provides a more detailed description of the photovoltaic power generation prediction method of this application through another embodiment of the photovoltaic power generation prediction method.

[0117] The time series collected within the interval is denoted as , where is the time series data at time within the interval , and is the dimension of the feature. Given the time series , predict the time series from time to time .

[0118] ​​To effectively learn the asynchronous spatio-temporal relationships of covariates, this application proposes a photovoltaic power prediction method, which is implemented by a set of models, denoted as the ISDFormer model. The ISDFormer model includes an embedding layer, an encoder, and a decoder. In the embedding layer, the aforementioned steps S101 to S108 are executed. In the embedding layer, a new embedding method is proposed to independently encode each variable in the same time and space, forming a decoupled form of the embedding vector. Meanwhile, frequency domain information is introduced to enhance the ability of the ISDFormer model to suppress unexpected fluctuations. In the encoder part (corresponding to the execution of the aforementioned step S109), a two-dimensional vector enhanced attention mechanism is proposed to decouple and process useful synchronous and asynchronous spatio-temporal information for prediction, emphasizing the information useful for prediction and reducing redundant calculations in the aggregation operation. In the decoder part (corresponding to the execution of the aforementioned step S110), a spatio-temporal cross-aggregation mechanism is proposed to aggregate the features obtained in the encoder stage from both the time and space dimensions, and finally obtain the prediction result. Specifically:

[0119] 1. A new embedding method is proposed in the embedding layer.

[0120] In previous time series prediction work, it was assumed that , where is a time series with dimension within the interval , . denotes the time series data with dimension at time

[0121] After being processed by the embedding layer, the embedding representation of the given time series can be obtained, where denotes the embedding result at time , and

[0122]

[0123] where , denotes the feature vector of the time series data at time . As an example, in the photovoltaic scenario, the feature vector may include relevant data of photovoltaic power generation, such as various meteorological factors such as sunlight intensity, temperature, humidity, wind speed, wind direction, cloud cover, etc. denotes at time the th dimension of the feature vector, moment, the eigenvector of the first dimension, Indicates moment, the eigenvector of the second dimension, Indicates connecting multiple feature vectors in the feature dimension (i.e., splicing operation), represents a one-dimensional convolution operation applied along the time axis, with input features and model depth output features, represents position embedding, represents the embedding of date features, Represents specific date data. Date features combined with specific date information, such as year, month, and day, can help the model learn the impact of seasonal changes on photovoltaic power generation.

[0124] The embedding method in the ISDFormer model proposed in this application has been applied to time series forecasting. However, this method couples variables of different dimensions, making it difficult for the model to exploit asynchronous spatiotemporal relationships between variables and to distinguish outliers.

[0125] To solve this problem, this application proposes a new embedding method, which is an improved two-dimensional embedding method, mainly divided into three parts: value embedding, position embedding and time embedding.

[0126] In the value embedding component, the value at each moment and dimension is encoded along with the corresponding frequency domain information. First, to achieve spatial vector decoupling, each variable in the historical PV power time series at the same moment is treated as an independent entity, while variables at different moments maintain their sequential relationship. The historical PV power time series is truncated into two dimensions from a temporal perspective. Then, to enable the model to detect isolated points in the historical PV power time series, a FreqEmbedding module is designed to obtain frequency distribution information. The core work of the FreqEmbedding module is to perform a windowed frequency domain decomposition of the time series. To ensure that the first and last positions in the time series are located in the middle of the FFT (Fast Fourier Transform) decomposition segment, mirror padding with half the optimal window length is applied at both ends of the time series. The window size affects the accuracy of the frequency decomposition. In practical applications, the optimal window length should be the minimum positive period of the signal variation. Using a different period may cause sidelobe leakage, which will adversely affect the decomposition results.

[0127] In practical applications, to obtain the optimal window length, first, the values of each variable are processed by FFT to obtain frequency components, and the frequency component with the largest amplitude is found. Dividing the length of the input sequence by the peak index of the non-zero index can obtain the minimum positive period, that is, the optimal window length, denoted as w , which is used to define the time series interval covered by each frequency domain analysis. In addition, to ensure that the results of different time series decompositions have the same order of magnitude, this application also designs an amplitude normalization function to normalize the results of frequency domain decomposition according to the amplitude size, and a set of tensors can be obtained, which can be expressed as:

[0128] , which is used to reflect the change of frequency. Among them, can be expressed as:

[0129]

[0130] Among them, , represents the amplitude of the frequency component at the time dimension , represents frequency amplitude normalization, represents the first half of represents the window function, which applies weights to a finite time period of the signal to reduce spectral leakage in spectral analysis, represents the time series data at the time dimension , represents the time series data at the time dimension . Then, the variables at each moment and each dimension are concatenated with the amplitude of the frequency component obtained from the decomposition to obtain the value embedding result: . Then, the variables at each moment and each dimension are concatenated with the amplitude of the frequency component obtained from the decomposition to obtain the value embedding result:

[0131]

[0132] Among them, represents the value embedding result. The value embedding result can also be processed by two-dimensional convolution. The above steps are taken as the content executed in the FreqEmbedding module.

[0133] For position embedding, a three-dimensional tensor is set, and the three dimensions respectively refer to time, space, and model depth. In practical applications, only the variables at different times need to be distinguished. Therefore, only position encoding needs to be performed on the time axis. Specifically, a two-dimensional convolution with a kernel size of 1x3 can be used. To ensure the consistency of the results, it can be extended on the time axis. It also means that the convolution is applied along the time axis without any processing in the spatial domain.

[0134] Next, perform time embedding on the historical photovoltaic power generation time series to obtain the time embedding result. The current value in the time series is not only related to several previous values but also related to the time intervals between these values. Through time embedding, one-dimensional time series data can be converted into a multi-dimensional vector space, where each vector contains information of multiple time points in the original time series.

[0135] Finally, add the results of position embedding, time embedding, and value embedding to obtain the added embedding result:

[0136]

[0137] where denotes the time dimension for the added embedding result, denotes a two-dimensional convolution operation on the input data, denotes the input vector after being extended on the time axis. Then the total embedding result denotes the total embedding result of the historical photovoltaic power generation time series collected within the time period .

[0138] 2. Vector-enhanced attention processing (encoder).

[0139] After obtaining the total embedding result, in order to avoid redundant context information brought by the aggregation operation of the traditional attention mechanism and emphasize the vectors useful for the predicted value, this application designs a two-dimensional vector-enhanced attention method.

[0140] First, for the variable to be predicted, represent it as the target , and generate a query vector through a linear layer:

[0141]

[0142] where denotes the linear transformation function for generating the query vector. For all variables, the key vector and the value vector are respectively generated through a linear layer:

[0143]

[0144]

[0145] where denotes the linear transformation function for generating the key vector, Represents a linear transformation function for generating a numerical vector. Using to query all and obtain the attention weights. For each query vector , a two-dimensional attention weight can be generated, and thus groups of attention weights can be generated, denoted as :

[0146]

[0147] where: , represents the attention weight at time represents the query vector at time

[0148] According to the position of the query vector , these attention weights are added and normalized to obtain a group of two-dimensional attention weights. This two-dimensional attention weight represents the importance of each covariate for the variable to be predicted:

[0149]

[0150] where, represents the weight learned in the vector-enhanced attention, represents two-dimensional normalization processing. Finally, to emphasize the degree of importance, different from the traditional attention aggregation operation, this application directly multiplies the attention weight by to highlight some important features, that is, the attention aggregation feature . Specifically:

[0151]

[0152] Since the vector-enhanced attention processing of this application has inputs and outputs of the same shape and size, it can seamlessly replace the self-attention mechanism. Although this mechanism is designed for two-dimensional features, by making , it can also be applied to the case of only one feature.

[0153] 3. Spatiotemporal cross-aggregation (decoder).

[0154] When embedding vectors, there are potential associations between spatial features, so they cannot be directly coupled and encoded. After obtaining these independent spatial encoding vectors, the next step is to process them in the most appropriate way to aggregate features. For this purpose, this application designs spatiotemporal cross-aggregation for aggregating two-dimensional spatiotemporal information.

[0155] At the decoder stage, first initialize a zero tensor with the same length as the prediction length, denoted as . Then, perform value embedding, positional embedding, and temporal embedding on this zero tensor to obtain a sequence containing positional and temporal information. Since the finally obtained tensor is one-dimensional, this process can follow the original embedding process. This application will use The initialized variable is denoted as and is formalized as:

[0156]

[0157] where represents the result obtained by performing value embedding.

[0158] For the result output by the encoder, generate a set of key vectors through two fully connected layers respectively:

[0159]

[0160] where represents the result of performing downsampling convolution operation on . Then generate a series of query vectors from the embedding result through a fully connected layer:

[0161]

[0162] These are used to synchronously query the spatio-temporal features generated in the encoder stage to obtain a series of attention scores:

[0163]

[0164] where , represents the weight learned by the th query vector , represents the time step in the time series, represents the th query vector , represents the feature dimension, represents the query vector sequence number. Finally, use the attention scores to aggregate all features to obtain the output result:

[0165]

[0166] where represents the prediction result.

[0167] Compared with the classical cross-attention mechanism, the method of the present application extends the attention from one dimension to two dimensions, further enriching the feature mining and expression methods of the attention mechanism and bringing a more flexible expression effect.

[0168] To prove the technical effect of the present application, the photovoltaic power prediction method of the present application was verified, and the following experiments were carried out:

[0169] On a self-built Solar dataset, the photovoltaic power prediction method proposed in the present application was trained and tested. The Solar dataset was established by deploying data acquisition terminals at a certain photovoltaic power station in northern China, recording the photovoltaic power generation situation from June 30, 2020 to June 30, 2023. Each record has 8 attributes, including power generation, total radiation, direct radiation, scattered radiation, temperature, air pressure, wind direction and wind speed, sampled every 15 minutes, with a total of 105,216 records. The training set, validation set and test set of each dataset were divided according to the ratio of 0.7, 0.1 and 0.2.

[0170] Experimental details: During the training process, the L2 loss function (i.e., the squared loss function or the Euclidean loss function) was used, and the Adam (Adaptive Moment Estimation) optimizer was adopted. The initial learning rate was set to 0.0001, and the batch size was set to 32. The basic hyperparameters include the number of encoder layers, decoder layers, feature dimensions, number of attention heads and convolution function dimensions, which were set to 2, 1, 512, 8 and 2048 respectively. The experiment was repeated 3 times, and the average value of the indicators was reported. In this embodiment, two evaluation indicators, MSE (Mean Squared Error) and MAE (Mean Absolute Error), were used. To objectively evaluate the method of the present application, 6 popular time series prediction models based on the Transformer model were selected as benchmark models, namely: Transformer model, Informer model, Autoformer model, FEDformer (Frequency Enhanced Decomposed Transformer) model, TimesNet model and Crossformer model. The experimental results of all benchmark models for comparison were trained, tested and verified using the open-source benchmark model code. To ensure the fairness of the comparison, the time series input and output scales of all models were standardized and kept consistent.

[0171] Comparison of experimental results:

[0172] The input length of all models is 384, and the prediction lengths are 96, 384, 672, and 1344 respectively, which are used to simulate short-term prediction, medium-term prediction, medium-long-term prediction, and long-term prediction.

[0173] Univariate-to-univariate prediction: Univariate-to-univariate prediction experiments were conducted on the Solar dataset, and the results are shown in Table 1. It can be seen that the method proposed in this application (the ISDFormer model in Table 1) has significantly lower MSE and MAE than other methods. This indicates that the learning paradigm of the ISDFormer model can effectively capture the potential asynchronous characteristics of time series variables, thereby improving the prediction performance of the model. It should be noted that in Table 1, the "Methods" row represents the models used, and the "Metric" row represents the evaluation metrics. The same applies to subsequent tables.

[0174] Table 1 Comparison table of univariate-to-univariate prediction results

[0175]

[0176] Multivariate-to-univariate prediction: Multivariate-to-univariate prediction experiments were conducted on the Solar dataset, and the results are shown in Table 2. It can be seen that when conducting multivariate-to-univariate prediction experiments, the ISDFormer model performs excellently at different prediction lengths (such as 96, 384, 672, 1344).

[0177] Table 2 Comparison table of multivariate-to-univariate prediction results

[0178]

[0179] Analyzing Table 1 and Table 2, on the Solar dataset, the average MSE of the multivariate-to-univariate task only differs by 3.6% from the average MSE of the univariate-to-univariate task. This is because the Solar dataset is more regular, and introducing covariates does not help much in improving the model performance.

[0180] Figures 2(a) to 2(c) are schematic diagrams of the visualization results of the ISDFormer model, Transformer model, and Autoformer model on the Solar dataset in the multi-variable to single-variable prediction task. In Figure 2, the red line represents the prediction result, and the blue line represents the actual situation. Among them, Figure 2(a) corresponds to the ISDFormer model, Figure 2(b) corresponds to the Autoformer model, and Figure 2(c) corresponds to the Transformer model. It can be seen that compared with other models, due to the ISDFormer model adopting the methods of cross-dimensional decoupling and frequency-domain learning, the multi-periodicity and global trend of the time series are more effectively learned. It should be noted that in Figures 2(a) to 2(c), the abscissa represents time, that is, the time dimension in the time series data, the ordinate represents the power value corresponding to the time point, the blue line represents the prediction situation, and the red line represents the real situation.

[0181] In the method of this application, there are three key factors: frequency and variable embedding, vector-enhanced attention, and spatio-temporal cross-aggregation. Frequency and variable embedding contains two core points: one is to encode each variable independently to capture the potential connections between variables at different times and within the same time; the other is to add frequency-domain information to the encoding result. The core idea of the vector-enhanced attention mechanism is to emphasize the components useful for the prediction result. And spatio-temporal cross-aggregation is a feature aggregation mechanism designed to cooperate with independent encoding. Correspondingly, this application aims to solve three problems: the interference problem existing in the data, the problem of capturing asynchronous spatio-temporal relationships, and the problem of how to enhance features. In order to verify the performance of the three key factors on the dataset, four models were designed for comparative experiments. Among them, Group A uses the complete ISDFormer model, Group B uses the ISDFormer model without frequency components, Group C uses the standard Transformer model for time series prediction, and Group D uses the Transformer model with vector-enhanced attention. All four models are trained on the Solar dataset, and their input and output lengths are both 384. The test results are shown in Table 3. In Table 3, the "Group" column represents the experimental group, and the "Method" column represents the model used.

[0182] "ISDFormer without frequency" represents the ISDFormer model without frequency components, and "Transformer with VEA" represents the Transformer model with vector-enhanced attention.

[0183] Table 3 Comparison table of test results of four models

[0184]

[0185] It can be seen that in the comparison between Group A and Group B, Group A achieved better results, indicating that frequency encoding has an enhancing effect on the results, that is, it solves the data interference problem mentioned above. In the comparative experiment between Group C and Group D, Group D achieved better results, indicating that vector-enhanced attention has a good effect on feature enhancement. In the comparative experiment between Group B and Group D, since the Transformer with vector-enhanced attention mechanism is equivalent to removing the frequency component and performing the coupled coding of ISDFormer, therefore, their comparative relationship shows that ISDFormer can capture asynchronous spatio-temporal relationships.

[0186] In addition, to illustrate the role of the embedding layer of this application, a pair of experiments were designed. The experiment was divided into two groups. Group E was the complete ISDFormer. As a comparison, Group F was the ISDFormer without frequency domain embedding. Both groups of experiments were trained for single-variable to single-variable prediction on the Solar dataset and the sine dataset. Using single-variable to single-variable prediction was to eliminate the influence of covariates on the prediction results. The results showed that there was almost no performance difference between the two groups on the sine dataset, while on the photovoltaic dataset, the performance of Group E was better than that of Group F, which could also be seen in the ablation experiment. This is because the sine dataset is very regular and the role of frequency information on the model can be almost ignored; while the fluctuations of the Solar dataset are more obvious, and the frequency domain information can help the model distinguish which points are abnormal.

[0187] To find out how the model internally uses this frequency information, the photovoltaic power generation data of two days were respectively input into Group E and Group F, and let them predict the power generation of one day. The input data came from the Solar dataset. In the example, there were many accidental disturbances in the power generation data of the first day, while the power generation data of the second day was relatively stable.

[0188] Group E introduced a vector with 49 features (this value is half of the window length plus one) in the embedding stage to additionally describe the surrounding information. Since the frequency components have been amplitude-normalized, their color depth can reflect the frequency distribution and is not affected by the energy size. In the relatively stable second half, its frequency distribution is more concentrated in the low frequency, while in the first half, the low-frequency components are relatively few. In the feature aggregation stage of the decoder, cross-aggregation attempts to aggregate the features emphasized by the encoder. The attention assigned to these features in the experiments of Group E and Group F is shown in Figures 3(a) and 3(b). Figure 3(a) shows the attention map assigned to these features in the experiment of Group E, and Figure 3(b) shows the attention map assigned to these features in the experiment of Group F. Among them, in Figures 3(a) and 3(b), the vertical axis represents the features emphasized by the encoder, and the horizontal axis represents the timestamp sequence to be predicted.

[0189] Since there is no swapping of time series in the encoder of the model (except for convolutions along the time axis), although the features output by the encoder are already difficult to interpret, the relative order relationship in time is still maintained. Therefore, it can be seen that in the decoder model of Group E, it assigns greater weights to the more stable points in the second half, that is, the model is more inclined to use the more stable points for prediction. In contrast, Group F has difficulty in emphasizing this feature. The prediction results of Group E and Group F are shown in Figures 4(a) and 4(b). The abscissa in Figures 4(a) and 4(b) represents time, that is, the time dimension in the time series data, and the ordinate represents the power value corresponding to the time point. The blue line represents the prediction situation, and the red line represents the real situation.

[0190] Furthermore, to illustrate the role of vector-enhanced attention, a set of control experiments was further designed. Group G uses vector-enhanced attention, and as a control, Group H uses the traditional self-attention mechanism represented by aggregation operations. In the decoder part, since this mechanism needs to be combined with the spatial aggregation attention mechanism, Group G is a complete ISDFormer structure. Group H uses the classical Transformer architecture with a full attention mechanism. It is trained on the Solar dataset, and the loss of the test set is monitored. Training stops when the best effect is achieved.

[0191] The main work of the encoder part is feature extraction, while the main work of the decoder part is feature aggregation. To observe which features are extracted by the encoder part, the 8-head attention scores of Group H are plotted respectively. It can be concluded that multiple consecutive query vectors focus on the same key vector and assign greater weights to it. In the 1st, 2nd, 4th, 6th, and 8th head attentions, the information between 260 - 270 in the sequence is particularly concerned. In the self-attention mechanism, the next step after calculating the attention scores is to aggregate features. Therefore, multiple consecutive tokens are replaced with similar or even identical content. In this way, the model will only focus on very limited content.

[0192] If a feature enhancement mechanism is used, this situation can be alleviated. First, since the model aggregates features in the decoder stage, the decoder will guide the feature enhancement attention mechanism to highlight the required features. Second, since there is no replacement operation when extracting features, even if only a limited part of the information is emphasized, the original information is not completely lost, and all features can still be obtained during the feature aggregation in the decoder.

[0193] Different from the self-attention mechanism, the model will emphasize all features in a relatively "uniform" way without overemphasizing a certain feature. In this way, the loss of information is avoided, and the model avoids over-focusing on limited information.

[0194] As Figure 5As shown, it is a schematic diagram of a photovoltaic power generation prediction system of the present application, which may include:

[0195] An acquisition module, configured to acquire a historical photovoltaic power generation power time series;

[0196] A disassembling module, configured to disassemble the historical photovoltaic power generation power time series into a plurality of two-dimensional time series according to time;

[0197] A transformation module, configured to perform Fourier transform on each two-dimensional time series to obtain the frequency components of the two-dimensional time series;

[0198] A normalization module, configured to perform normalization processing on the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation power time series;

[0199] A value embedding module, configured to extract features from the set of tensors to obtain a value embedding result;

[0200] A position embedding module, configured to perform position encoding on the historical photovoltaic power generation power time series on the time axis and then expand it on the time axis to obtain a position embedding result;

[0201] A time embedding module, configured to perform time embedding on the historical photovoltaic power generation power time series to obtain a time embedding result;

[0202] An addition module, configured to add the value embedding result, the position embedding result and the time embedding result to obtain a total embedding result;

[0203] An enhancement module, configured to perform vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result;

[0204] An aggregation module, configured to perform spatio-temporal cross aggregation on the attention enhanced result to obtain a prediction result.

[0205] In addition, an embodiment of the present application further provides a computer program product. When the computer program product runs on a computer or a processor, it causes the computer or the processor to execute the above related steps to implement any one of the photovoltaic power generation prediction methods described above.

[0206] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A photovoltaic power generation power prediction method, characterized in that: include: Obtain historical photovoltaic power generation time series; Decompose the historical photovoltaic power generation time series into multiple two-dimensional time series; Perform Fourier transform on each two-dimensional time series to obtain the frequency components of the two-dimensional time series; Normalize the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series; The normalizing process of the frequency components of the two-dimensional time series includes: in, represents the normalization result of the frequency component of the two-dimensional time series, Represents the total number of moments in a two-dimensional time series, Represents a specific time point in a two-dimensional time series, ranging from 1 to , express Moment dimension the amplitude of the frequency component; in, represents frequency amplitude normalization, represents the first half of the Fourier transform, represents a window function, express Moment dimension Two-dimensional time series data, express Moment dimension Two-dimensional time series data, express Moment dimension Two-dimensional time series data, Indicates the optimal window length; Extract features from the frequency components of the historical photovoltaic power generation time series and obtain value embedding results; Perform position encoding on the time axis for the historical photovoltaic power generation time series, and then expand it on the time axis to obtain the position embedding result; Time embedding is performed on the historical photovoltaic power generation time series to obtain the time embedding result; Add the value embedding result, position embedding result and time embedding result to get the total embedding result; Performing vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result; Performing spatiotemporal cross aggregation on the attention enhancement results to obtain a prediction result; Perform vector-enhanced attention processing on the total embedding result, including: S2-1, generate query vectors through a linear layer respectively , key vector and a numeric vector : in, represents the linear transformation function used to generate the query vector, represents the prediction target, represents the linear transformation function used to generate the key vector, represents a linear transformation function used to generate a numeric vector, express Moment dimension against The embedding result of S2-2, using query vector Query key vector The component vectors of , and get the attention weights: in, express Group attention weight; in, express The attention weight of the moment, express The query vector at the moment; S2-3, yes The group attention weights are summed and normalized to obtain a set of two-dimensional attention weights; S2-4, the attention aggregation feature is obtained by the following formula , that is, the attention enhancement result: in, represents a set of two-dimensional attention weights obtained in step S2-3; Performing spatiotemporal cross aggregation on the attention enhancement results, including: S3-1, generate a set of key vectors through two fully connected layers : in, Express The result of the downsampling convolution operation; S3-2, initialize a zero tensor consistent with the predicted target length ; S3-3, zero tensor Perform value embedding, position embedding, and time embedding to obtain a sequence containing position information and time information ; S3-4, according to the sequence , through a fully connected layer, generate a series of query vectors ; S3-5, through a series of query vectors , query the spatiotemporal features generated in the spatiotemporal cross-aggregation encoding phase and obtain the corresponding attention scores: in, Indicates the query vector The learned weights, represents the feature dimension, Indicates the query vector , Represents the query vector Serial number; S3-6, aggregation using attention scores: in, Indicates the prediction result.

2. A photovoltaic power generation power prediction method according to claim 1, characterized in that: The historical photovoltaic power generation time series is decomposed into multiple two-dimensional time series, including: Each variable at the same moment in the historical photovoltaic power generation time series is regarded as an independent individual, so that different variables at different moments maintain a sequential relationship, and the historical photovoltaic power generation time series is disassembled.

3. A photovoltaic power generation power prediction method according to claim 2, characterized in that: The feature extraction from the frequency component of the historical photovoltaic power generation time series includes: A two-dimensional convolution operation is performed on the frequency components of the historical photovoltaic power generation time series to extract features.

4. A photovoltaic power generation power prediction method according to claim 3, characterized in that: Before performing Fourier transform on each two-dimensional time series, the method further includes padding each two-dimensional time series: S1-1, perform Fourier transform on the value of each variable in the two-dimensional time series to obtain the frequency component corresponding to the value of each variable; S1-2, obtain the frequency component with the largest amplitude among the frequency components corresponding to the value of each variable, and record it as the frequency component with the largest amplitude; S1-3, divide the length of the two-dimensional time series corresponding to the frequency component with the maximum amplitude by the peak index of the non-zero index to obtain the minimum positive period as the optimal window length; S1-4, perform mirror padding of half the optimal window length at both ends of each two-dimensional time series.

5. The photovoltaic power generation prediction method according to claim 1, characterized in that: The position encoding of the historical photovoltaic power generation time series on the time axis includes: A two-dimensional convolution with a kernel size of 1×3 is used to perform position encoding on the time axis for the historical photovoltaic power generation time series.

6. A photovoltaic power generation power prediction system, characterized in that: include: Acquisition module, used to obtain the historical photovoltaic power generation time series; A disassembly module is used to disassemble the historical photovoltaic power generation time series into multiple two-dimensional time series; A transformation module is used to perform Fourier transformation on each two-dimensional time series to obtain the frequency components of the two-dimensional time series; The normalization module is used to normalize the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series; The normalizing process of the frequency components of the two-dimensional time series includes: in, represents the normalization result of the frequency component of the two-dimensional time series, Represents the total number of moments in a two-dimensional time series, Represents a specific time point in a two-dimensional time series, ranging from 1 to , express Moment dimension the amplitude of the frequency component; in, represents frequency amplitude normalization, represents the first half of the Fourier transform, represents a window function, express Moment dimension Two-dimensional time series data, express Moment dimension Two-dimensional time series data, express Moment dimension Two-dimensional time series data, Indicates the optimal window length; The value embedding module is used to extract features from the frequency components of the historical photovoltaic power generation time series and obtain value embedding results; The position embedding module is used to position encode the historical photovoltaic power generation time series on the time axis, and then expand it on the time axis to obtain the position embedding result; The time embedding module is used to perform time embedding on the historical photovoltaic power generation time series to obtain the time embedding result; The summation module is used to add the value embedding results, position embedding results and time embedding results to obtain the total embedding result; An enhancement module is configured to perform vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result; An aggregation module, configured to perform spatiotemporal cross aggregation on the attention enhancement results to obtain a prediction result; Perform vector-enhanced attention processing on the total embedding result, including: S2-1, generate query vectors through a linear layer respectively , key vector and a numeric vector : in, represents the linear transformation function used to generate the query vector, represents the prediction target, represents the linear transformation function used to generate the key vector, represents a linear transformation function used to generate a numeric vector, express Moment dimension against The embedding result of S2-2, using query vector Query key vector The component vectors of , and get the attention weights: in, express Group attention weight; in, express The attention weight of the moment, express The query vector at the moment; S2-3, yes The group attention weights are summed and normalized to obtain a set of two-dimensional attention weights; S2-4, the attention aggregation feature is obtained by the following formula , that is, the attention enhancement result: in, represents a set of two-dimensional attention weights obtained in step S2-3; Performing spatiotemporal cross aggregation on the attention enhancement results, including: S3-1, generate a set of key vectors through two fully connected layers : in, Express The result of the downsampling convolution operation; S3-2, initialize a zero tensor consistent with the predicted target length ; S3-3, zero tensor Perform value embedding, position embedding, and time embedding to obtain a sequence containing position information and time information ; S3-4, according to the sequence , through a fully connected layer, generate a series of query vectors ; S3-5, through a series of query vectors , query the spatiotemporal features generated in the spatiotemporal cross-aggregation encoding phase and obtain the corresponding attention scores: in, Indicates the query vector The learned weights, represents the feature dimension, Indicates the query vector , Represents the query vector Serial number; S3-6, aggregation using attention scores: in, Indicates the prediction result.

7. A computer program product, characterized in that: The computer program product comprises instructions, and when the instructions are executed by a processor, a photovoltaic power generation power prediction method according to any one of claims 1 to 5 is implemented.

Images