Photovoltaic power generation power prediction method and related device

Abstract

The invention belongs to a prediction method, and provides a photovoltaic power generation power prediction method and a related device for solving the technical problems that when a Transform model is adopted for time sequence prediction at present, coupling embedding of synchronous multi-dimensional variables limits expression of an asynchronous space-time relation, and coupling decoding of the synchronous multi-dimensional variables limits learning of asynchronous space-time features. A historical photovoltaic power generation power time sequence is obtained, value embedding, position embedding and time embedding are carried out respectively, a value embedding result, a position embedding result and a time embedding result are added to obtain a total embedding result, vector attention enhancement processing is carried out on the total embedding result to obtain an attention enhancement result, and finally, the attention enhancement result is obtained. And carrying out space-time cross aggregation on the attention enhancement result to obtain a prediction result.

Description

Technical Field

[0001] The present application relates to a prediction method, and specifically to a photovoltaic power generation power prediction method and related devices. Background Art

[0002] Photovoltaic power generation, as a sustainable and clean energy technology, has been widely used in the global energy supply. However, since photovoltaic power generation depends on meteorological conditions (such as solar radiation, temperature, wind speed, etc.) and has significant randomness and volatility, the accurate prediction of photovoltaic power generation has become a complex and important research issue. In practical applications, grid managers and photovoltaic power station operators need high-precision power prediction models to achieve balanced scheduling and stable supply of electricity, reduce the demand for spare capacity and operating costs.

[0003] Currently, time series forecasting methods are widely used in photovoltaic power generation forecasting, especially Transformer-based models, which have become a mainstream choice due to their superiority in processing long sequence data and capturing spatiotemporal characteristics. However, in the photovoltaic power generation scenario, such models still have the following two limitations in capturing complex asynchronous spatiotemporal relationships and mitigating accidental fluctuations: (1) The coupled embedding of synchronous multidimensional variables limits the expression of asynchronous spatiotemporal relationships.

[0004] (2) Synchronous multi-dimensional variable coupling decoding limits the learning of asynchronous spatiotemporal features. Summary of the invention

[0005] The present application aims to provide a photovoltaic power generation power prediction method and related devices to address the technical problems that when the Transformer model is currently used for time series prediction, the coupled embedding of synchronous multidimensional variables limits the expression of asynchronous spatiotemporal relationships, and the coupled decoding of synchronous multidimensional variables limits the learning of asynchronous spatiotemporal features.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions: In a first aspect, the present application proposes a photovoltaic power generation power prediction method, comprising: Obtain historical photovoltaic power generation time series; Decompose the historical photovoltaic power generation time series into multiple two-dimensional time series; Perform Fourier transform on each two-dimensional time series to obtain the frequency components of the two-dimensional time series; The frequency components of the two-dimensional time series are normalized to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series; Extract features from the frequency components of the historical photovoltaic power generation time series and obtain value embedding results; The historical photovoltaic power generation time series is position-encoded on the time axis, and then expanded on the time axis to obtain the position embedding result; Time embedding is performed on the historical photovoltaic power generation time series to obtain the time embedding result; Add the value embedding result, position embedding result and time embedding result to get the total embedding result; Performing vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result; The attention enhancement results are cross-aggregated in time and space to obtain prediction results.

[0007] Furthermore, the historical photovoltaic power generation time series is decomposed into multiple two-dimensional time series, including: Each variable at the same time in the historical photovoltaic power generation time series is regarded as an independent individual, so that different variables at different times maintain a sequential relationship, and the historical photovoltaic power generation time series is disassembled.

[0008] Furthermore, the extracting of features from the frequency component of the historical photovoltaic power generation time series includes: A two-dimensional convolution operation is performed on the frequency components of the historical photovoltaic power time series to extract features.

[0009] Furthermore, before performing Fourier transform on each two-dimensional time series, the method further includes padding each two-dimensional time series: S1-1, perform Fourier transform on the value of each variable in the two-dimensional time series to obtain the frequency component corresponding to the value of each variable; S1-2, obtain the frequency component with the largest amplitude among the frequency components corresponding to the value of each variable, and record it as the frequency component with the largest amplitude; S1-3, divide the length of the two-dimensional time series corresponding to the frequency component with the maximum amplitude by the peak index of the non-zero index to obtain the minimum positive period as the optimal window length; S1-4, mirror padding with half the optimal window length is performed at both ends of each two-dimensional time series.

[0010] Furthermore, the normalizing process of the frequency component of the two-dimensional time series includes:

[0011] in, represents the normalization result of the frequency component of the two-dimensional time series, Represents the total number of moments in a two-dimensional time series, Represents a specific time point in a two-dimensional time series, ranging from 1 to , express Time dimension the amplitude of the frequency component;

[0012] in, represents frequency amplitude normalization, represents the first half of the Fourier transform, represents a window function, express Time dimension Two-dimensional time series data, express Moment Dimension Two-dimensional time series data, express Time dimension Two-dimensional time series data, Indicates the optimal window length.

[0013] Furthermore, the position encoding of the historical photovoltaic power generation time series on the time axis includes: A two-dimensional convolution with a kernel size of 1×3 is used to positionally encode the historical photovoltaic power time series on the time axis.

[0014] Further, vector enhanced attention processing is performed on the total embedding result, including: S2-1, generate query vectors through a linear layer respectively , key vector and a numeric vector :

[0015]

[0016]

[0017] in, represents the linear transformation function used to generate the query vector, represents the prediction target, represents the linear transformation function used to generate the key vector, represents a linear transformation function used to generate a numeric vector, express Moment Dimension against The embedding result of S2-2, using query vector Query key vector The component vectors of , and get the attention weights:

[0018] in, express Group attention weight;

[0019] in, express The attention weight of the moment, express The query vector at the moment; S2-3, yes The group attention weights are added and normalized to obtain a set of two-dimensional attention weights; S2-4, the attention aggregation feature is obtained by the following formula , that is, the attention enhancement result:

[0020] in, Represents a set of two-dimensional attention weights obtained in step S2-3.

[0021] Furthermore, the attention enhancement result is subjected to spatiotemporal cross aggregation, including: S3-1, generate a set of key vectors through two fully connected layers :

[0022] in, Express The result of the downsampling convolution operation; S3-2, initialize a zero tensor consistent with the predicted target length ; S3-3, Zero Tensor Perform value embedding, position embedding, and time embedding to obtain a sequence containing position information and time information ; S3-4, according to the sequence , through a fully connected layer, generating a series of query vectors ; S3-5, through a series of query vectors , query the spatiotemporal features generated in the spatiotemporal cross-aggregation encoding phase and obtain the corresponding attention scores:

[0023] in, Indicates query vector The learned weights, represents the feature dimension, Indicates query vector , Represents the query vector Serial number; S3-6, aggregation using attention scores:

[0024] in, Indicates the prediction result.

[0025] In a second aspect, the present application proposes a photovoltaic power generation power prediction system, comprising: An acquisition module is used to obtain the historical photovoltaic power generation time series; A disassembly module is used to disassemble the historical photovoltaic power generation time series into multiple two-dimensional time series according to time; A transformation module, used for performing Fourier transformation on each two-dimensional time series to obtain the frequency components of the two-dimensional time series; A normalization module is used to normalize the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series; A value embedding module, used for extracting features from the set of tensors to obtain a value embedding result; The position embedding module is used to position encode the historical photovoltaic power generation time series on the time axis, and then expand it on the time axis to obtain the position embedding result; The time embedding module is used to embed the historical photovoltaic power generation time series to obtain the time embedding result; A summing module is used to add the value embedding result, the position embedding result and the time embedding result to obtain the total embedding result; An enhancement module is used to perform vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result; The aggregation module is used to perform spatiotemporal cross aggregation on the attention enhancement results to obtain a prediction result.

[0026] In a third aspect, the present application proposes a computer program product, which includes instructions, and when the instructions are executed by a processor, the above-mentioned photovoltaic power generation power prediction method is implemented.

[0027] Compared with the prior art, this application has the following beneficial effects: The present application proposes a photovoltaic power prediction method, which decomposes the historical photovoltaic power time series into multiple two-dimensional time series, performs Fourier transform on each two-dimensional time series, and then normalizes the frequency components of the two-dimensional time series. Then, features are extracted from the set of tensors of the frequency components of the historical photovoltaic power time series to obtain value embedding results, and the historical photovoltaic power time series is position-encoded on the time axis, and then expanded on the time axis to obtain position embedding results. The present application encodes each variable independently in the same time and space to form an embedding vector in a decoupled form. At the same time, the frequency domain information is introduced to suppress unexpected fluctuations. Then, the total embedding result obtained by adding the value embedding result, the position embedding result and the time embedding result is vector-enhanced attention processing to obtain an attention enhancement result. A two-dimensional vector-enhanced attention mechanism is used to decouple and process useful synchronous and asynchronous spatiotemporal information for prediction, emphasizing information useful for prediction, and reducing redundant calculations in aggregation operations. Finally, a spatiotemporal cross-aggregation mechanism is proposed to aggregate features from two dimensions of time and space to finally obtain a prediction result. The present application makes full use of time and space information to ensure the accuracy of prediction.

[0028] The present application also proposes a photovoltaic power generation prediction system, an electronic device and a computer storage medium, which have all the advantages of the above-mentioned photovoltaic power generation prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 This is a schematic diagram of the first process of the photovoltaic power generation prediction method of this application; Figures 2(a) to 2(c) are schematic diagrams of visualization results of the ISDFormerr model, the Transformerr model, and the Autoformer model on the Solar dataset in the multivariate to univariate prediction task in an embodiment of the present application; wherein Figure 2(a) corresponds to the ISDFormer model, Figure 2(b) corresponds to the Autoformer model, and Figure 2(c) corresponds to the Transformer model; FIG3 (a) and FIG3 (b) are schematic diagrams of feature attention allocated to two groups of experiments when verifying the embedding layer experiment in an embodiment of the present application; wherein FIG3 (a) is the E group experiment, and FIG3 (b) is the F group experiment; FIG4 (a) and FIG4 (b) are schematic diagrams of the predicted results of two groups of experiments in verifying the embedding layer experiment in an embodiment of the present application; among them, FIG4 (a) is group E, and FIG4 (b) is group F.

[0031] Figure 5 A schematic diagram of a photovoltaic power generation prediction system for this application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0033] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0034] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0035] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0036] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0037] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0038] The accuracy of photovoltaic power generation prediction is crucial to the stable operation and efficient management of the power grid. Although the Transformer-based time series prediction model has shown strong capabilities in processing sequence data and spatiotemporal features, it still faces some specific challenges and limitations in the specific application scenario of photovoltaic power generation: (1) The coupled embedding of synchronous multidimensional variables limits the expression of asynchronous spatiotemporal relationships.

[0039] In photovoltaic power generation prediction, there are many factors that affect power output, including but not limited to light intensity, temperature, wind speed, humidity, cloud coverage, etc. In the embedding stage, the traditional Transformer-based model will simultaneously map all variables that affect power output into a coupled vector. This coupled embedding method hinders the model from effectively learning the complex asynchronous spatiotemporal relationships between various variables. For example, different meteorological factors have different effects on power generation at different time points, and the coupled embedding method cannot fully express these asynchronous effects, which in turn limits the accurate prediction of photovoltaic power generation. In addition, the direct point-by-point encoding method ignores the randomness in the time series data, making it impossible for the attention mechanism to effectively detect and process the negative impact of abnormal meteorological conditions on power forecasting.

[0040] (2) Synchronous multi-dimensional variable coupling decoding limits the learning of asynchronous spatiotemporal features.

[0041] In the encoding stage, the self-attention mechanism in the Transformer model aims to capture the contextual relationship between various variables. However, in the photovoltaic power generation prediction scenario, the learning of such contextual relationship is often too complex and may not have practical application value. In addition, in the decoding stage, the commonly used one-dimensional aggregation operation cannot effectively capture the differences in time and space of various meteorological factors when processing the two-dimensional asynchronous spatiotemporal relationship of photovoltaic data, which will affect the prediction results.

[0042] Based on the above situation, the present application proposes a photovoltaic power generation power prediction method and related devices, which are described in detail below in conjunction with embodiments and drawings.

[0043] like Figure 1As shown, it is a schematic diagram of the first process of the photovoltaic power prediction method of the present application, which may include: S101, obtaining a historical photovoltaic power generation time series.

[0044] In practical applications, historical data can be obtained through the monitoring system of photovoltaic power stations, smart grid databases or meteorological data combined with photovoltaic panel performance to form a historical photovoltaic power generation time series. In order to improve the accuracy of subsequent predictions, the acquired data can also be cleaned to remove outliers and missing values, or smoothed to reduce noise.

[0045] S102, decomposing the historical photovoltaic power generation time series into multiple two-dimensional time series.

[0046] S103, performing Fourier transform on each two-dimensional time series to obtain the frequency component of the two-dimensional time series.

[0047] Through Fourier transform, the two-dimensional time series can be converted from the time domain to the frequency domain to reveal the hidden periodic characteristics. The frequency components of the two-dimensional time series obtained after the transformation can reflect the signal strength at different frequencies.

[0048] S104, normalizing the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series.

[0049] In this application, the dimensions of different frequency components are unified to facilitate subsequent processing. In practical applications, methods such as minimum-maximum normalization can be used to convert frequency components into values ​​within a standard range and organize them into a tensor form.

[0050] S105, extracting features from the set of tensors of the frequency components of the historical photovoltaic power generation time series to obtain a value embedding result.

[0051] Extracting key features from tensors can provide valuable information. In practical applications, convolutional neural networks, feature selection algorithms, or embedding layers in deep learning can be used to convert high-dimensional tensors into low-dimensional, dense vector representations.

[0052] S106, position encoding the historical photovoltaic power generation time series on the time axis, and then expanding it on the time axis to obtain a position embedding result.

[0053] Position encoding can give unique position information to each time point in the historical PV power time series, which helps to understand the time sequence.

[0054] S107, time embedding is performed on the historical photovoltaic power generation time series to obtain a time embedding result.

[0055] Time embedding takes into account the periodic characteristics of time itself (such as day, week, season, etc.), which can provide additional time context. In practical applications, time embedding vectors can be generated through periodic functions (such as sine or cosine functions) or time feature engineering (such as date, week, month, etc.).

[0056] S108, adding the value embedding result, the position embedding result and the time embedding result to obtain a total embedding result.

[0057] By integrating all the embedded information, a comprehensive vector representation can be formed for subsequent prediction tasks.

[0058] S109, performing vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result.

[0059] Through the attention mechanism, we can dynamically focus on the most important information for prediction. When performing attention aggregation operations, we can calculate the attention weight and multiply it with the attention enhancement result to enhance the key information.

[0060] S110, performing spatiotemporal cross aggregation on the attention enhancement results to obtain a prediction result.

[0061] This application combines the information of time dimension and space dimension to make the final prediction. Through time-space cross-aggregation, further feature extraction and sequence modeling are performed on the attention enhancement results, and finally the prediction result of photovoltaic power generation can be output.

[0062] The photovoltaic power generation prediction method of the present application is described in more detail below through another embodiment of the photovoltaic power generation prediction method.

[0063] Will be in the interval Time series collected within Recorded as ,in, , is the interval Inside Time series data at the moment, is the dimension of the feature. ,predict Time has come Time series of moments .

[0064] In order to effectively learn the asynchronous spatiotemporal relationship of covariates, the present application proposes a photovoltaic power generation prediction method, which implements the above method through a set of models, and the model is recorded as an ISDFormer model. The ISDFormer model includes an embedding layer, an encoder, and a decoder. In the embedding layer, the aforementioned steps S101 to S108 are executed. In the embedding layer, a new embedding method is proposed to encode each variable independently in the same time and space to form an embedding vector in a decoupled form. At the same time, frequency domain information is introduced to enhance the ability of the ISDFormer model to suppress unexpected fluctuations. In the encoder part (corresponding to the aforementioned step S109), a two-dimensional vector enhanced attention mechanism is proposed to decouple and process useful synchronous and asynchronous spatiotemporal information for prediction, emphasize information useful for prediction, and reduce redundant calculations in aggregation operations. In the decoder part (corresponding to the aforementioned step S110), a spatiotemporal cross-aggregation mechanism is proposed to aggregate the features obtained in the encoder stage from the two dimensions of time and space, and finally obtain the prediction result. Specifically: 1. Propose a new embedding method in the embedding layer.

[0065] In previous time series forecasting work, it is assumed that ,in, Is the interval The inner dimension is time series, . express The time dimension is time series data.

[0066] After processing by the embedding layer, the embedding representation of a given time series can be obtained ,in, ,express The embedding result at the moment, Represents the model depth, which can be expressed as:

[0067] in, , Represents time series data at time As an example, in the photovoltaic scenario, the eigenvector It may include relevant data of photovoltaic power generation, such as sunshine intensity, temperature, humidity, wind speed, wind direction, cloud cover and other meteorological factors. Indicated in Moment, The feature vector of dimension Indicated in moment, the feature vector of the first dimension, Indicated in moment, the feature vector of the second dimension, Indicates connecting multiple feature vectors in the feature dimension (i.e., concatenation operation). represents a one-dimensional convolution operation applied along the time axis, with input features and model depth output features, represents position embedding, represents the embedding of date features, Representing specific date data, date features combined with specific date information, such as year, month, and day, can help the model learn the impact of seasonal changes on photovoltaic power generation.

[0068] The embedding method in the ISDFormer model proposed in this application is applied to time series prediction. However, this method couples variables of different dimensions together, making it difficult for the model to mine asynchronous spatiotemporal relationships between variables and difficult to distinguish outliers.

[0069] To solve this problem, this application proposes a new embedding method, which is an improved two-dimensional embedding method, mainly divided into three parts: value embedding, position embedding and time embedding.

[0070] In the value embedding part, the value of each moment and each dimension can be encoded together with the corresponding frequency domain information. First, in order to achieve the purpose of decoupling the spatial vector, each variable at the same moment in the historical photovoltaic power generation time series is regarded as an independent individual, while different variables at different moments still maintain their order relationship, and the historical photovoltaic power generation time series is truncated into two dimensions from the time perspective. Then, in order to enable the model to perceive the situation of isolated points in the historical photovoltaic power generation time series, a FreqEmbedding (frequency embedding) module is also designed, through which the frequency distribution information can be obtained. In the FreqEmbedding module, the core work is to perform windowed frequency domain decomposition of the time series. In order to ensure that the first and last positions in the time series are located in the middle of the FFT (Fast Fourier Transform) decomposition segment, half of the optimal window length can be mirrored and filled at both ends of the time series. The size of the window will affect the accuracy of frequency decomposition. In practical applications, the optimal window length should be the minimum positive period of signal change. If different periods are used, sidelobe leakage may occur, which will have an adverse effect on the decomposition results.

[0071] In practical applications, in order to obtain the optimal window length, first perform FFT processing on the value of each variable to obtain the frequency component, find the frequency component with the largest amplitude, and divide the length of the input sequence by the peak index of the non-zero index to obtain the minimum positive period, that is, the optimal window length, which is recorded as w , which is used to define the time series interval covered by each frequency domain analysis. In addition, in order to ensure that the results of different time series decompositions have the same order of magnitude, this application also designs an amplitude normalization function, which normalizes the results of frequency domain decomposition according to the amplitude to obtain a set of tensors, which can be expressed as: , used to reflect the change of frequency. Among them, It can be expressed as:

[0072] in, ,express Moment Dimension The amplitude of the frequency component, represents frequency amplitude normalization, express The first half of Represents the window function, which reduces spectrum leakage in spectrum analysis by applying weights to a limited period of time of the signal. express Moment Dimension Time series data, express Moment Dimension Time series data The variables at each moment and each dimension are then concatenated with the magnitude of the frequency components obtained from the decomposition to obtain the value embedding result:

[0073] in, Indicates the value embedding result. You can also perform two-dimensional convolution on the value embedding result. The above steps are taken as the content executed in the FreqEmbedding module.

[0074] For position embedding, a three-dimensional tensor is set, where the three dimensions refer to time, space, and model depth. In practical applications, we only need to distinguish variables at different times, so we only need to encode positions on the time axis. So we can use a two-dimensional convolution with a kernel size of 1x3. To ensure the consistency of the results, we can expand it on the time axis. This also means that the convolution is applied along the time axis, and no processing is performed in the spatial domain.

[0075] Then, the historical photovoltaic power generation time series is time-embedded to obtain the time embedding result. The current value in the time series is not only related to the previous values, but also to the time interval between these values. Through time embedding, the one-dimensional time series data can be converted into a multi-dimensional vector space, in which each vector contains the information of multiple time points in the original time series.

[0076] Finally, add the results of position embedding, time embedding, and value embedding to get the added embedding result:

[0077] in, express Moment Dimension Time Target The embedded result after addition is Indicates a two-dimensional convolution operation on the input data. represents the input vector after expansion on the time axis. Then the total embedding result is Indicates the time period The total embedding result of the historical photovoltaic power time series collected within.

[0078] 2. Vector-enhanced attention processing (encoder).

[0079] After obtaining the total embedding result, in order to avoid the redundant context information brought by the aggregation operation of the traditional attention mechanism and emphasize the vectors that are useful for predicting values, this application designs a two-dimensional vector enhanced attention method.

[0080] First, for the variable to be predicted, it is expressed as the target , and generate the query vector through a linear layer :

[0081] in, represents the linear transformation function used to generate the query vector. For all variables, the key vector and a numeric vector Each is generated by a linear layer:

[0082]

[0083] in, represents the linear transformation function used to generate the key vector, Represents a linear transformation function used to generate a numeric vector. To query all , and get the attention weight. For each query vector , can generate a two-dimensional attention weight, so that Group attention weight, expressed as :

[0084] in: , express The attention weight of the moment, express The query vector at the moment.

[0085] Based on the query vector , add these attention weights and normalize them to get a set of two-dimensional attention weights. This two-dimensional attention weight represents the importance of each covariate to the variable to be predicted:

[0086] in, represents the weights learned in vector-enhanced attention, Finally, in order to emphasize the importance, unlike the traditional attention aggregation operation, this application directly combines the attention weight with Multiply, highlight Some important features of Specifically:

[0087] Since the vector-enhanced attention of this application processes inputs and outputs with the same shape and size, it can seamlessly replace the self-attention mechanism. Although this mechanism is designed for two-dimensional features, by using , can also be applied to the case where there is only one feature.

[0088] 3. Space-time cross-aggregation (decoder).

[0089] When embedding vectors, spatial features have potential correlations and therefore cannot be directly coupled and encoded. After obtaining these independent spatial encoding vectors, the next step is to process them in the most appropriate way to aggregate features. To this end, this application designs spatiotemporal cross aggregation for aggregating two-dimensional spatiotemporal information.

[0090] In the decoder stage, we first initialize a zero tensor that is consistent with the prediction length, expressed as Then, this zero tensor is embedded by value, position and time to obtain a sequence containing position and time information. Since the final tensor is one-dimensional, this process can follow the original embedding process. The initialized variables are represented as , and formalize it as:

[0091] in, Represents the result of value embedding.

[0092] For the encoder output, a set of key vectors are generated through two fully connected layers. :

[0093] in, Express The result of the downsampling convolution operation. Then a series of query vectors are generated from the embedding result through a fully connected layer :

[0094] These It is used to synchronously query the spatiotemporal features generated by the encoder stage to obtain a series of attention scores:

[0095] in, , indicating the query vector The learned weights, represents the time step in the time series, Indicates query vector , represents the feature dimension, Represents the query vector Finally, all features are aggregated using the attention score to obtain the output result:

[0096] in, , indicating the prediction result.

[0097] Compared with the classic cross-attention mechanism, the method of this application expands the attention from one dimension to two dimensions, further enriches the feature mining and expression methods of the attention mechanism, and brings more flexible expression effects.

[0098] In order to prove the technical effect of the present application, the photovoltaic power generation prediction method of the present application was verified and the following experiments were carried out: The photovoltaic power prediction method proposed in this application was trained and tested on a self-built Solar dataset. The Solar dataset was established by deploying data acquisition terminals in a photovoltaic power station in northern China. It records photovoltaic power generation from June 30, 2020 to June 30, 2023. Each record has 8 attributes, including power generation, total radiation, direct radiation, diffuse radiation, temperature, air pressure, wind direction and wind speed. It is sampled every 15 minutes, with a total of 105,216 records. The training set, validation set and test set of each dataset are divided according to the ratio of 0.7, 0.1 and 0.2.

[0099] Experimental details: During the training process, the L2 loss function (i.e., square loss function or Euclidean loss function) is used, and the Adam (Adaptive Moment Estimation) optimizer is adopted. The initial learning rate is set to 0.0001 and the batch size is set to 32. The basic hyperparameters include the number of encoder layers, the number of decoder layers, the feature dimension, the number of attention heads, and the convolution function dimension, which are set to 2, 1, 512, 8, and 2048, respectively. The experiment was repeated 3 times, and the average value of the index was reported. In this embodiment, two evaluation indicators MSE (Mean Squared Error) and MAE (Mean Absolute Error) are used. In order to objectively evaluate the method of this application, 6 popular time series prediction models based on the Transformer model are selected as benchmark models, namely: Transformer model, Informer model, Autoformer model, FEDformer (Frequency Enhanced Decomposed Transformer) model, TimesNet model, and Crossformer model. All experimental results of the benchmark models used for comparison are trained, tested, and validated using the open source benchmark model code. To ensure a fair comparison, the time series input and output scales of all models are standardized and kept consistent.

[0100] Comparison of experimental results: The input length of all models is 384, and the prediction lengths are 96, 384, 672, and 1344, respectively. They are used to simulate short-term prediction, medium-term prediction, medium- to long-term prediction, and long-term prediction.

[0101] Single variable to single variable prediction: A single variable to single variable prediction experiment was conducted on the Solar dataset, and the results are shown in Table 1. It can be seen that the MSE and MAE of the method proposed in this application (ISDFormer model in Table 1) are significantly better than other methods. This shows that the learning paradigm of the ISDFormer model can effectively capture the potential asynchronous characteristics of time series variables, thereby improving the prediction performance of the model. It should be noted that the "Methods" row in Table 1 indicates the model used, and the "Metric" row indicates the evaluation indicator. The meaning is the same in other subsequent tables.

[0102] Table 1 Comparison of single variable to single variable prediction results

[0103] Multivariate to univariate prediction: Multivariate to univariate prediction experiments were conducted on the Solar dataset, and the results are shown in Table 2. It can be seen that when conducting multivariate to univariate prediction experiments, the ISDFormer model performs well under different prediction lengths (such as 96, 384, 672, and 1344).

[0104] Table 2 Comparison of multivariate to univariate prediction results

[0105] Analyzing Tables 1 and 2, on the Solar dataset, the average MSE of the multivariate to univariate task differs from the average MSE of the univariate to univariate task by only 3.6%. This is because the Solar dataset is more regular, and the introduction of covariates does not help much in improving the model performance.

[0106] Figures 2 (a) to 2 (c) are schematic diagrams of the visualization results of the ISDFormer model, Transformer model, and Autoformer model on the Solar dataset in the multivariate to univariate prediction task. The red line in Figure 2 represents the prediction result, and the blue line represents the actual situation. Figure 2 (a) corresponds to the ISDFormer model, Figure 2 (b) corresponds to the Autoformer model, and Figure 2 (c) corresponds to the Transformer model. It can be seen that compared with other models, the multi-periodicity and global trend of the time series are more effectively learned because the ISDFormer model adopts the method of cross-dimensional decoupling and frequency domain learning. It should be noted that in Figures 2 (a) to 2 (c), the horizontal axis represents time, that is, the time dimension in the time series data, the vertical axis represents the power value corresponding to the time point, the blue line represents the prediction, and the red line represents the actual situation.

[0107] In the method of the present application, there are three key factors: frequency and variable embedding, vector enhanced attention and spatiotemporal cross aggregation. Frequency and variable embedding contains two core points: one is to encode each variable independently in order to capture the potential connection between variables at different times and the same time; the other is to add frequency domain information to the encoding result. The core idea of ​​the vector enhanced attention mechanism is to emphasize components that are useful for predicting results. Spatiotemporal cross aggregation is a feature aggregation mechanism designed to cooperate with independent encoding. Correspondingly, the present application aims to solve three problems: the interference problem in the data, the problem of capturing asynchronous spatiotemporal relationships, and how to enhance features. In order to verify the performance of the three key factors on the data set, four models were designed for comparative experiments. Among them, Group A uses the complete ISDFormer model, Group B uses the ISDFormer model without the frequency component, Group C uses the standard Transformer model for time series prediction, and Group D uses the Transformer model with vector enhanced attention. All four models are trained on the Solar data set, and their input and output lengths are 384. The test results are shown in Table 3. The "Group" column in Table 3 represents the experimental group, and the "Method" column represents the model used. “ISDFormer without frequency” means using the ISDFormer model without a frequency component, and “Transformer with VEA” means the Transformer model with vector-enhanced attention.

[0108] Table 3 Comparison of test results of four models

[0109] It can be seen that in the comparison between Group A and Group B, Group A achieved better results, indicating that frequency encoding has an enhancing effect on the results, that is, it solves the data interference problem mentioned above. In the comparative experiment between Group C and Group D, Group D achieved better results, indicating that vector-enhanced attention has a good effect on feature enhancement. In the comparative experiment between Group B and Group D, since the Transformer with vector-enhanced attention mechanism is equivalent to ISDFormer with frequency components removed and coupled encoding performed, their comparative relationship shows that ISDFormer can capture asynchronous spatiotemporal relationships.

[0110] In addition, in order to illustrate the role of the embedding layer of this application, a pair of experiments were designed. The experiments are divided into two groups, Group E is the complete ISDFormer, and for comparison, Group F is the ISDFormer without frequency domain embedding. Both groups of experiments performed univariate to univariate prediction training on the Solar dataset and the sinusoidal dataset. The use of univariate to univariate prediction is to eliminate the influence of covariates on the prediction results. The results show that there is almost no performance difference between the two groups on the sinusoidal dataset, while on the photovoltaic dataset, Group E performs better than Group F, which can also be seen in the ablation experiment. This is because the sinusoidal dataset is very regular, and the effect of frequency information on the model is almost negligible; while the fluctuations in the Solar dataset are more obvious, and frequency domain information can help the model distinguish which points are abnormal.

[0111] In order to find out how the model uses this frequency information internally, two days of photovoltaic power generation data are input into Group E and Group F respectively, and they are asked to predict one day's power generation. The input data comes from the Solar dataset. The power generation data on the first day of the example has many accidental interferences, while the power generation data on the second day is relatively stable.

[0112] Group E introduces a vector with 49 features (this value is half the window length plus one) in the embedding stage to additionally describe the surrounding information. Since the frequency components have been amplitude normalized, their color depth can reflect the frequency distribution without being affected by the energy size. In the relatively stable second half, its frequency distribution is more concentrated in low frequencies, while in the first half, there are relatively few low-frequency components. In the feature aggregation stage of the decoder, cross aggregation attempts to aggregate the features emphasized by the encoder. The attention assigned to these features in the experiments of Group E and Group F is shown in Figure 3 (a) and Figure 3 (b). Figure 3 (a) shows the attention map assigned to these features in the experiments of Group E, and Figure 3 (b) shows the attention map assigned to these features in the experiments of Group F. Among them, in Figure 3 (a) and Figure 3 (b), the vertical axis represents the features emphasized by the encoder, and the horizontal axis represents the timestamp sequence to be predicted.

[0113] Since there is no exchange of time series in the encoder of the model (except for convolution along the time axis), although the features output by the encoder are difficult to interpret, the relative order relationship in time is still maintained. Therefore, it can be seen that in the decoder model of group E, it assigns greater weights to the more stable points in the second half, that is, the model is more inclined to use more stable points for prediction. In contrast, group F has difficulty in emphasizing this feature. The prediction results of groups E and F are shown in Figures 4 (a) and 4 (b). The horizontal axis in Figures 4 (a) and 4 (b) represents time, that is, the time dimension in the time series data, and the vertical axis represents the power value corresponding to the time point. The blue line represents the prediction situation, and the red line represents the actual situation.

[0114] Furthermore, in order to illustrate the role of vector-enhanced attention, a set of control experiments was further designed. Group G uses vector-enhanced attention, while as a control, Group H uses the traditional self-attention mechanism represented by the aggregation operation. In the decoder part, since this mechanism needs to be used in conjunction with the spatial aggregation attention mechanism, Group G is a complete ISDFormer structure. Group H uses the classic Transformer architecture with a full attention mechanism. Training is performed on the Solar dataset, and the loss of the test set is monitored. Training stops when the best effect is achieved.

[0115] The main work of the encoder part is feature extraction, while the main work of the decoder part is feature aggregation. In order to observe which features are extracted by the encoder part, the 8-head attention scores of group H are plotted separately. It can be concluded that multiple consecutive query vectors pay attention to the same key vector and assign greater weight to it. In the 1st, 2nd, 4th, 6th and 8th attention heads, special attention is paid to the information between 260-270 in the sequence. In the self-attention mechanism, the next step after calculating the attention score is to aggregate features. Therefore, multiple consecutive tokens are replaced with similar or even identical content. In this way, the model will only focus on very limited content.

[0116] This situation can be alleviated if a feature enhancement mechanism is used. First, since the model aggregates features in the decoder stage, the decoder will guide the feature enhancement attention mechanism to highlight the desired features. Second, since there is no replacement operation when extracting features, even if only a limited part of the information is emphasized, the original information is not completely lost, and all features can still be obtained during feature aggregation in the decoder.

[0117] Unlike the self-attention mechanism, the model will emphasize all features in a relatively "even" manner without overemphasizing a certain feature. This avoids the loss of information, and the model avoids over-focusing on limited information.

[0118] like Figure 5 As shown, it is a schematic diagram of a photovoltaic power generation prediction system of the present application, which may include: An acquisition module is used to obtain the historical photovoltaic power generation time series; A disassembly module is used to disassemble the historical photovoltaic power generation time series into multiple two-dimensional time series according to time; A transformation module, used for performing Fourier transformation on each two-dimensional time series to obtain the frequency components of the two-dimensional time series; A normalization module is used to normalize the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series; A value embedding module, used for extracting features from the set of tensors to obtain a value embedding result; The position embedding module is used to position encode the historical photovoltaic power generation time series on the time axis, and then expand it on the time axis to obtain the position embedding result; The time embedding module is used to embed the historical photovoltaic power generation time series to obtain the time embedding result; A summing module is used to add the value embedding result, the position embedding result and the time embedding result to obtain the total embedding result; An enhancement module is used to perform vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; the attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result; The aggregation module is used to perform spatiotemporal cross aggregation on the attention enhancement results to obtain a prediction result.

[0119] In addition, an embodiment of the present application also provides a computer program product. When the computer program product runs on a computer or a processor, the computer or the processor executes the above-mentioned related steps to implement any of the photovoltaic power generation prediction methods introduced above.

[0120] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A photovoltaic power generation power prediction method, characterized in that: include: Obtain historical photovoltaic power generation time series; Decompose the historical photovoltaic power generation time series into multiple two-dimensional time series; Perform Fourier transform on each two-dimensional time series to obtain the frequency components of the two-dimensional time series; The frequency components of the two-dimensional time series are normalized to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series; Extract features from the frequency components of the historical photovoltaic power generation time series and obtain value embedding results; The historical photovoltaic power generation time series is position-encoded on the time axis, and then expanded on the time axis to obtain the position embedding result; Time embedding is performed on the historical photovoltaic power generation time series to obtain the time embedding result; Add the value embedding result, position embedding result and time embedding result to get the total embedding result; Performing vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; The attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result; The attention enhancement results are cross-aggregated in time and space to obtain prediction results.

2. A photovoltaic power generation prediction method according to claim 1, characterized in that: The historical photovoltaic power generation time series is decomposed into multiple two-dimensional time series, including: Each variable at the same time in the historical photovoltaic power generation time series is regarded as an independent individual, so that different variables at different times maintain a sequential relationship, and the historical photovoltaic power generation time series is disassembled.

3. A photovoltaic power generation prediction method according to claim 2, characterized in that: The extracting of features from the frequency component of the historical photovoltaic power generation time series includes: A two-dimensional convolution operation is performed on the frequency components of the historical photovoltaic power time series to extract features.

4. A photovoltaic power generation prediction method according to claim 3, characterized in that: Before performing Fourier transform on each two-dimensional time series, the method further includes filling each two-dimensional time series: S1-1, perform Fourier transform on the value of each variable in the two-dimensional time series to obtain the frequency component corresponding to the value of each variable; S1-2, obtain the frequency component with the largest amplitude among the frequency components corresponding to the value of each variable, and record it as the frequency component with the largest amplitude; S1-3, divide the length of the two-dimensional time series corresponding to the frequency component with the maximum amplitude by the peak index of the non-zero index to obtain the minimum positive period as the optimal window length; S1-4, mirror padding with half the optimal window length is performed at both ends of each two-dimensional time series.

5. A photovoltaic power generation prediction method according to claim 4, characterized in that: The normalizing process of the frequency component of the two-dimensional time series includes: in, represents the normalization result of the frequency component of the two-dimensional time series, Represents the total number of moments in a two-dimensional time series, Represents a specific time point in a two-dimensional time series, ranging from 1 to , express Time dimension the amplitude of the frequency component; in, represents frequency amplitude normalization, represents the first half of the Fourier transform, represents a window function, express Time dimension Two-dimensional time series data, express Time dimension Two-dimensional time series data, express Time dimension Two-dimensional time series data, Indicates the optimal window length.

6. A photovoltaic power generation power prediction method according to claim 5, characterized in that: The position encoding of the historical photovoltaic power generation time series on the time axis includes: A two-dimensional convolution with a kernel size of 1×3 is used to positionally encode the historical photovoltaic power time series on the time axis.

7. A photovoltaic power generation power prediction method according to claim 6, characterized in that: Performing vector enhanced attention processing on the total embedding result includes: S2-1, generate query vectors through a linear layer respectively , key vector and a numeric vector : in, represents the linear transformation function used to generate the query vector, represents the prediction target, represents the linear transformation function used to generate the key vector, represents a linear transformation function used to generate a numeric vector, express Time dimension against The embedding result of S2-2, using query vector Query key vector The component vectors of and get the attention weights: in, express Group attention weight; in, express The attention weight of the moment, express The query vector at the moment; S2-3, yes The group attention weights are added and normalized to obtain a set of two-dimensional attention weights; S2-4, the attention aggregation feature is obtained by the following formula , that is, the attention enhancement result: in, Represents a set of two-dimensional attention weights obtained in step S2-3.

8. A photovoltaic power generation power prediction method according to claim 7, characterized in that: Performing spatiotemporal cross aggregation on the attention enhancement results, including: S3-1, generate a set of key vectors through two fully connected layers : in, Express The result of the downsampling convolution operation; S3-2, initialize a zero tensor consistent with the predicted target length ; S3-3, Zero Tensor Perform value embedding, position embedding, and time embedding to obtain a sequence containing position information and time information ; S3-4, according to the sequence , through a fully connected layer, generating a series of query vectors ; S3-5, through a series of query vectors , query the spatiotemporal features generated in the spatiotemporal cross-aggregation encoding phase and obtain the corresponding attention scores: in, Indicates query vector The learned weights, represents the feature dimension, Indicates query vector , Represents the query vector Serial number; S3-6, aggregation using attention scores: in, Represents the prediction result.

9. A photovoltaic power generation power prediction system, characterized in that: include: An acquisition module is used to obtain the historical photovoltaic power generation time series; A disassembly module is used to disassemble the historical photovoltaic power generation time series into multiple two-dimensional time series according to time; A transformation module, used for performing Fourier transformation on each two-dimensional time series to obtain the frequency components of the two-dimensional time series; A normalization module is used to normalize the frequency components of the two-dimensional time series to obtain a set of tensors as the frequency components of the historical photovoltaic power generation time series; A value embedding module, used for extracting features from the set of tensors to obtain a value embedding result; The position embedding module is used to position encode the historical photovoltaic power generation time series on the time axis, and then expand it on the time axis to obtain the position embedding result; The time embedding module is used to embed the historical photovoltaic power generation time series to obtain the time embedding result; A summing module is used to add the value embedding result, the position embedding result and the time embedding result to obtain the total embedding result; An enhancement module, used for performing vector enhanced attention processing on the total embedding result to obtain an attention enhanced result; The attention aggregation operation in the vector enhanced attention processing is: multiplying the attention weight by the total embedding result; The aggregation module is used to perform spatiotemporal cross aggregation on the attention enhancement results to obtain a prediction result.

10. A computer program product, characterized in that: The computer program product comprises instructions, and when the instructions are executed by a processor, a photovoltaic power generation power prediction method according to any one of claims 1 to 8 is implemented.

Images