A method for ultra-short-term photovoltaic power prediction based on multimodal learning

Abstract

This invention provides an ultra-short-term photovoltaic power prediction method based on multimodal learning, involving the fields of multimodal and deep learning. It utilizes deep learning methods to design a power prediction model to predict power output in the ultra-short term, thereby promoting the practical application of photovoltaic power generation. Based on multimodal learning, this invention simultaneously inputs and processes historical power data and historical ground-based cloud image data, enabling the prediction model to learn more comprehensive information and obtain more accurate predictions. This invention also improves the ability to cope with different weather conditions.

Description

Technical Field

[0001] This invention relates to the fields of multimodal and deep learning, and specifically to an ultra-short-term photovoltaic power prediction method based on multimodal learning. Background Technology

[0002] Rising carbon dioxide concentrations lead to various harms, including global warming and reduced crop yields. Against this backdrop, the goal of "carbon neutrality"—aiming for energy conservation and emission reduction—has been proposed. To achieve this goal, clean energy sources with renewable and low-pollution characteristics are crucial, as they can replace fossil fuels and reduce carbon emissions at their source. Examples include wind, solar, and nuclear power. As one of the most promising clean energy sources, solar energy utilization has undergone substantial improvements in recent decades, and the photovoltaic industry has gradually matured after half a century of development.

[0003] However, effectively utilizing photovoltaic (PV) power generation in the power grid is not easy. Due to the diversity of weather conditions, such as rainy days, cloudy days, sunny days, and snowy days, the power output of PV power generation fluctuates, making it difficult to provide stable power. To address this problem, PV power forecasting has become an effective solution. Accurate PV power forecasting can optimize the utilization of solar energy resources. By understanding the expected power output, system administrators can better plan grid operation, energy dispatch, and energy storage strategies to maximize the efficiency of solar power generation systems. Furthermore, PV power forecasting helps ensure the stability and reliability of the power system. The power system needs to adjust the balance between power generation and supply based on predicted power changes to avoid overload or insufficient supply. PV power forecasting also allows for the early identification and prevention of potential faults and problems, facilitating timely maintenance, reducing downtime, and improving the reliability and continuity of PV power generation systems. Accurate PV power forecasting also helps in developing reasonable energy procurement plans and sales strategies, optimizing energy market participation, electricity price management, and electricity trading, thereby improving the economic efficiency of the power generation system. Therefore, PV power forecasting is of great significance in achieving efficient operation of PV power generation systems and the stability and economic efficiency of the power system.

[0004] From the perspective of model input, ultra-short-term photovoltaic power prediction methods can be divided into two groups: data-based methods and image-based methods.

[0005] Data-driven approaches rely on historical data, such as solar irradiance, temperature, numerical weather prediction (NWP) data, and previous photovoltaic (PV) output, to train predictive models. These methods typically utilize machine learning algorithms such as regression, stochastic prediction (RF), support vector machines (SVM), extreme learning machines (ELM), or artificial neural networks (ANN) to capture the relationship between input features and PV output, enabling predictions of future data. The development of deep learning, such as traditional neural networks (CNN) and long short-term memory (LSTM) networks, has also injected new vitality into PV power prediction. Due to their powerful ability to extract high-dimensional features, deep learning methods generally outperform traditional machine learning methods in most cases. However, for these data-driven approaches, it is nearly impossible to capture the dynamic changes in cloud cover during the prediction period, which is the most critical factor affecting short-term power variations.

[0006] Recent advancements in remote sensing and imaging technologies have provided the field with more data sources, such as ground-based and satellite cloud images, giving rise to image-based methods. Specifically, these images can provide a detailed visual representation of cloud movement, which is highly helpful for prediction. Compared to satellite cloud images, ground-based cloud images offer more detailed information on local cloud cover, capturing details such as cloud thickness and color, thus providing a more comprehensive understanding of cloud characteristics. Furthermore, the sampling time interval of ground-based cloud images can reach the second level, capturing rapid changes in cloud cover over short periods. These capabilities have made ground-based cloud images increasingly important in ultra-short-term photovoltaic power prediction. Moreover, the rapid development of computer vision has made the analysis of ground-based cloud images highly efficient, making the prediction of cloud changes easier and more accurate. In recent years, numerous studies have also demonstrated the feasibility and effectiveness of image-based methods. However, relying solely on images also has limitations; for example, images cannot provide current photovoltaic power data, solar irradiance, ambient temperature, and other information that can affect photovoltaic power output. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides an ultra-short-term photovoltaic power prediction method based on multimodal learning. It utilizes deep learning to design a power prediction model to predict power output in the ultra-short term, thereby aiding in the practical application of photovoltaic power generation. Based on multimodal learning, this invention simultaneously inputs and processes historical power data and historical ground-based cloud map data, enabling the prediction model to learn more comprehensive information and obtain more accurate predictions.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: A method for predicting ultra-short-term photovoltaic power based on multimodal learning includes the following steps: Step 1: Collect time-aligned cloud images and power data pairs of the foundation to construct a photovoltaic power prediction dataset for training a multimodal photovoltaic power prediction model; Step 2: Preprocess the power data and cloud map; the preprocessing includes data normalization and image cropping. Step 3: Read the power data sequence and cloud map sequence within a certain time period T in batch order, input them into the designed multi-mode photovoltaic power prediction model, and output the power data sequence within the future time period T'; Step 4: Compare the power data sequence for the future time period T' output by the multimodal photovoltaic power prediction model with the actual power data sequence for the future time period T', calculate the loss function value, and train the network using error backpropagation; Step 5: Determine whether the validation set metrics meet the requirements. Specifically, determine whether the error between the predicted value and the actual value for the future time period T' on the validation set meets the design requirements. If not, adjust the parameters and retrain. If the design requirements are met, the trained multimodal photovoltaic power prediction model is obtained.

[0009] Beneficial effects: 1) This invention proposes a novel end-to-end modular multimodal deep learning model for ultra-short-term photovoltaic power prediction. This model takes both historical cloud map sequences and power data sequences as inputs and predicts future power data sequences over a certain time range end-to-end.

[0010] 2) This invention designs a multimodal fusion module based on a convolutional neural network (CNN). By fusing spatiotemporal attention features of cloud map and power data, the model can effectively learn the correlation between cloud map and power data, thereby improving the model's predictive ability.

[0011] 3) This invention proposes a dynamic adaptive decoder. The diversity of weather changes brings great challenges to photovoltaic power prediction. Cloud maps can reflect weather information to a certain extent. This decoder can dynamically adjust the decoder parameters according to the input cloud map to improve its ability to cope with different weather conditions. Attached Figure Description

[0012] Figure 1 This is a flowchart of an ultra-short-term photovoltaic power prediction method based on multimodal learning according to the present invention; Figure 2 This is a diagram illustrating the preprocessing of the ground cloud image; where A is the original cloud image collected, B is the cloud image after cropping, C is the pre-designed mask image after removing background factors, and D is the cloud image after mask processing. Figure 3 This is a structural diagram of the ultra-short-term photovoltaic power prediction model based on multimodal learning of the present invention; Figure 4 Here is a diagram of the weight generator's structure; Figure 5 This is a schematic diagram of the structural unit; Figure 6a , Figure 6b This is a schematic diagram illustrating the prediction performance of the model proposed in this invention over a 15-minute time period on the test set; wherein, Figure 6a The weather was sunny. Figure 6b The weather is mostly cloudy. Figure 7a , Figure 7b This is a schematic diagram illustrating the photovoltaic power prediction effect over 15 minutes; where, Figure 7a This is a comparison between the predicted and actual values ​​for a specific sunny day. Figure 7b The result is a comparison between the predicted value and the actual value for a certain cloudy day. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0014] like Figure 1 As shown, the ultra-short-term photovoltaic power prediction method based on multimodal learning of the present invention includes the following steps: Step 1: Use the equipment to collect time-aligned ground cloud maps and power data pairs to construct a photovoltaic power prediction dataset for training a multimodal photovoltaic power prediction model; Step 2: Preprocess the power data and cloud map; the preprocessing includes data normalization, image cropping, etc. Step 3: Read a batch sequentially The power data sequence and cloud map sequence within a time period are input into the trained multimodal photovoltaic power prediction model, and the next output is... Power data sequence within a time period; Step 4: Output the future from the model Power data sequences within a time period and the actual future The power data sequences within a time period are compared, the loss function value is calculated, and the network is trained; in particular, the network is trained using error backpropagation. Step 5: Determine if the validation set metrics meet the requirements, that is, determine the future... If the error between the predicted and actual values ​​over a given time period meets the design requirements, then the parameters are adjusted and the model is retrained. If the error meets the requirements, a well-trained multimodal photovoltaic power prediction model is obtained.

[0015] Specifically, in Step 2, for the cloud map, as follows: Figure 2 As shown, the edges contain irrelevant background elements. Therefore, a four-sided cropping is performed first, followed by the removal of irrelevant background content using a mask. Finally, the image is downsampled to 64×64 pixels, balancing training efficiency and model performance. For power data, normalization is used to reduce the impact of data scale. ; in, , , and These represent the original power data, the minimum value of the original power data, the maximum value of the original power data, and the normalized power data, respectively.

[0016] like Figure 3 The diagram shown is a structural diagram of the ultra-short-term photovoltaic power prediction model based on multimodal learning of the present invention; wherein, Given the lengths of the input image sequence and power data sequence, To input the number of channels in the cloud map, and The height and width of the cloud map, The length of the feature dimension. and The height and width of the feature map, The length of the output power data sequence.

[0017] like Figure 3 As shown, the multimodal photovoltaic power prediction model described in Step 3 includes a power data encoder, a cloud map encoder, a multimodal fusion module, a spatiotemporal inference module, and a dynamically adaptive decoder.

[0018] The power data encoder is used to encode power data to obtain power data features. It employs sine and cosine encoding, as proposed in the Transformer model, which can encode continuous data and reflect relative magnitude relationships. ; ; in, Represents the encoded value. This represents the feature dimension label after encoding. Using this encoding method can reduce the number of training parameters while still achieving good results.

[0019] The cloud map is encoded using a cloud map encoder. A module using 2D convolution, batch normalization, and ReLU activation (activation layer) extracts image features. The number of channels in the features is consistent with the dimension of the encoded power data features, which is 2D. Then, at the channel level, each cloud map feature is stitched together in chronological order to obtain the features of the cloud map sequence.

[0020] The multimodal fusion module functions as follows: After obtaining the features of the power data sequence and the cloud image sequence, to align these two modal features, global adaptive pooling is applied to the features of the concatenated cloud image sequence to obtain a one-dimensional global sequence feature. This global sequence feature is then concatenated with the power data features at specific time points to obtain a preliminary fused one-dimensional feature. Next, two fully connected layers with interleaved ReLU activation functions (activation layers) are used to learn the correlation between modalities, and a set of weights is obtained through a Sigmoid activation function (activation layer). These weights are then multiplied by the channels of the cloud image sequence features to obtain the fused feature after modal interaction. The weights are calculated based on the input image; therefore, different inputs result in different weights.

[0021] The spatiotemporal inference module consists of several structural units containing grouped convolutions, used to learn spatiotemporal dependencies. Preferably, as follows: Figure 5 As shown, the structural unit includes two-dimensional convolution and multiple two-dimensional grouped convolutions.

[0022] The function of the dynamically adaptive decoder is to design a dynamically adaptive decoder to obtain the final result for the features after the spatiotemporal inference module. First, global adaptive pooling is used to obtain one-dimensional features, and then... A parallel sub-decoder obtains Each sub-decoder contains two sets of fully connected layers plus ReLU activation function (activation layer) operations. To better integrate these results, a weight generator is designed to adaptively generate weights based on the features of the contour sequence. The weights corresponding to each result ( The structure of the weight generator is as follows: Figure 4 As shown. Then the above The prediction result is obtained by weighting the results together. This allows the decoder to adjust its parameters (i.e., weights) according to different weather conditions for better robustness. Finally, the results are concatenated with the input power data and passed through a fully connected layer to obtain the final prediction result. Figure 4 As shown, the weight generator sequentially includes a convolutional layer + batch normalization + activation layer, a global adaptive pooling layer, a fully connected layer, an activation layer, a fully connected layer, and another activation layer.

[0023] Mean squared error (MSE) was used as the loss function during network training. Table 1 (Performance results of the proposed ultra-short-term photovoltaic power prediction model over a 15-minute time period) shows that the proposed method offers a significant improvement over traditional methods. Figure 6a , Figure 6b This indicates that the method of the present invention can predict power results well in the short term, especially under clear weather conditions. Among other things, Figure 6a The weather was sunny. Figure 6b The weather is mostly cloudy.

[0024] The ablation experiment results in Table 2 (Ablation Experiment Performance Table) show that the multimodal fusion method and the dynamically adaptive decoder proposed in this invention both improve the prediction results.

[0025] Table 1

[0026] Here, FS stands for Forecast Skill, which represents the performance result compared to the traditional method, Smart Persistent Model.

[0027] Table 2

[0028] Figure 7a , Figure 7b This is a schematic diagram illustrating the photovoltaic power prediction effect over 15 minutes; where, Figure 7a This is a comparison between the predicted and actual values ​​for a specific sunny day. Figure 7b The result is a comparison between the predicted value and the actual value for a certain cloudy day.

Claims

1. A method for predicting ultra-short-term photovoltaic power based on multimodal learning, characterized in that, Includes the following steps: Step 1: Collect time-aligned cloud images and power data pairs of the foundation to construct a photovoltaic power prediction dataset for training a multimodal photovoltaic power prediction model; Step 2: Preprocess the power data and cloud map; the preprocessing includes data normalization and image cropping; Step 3: Read the power data sequence and cloud map sequence within a certain time period T in batch order, input them into the designed multi-mode photovoltaic power prediction model, and output the power data sequence within the future time period T'; The multimodal photovoltaic power prediction model described in Step 3 includes a power data encoder, a multimodal fusion module, a spatiotemporal inference module, a cloud map encoder, and a dynamically adaptive decoder. The power data encoder is used to encode power data to obtain power data features. It uses sine and cosine encoding proposed in Transformer for encoding. Sine and cosine encoding encodes continuous data and reflects relative magnitude relationships. ； ； in, Represents the encoded value. Represents the encoded feature dimension label. , Represents the length of the feature dimension. This represents the power data after normalization; The cloud image encoder is used to encode cloud images, using... The module, which includes two-dimensional convolution, batch normalization, and ReLU activation function, extracts cloud map features. Then, at the channel level, each cloud map feature is concatenated together in chronological order to obtain the features of the cloud map sequence. The spatiotemporal reasoning module consists of several structural units containing grouped convolutions, used to learn spatiotemporal dependencies; For the features after the spatiotemporal inference module, a dynamically adaptive decoder is designed to obtain the final prediction result, including: first, using global adaptive pooling to obtain one-dimensional features, and then using... A parallel sub-decoder obtains Each sub-decoder contains two sets of fully connected layers and ReLU activation function operations. A weight generator is designed to adaptively generate weights based on the features of the contour sequence. The weights corresponding to each result; then the above The weighted average of the results yields the prediction result, enabling the dynamically adaptive decoder to adjust its parameters according to different weather conditions. Finally, the prediction result is concatenated with the input power data and passed through a fully connected layer to obtain the final prediction result. Step 4: Compare the power data sequence for the future time period T' output by the multimodal photovoltaic power prediction model with the actual power data sequence for the future time period T', calculate the loss function value, and train the network using error backpropagation; Step 5: Determine whether the validation set metrics meet the requirements. Specifically, determine whether the error between the predicted value and the actual value for the future time period T' on the validation set meets the design requirements. If not, adjust the parameters and retrain. If the design requirements are met, the trained multimodal photovoltaic power prediction model is obtained.

2. The ultra-short-term photovoltaic power prediction method based on multimodal learning according to claim 1, characterized in that, In Step 2, for the cloud image, first, four sides are cropped, then an irrelevant background content is removed using a mask, and finally, it is downsampled to a size of 64×64 pixels; for the power data, a normalization operation is performed. ； in, , , and These represent the original power data, the minimum value of the original power data, the maximum value of the original power data, and the normalized power data, respectively.

3. The ultra-short-term photovoltaic power prediction method based on multimodal learning according to claim 1, characterized in that, The function of the multimodal fusion module is as follows: after obtaining the features of the power data sequence and the cloud map sequence, global adaptive pooling is applied to the features of the concatenated cloud map sequence to obtain a one-dimensional global sequence feature; then, the global sequence feature is combined and concatenated with the power data sequence according to time step to obtain a preliminary fused one-dimensional feature; then, the correlation between modes is learned by using two fully connected layers with interleaved ReLU activation functions, and a set of weights is obtained by using the Sigmoid activation function; then, the weights are multiplied with the channels of the cloud map sequence features to obtain the fused feature after modal interaction.

Images