Multi-mode photovoltaic power prediction method based on cloud picture segmentation and cloud genus cloud cover guidance

Abstract

The invention discloses a multi-mode photovoltaic power prediction method based on cloud picture segmentation and cloud genus cloud cover guidance. The foundation cloud picture passes through a foundation cloud picture fine-grained segmentation model to obtain a foundation cloud picture segmentation mask; according to the ground-based cloud picture segmentation mask, the cloud amount of each cloud genus is calculated, and the cloud amounts of all the cloud genus form a cloud genus cloud amount distribution feature tag; the photovoltaic power prediction model comprises a weight generator module, a cloud picture coding module, a numerical data coding module, a cross-modal fusion module and a hybrid predictor module; the weight generator module generates weights of sub-predictors according to cloud genus cloud quantity distribution feature tags of historical time steps, each sub-predictor of the hybrid predictor module performs photovoltaic power prediction according to cross-modal fusion features, and weighted summation is performed on predicted power of each sub-predictor to obtain final predicted photovoltaic power. And the prediction power of the sub-predictors is fused by considering a cloud layer semantic environment and taking the cloud genus cloud amount as guidance, so that the prediction performance under different cloud layer conditions is improved.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power prediction technology, specifically to a multimodal photovoltaic power prediction method based on cloud map segmentation and cloud attribute / volume guidance. Background Technology

[0002] Photovoltaic (PV) power generation exhibits significant intermittency, randomness, and volatility, leading to substantial impacts on power quality and dispatch after large-scale grid integration. Therefore, accurate prediction of PV power generation is crucial for addressing grid integration issues. PV power generation primarily depends on solar irradiance, while cloud cover significantly and intermittently affects solar radiation, causing fluctuations in PV power plant output. Furthermore, different types of clouds attenuate solar irradiance to varying degrees. Therefore, accurate cloud observation and cloud cover information are of paramount importance for the precise prediction of PV power generation.

[0003] Currently, photovoltaic (PV) power prediction methods are mainly divided into data-driven and image-driven approaches. Data-driven methods establish a mapping relationship between historical time-series data and PV power based on historical observation data such as PV output data and meteorological data. However, this method ignores the spatiotemporal information of clouds, resulting in limited prediction accuracy under cloudy weather conditions. Image-driven methods combine cloud images for prediction, but most studies directly use cloud images as model input or manually extract image features and then use them as model input, resulting in a predominance of single-modal models and insufficient feature mining of multi-source data. In addition, existing studies ignore the impact of different cloud genera on PV power generation, lack a more detailed analysis of the physical meaning of cloud images, and lack characterization of the impact of cloud layer information and evolution trends on prediction results. Summary of the Invention

[0004] To address the shortcomings of existing research, the technical problem this invention aims to solve is a multimodal photovoltaic power prediction method based on cloud map segmentation and cloud attribute / volume guidance.

[0005] The present invention solves the aforementioned technical problem by adopting the following technical solution: A multimodal photovoltaic power prediction method based on cloud map segmentation and cloud attribute / volume guidance, characterized by the following steps: Step 1: Construct a ground-based cloud map segmentation dataset and a multimodal photovoltaic power prediction dataset; Step 2: Construct a fine-grained segmentation model for the ground cloud map; the ground cloud map is processed through the fine-grained segmentation model to obtain a ground cloud map segmentation mask; based on the ground cloud map segmentation mask, the number of pixels for each cloud species is counted, and the ratio of the number of pixels for each cloud species to the total number of pixels in the ground cloud map segmentation mask is the cloud species quantity. All cloud species quantities form a cloud species quantity distribution feature label. Step 3: Construct a photovoltaic power prediction model, including a weight generator module, a cloud map encoding module, a numerical data encoding module, a cross-modal fusion module, and a hybrid predictor module; In the weight generator module, the cloud type and cloud amount distribution feature labels of the historical time steps are concatenated to obtain the cloud type and cloud amount distribution feature label matrix of the historical time steps. The cloud type and cloud amount distribution feature label matrix of the historical time steps is then passed through a fully connected layer, a ReLU activation function, another fully connected layer, and another ReLU activation function to generate a time decay factor. The weights of each historical time step are calculated based on the time decay factor.

[0006] In the formula, Representing historical moments The weight, Indicates the time decay factor. Indicates the time step. Indicates time; The weights of each historical moment are summed with the cloud type and cloud amount distribution feature labels to obtain the time-weighted cloud type and cloud amount distribution feature labels. The time-weighted cloud type and cloud amount distribution feature labels are then passed through a fully connected layer and a Softmax activation function to generate the weights of the sub-predictors. The ground-based cloud images from historical time steps are processed by a cloud image encoding module to extract visual features, while the photovoltaic power and meteorological data from historical time steps are processed by a numerical data encoding module to extract numerical time-series features. The cloud image visual features and numerical time-series features are then processed by a cross-modal fusion module to obtain cross-modal fused features. In the hybrid predictor module, the cross-modal fused features are processed by global average pooling and then by multiple parallel sub-predictors to predict photovoltaic power. Based on the weights of each sub-predictor, the predicted photovoltaic power of each sub-predictor is weighted and summed to obtain the final predicted photovoltaic power. Step 4: Train a fine-grained segmentation model for ground cloud maps using the ground cloud map segmentation dataset, train a photovoltaic power prediction model using the multimodal photovoltaic power prediction dataset, and use the trained fine-grained segmentation model for ground cloud maps and photovoltaic power prediction model for photovoltaic power prediction.

[0007] Furthermore, the ground-based cloud map fine-grained segmentation model includes an encoder, a decoder, and a segmentation head. The encoder includes multiple encoding layers and a parallel convolutional attention module. The decoder includes multiple decoding layers and a wavelet enhancement feature fusion module. The output feature map of the previous encoding layer is used as the input feature map of the next encoding layer after passing through the parallel convolutional attention module. Each parallel convolutional attention module is skipped to the corresponding decoding layer. The wavelet enhancement feature fusion module is embedded in the skip connection. The output feature map of the wavelet enhancement feature fusion module and the output feature map of the previous decoding layer are used as the input feature map of the next decoding layer. For the wavelet enhancement feature fusion module on the last skip connection, the output feature map of the last parallel convolutional attention module is max-pooled and then passed through the wavelet enhancement feature fusion module with the output feature map of the last encoding layer to serve as the input feature map of the first decoding layer. For the wavelet enhancement feature fusion modules on the remaining skip connections, the output feature map of the parallel convolutional attention module and the output feature map of the previous decoding layer are used as the input feature map. Figure 1 It serves as the input feature map for the wavelet-enhanced feature fusion module; In the parallel convolutional attention module, the input feature map passes through the parallel convolutional module and the dual convolutional attention module respectively. The output feature map of the parallel convolutional module is multiplied element-wise with the output feature map of the dual convolutional attention module to obtain the parallel dual convolutional attention fusion feature map. The two parallel dual convolutional attention fusion feature maps are added element-wise and then passed through a 1×1 depthwise separable convolution to obtain the output feature map of the parallel convolutional attention module. In the dual convolutional attention module, the input feature map undergoes 7×7 average pooling and 1×1 depthwise separable convolution, followed by 1×7 and 7×1 depthwise separable convolutions to obtain horizontal and vertical feature maps. The horizontal and vertical feature maps are then processed by batch normalization and the Sigmoid activation function to generate spatial horizontal and spatial vertical attention weight matrices, respectively. These matrices are then multiplied element-wise with the output feature map of the parallel convolution module, followed by element-wise addition. The resulting feature map is then processed by a 1×1 depthwise separable convolution to obtain the output feature map of the dual convolutional attention module.

[0008] Furthermore, in the parallel convolution module, the input feature map is processed through four parallel convolution branches. The first convolution branch includes 3×3 depthwise separable convolutions, the second convolution branch includes 3×3 and 5×5 depthwise separable convolutions, the third convolution branch includes 3×3, 5×5 and 7×7 depthwise separable convolutions, and the fourth convolution branch includes 3×3, 5×5, 7×7 and 9×9 depthwise separable convolutions. The output feature maps of the four convolution branches are summed element-wise and then processed through a 1×1 depthwise separable convolution to obtain the output feature map of the parallel convolution module.

[0009] Furthermore, in the wavelet-enhanced feature fusion module, the output feature map of the parallel convolutional attention module is convolved with wavelet transform to obtain low-frequency and high-frequency feature maps; the low-frequency and high-frequency feature maps are concatenated and then convolved with inverse wavelet transform to obtain the restored feature map; the low-frequency feature map is fused with the output feature map of the previous decoding layer through the local multi-directional feature fusion module, and the output feature map of the local multi-directional feature fusion module is purified by multiple superimposed wavelet transform-inverse wavelet transform convolutions to obtain the purified low-frequency feature map; the purified low-frequency feature map is passed through the upsampling module to obtain the upsampled feature map; the upsampled feature map is concatenated with the restored feature map to obtain the output feature map of the wavelet-enhanced feature fusion module.

[0010] Furthermore, in the local multi-directional feature fusion module, the low-frequency feature map is added element-wise to the output feature map of the previous decoding layer to obtain the encoder-decoder fusion feature map. The encoder-decoder fusion feature map is then passed through parallel depthwise separable convolution branches and multi-directional decay branches. The output feature maps of the two branches are concatenated and then passed through a sigmoid activation function to generate a spatial attention weight matrix. The spatial attention weight matrix is ​​used as the weight of the low-frequency feature map, and the difference matrix between the identity matrix and the spatial attention weight matrix is ​​used as the weight of the output feature map of the previous decoding layer. The low-frequency feature map and the output feature map of the previous decoding layer are then weighted and fused to obtain the output feature map of the local multi-directional feature fusion module.

[0011] Furthermore, the depthwise separable convolution branch includes two 1×1 depthwise separable convolutions, each followed by batch normalization and a ReLU activation function.

[0012] Furthermore, the input feature map of the multi-directional decay branch is sequentially processed by 7×7 average pooling and 1×1, 1×7 and 7×1 depthwise separable convolutions, and then batch normalization and ReLU activation function to obtain the output feature map of the multi-directional decay branch.

[0013] Furthermore, in the cloud image encoding module, the ground cloud image is processed by a feature extraction network to extract spatial features. The spatial features are flattened according to the spatial dimensions, and after adding position encoding, they are processed by a Transformer encoder for temporal encoding. The output feature map of the Transformer encoder is then processed by global average pooling to obtain the visual features of the cloud image.

[0014] Furthermore, in the numerical data encoding module, the photovoltaic power and meteorological data of historical time steps are concatenated to obtain the power-meteorological data fusion feature; the power-meteorological data fusion feature is then enhanced by a channel attention mechanism after passing through a fully connected layer to obtain the channel-enhanced power-meteorological data high-dimensional feature; the channel-enhanced power-meteorological data high-dimensional feature is then transposed and passed through a temporal convolutional network to obtain the numerical time-series feature.

[0015] Furthermore, in the cross-modal fusion module, the cloud map visual features and numerical time-series features are unified in dimension through a linear mapping layer, and then position encoding is added to obtain position-encoded cloud map visual features and numerical time-series features; the position-encoded cloud map visual features and numerical time-series features are interactively enhanced through a Cross Transformer network; the interactively enhanced cloud map visual features and numerical time-series features are concatenated to obtain cross-modal fusion features.

[0016] Compared with the prior art, the present invention has the following beneficial results: 1. A fine-grained segmentation model for ground-based cloud images performs fine-grained segmentation to obtain a ground-based cloud image segmentation mask. Based on the ground-based cloud image segmentation mask, cloud images of each cloud genus are calculated, and cloud quantity distribution feature labels are constructed accordingly, thereby transforming the visual image into semantic soft labels containing physical information about cloud quantity. The photovoltaic power prediction model uses cloud quantity of each cloud genus as a guide to generate the weights of each sub-predictor. That is, the weights of each sub-predictor are calculated based on the cloud quantity distribution feature labels, and the predicted power of each sub-predictor is weighted and summed to obtain the final predicted photovoltaic power. This method fully considers the impact of fine-grained cloud attributes on photovoltaic power prediction. The photovoltaic power prediction model can dynamically adjust the contribution of each sub-predictor based on the current cloud semantic environment, adaptively fuse the predicted power of multiple sub-predictors, achieve accurate power prediction, and improve prediction performance under different cloud coverage conditions.

[0017] 2. A parallel convolutional attention module is introduced into the encoder of the ground-based cloud image fine-grained segmentation model. By extracting local contextual semantic information at multiple scales and directions and enhancing global perception using an attention mechanism, accurate cloud classification can be achieved. Addressing the characteristics of ground-based cloud images, such as gradual changes, blurred boundaries, and abundant low-frequency information, a wavelet-enhanced feature fusion module is introduced into the skip connections between the encoder and decoder of the ground-based cloud image fine-grained segmentation model to effectively extract cloud information at different scales and enhance the segmentation capability for blurred cloud boundaries.

[0018] 3. For multimodal data, the photovoltaic power prediction model extracts visual cloud features and numerical time-series features through cloud image encoding and numerical data encoding modules, respectively. The cross-modal fusion module utilizes a cross-attention mechanism to achieve deep interaction between visual cloud features and numerical time-series features, realizing full fusion of multimodal data. The weight generator module calculates the weights at each time step based on a learnable time decay mechanism, and then performs a weighted summation of the cloud type and cloud cover distribution feature labels at each time step. This follows the physical law that "the recent state has a significant impact, and the long-term impact decays" in the cloud evolution process, enhancing the model's adaptability to variable weather conditions. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the structure of the fine-grained segmentation model of the ground cloud map of the present invention; Figure 3 This is a schematic diagram of the parallel convolutional attention module of the present invention; Figure 4 This is a schematic diagram of the wavelet enhancement feature fusion module of the present invention; Figure 5 This is a schematic diagram of the convolutional branch and multi-directional decay branch of the local multi-directional feature fusion module of the present invention; Figure 6 This is a schematic diagram of the photovoltaic power prediction model of the present invention. Detailed Implementation

[0020] Specific embodiments are given below with reference to the accompanying drawings. These specific embodiments are only used to describe the technical solution of the present invention in detail, and are not intended to limit the scope of protection of this application.

[0021] This invention provides a multimodal photovoltaic power prediction method based on cloud map segmentation and cloud attribute / cloud amount guidance, such as... Figure 1 As shown, it includes the following steps: Step 1: Construct a ground-based cloud map segmentation dataset and a multimodal photovoltaic power prediction dataset; The Ineichen model was used to calculate the theoretical direct normal irradiance (DNI), total horizontal irradiance (GHI), and diffuse irradiance (DHI) under clear sky conditions. Subsequently, based on the photovoltaic module parameters, the theoretical solar irradiance that the photovoltaic module should receive under clear sky conditions was calculated using the Perez tilted surface irradiance model. For a cloudy ground-based cloud map, several clear sky ground-based cloud maps near the same time were selected, and the difference between the theoretical and measured irradiance of the photovoltaic module in the clear sky ground-based cloud map was recorded. This difference represents the attenuation effect of aerosols on solar irradiance. The recorded differences were then supplemented using the moving average method to obtain the attenuation magnitude of solar irradiance due to aerosols at the time of the cloudy ground-based cloud map. Subtracting the attenuation magnitude of aerosols and the actual solar irradiance from the theoretical solar irradiance of the photovoltaic module yields the degree of attenuation of solar irradiance by clouds. The degree of attenuation of solar irradiance by cloud clusters is used as the basis for cloud cluster classification to obtain the cloud genus of the ground-based cloud map. In this embodiment, five categories are included: cumulonimbus and nimbostratus, cumulus, stratus, cirrus, and clear sky. The ground-based cloud map is manually labeled to construct a ground-based cloud map segmentation dataset.

[0022] Historical photovoltaic power, meteorological data, and ground-based cloud maps are acquired and preprocessed, mainly including outlier screening and normalization, cropping and masking of ground-based cloud maps, and timestamp alignment, in order to construct a multimodal photovoltaic power prediction dataset.

[0023] Step 2: Construct a fine-grained segmentation model for the ground cloud map. The ground cloud map is processed through the fine-grained segmentation model to obtain a ground cloud map segmentation mask. The cloud amount of each cloud species is calculated based on the ground cloud map segmentation mask to obtain the cloud species and cloud amount distribution feature labels. like Figure 2 As shown, based on the problems of blurred cloud boundaries, unclear class transformation, and severe feature aliasing, the ground cloud map fine-grained segmentation model includes an encoder, a decoder, and a segmentation head. The encoder continuously mines the abstract features of the ground cloud map from low-level texture to high-level semantics through convolution. The decoder gradually restores the image resolution through upsampling and feature fusion. The output feature map of the decoder is passed through the segmentation head to obtain the ground cloud map segmentation mask.

[0024] The encoder includes multiple encoding layers and a parallel convolutional attention (PCA) module. The decoder includes multiple decoding layers and a wavelet-enhanced feature fusion (WE-FF) module. The output feature map of the previous encoding layer is passed through a parallel convolutional attention module and used as the input feature map of the next encoding layer. Each parallel convolutional attention module is skipped to its corresponding decoding layer, and a wavelet-enhanced feature fusion module is embedded in the skip connection. The output feature map of the wavelet-enhanced feature fusion module is combined with the output feature map of the previous decoding layer. Figure 1 The output feature map of the last parallel convolutional attention module, after max pooling, is used as the input feature map for the next decoding layer. For the wavelet enhancement feature fusion module on the last skip connection, the output feature map of the last parallel convolutional attention module, after being processed by the wavelet enhancement feature fusion module, is combined with the output feature map of the last coding layer and then used as the input feature map for the first decoding layer. For the wavelet enhancement feature fusion modules on the remaining skip connections, the output feature map of the parallel convolutional attention module and the output feature map of the previous decoding layer are used as the input feature map for the first decoding layer. Figure 1 The input feature map is used as the input feature map for the wavelet-enhanced feature fusion module. The encoding layer achieves dimensionality transformation of the feature map and multi-level semantic information mining through stacked convolution operations; the parallel convolutional attention module focuses on capturing multi-scale and multi-directional contextual semantic information of the cloud layer and effectively expands the receptive field to improve the model's ability to capture complex cloud structures and multi-scale features; considering the characteristics of ground-based cloud maps such as smooth changes, blurred boundaries, and rich low-frequency information, the wavelet-enhanced feature fusion module extracts low-frequency information and achieves feature fusion to enhance the ability to model the global structure of the cloud layer.

[0025] like Figure 3 As shown in (a), the parallel convolutional attention module includes a parallel convolutional module (PCM) and a dual convolutional attention module (DCM). By extracting local contextual semantic information at multiple scales and directions and enhancing global perception using an attention mechanism, it can achieve accurate classification of clouds. The input feature map of the parallel convolutional attention module is processed by the parallel convolutional module and the dual convolutional attention module respectively. The output feature map of the parallel convolutional module is multiplied element-wise with the output feature map of the dual convolutional attention module to obtain the parallel dual convolutional attention fusion feature map. After the two parallel dual convolutional attention fusion feature maps are added element-wise, they are then subjected to a 1×1 depthwise separable convolution (DWConv) to obtain the output feature map of the parallel convolutional attention module.

[0026] like Figure 3As shown in (b), for the dual convolutional attention module, the input feature map first aggregates local contextual semantic information through a 7×7 average pooling operation (AvgPool), and then performs feature compression through a 1×1 depthwise separable convolution. Next, spatial dependencies are extracted from the horizontal and vertical directions using 1×7 and 7×1 depthwise separable convolutions, respectively, to obtain horizontal and vertical feature maps. The horizontal feature map is then processed by batch normalization (BN) and sigmoid activation functions to generate a spatial horizontal attention weight matrix. The vertical feature map is also processed by batch normalization (BN) and sigmoid activation functions to generate a spatial vertical attention weight matrix. The spatial horizontal attention weight matrix and the spatial vertical attention weight matrix are then multiplied element-wise with the output feature map of the parallel convolution module, which significantly improves the model's ability to capture cloud morphology and spatial distribution features. The resulting two feature maps are then fused element-wise. The resulting feature map is then integrated through a 1×1 depthwise separable convolution to obtain the output feature map of the dual convolutional attention module.

[0027] like Figure 3 As shown in (c), for the parallel convolution module, the input feature map extracts features from different scales through four parallel convolution branches. The output feature maps of the four convolution branches are added element-wise, and then further aggregated and fused through 1×1 depthwise separable convolutions, enhancing the model's multi-scale information fusion capability and obtaining the output feature map of the parallel convolution module. The first convolution branch includes 3×3 depthwise separable convolutions, the second convolution branch includes 3×3 and 5×5 depthwise separable convolutions, the third convolution branch includes 3×3, 5×5 and 7×7 depthwise separable convolutions, and the fourth convolution branch includes 3×3, 5×5, 7×7 and 9×9 depthwise separable convolutions.

[0028] like Figure 4 As shown, for the wavelet enhancement feature fusion module, the output feature map of the parallel convolutional attention module is processed by wavelet transform convolution (WTConv) to obtain low-frequency and high-frequency feature maps. The low-frequency feature map represents the low-frequency components of the overall morphology and spatial distribution of the cloud layer, while the high-frequency feature map represents noise in the ground-based cloud map. Subsequent processing mainly focuses on the low-frequency feature map to enhance the cloud structure representation, while the high-frequency feature map only participates in feature map recovery. This involves concatenating the low-frequency and high-frequency feature maps and then performing inverse wavelet transform convolution to recover the output feature map of the parallel convolutional attention module, resulting in the recovered feature map. The low-frequency feature map is then combined with the output feature map of the previous decoding layer. Figure 1The input is fused using the Local Multidirectional Feature Fusion (LM-DAFF) module. The output feature map of the LM-DAFF module is then purified by multiple superimposed wavelet transform-inverse wavelet transform convolutions (WTConv&i-WTConv) to obtain a purified low-frequency feature map, which further suppresses high-frequency components from the decoder feature map. The purified low-frequency feature map is then passed through an upsampling module to obtain an upsampled feature map. The upsampled feature map is then concatenated with the restored feature map to obtain the output feature map of the wavelet enhancement feature fusion module.

[0029] For the LM-DAFF module, the low-frequency feature map is first element-wise added to the output feature map of the previous decoding layer to obtain the encoder-decoder fusion feature map. This fusion feature map is then processed by parallel depthwise separable convolutional branches and multi-directional decay branches. The output feature maps of the two branches are concatenated and then processed by a sigmoid activation function to generate a spatial attention weight matrix. This spatial attention weight matrix is ​​used as the weight of the low-frequency feature map, and the difference between the identity matrix and the spatial attention weight matrix is ​​used as the weight of the output feature map of the previous decoding layer. This weighted fusion of the low-frequency feature map and the output feature map of the previous decoding layer integrates local details and global features, enhancing regional and overall cloud features and effectively mitigating boundary blurring, resulting in the output feature map of the LM-DAFF module. Figure 5 As shown in (a), the depthwise separable convolution branch includes two 1×1 depthwise separable convolutions. Each 1×1 depthwise separable convolution is followed by batch normalization and a ReLU activation function to compress the feature dimension and extract local context information. The multi-directional decay branch first uses 7×7 average pooling to compress the encoder-decoder fusion feature map, and then sequentially passes it through 1×1, 1×7, and 7×1 depthwise separable convolutions to extract local attention weights from multiple directions, enhancing the feature representation in different directions. Finally, batch normalization and a ReLU activation function are used to obtain the output features of the multi-directional decay branch.

[0030] like Figure 5 (b) For the upsampling module, the purified low-frequency feature map is first upsampled by the nearest neighbor interpolation algorithm, then sequentially passed through 3×3 atrous convolution (AtrousConv), batch normalization and ReLU activation function, then channel rearrangement, and finally through 1×1 convolution to aggregate information to achieve effective upsampling, thereby allowing the upsampled features to capture a wider range of contextual information.

[0031] Based on the ground-based cloud image segmentation mask, the number of pixels for each cloud genus is counted. The ratio of the number of pixels for each cloud genus to the total number of pixels in the ground-based cloud image segmentation mask is the cloud amount for that cloud genus. All cloud amounts for each cloud genus form a cloud genus cloud amount distribution feature label, at any given time. The cloud distribution characteristics are tagged as follows: , Indicates cloud species Cloud cover, This indicates the total number of cloud species.

[0032] Step 3: Construct a photovoltaic power prediction model, such as Figure 6 As shown, the photovoltaic power prediction model includes a weight generator module, a cloud map encoding module, a numerical data encoding module, a cross-modal fusion module, and a hybrid predictor module. The weight generator module generates predictor weight vectors based on cloud type and cloud amount distribution feature labels from historical time steps. The cloud map encoding module extracts cloud map visual features based on ground-based cloud maps from historical time steps. The numerical data encoding module extracts numerical time-series features based on photovoltaic power and meteorological data from historical time steps. The cross-modal fusion module fuses the cloud map visual features and numerical time-series features to obtain cross-modal fusion features. Each sub-predictor in the hybrid predictor module predicts photovoltaic power based on the cross-modal fusion features. The predicted power of each sub-predictor is weighted and summed based on the predictor weight vector to obtain the final predicted photovoltaic power.

[0033] To effectively transform the prior physical knowledge inherent in ground-based cloud images into dynamic control signals guiding power prediction and to address the time-varying non-stationarity caused by cloud motion, a weight generator module with a time decay mechanism was designed. The weight generator module takes the cloud type and cloud amount distribution feature labels from historical time steps as input. First, it concatenates these feature labels to obtain a matrix of cloud type and cloud amount distribution feature labels from historical time steps. , Indicates the time step. This represents the transpose operation. Considering the physical law that "the recent state has a significant impact, while the long-term impact diminishes" during cloud evolution, a learnable time decay mechanism was designed. The cloud genus and cloud amount distribution feature label matrix of historical time steps is sequentially passed through a fully connected layer, a ReLU activation function, another fully connected layer, and another ReLU activation function to adaptively fit the time decay factor. The time decay factor is dynamically adjusted according to the intensity of cloud movement; the weight of each historical moment is calculated based on the time decay factor, realizing dynamic recalibration of the importance of historical information. The calculation formula is as follows:

[0034] In the formula, Representing historical moments The weights; The weights of each historical moment are summed with the cloud type and cloud amount distribution feature labels of that historical moment to obtain the time-weighted cloud type and cloud amount distribution feature labels. The time-weighted cloud cover distribution feature labels are sequentially passed through a fully connected layer and a Softmax activation function to generate the predictor weight vector. , Indicates the first The weights of each sub-predictor are fused with the decisions of the sub-predictors, enabling the model to dynamically adjust the contribution of each sub-predictor based on real-time weather conditions. This ensures that the prediction results can respond sensitively to subtle changes in cloud cover, improving the model's adaptability under various variable weather conditions, especially cloudy weather, thereby enhancing the model's prediction accuracy.

[0035] For the cloud image encoding module, the ground cloud image at historical time steps is processed by a spatial feature extraction network to extract spatial features. These spatial features are flattened according to spatial dimensions, and after adding positional encoding, they are fed into a Transformer encoder for temporal encoding. The output feature map of the Transformer encoder is then subjected to global average pooling to obtain the visual features of the cloud image. ;in, This indicates the batch size. The cloud map encoding module combines the feature extraction capabilities of the feature extraction network with the ability of the Transformer encoder to model long-sequence temporal dependencies, enabling it to simultaneously preserve both local and global morphological features and global evolution trends of the cloud map. In this embodiment, the feature extraction network uses a pre-trained ResNet50.

[0036] For the numerical data encoding module, photovoltaic power and meteorological data at historical time steps are concatenated to obtain power-meteorological data fusion features. These fusion features are mapped to a high-dimensional space through a fully connected layer, and then a channel attention mechanism is used to enhance the dependence on important feature channels while suppressing unimportant feature channels, resulting in channel-enhanced high-dimensional power-meteorological data features. After transposing, these channel-enhanced high-dimensional power-meteorological data features are passed through a temporal convolutional network (TCN) to obtain a larger receptive field to capture long-term dependencies, thus obtaining numerical time-series features. .

[0037] To achieve dynamic interaction and alignment between cloud image visual features and numerical time-series features, in the cross-modal fusion module, cloud image visual features and numerical time-series features are unified in dimension through a linear mapping layer, and then positional encoding is added to obtain position-encoded cloud image visual features and numerical time-series features. The position-encoded cloud image visual features and numerical time-series features are then enhanced through a Cross Transformer network to achieve interaction between different modal features. The Cross Transformer network is a variant of the Transformer, differing from the standard Transformer in that it replaces the self-attention mechanism in the Transformer encoder with a cross-attention mechanism. The interactively enhanced cloud image visual features and numerical time-series features are then concatenated to fully aggregate multimodal complementary information containing physical visual information and numerical meteorological observation information, resulting in cross-modal fused features. The cross-modal fusion module allows the model to dynamically focus on regions with key shading effects in the cloud image based on the fluctuation trend of historical photovoltaic power, thereby achieving adaptive semantic alignment between visual and numerical information. It has strong modeling capabilities for long-distance, many-to-many dependencies and is more suitable for processing heterogeneous data.

[0038] Because photovoltaic power is affected by various meteorological factors, a single regression network often struggles to simultaneously fit a stable sunny weather curve and a drastically fluctuating cloudy weather curve. Therefore, in the hybrid predictor module, the cross-modal fusion features are first converted into a one-dimensional vector through global average pooling. Then, multiple parallel sub-predictors (MLPs) are used to obtain the predicted power of each sub-predictor. The number of sub-predictors is the same as the total number of cloud genera. During training, different sub-predictors automatically learn the data distribution characteristics of different cloud genera, making each sub-predictor sensitive to different cloud genera, thereby improving the overall expressive power of the model. Based on the predictor weight vector, the predicted power of each sub-predictor is weighted and summed to obtain the final predicted photovoltaic power. Therefore, the model can dynamically adjust the contribution of each sub-predictor based on the current cloud cover information and evolution trend, thus achieving optimal prediction performance in different scenarios.

[0039] Step 4: Train a fine-grained segmentation model for ground cloud maps using the ground cloud map segmentation dataset, train a photovoltaic power prediction model using the multimodal photovoltaic power prediction dataset, and use the trained fine-grained segmentation model for ground cloud maps and photovoltaic power prediction model for photovoltaic power prediction.

[0040] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A multi-modal photovoltaic power prediction method based on cloud map segmentation and cloud genus cloud cover guidance, characterized in that, Comprising the following steps: Step one: constructing a ground-based cloud image segmentation dataset and a multi-modal photovoltaic power prediction dataset; Step two: constructing a ground-based cloud image fine-grained segmentation model; the ground-based cloud image is subjected to the ground-based cloud image fine-grained segmentation model to obtain a ground-based cloud image segmentation mask; according to the ground-based cloud image segmentation mask, the pixel number of each cloud genus is counted, and the ratio of the pixel number of the cloud genus to the total pixel number of the ground-based cloud image segmentation mask is the cloud amount of the cloud genus, and all cloud genus cloud amounts constitute a cloud genus cloud amount distribution feature label; Step three: constructing a photovoltaic power prediction model, including a weight generator module, a cloud image encoding module, a numerical data encoding module, a cross-modal fusion module and a hybrid predictor module; In the weight generator module, the cloud genus cloud amount distribution feature labels of the historical time steps are spliced to obtain a cloud genus cloud amount distribution feature label matrix of the historical time steps; the cloud genus cloud amount distribution feature label matrix of the historical time steps is sequentially subjected to a fully connected layer, a ReLu activation function, a fully connected layer and a ReLu activation function to generate a time decay factor; the weight of each historical time is calculated according to the time decay factor: ， wherein denotes a history time instant a weight, denotes a time decay factor, denotes a time step, denotes a time instant; The weights of each historical time are weighted and summed with the cloud genus cloud amount distribution feature labels to obtain a time-weighted cloud genus cloud amount distribution feature label; The time-weighted cloud genus cloud amount distribution feature label is sequentially subjected to a fully connected layer and a Softmax activation function to generate the weight of the sub-predictor; The ground-based cloud image of the historical time step is subjected to the cloud image encoding module to extract cloud image visual features, and the photovoltaic power and meteorological data of the historical time step are subjected to the numerical data encoding module to extract numerical time series features; the cloud image visual features and the numerical time series features are subjected to the cross-modal fusion module to obtain cross-modal fusion features; in the hybrid predictor module, the cross-modal fusion features are subjected to global average pooling and then subjected to multiple parallel sub-predictors to predict photovoltaic power; the predicted photovoltaic power of each sub-predictor is weighted and summed according to the weight of each sub-predictor to obtain the final predicted photovoltaic power; Step four: training the ground-based cloud image fine-grained segmentation model using the ground-based cloud image segmentation dataset, and training the photovoltaic power prediction model using the multi-modal photovoltaic power prediction dataset, and using the trained ground-based cloud image fine-grained segmentation model and photovoltaic power prediction model for photovoltaic power prediction.

2. The method for multi-modal photovoltaic power forecasting based on cloud fraction segmentation and cloud genus cloud cover guidance according to claim 1, characterized in that, The foundation cloud picture fine-grained segmentation model comprises an encoder, a decoder and a segmentation head; the encoder comprises a plurality of encoding layers and parallel convolution attention modules, and the decoder comprises a plurality of decoding layers and wavelet enhanced feature fusion modules; the output feature map of the previous encoding layer is taken as the input feature map of the next encoding layer after the output feature map of the previous encoding layer passes through the parallel convolution attention module; each parallel convolution attention module is connected with the corresponding decoding layer in a jump connection mode, and a wavelet enhanced feature fusion module is embedded in the jump connection; the output feature map of the wavelet enhanced feature fusion module and the output feature map of the previous decoding layer are taken as the input feature map of the next decoding layer; for the wavelet enhanced feature fusion module on the last jump connection, the output feature map of the last parallel convolution attention module is taken as the input feature map of the first decoding layer after the output feature map of the last parallel convolution attention module passes through a maximum pooling operation and the output feature map of the last encoding layer passes through the wavelet enhanced feature fusion module; for the wavelet enhanced feature fusion modules on the remaining jump connections, the output feature map of the parallel convolution attention module and the output feature map of the previous decoding layer are taken as the input feature map of the wavelet enhanced feature fusion module. In the parallel convolution attention module, the input feature map passes through a parallel convolution module and a double convolution attention module respectively, the output feature map of the parallel convolution module and the output feature map of the double convolution attention module are multiplied element by element to obtain a parallel double convolution attention fusion feature map; the two parallel double convolution attention fusion feature maps are added element by element, and then pass through a 1*1 depth separable convolution to obtain the output feature map of the parallel convolution attention module. In the double convolution attention module, the input feature map passes through a 7*7 average pooling and a 1*1 depth separable convolution, and then passes through a 1*7 and a 7*1 depth separable convolution respectively to obtain a horizontal direction feature map and a vertical direction feature map; the horizontal direction feature map and the vertical direction feature map pass through batch normalization and Sigmoid activation function in turn respectively to generate a spatial horizontal attention weight matrix and a spatial vertical attention weight matrix; the spatial horizontal attention weight matrix and the spatial vertical attention weight matrix are multiplied element by element with the output feature map of the parallel convolution module respectively, and then added element by element, and the feature map obtained by the addition passes through a 1*1 depth separable convolution to obtain the output feature map of the double convolution attention module.

3. The multi-modal photovoltaic power forecasting method based on cloud map segmentation and cloud genus cloud cover guidance according to claim 2, characterized in that, In the parallel convolution module, the input feature map passes through four parallel convolution branches respectively; the first convolution branch comprises a 3*3 depth separable convolution, the second convolution branch comprises a 3*3 and a 5*5 depth separable convolution, the third convolution branch comprises a 3*3, a 5*5 and a 7*7 depth separable convolution, and the fourth convolution branch comprises a 3*3, a 5*5, a 7*7 and a 9*9 depth separable convolution; the output feature maps of the four convolution branches are added element by element, and then pass through a 1*1 depth separable convolution to obtain the output feature map of the parallel convolution module.

4. The multi-modal photovoltaic power prediction method based on cloud map segmentation and cloud genus cloud cover guidance according to claim 2 or 3, characterized in that, In the wavelet enhanced feature fusion module, the output feature map of the parallel convolution attention module is subjected to wavelet transform convolution to obtain a low-frequency feature map and a high-frequency feature map; the low-frequency feature map and the high-frequency feature map are spliced, and then subjected to inverse wavelet transform convolution to obtain a restored feature map; the low-frequency feature map and the output feature map of the previous decoding layer are fused through the local multi-directional feature fusion module, the output feature map of the local multi-directional feature fusion module is subjected to multiple superimposed wavelet transform-inverse wavelet transform convolution to purify the low-frequency feature map, and a purified low-frequency feature map is obtained; the purified low-frequency feature map is subjected to an upsampling module to obtain an upsampled feature map; the upsampled feature map and the restored feature map are spliced to obtain an output feature map of the wavelet enhanced feature fusion module.

5. The method of claim 4, wherein the cloud base height is determined by using a cloud base height estimation algorithm. In the local multi-directional feature fusion module, the low-frequency feature map and the output feature map of the previous decoding layer are element-wise added to obtain an encoding-decoding fusion feature map; the encoding-decoding fusion feature map is subjected to parallel depth separable convolution branches and multi-directional attenuation branches respectively, and the output feature maps of the two branches are spliced and then subjected to a Sigmoid activation function to generate a spatial attention weight matrix; The spatial attention weight matrix is used as the weight of the low-frequency feature map, and the difference matrix of the unit matrix and the spatial attention weight matrix is used as the weight of the output feature map of the previous decoding layer, and the low-frequency feature map and the output feature map of the previous decoding layer are fused by weighting to obtain an output feature map of the local multi-directional feature fusion module.

6. The method for multi-modal photovoltaic power prediction based on cloud fraction segmentation and cloud genus cloud cover guidance according to claim 5, characterized in that, The depth separable convolution branch includes two 1x1 depth separable convolutions, each followed by batch normalization and a ReLu activation function.

7. The multi-modal photovoltaic power prediction method based on cloud map segmentation and cloud genus cloud cover guidance according to claim 5 or 6, characterized in that, The input feature map of the multi-directional attenuation branch is sequentially subjected to 7x7 average pooling and 1x1, 1x7 and 7x1 depth separable convolutions, and then subjected to batch normalization and a ReLu activation function to obtain an output feature map of the multi-directional attenuation branch.

8. The method for multi-modal photovoltaic power forecasting based on cloud fraction segmentation and cloud genus cloud cover guidance according to claim 1, characterized in that, In the cloud image encoding module, the ground-based cloud image is subjected to a feature extraction network to extract spatial features, the spatial features are flattened according to the spatial dimension, and then subjected to position encoding, and then subjected to a Transformer encoder for time series encoding, and the output feature map of the Transformer encoder is subjected to global average pooling to obtain a cloud image visual feature.

9. The method of claim 1, wherein the cloud base height is determined by using a cloud base height estimation algorithm. In the numerical data encoding module, the photovoltaic power and meteorological data of the historical time step are spliced to obtain power-meteorological data fusion features; the power-meteorological data fusion features are subjected to a fully connected layer and then subjected to a channel attention mechanism for enhancement to obtain channel-enhanced power-meteorological data high-dimensional features; The channel-enhanced power-meteorological data high-dimensional features are subjected to transposition and then subjected to a temporal convolution network to obtain numerical time series features.

10. The method of claim 1, wherein the cloud base height is determined by using a cloud base height estimation algorithm. In the cross-modal fusion module, the cloud image visual feature and the numerical time series feature are unified in dimension through a linear mapping layer, and then subjected to position encoding to obtain position-encoded cloud image visual features and numerical time series features; The position coded cloud image visual features and the numerical time sequence features are interactively enhanced through a Cross Transformer network; and the interactively enhanced cloud image visual features and the numerical time sequence features are spliced to obtain cross-modal fusion features.

Images