Ultra-short-term solar irradiance forecasting method based on physical information and multi-modal spatio-temporal attention network

Abstract

The application discloses a kind of based on physical information and multimodal space-time attention network's ultra-short-term solar irradiance prediction method, obtains and aligns historical ground observation time series data and all-sky image sequence;According to clear sky total horizontal irradiance, original total horizontal irradiance is converted into clear sky normalized irradiance index;Based on all-sky image red blue ratio, learnable cloud soft mask is constructed, and based on solar disc position, solar attention map is constructed, and solar sensitive cloud coverage is calculated to reweight training loss;Numerical time series features and image space-time features are extracted using time series encoder and space-time decoupling hierarchical serial visual encoder respectively;By bidirectional cross-modal attention and dynamic cross-modal channel gate module, heterogeneous features are adaptively fused;The clear sky normalized irradiance index at future time is output using multi-step prediction decoder, and the future total horizontal irradiance prediction result is obtained by denormalization.The application improves the prediction accuracy and physical consistency of ultra-short-term solar irradiance.

Description

Technical Field

[0001] This invention belongs to the field of irradiance prediction technology, specifically relating to an ultra-short-term solar irradiance prediction method based on physical information and a multimodal spatiotemporal attention network. Background Technology

[0002] As the penetration rate of photovoltaic power generation in modern power systems continues to increase, the randomness, volatility, and intermittency of solar irradiance have become key uncertainties in real-time dispatch and automatic generation control. The main physical driver of photovoltaic output is the surface solar irradiance. When clouds rapidly pass through the area near the solar disk, the total horizontal irradiance can drop or rise sharply within minutes, potentially causing voltage flicker, frequency deviation, and pressure on reserve capacity regulation in the distribution network.

[0003] Traditional statistical forecasting methods typically model irradiance as a one-dimensional time series, making it difficult to capture the spatial structure and rapid evolution of clouds. While numerical weather prediction methods have a physical basis, their spatiotemporal resolution is relatively coarse, making it difficult to reflect local cloud cover in short-term forecasts down to the minute level. In recent years, convolutional neural networks, recurrent neural networks, and self-attention network models have been used for irradiance prediction, and further, a multimodal prediction framework integrating all-sky imagery and surface meteorological time series has been developed.

[0004] However, existing multimodal irradiance prediction methods still have the following shortcomings: First, many methods directly regress the original total horizontal irradiance, causing the model to simultaneously fit the clear sky background determined by the solar altitude angle and seasonal changes, as well as the random attenuation caused by cloud cover. These two types of factors are intertwined, resulting in insufficient sensitivity of the model to sudden slope events. Second, image branches usually extract global visual features from the entire sky hemisphere, without explicitly distinguishing between clouds that truly block the sun's path and clouds located far from the sun with weaker radiation impact. Third, directly engaging in deep interaction between high-dimensional image features and low-dimensional temporal features can lead to computational burden, while excessive compression of image features can result in the loss of cloud morphology and motion information.

[0005] Therefore, there is an urgent need to propose an ultra-short-term solar irradiance prediction method that can take into account physical consistency, solar shading sensitivity, and multimodal fusion efficiency, so as to improve the minute-level prediction accuracy and reliability in practical applications of power systems. Summary of the Invention

[0006] To overcome the problems of existing technologies, such as the entanglement between the original irradiance target and clear-sky variations, the lack of solar occlusion attention in all-sky images, and insufficient multimodal fusion efficiency, this invention provides an ultra-short-term solar irradiance prediction method based on physical information and a multimodal spatiotemporal attention network. This method achieves accurate modeling of cloud-induced irradiance attenuation through a clear-sky normalized irradiance index, learnable cloud soft masks, solar perception loss, a spatiotemporally decoupled visual encoder, and a dynamic cross-modal channel gating mechanism.

[0007] The technical solution adopted by this invention to solve its technical problem is: A method for predicting ultra-short-term solar irradiance based on physical information and a multimodal spatiotemporal attention network, the method comprising the following steps: Step 1: Acquire and time-align historical ground observation time-series data and all-sky image sequences. The historical ground observation time-series data shall include at least solar irradiance data, solar geometric parameters, and meteorological parameters. Step 2: Calculate or read the total horizontal irradiance under clear skies based on the observation time, geographical location, and solar geometric information. Normalize the original total horizontal irradiance under clear skies to obtain the normalized irradiance index under clear skies. Step 3: Construct a learnable cloud soft mask based on the all-sky image sequence, and concatenate the learnable cloud soft mask with the all-sky image sequence to obtain a mask-enhanced image sequence; Step 4: Construct a solar attention map based on the position of the solar disk, calculate the solar sensitive cloud coverage using the learnable cloud soft mask, and reweight the prediction loss at the sample level using the solar sensitive cloud coverage during the training process. Step 5: Input the historical ground observation time series data into the time series encoder to extract numerical time series features, and input the masked enhanced image sequence into the space-time decoupled hierarchical serial visual encoder to extract image spatiotemporal features; Step 6: Input the numerical time-series features and the image spatiotemporal features into the cross-modal interaction module for bidirectional feature exchange, and then perform adaptive fusion through the dynamic cross-modal channel gating module to obtain fused features; Step 7: Input the fused features into the multi-step prediction decoder, output the predicted values ​​of the clear sky normalized irradiance index for several future times, and restore the predicted values ​​of the clear sky normalized irradiance index to the predicted results of the future total horizontal irradiance based on the total clear sky horizontal irradiance.

[0008] The beneficial effects of this invention are mainly reflected in: (1) By using the clear sky normalized irradiance index, the deterministic background changes caused by solar geometry are decoupled from the random attenuation caused by cloud cover, thereby reducing the bias of day and night and seasonal changes and improving the response capability to rapid changes in irradiance.

[0009] (2) By using learnable cloud soft masks and solar attention maps, the visual branch can be shifted from generalized cloud recognition to irradiance modeling related to solar disk occlusion, thus avoiding interference from clouds far from the sun on the training target.

[0010] (3) By first performing intra-frame spatial convergence and then cross-frame temporal modeling through a spatiotemporally decoupled hierarchical serial visual encoder, the computational complexity of direct interaction between high-dimensional image sequences is reduced while maintaining cloud morphology and motion cues.

[0011] (4) By using a dynamic cross-modal channel gating mechanism, numerical evidence and visual evidence are adaptively selected according to different weather conditions and sensor noise, thereby improving robustness in various scenarios such as clear skies, scattered clouds, and thick clouds.

[0012] (5) By generating a multi-step decoding structure, multiple future prediction time domains are output at once, reducing the accumulation of errors in autoregressive stepwise prediction, which is suitable for ultra-short-term scheduling scenarios such as 10 minutes, 20 minutes and 30 minutes. Attached Figure Description

[0013] Figure 1 This is a flowchart of the ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention network of the present invention. Figure 2 This is a schematic diagram of the multimodal spatiotemporal attention prediction network structure of the present invention; Figure 3 This is a schematic diagram of the learning mechanism of solar sensing cloud mask and clear sky normalized irradiance index in this invention. Figure 4 This is a schematic diagram of the overall architecture and signal flow of the multimodal spatiotemporal attention prediction network of the present invention. Detailed Implementation

[0014] The present invention will now be further described with reference to the accompanying drawings.

[0015] Reference Figures 1-4 A method for predicting ultra-short-term solar irradiance based on physical information and a multimodal spatiotemporal attention network, such as... Figure 2 As shown, the ultra-short-term solar irradiance prediction network constructed in this embodiment includes a data acquisition and preprocessing module, a clear-sky normalization module, a solar sensing cloud mask learning module, a temporal encoder, a spatiotemporally decoupled hierarchical serial visual encoder, a cross-modal interaction module, a dynamic cross-modal channel gating module, and a multi-step prediction decoder. The overall network can be called a multimodal spatiotemporal attention prediction network.

[0016] Figure 4The solid line represents data processing and feature flow, indicating the flow of input information such as historical ground observation time series data, all-sky image sequences, and solar position through the solar sensing cloud mask index learning module, temporal encoder, spatiotemporal decoupling serial visual encoder, cross-modal interaction module, dynamic cross-modal channel gating module, and predictive decoder to form the future irradiance prediction result. The dashed line represents the solar sensitivity loss reweighting signal during the training phase, which is used to incorporate the solar attention map, cloud soft mask, and solar sensitive cloud coverage into the loss function weight adjustment, thereby improving the model's learning intensity for solar disk occlusion samples.

[0017] The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks in this embodiment includes the following steps: Step 1: Data Acquisition and Preprocessing: Acquire and time-align historical ground observation time-series data and all-sky image sequences. The historical ground observation time-series data includes at least solar irradiance data, solar geometric parameters, and meteorological parameters. The historical ground observation time series data includes one or more of the following: total horizontal irradiance, direct normal irradiance, diffuse horizontal irradiance, solar zenith angle, solar azimuth angle, atmospheric mass, average wind speed, peak wind speed, wind direction, and cloud cover detection data; the all-sky image sequence is acquired by an all-sky imager and undergoes fisheye distortion correction, geometric registration, normalization, and downsampling processing.

[0018] In this embodiment, the all-sky image is acquired by an all-sky imager, and the image and time-series observations are registered according to a unified timestamp.

[0019] For all-sky images, fisheye distortion correction or dual-dimensional geometric correction is first performed to align the sky region with the sun's position in the image coordinates. Then, the image is downsampled to a predetermined size, such as 100×100 pixels, to reduce computational complexity. Finally, the red, green, and blue channels are normalized. For time-series data, missing and outlier values ​​are removed or interpolated, and the data is standardized according to the training set statistics.

[0020] Suppose a training sample contains A historical moment and For each future predicted moment, historical time-series data is represented as follows: The whole-sky image sequence is represented as ,in For batch size, For numerical variables, For the number of image frames, and This refers to the image height and width.

[0021] Step 2, Clear Sky Normalized Irradiance Index Learning: Calculate or read the total horizontal irradiance under clear sky based on the observation time, geographical location and solar geometry information, and normalize the original total horizontal irradiance under clear sky to obtain the clear sky normalized irradiance index. In this embodiment, directly regressing the original total horizontal irradiance would cause the model to simultaneously fit both clear-sky background and cloud-induced attenuation. To avoid this problem, this invention reconstructs the prediction target as the clear-sky normalized irradiance index. : ; In the formula, for Original total horizontal irradiance at any given time Total horizontal irradiance under clear skies, To prevent division by zero constants, This is a truncation function. and This represents the upper and lower bounds of the index range. The index reflects the rate of decrease in current actual irradiance relative to clear-sky conditions, thus focusing the learning objective more on the relative changes caused by cloud cover.

[0022] During the model inference phase, after predicting the clear-sky normalized irradiance index for future times, the total horizontal irradiance is recovered according to the following formula: ; in This is the predicted value of the normalized normalized irradiance index for future clear skies output by the model. This represents the predicted total future horizontal irradiance after recovery.

[0023] Step 3: Construction of solar sensing cloud soft mask: Construct a learnable cloud soft mask based on the full-sky image sequence, and concatenate the learnable cloud soft mask with the full-sky image sequence to obtain a mask-enhanced image sequence; This invention utilizes the differences between clouds and clear sky in the red and blue channels of a full-sky image to construct a learnable cloud soft mask. For the inversely normalized red, green, and blue three-channel image, at the pixel location... Calculate the red-to-blue ratio: ; in and These represent the pixel intensities of the red and blue channels, respectively. Unlike fixed-threshold binary cloud detection, this invention uses a learnable continuous function to obtain the cloud probability map: ; in, For the smoothing threshold function, As a learning transition center, This provides a learnable transition sharpness. Through this method, cloud masking not only preserves end-to-end training capabilities but also adaptively adjusts the boundary between clouds and clear sky based on dataset characteristics; subsequently... As an additional channel, it is stitched together with the red, green, and blue three-channel image to obtain a four-channel masked enhanced image: .

[0024] Step 4: Reweighting the solar sensitive cloud coverage and loss: Construct a solar attention map based on the position of the solar disk, calculate the solar sensitive cloud coverage using the learnable cloud soft mask, and reweight the prediction loss at the sample level using the solar sensitive cloud coverage during training. Since clouds near the solar disk have a much greater impact on surface irradiance than clouds at the sky's edge, this embodiment utilizes the solar disk's position to construct a solar attention map. This is achieved through solar detection maps, aligned images corrected for solar disk positioning, or geometric calculations of the solar position. The Sun Center at Moment Then construct the Gaussian solar attention map: ; in Controlling the spatial diffusion range of the solar-affected region. Defining solar-sensitive cloud coverage based on cloud soft masking and solar attention maps: ; This quantity measures cloud cover in a solar-related region, rather than the average cloud cover across the entire hemisphere. During training, a solar-perception-weighted smoothed absolute error loss is used. ; in The weighting coefficients are as follows: The higher the cloud cover near the sun, the greater the weight of the training samples, thus making the model pay more attention to the rapid changes in irradiance caused by cloud obstruction of the solar disk.

[0025] Step 5: Timing encoder and spatiotemporal decoupled hierarchical serial visual encoder: Input the historical ground observation time series data into the timing encoder to extract numerical temporal features, and input the masked enhanced image sequence into the spatiotemporal decoupled hierarchical serial visual encoder to extract image spatiotemporal features; The temporal encoder includes a temporal embedding layer, a positional encoding layer, several layers of probabilistic sparse self-attention layers, and a sequence distillation layer. The probabilistic sparse self-attention layer selects the dominant query with the highest sparsity score for attention calculation to reduce the computational complexity when the historical time series length is large. The sequence distillation layer compresses the sequence length step by step through one-dimensional convolution and max pooling while retaining key temporal features. The spatiotemporally decoupled hierarchical serial visual encoder includes an image patch embedding layer, a spatial self-attention encoder, a frame-level pooling layer, and a temporal self-attention encoder. The image patch embedding layer divides each frame of masked enhanced image into several non-overlapping image patches and maps them to a unified dimension. The spatial self-attention encoder extracts cloud morphology and local occlusion structures frame by frame. The frame-level pooling layer performs mean pooling on the image patch features of each frame to obtain frame-level representations. The temporal self-attention encoder models cloud motion and morphological evolution based on the frame-level representation sequence.

[0026] The temporal encoder in this embodiment is used to model the long-term dependencies between historical irradiance, meteorological variables, and solar geometric variables. The historical temporal input is linearly embedded and positionally encoded before entering a multilayer self-attention encoder. Its basic attention calculation form is as follows: ; To reduce the computational complexity of long sequences, the time encoder employs a probabilistic sparse self-attention mechanism, selecting only the dominant queries with higher sparsity scores for attention computation, thus reducing the complexity from... Reduced to approximate Sequence distillation units, consisting of one-dimensional convolution and max pooling, are placed between adjacent coding layers to progressively compress the time length while retaining key change information, ultimately obtaining numerical temporal features. .

[0027] like Figure 2 As shown, the visual branch adopts a spatiotemporally decoupled hierarchical serial structure, that is, first extracting the spatial cloud structure within each frame, and then extracting the cloud motion and morphological evolution between frame sequences. For a four-channel mask-enhanced image sequence... First, divide each frame into A number of non-overlapping image patches are obtained, and the image patch embedding is obtained through learnable projection: ; in For learnable linear projection matrices, For location embedding, each frame's image patch is embedded into a spatial self-attention encoder: ; After obtaining the context-enhanced image patch features, frame-level representations are formed through mean pooling: ; Finally, the frame-level representation sequence Input a temporal self-attention encoder to obtain the spatiotemporal features of the image: ; This design avoids the computational explosion that would result from directly unfolding all image patches in both time and space dimensions, while retaining important clues about cloud movement and morphological changes for irradiance prediction.

[0028] Step 6, Cross-modal interaction and dynamic cross-modal channel gating: The numerical temporal features and the image spatiotemporal features are input into the cross-modal interaction module for bidirectional feature exchange, and then adaptively fused through the dynamic cross-modal channel gating module to obtain fused features; The cross-modal interaction module adopts a symmetrical cross-attention structure, using numerical temporal features as queries and image spatiotemporal features as keys and values ​​to obtain image-guided numerical features, and simultaneously using image spatiotemporal features as queries and numerical temporal features as keys and values ​​to obtain time-guided image features.

[0029] To fully utilize the complementarity between numerical time-series data and image data, this invention employs a symmetrical cross-attention structure for intermodal information exchange: ; The first cross-attention calculation uses numerical features as the query and image features as the key and value, while the second cross-attention calculation uses image features as the query and numerical features as the key and value. Subsequently, the dynamic cross-modal channel gating module concatenates the two types of features. ; The channel gating coefficient is obtained through the gating network: ; The fusion features are calculated as follows: ; in The model adaptively adjusts the importance of the two modalities along the channel dimension. In clear skies or with poor image quality, the model can increase the weight of numerical temporal evidence; when cloud cover changes rapidly or when a slope event occurs, the model can enhance the weight of visual evidence.

[0030] Step 7: Multi-step prediction decoder and output: Input the fused features into the multi-step prediction decoder, output the predicted values ​​of the normalized irradiance index for clear skies at several future times, and restore the predicted values ​​of the normalized irradiance index for clear skies to the predicted results of the future total horizontal irradiance based on the total horizontal irradiance for clear skies.

[0031] The multi-step prediction decoder adopts a generative non-autoregressive structure. The decoding input is formed by splicing historical label fragments and future placeholder fragments. Each decoder layer includes a mask self-attention layer, a cross-attention layer with fused features, a feedforward network layer, and a normalization layer. Finally, it outputs the predicted values ​​of clear sky normalized irradiance index for one or more prediction time domains in the next 10 minutes, 20 minutes, and 30 minutes through linear projection.

[0032] In this embodiment, the predictive decoder employs a generative multi-step structure to reduce error accumulation caused by stepwise autoregression. The decoding input consists of known label fragments. and future placeholder fragments Formed by splicing: ; Each layer of decoder first... Perform masked self-attention, then fuse features. Perform cross attention as key and value: ; Finally, the predicted values ​​of the clear sky normalized irradiance index for multiple future moments are output through linear projection, and the total horizontal irradiance is recovered using equation (2). Depending on the actual application needs, the prediction time domain can be set to 10 minutes, 20 minutes, 30 minutes or other minute-level time scales.

[0033] In one specific embodiment, publicly available solar radiation research laboratory data is used for model training and validation. The data includes synchronous ground meteorological records and all-sky images, which are processed to form training samples after missing value removal, time alignment, and image correction. The training set, validation set, and test set can be divided in a ratio of 80%, 10%, and 10%, respectively. All comparative models are trained under the same data partitioning, preprocessing procedures, and optimization protocols.

[0034] The evaluation indicators include mean absolute error, mean absolute percentage error, normalized root mean square error, root mean square error, coefficient of determination, and systematic bias. Mean absolute error measures the average absolute deviation between predicted and actual values; mean absolute percentage error measures the relative percentage error; normalized root mean square error and root mean square error measure the level of root mean square error; the coefficient of determination measures the goodness of fit between predicted and actual values; and systematic bias reflects systematic overestimation or underestimation.

[0035] Tables 1 to 3 show a comparison of the controlled performance of the present invention and some strong multimodal baseline models in the 10-minute, 20-minute, and 30-minute prediction time domains.

[0036] Table 1 shows the controlled comparison: MAE (W / m 2 (The smaller the better) Table 2 shows the controlled comparison: MAPE (%, the smaller the better); Table 3 shows the controlled comparison: NRMSE (%, the smaller the better); As shown in Tables 1 to 3, this invention achieves the lowest mean absolute error, lowest mean absolute percentage error, and lowest normalized root mean square error in all three prediction time domains. Compared to the best competing model in the controlled baseline, this invention achieves relative reductions of approximately 11.00%, 10.94%, and 10.94% in mean absolute error at 10-minute, 20-minute, and 30-minute intervals, respectively, indicating a synergistic gain among clear-sky normalization, solar-sensing cloud masking, spatiotemporal decoupling visual coding, and dynamic channel gating.

[0037] Furthermore, in the 10-minute prediction scatter analysis, the coefficient of determination of this invention was 0.889, the root mean square error was 62.0 W / m², the systematic bias was 4.8 W / m², the mean prediction error was 4.800, the median error was 3.501, and the standard deviation of the error was 61.857. This indicates that this invention can not only reduce the average prediction error, but also reduce the systematic bias and improve the stability of the prediction results.

[0038] Table 4 presents the ablation results of key components, illustrating the contributions of SACMK, ST-Serial, and DCCG in this invention. Table 4 shows the experimental results of core component ablation; As shown in Table 4, introducing solar-sensing cloud mask learning alone can significantly reduce errors. Building upon this, the spatiotemporally decoupled visual encoder provides cloud evolution information, and dynamic cross-modal channel gating improves the quality of heterogeneous feature fusion. The complete model achieves the best results. The ablation experiment simultaneously statistically analyzed the mean absolute percentage error (MAE). The complete model's MAEs in the 10-minute, 20-minute, and 30-minute prediction time domains were 22.84%, 27.31%, and 32.44%, respectively, all superior to the results after removing any core module.

[0039] The prediction time domain, image size, number of self-attention network layers, embedding dimension, loss weights, clear sky model, and input meteorological variables in the above embodiments can all be adjusted according to specific application scenarios. For example, the prediction time domain can be extended to 5 minutes, 15 minutes, or 60 minutes, and the all-sky image can be replaced with satellite cloud images, ground cloud camera images, or multispectral cloud images. As long as they are modeled collaboratively with time-series data through solar sensing cloud masking and cross-modal gating mechanisms, they are all equivalent implementations of this invention.

[0040] The embodiments described above merely illustrate more specific and detailed implementations of the present invention, but should not be construed as limiting the scope of the present invention. For those skilled in the art, various modifications, substitutions, and improvements can be made without departing from the concept of the present invention, and all such modifications, substitutions, and improvements should fall within the protection scope of the present invention. The protection scope of the present invention is defined by the claims.

Claims

1. A method for predicting ultra-short-term solar irradiance based on physical information and a multimodal spatiotemporal attention network, characterized in that, The method includes the following steps: Step 1: Acquire and time-align historical ground observation time-series data and all-sky image sequences. The historical ground observation time-series data shall include at least solar irradiance data, solar geometric parameters, and meteorological parameters. Step 2: Calculate or read the total horizontal irradiance under clear skies based on the observation time, geographical location, and solar geometric information. Normalize the original total horizontal irradiance under clear skies to obtain the normalized irradiance index under clear skies. Step 3: Construct a learnable cloud soft mask based on the all-sky image sequence, and concatenate the learnable cloud soft mask with the all-sky image sequence to obtain a mask-enhanced image sequence; Step 4: Construct a solar attention map based on the position of the solar disk, calculate the solar sensitive cloud coverage using the learnable cloud soft mask, and reweight the prediction loss at the sample level using the solar sensitive cloud coverage during the training process. Step 5: Input the historical ground observation time series data into the time series encoder to extract numerical time series features, and input the masked enhanced image sequence into the space-time decoupled hierarchical serial visual encoder to extract image spatiotemporal features; Step 6: Input the numerical time-series features and the image spatiotemporal features into the cross-modal interaction module for bidirectional feature exchange, and then perform adaptive fusion through the dynamic cross-modal channel gating module to obtain fused features; Step 7: Input the fused features into the multi-step prediction decoder, output the predicted values ​​of the clear sky normalized irradiance index for several future times, and restore the predicted values ​​of the clear sky normalized irradiance index to the predicted results of the future total horizontal irradiance based on the total clear sky horizontal irradiance.

2. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1, characterized in that, In step one, the historical ground observation time series data includes one or more of the following: total horizontal irradiance, direct normal irradiance, diffuse horizontal irradiance, solar zenith angle, solar azimuth angle, atmospheric mass, average wind speed, peak wind speed, wind direction, and cloud cover detection data; the all-sky image sequence is acquired by an all-sky imager and undergoes fisheye distortion correction, geometric registration, normalization, and downsampling processing.

3. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1 or 2, characterized in that, In step two, the clear sky normalized irradiance index Calculate as follows: ； in, for Observe the total horizontal irradiance at all times. for Total horizontal irradiance under clear skies at any given time It is the numerical stability constant. and To truncate the upper and lower bounds, the total horizontal irradiance is restored during the inference phase as follows: ； in This is the predicted value of the normalized normalized irradiance index for future clear skies, output by the model. This represents the predicted total future horizontal irradiance after recovery.

4. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1 or 2, characterized in that, In step three, the learnable cloud soft mask is constructed based on the red-blue ratio of the entire sky image; for pixel positions... and time The red-blue ratio is: ； The learnable cloud software mask is: ； in and These represent the intensity of the red and blue channels, respectively. For the smoothing threshold function, and These are learnable parameters.

5. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1 or 2, characterized in that, In step four, the solar attention map is constructed using the centroid of the solar disk pixels; if the solar center is... The solar attention diagram is then: ； in The parameter represents the range of solar influence; the solar-sensitive cloud coverage is: ； The training loss is: ； in, These are the weighted coefficients.

6. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1 or 2, characterized in that, In step five, the temporal encoder includes a temporal embedding layer, a positional coding layer, several layers of probabilistic sparse self-attention layers, and a sequence distillation layer. The probabilistic sparse self-attention layer selects the dominant query with the highest sparsity score for attention calculation to reduce the computational complexity when the historical time series length is large; the sequence distillation layer compresses the sequence length step by step through one-dimensional convolution and max pooling while retaining key temporal features. The spatiotemporally decoupled hierarchical serial visual encoder includes an image patch embedding layer, a spatial self-attention encoder, a frame-level pooling layer, and a temporal self-attention encoder. The image patch embedding layer divides each frame of masked enhanced image into several non-overlapping image patches and maps them to a unified dimension. The spatial self-attention encoder extracts cloud morphology and local occlusion structures frame by frame. The frame-level pooling layer performs mean pooling on the image patch features of each frame to obtain frame-level representations. The temporal self-attention encoder models cloud motion and morphological evolution based on the frame-level representation sequence.

7. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1 or 2, characterized in that, In step six, the cross-modal interaction module adopts a symmetrical cross-attention structure, using numerical temporal features as queries and image spatiotemporal features as keys and values ​​to obtain image-guided numerical features, and simultaneously using image spatiotemporal features as queries and numerical temporal features as keys and values ​​to obtain time-guided image features.

8. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 7, characterized in that, In step six, the dynamic cross-modal channel gating module concatenates the interacted numerical features and image features in the channel dimension, and generates channel gating coefficients through linear mapping or one-dimensional convolution and smooth activation. ; The fusion features are obtained according to the following formula: ； in, For numerical features, For image features, For residual connectivity features, This indicates element-wise multiplication.

9. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1 or 2, characterized in that, In step seven, the multi-step prediction decoder adopts a generative non-autoregressive structure. The decoding input is formed by splicing historical label fragments and future placeholder fragments. Each decoder layer includes a mask self-attention layer, a cross-attention layer with fused features, a feedforward network layer, and a normalization layer. Finally, it outputs the predicted values ​​of clear sky normalized irradiance index for one or more prediction time domains in the next 10 minutes, 20 minutes, and 30 minutes through linear projection.

10. The ultra-short-term solar irradiance prediction method based on physical information and multimodal spatiotemporal attention networks as described in claim 1 or 2, characterized in that, The training process in step four uses the clear-sky normalized irradiance index as the supervision target and is optimized using a smoothed absolute error loss with reweighted solar-sensitive cloud coverage. In each training sample, the stronger the cloud soft mask response near the solar disk, the greater the sample loss weight, so as to improve the model's ability to learn about irradiance abrupt events caused by the shading of the solar disk.

Images