Multi-mode photovoltaic power prediction method, system and equipment based on improved ViT, and medium

Abstract

The invention discloses a multi-mode photovoltaic power prediction method, system and device based on improved ViT and a medium, and belongs to the technical field of photovoltaic power generation system monitoring, and the method comprises the steps: processing a sky image sequence and historical meteorological data: extracting the spatial-temporal dynamic characteristics of a cloud layer through Transform layering, mining the high-dimensional law of meteorological data through a deep network, and obtaining the spatial-temporal dynamic characteristics of the cloud layer; a single factor prediction is generated through LSTM. And fusing the image and the meteorological features by adopting a cross-modal attention mechanism, inputting the fused image and meteorological features into a MambaTS network, and carrying out adaptive fusion on a single-factor prediction result and a multi-modal prediction result. According to the invention, through spatial-temporal feature extraction of a sky image sequence, high-dimensional feature mining of historical meteorological data, cross-modal attention fusion and adaptive result integration, a multi-level and collaborative optimization photovoltaic power prediction framework is constructed. According to the framework, the comprehensive sensing ability of local details, global trends, short-term fluctuation and multi-mode association is unified.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power generation system monitoring technology, specifically to a multimodal photovoltaic power prediction method, system, equipment, and medium based on improved ViT. Background Technology

[0002] Against the backdrop of global energy shortages, the drawbacks of fossil fuels are becoming increasingly apparent. As a non-renewable resource, fossil fuels not only face the risk of depletion but also emit large amounts of greenhouse gases during use, polluting the environment and exacerbating climate change. In contrast, solar energy, as a clean and renewable energy source, has demonstrated enormous potential. It causes virtually no environmental pollution, aligning with the global demand for green development.

[0003] In recent years, the total amount of new energy power generation has been continuously increasing, which meets the specific requirements of my country's sustainable development and can effectively curb environmental pollution. However, in the process of continuous development of new energy, power grid connection has significantly affected the grid's dispatch and stability during operation. Solar photovoltaic power generation is highly volatile and intermittent due to the influence of multiple factors such as solar irradiance, temperature, and humidity. In order to cope with the uncertainty of photovoltaic power generation and improve prediction accuracy, photovoltaic power prediction, as a key technology for photovoltaic energy grid connection, has made significant progress in methodological innovation and model optimization. The rapid development of photovoltaic power prediction technology is essentially an inevitable choice to address the contradiction between grid safety operation and efficient consumption of new energy. From passively bearing fluctuation losses to actively managing uncertainty, according to calculations by the China Electricity Council, every 1% improvement in prediction accuracy can reduce the loss of tens of millions of tons of abandoned power. Current technology is evolving from single power plant prediction to a full-link intelligent agent of "wide-area collaborative prediction - market trading - grid dispatch," becoming the core hub of the new power system.

[0004] The inherent volatility and intermittency of photovoltaic (PV) power generation, and the core challenge to its large-scale application, lies in the minute-level power fluctuations caused by factors such as irradiance, cloud movement, and seasonal climate. These anomalies can sometimes reach 80% of installed capacity, significantly increasing grid frequency regulation pressure and safety risks. Against this backdrop, high-precision PV power prediction has become a key technology for overcoming this predicament. Current research includes mining patterns from large amounts of historical meteorological data to predict PV power output and analyzing sky images to model the relationship between image and power output. Scholars have proposed numerous PV power prediction schemes based on traditional machine learning and deep learning methods, making significant contributions to improving the accuracy of PV power prediction.

[0005] With technological advancements, researchers have noticed that the core challenge in photovoltaic (PV) power prediction lies in accurately capturing the dynamic changes in solar irradiance, especially short-term fluctuations caused by cloud movement. They have begun using meteorological data to represent weather time series and sky images to represent cloud dynamics, achieving a finer characterization of the spatiotemporal heterogeneity of cloud dynamics by coordinating meteorological time series information and cloud dynamics information. Research methods have also gradually shifted from focusing on single-modal prediction based on data or images to multimodal prediction based on both data and images. However, existing multimodal fusion methods are insufficient in modeling long-range dependencies, and spatiotemporal feature interaction noise is not effectively suppressed, primarily limiting their application to short-term predictions. The core issue in multimodal learning for PV power prediction is designing an efficient spatiotemporal feature extraction and fusion mechanism to jointly process sky images and multi-source time series data.

[0006] To address the shortcomings of existing methods in capturing the spatiotemporal variability of cloud dynamics and the insufficient modeling ability of multimodal methods for long-term dependencies, this paper proposes a novel long-term photovoltaic (PV) power prediction method inspired by the patch mechanism of ViT. This method involves altering the arrangement of the Transformer encoder in ViT to create a two-layer structure for extracting temporal and spatial features from sky image sequences. Next, cross-modal attention is employed to fuse features from sky images and meteorological data. Finally, single-factor temporal prediction is used to capture short-term features, collaboratively constraining the PV power prediction curve within the sunrise and sunset time periods. By combining the advantages of single-modal and multimodal methods, the accuracy of long-term PV power prediction is further improved. Summary of the Invention

[0007] In view of the above-mentioned problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by this invention is that existing photovoltaic power prediction methods, especially multimodal methods for long-term prediction, are unable to effectively capture the spatiotemporal dynamic differences caused by cloud movement, and have insufficient ability to model the long-range dependency relationship between historical data and sky images, resulting in limited prediction accuracy for photovoltaic power with strong fluctuations and intermittency, especially for long-term trends.

[0009] To address the aforementioned technical problems, this invention provides the following technical solution: a multimodal photovoltaic power prediction method based on improved ViT, comprising,

[0010] The sky image sequence is preprocessed into a serialization sequence. An encoder structure is used to extract the spatiotemporal features of the preprocessed sky image sequence to capture local dependencies in the sky image sequence. A deep network is used to extract high-dimensional features from historical meteorological data. The historical meteorological data is input into a memory network LSTM to obtain single-factor prediction results. Cross-modal attention is used to fuse the spatiotemporal features of the sky image sequence and the high-dimensional features of historical meteorological data to obtain multimodal features. The multimodal features are input into a MambaTS network to obtain multimodal prediction results. The single-factor prediction results and multimodal prediction results are adaptively fused to obtain the final prediction result.

[0011] As a preferred embodiment of the multimodal photovoltaic power prediction method based on improved ViT described in this invention, the step of performing serialization preprocessing on the sky image sequence includes,

[0012] Each frame of the sky image sequence and its corresponding natural language description are abstracted into natural language statements.

[0013] The single-frame image in the sky image sequence is divided into image blocks.

[0014] Arrange all the image blocks segmented from each frame into a one-dimensional vector according to the temporal order of the image frames.

[0015] As a preferred embodiment of the improved ViT-based multimodal photovoltaic power prediction method of the present invention, wherein: the high-dimensional features extracted from historical meteorological data using deep networks include...

[0016] The original historical meteorological data is cleaned and standardized to eliminate noise and dimensional effects.

[0017] Multivariate statistical analysis is introduced to quantitatively analyze and characterize the nonlinear dependency structure among variables in historical meteorological data.

[0018] A deep dilated convolutional network is constructed to automatically extract high-dimensional spatiotemporal features from preprocessed historical meteorological data.

[0019] As a preferred embodiment of the multimodal photovoltaic power prediction method based on improved ViT described in this invention, the standardization preprocessing includes:

[0020] Outlier detection is performed on historical photovoltaic data sequences in historical meteorological data.

[0021] For the detected outliers, cubic spline interpolation is used for repair.

[0022] The same outlier detection and repair methods as those used for photovoltaic historical data were applied to process historical meteorological data.

[0023] Based on the irradiance values, the photovoltaic power data corresponding to the period from sunset to sunrise is manually set to zero.

[0024] As a preferred embodiment of the multimodal photovoltaic power prediction method based on improved ViT described in this invention, the high-dimensional features of the fusion of spatiotemporal features of sky image sequences and historical meteorological data include:

[0025] The sky image sequence was aligned with and standardized with historical photovoltaic data.

[0026] Feature extraction of the pre-processed sky image sequence includes:

[0027] The preprocessed sky image sequence I is input into the Modified ViT model to extract features.

[0028] The first layer of the Modified ViT extracts spatial features related to the shape, density, and coverage of the cloud layer through multiple Transformer Encoders.

[0029] If the sequence has T images, the spatial feature is taken as the output feature vector S-Feature of the last Encoder.

[0030] The second layer of Modified ViT contains a Transformer Encoder, which further processes the features proposed by the T Encoders in the previous layer. The final output captures the dynamic trend of cloud changes over time and is called the time feature T-Feature.

[0031] The beneficial effects of this preferred technical solution are that the first layer of multiple Transformer encoders can focus on a single frame image respectively, accurately extracting key spatial features related to cloud shape, density and coverage; the second layer of a single Transformer encoder integrates the spatial feature sequence of all frames on this basis, effectively capturing the dynamic trend and long-term dependence of cloud movement over time, thereby providing a more accurate and time-series discriminative spatiotemporal feature representation of cloud conditions for photovoltaic power prediction.

[0032] As a preferred embodiment of the improved ViT-based multimodal photovoltaic power prediction method of the present invention, the introduction of multivariate statistical analysis to quantitatively analyze and characterize the nonlinear dependency structure among variables in historical meteorological data includes:

[0033] The copula function is introduced to analyze the nonlinear correlation in historical meteorological data.

[0034] The input sequence is processed by dilated convolution to obtain historical data features W-Feature.

[0035] The beneficial effects of this preferred technical solution are that by introducing the Copula function to analyze the nonlinear correlation between variables in historical meteorological data and calculate the weights, the differential impact of different meteorological factors on photovoltaic power can be accurately quantified; combined with the use of a deep dilated convolutional network with a fixed convolution kernel and an increasing dilation rate, the model's ability to capture long-term dependencies and nonlinear high-dimensional features in historical data can be improved.

[0036] As a preferred embodiment of the improved ViT-based multimodal photovoltaic power prediction method described in this invention, the method involves using historical power data to infer single-factor prediction results through an LSTM model, denoted as follows:

[0037] PV2 = LSTM(P)

[0038] Where P represents historical power data;

[0039] The obtained multimodal prediction results include,

[0040] After dimensional unification, the input x has a dimension of K and a time step of L.

[0041] The x is represented as,

[0042]

[0043] Divide x into blocks every s time steps to obtain

[0044] Will Linearly mapped to D-dimensional tokens, we have

[0045] For the obtained K×M tokens, a scan along the time variable is performed to obtain...

[0046] The tokens are fed into an encoder consisting of N time-based Mamba blocks, where TMB is represented as:

[0047] h=SSM(Dropout(Linear(z)))+σ(Linear(z))

[0048] A linear transformation is performed in the decoder to map the encoder's result h to the predicted result. That is, multimodal prediction results;

[0049] The adaptive fusion is expressed as follows:

[0050] PV = PV1 × r + (1 - r) × PV2

[0051] in, M is xk The maximum number of blocks after partitioning It is a weight matrix, where D represents the dimension of the token. Represented as x k The value of the feature after linear mapping at index j, SSM is the state-space model operation, Linear represents the linear layer operation, σ represents the activation function, Dropout represents the regularization operation in the deep learning algorithm, and r represents the adaptive weight.

[0052] The beneficial effects of this preferred technical solution are as follows: by using an LSTM model to generate single-factor predictions to capture short-term patterns in historical power, and by employing a MambaTS network to process multimodal features after block division and labeling to capture long-term complex dependencies, the beneficial effect lies in combining the short-term memory advantage of LSTM with the efficiency of MambaTS in long-sequence modeling; finally, by adaptively fusion of the two prediction results, the model can dynamically balance short-term fluctuation characteristics and long-term spatiotemporal patterns, thereby improving the overall accuracy and robustness of long-sequence photovoltaic power prediction.

[0053] This invention provides a multimodal photovoltaic power prediction system based on an improved ViT.

[0054] To address the aforementioned technical problems, this invention provides the following technical solution: a multimodal photovoltaic power prediction system based on improved ViT, comprising: a data preprocessing module, a sky image spatiotemporal feature extraction module, a historical meteorological data feature extraction module, a single-factor prediction module, a multimodal feature fusion module, and an adaptive result fusion module.

[0055] The data preprocessing module is used to perform serialization preprocessing on the sky image sequence and to extract the spatiotemporal features of the serialized sky image sequence using an encoder structure in order to capture the local dependencies in the sky image sequence.

[0056] The sky image spatiotemporal feature extraction module is used to fuse the spatiotemporal features of the sky image sequence and the high-dimensional features of historical meteorological data using cross-modal attention to obtain multimodal features.

[0057] The historical meteorological data feature extraction module is used to extract high-dimensional features from historical meteorological data using deep networks.

[0058] The single-factor prediction module is used to input historical meteorological data into the LSTM memory network to obtain single-factor prediction results.

[0059] The multimodal feature fusion module is used to input multimodal features into the MambaTS network to obtain multimodal prediction results.

[0060] The adaptive result fusion module is used to adaptively fuse single-factor prediction results and multi-modal prediction results to obtain the final prediction result.

[0061] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the improved ViT-based multimodal photovoltaic power prediction method.

[0062] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the improved ViT-based multimodal photovoltaic power prediction method.

[0063] The beneficial effects of this invention are as follows: First, by using image sequence segmentation and a two-layer Transformer encoder design, it accurately captures both long-term and short-term dependencies in both temporal and spatial dimensions. Second, it employs a three-layer dilated convolution to efficiently extract high-dimensional features from historical meteorological data and cleverly introduces single-factor temporal prediction to collaboratively constrain multimodal predictions of short-term daytime and nighttime trends. Third, it achieves deep feature fusion through a cross-modal attention mechanism, effectively achieving fine-grained alignment between sky images and meteorological data and enhancing perceptual fusion capabilities under different weather scenarios. Finally, it adopts an adaptive fusion strategy to intelligently integrate the temporal model inference results with single-factor prediction results, further improving the robustness and accuracy of the final prediction. Attached Figure Description

[0064] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 The overall flowchart of a multimodal photovoltaic power prediction method based on an improved ViT provided in an embodiment of the present invention is shown below.

[0066] Figure 2 This is an overall framework diagram of a multimodal photovoltaic power prediction system based on an improved ViT, provided as an embodiment of the present invention. Detailed Implementation

[0067] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0068] Example 1, referring to Figure 1 This is one embodiment of the present invention, which provides a multimodal photovoltaic power prediction method based on an improved ViT, comprising:

[0069] S1. Perform serialization preprocessing on the sky image sequence, and use an encoder structure to extract the spatiotemporal features of the preprocessed sky image sequence in order to capture the local dependencies in the sky image sequence.

[0070] S2. Use deep networks to extract high-dimensional features from historical meteorological data, input the historical meteorological data into the memory network LSTM, and obtain single-factor prediction results.

[0071] S3. Cross-modal attention is used to fuse the spatiotemporal features of sky image sequences and the high-dimensional features of historical meteorological data to obtain multimodal features. The multimodal features are then input into the MambaTS network to obtain multimodal prediction results.

[0072] S4. Adaptively fuse the single-factor prediction results and the multimodal prediction results to obtain the final prediction result.

[0073] This invention integrates the spatiotemporal dynamic features of clouds extracted by improved ViT with the high-dimensional patterns of meteorological data mined by deep networks, and utilizes a cross-modal attention mechanism to achieve deep interaction of multimodal information. Finally, by combining the short-term memory advantage of LSTM with the long-range dependency modeling capability of MambaTS, an adaptive weighted fusion strategy is adopted to effectively unify the predictive advantages of short-term local fluctuations and long-term evolution trends, thereby improving the accuracy of photovoltaic power prediction in characterizing spatiotemporal heterogeneity under complex weather conditions and the robustness of long-term prediction.

[0074] Example 2, an embodiment of the present invention, provides a multimode photovoltaic power prediction method based on an improved ViT, based on the previous embodiment, including:

[0075] In this embodiment, the serialization preprocessing in S1 involves segmenting the sky image sequence into block sequences, abstracting the image sequence and each image into natural language. First, a single frame image is divided into 16×16 patches, and then the patches from each image are arranged into a vector according to frame order. Simultaneously, the Transformer encoder in ViT is arranged in two layers: the first layer of n Transformer encoders processes the n frames of the sequence to obtain spatial features, and the second layer of one Transformer encoder captures the long-term dependency information, i.e., the temporal features, contained in the image sequence.

[0076] In one alternative implementation, the serialization preprocessing may involve uniformly dividing a single frame image into 16×16 non-overlapping image patches, and then, according to the temporal order of the image frames, flattening and arranging all image patches of each frame in the sequence into a one-dimensional vector sequence.

[0077] In another alternative implementation, the serialization preprocessing can also employ a sliding window-based overlapping block strategy, dividing the image into 16×16 image blocks with a step size of 8 to preserve richer local spatial context information, and concatenating the sequence of each frame block into an overall input vector in chronological order.

[0078] Furthermore, the serialization preprocessing of the sky image sequence in S1 includes the following steps A1-A3:

[0079] A1. Abstract each frame of the sky image sequence and its corresponding natural language description into natural language statements.

[0080] A2. Divide the single-frame images in the sky image sequence into image blocks.

[0081] A3. Arrange all the image blocks segmented from each frame into a one-dimensional vector according to the time sequence of the image frames.

[0082] In this embodiment, the deep network in S2 is a 3-layer dilated convolution. First, the standardized historical meteorological data sequence is input into the first dilated convolution layer, which uses a convolution kernel of size 3 and a dilation rate of 2 to perform convolution operations to capture short-term local features. Second, the output features are input into the second dilated convolution layer, where the kernel size remains at 3 but the dilation rate is increased to 4 to expand the receptive field and capture medium-range dependencies. Finally, the feature sequence is input into the third dilated convolution layer, where the dilation rate is further increased to 8, thereby obtaining contextual information covering a wider time range without changing the number of parameters, and finally outputting a high-dimensional feature representation.

[0083] In one alternative implementation, the deep network can be a deep residual network (ResNet) composed of multiple stacked residual modules. The specific steps are as follows: First, the input meteorological data is subjected to one-dimensional convolution to increase the feature dimension; then, the features are input into multiple cascaded residual modules. Each module first undergoes a nonlinear transformation through a one-dimensional convolutional layer and a ReLU activation function. Then, the transformation result is directly added to the original input of the module through a shortcut connection to alleviate the gradient vanishing problem in deep network training; finally, all temporal features are integrated through a global pooling layer to form a robust high-dimensional feature representation.

[0084] In another alternative implementation, the deep network can also be a hybrid architecture combining one-dimensional convolutional layers and gated recurrent units (GRUs). The specific steps include: first, using one-dimensional convolutional layers to perform sliding convolutions in the time dimension to efficiently extract local patterns and short-term features from meteorological data; then, inputting the feature sequence output by the convolutional layers into multi-layer gated recurrent units (GRUs), using the gating mechanism of GRUs to selectively memorize and forget information, thereby effectively capturing long-term dynamic dependencies in the data; finally, taking the hidden state of the GRU at the last time step as the comprehensive high-dimensional feature output of the entire sequence.

[0085] Specifically, the extraction of high-dimensional features from historical meteorological data using deep networks in S2 includes the following steps B1-B3:

[0086] B1. Clean and standardize the original historical meteorological data to eliminate noise and dimensional effects.

[0087] B2. Introduce multivariate statistical analysis to quantitatively analyze and characterize the nonlinear dependency structure among variables in historical meteorological data.

[0088] B3. Construct a deep dilated convolutional network to automatically extract high-dimensional spatiotemporal features from preprocessed historical meteorological data.

[0089] In the embodiments of this application, the multivariate statistical analysis in B2, namely the copula function, is specifically implemented as follows: First, the marginal distributions of each variable in historical meteorological data are fitted using methods such as kernel density estimation. Then, an appropriate copula function (such as Gaussian Copula or t-Copula) is selected to model the nonlinear dependency structure between the marginal distributions of the variables, and the weight of each variable's influence on photovoltaic power is accurately calculated accordingly.

[0090] In one alternative implementation, multivariate statistical analysis can be performed by calculating the Pearson correlation coefficient matrix. Specifically, the Pearson correlation coefficient between any two variables in historical meteorological data is directly calculated. This coefficient is obtained by the quotient of the covariance and the standard deviation, and is used to measure the degree of linear correlation between the variables. Weights are then assigned to each variable based on the magnitude of the correlation coefficient.

[0091] In another alternative implementation, multivariate statistical analysis can also be based on mutual information theory for variable analysis. Specifically, the mutual information value between each variable in historical meteorological data and the target photovoltaic power sequence is calculated. A higher mutual information value indicates that the variable contains more information about power, and the variable weights are determined accordingly.

[0092] Furthermore, the standardization preprocessing in B1 includes the following steps B11-B14:

[0093] B11. Perform outlier detection on the historical photovoltaic data sequence in historical meteorological data.

[0094] B12. For detected outliers, cubic spline interpolation is used for repair.

[0095] B13. The same outlier detection and repair methods as those used for photovoltaic historical data should be applied to process historical meteorological data.

[0096] B14. Based on the irradiance values, manually reset the photovoltaic power data corresponding to the period from sunset to sunrise to zero.

[0097] In this embodiment, outlier detection involves preprocessing historical photovoltaic power data. First, outliers are detected and processed to ensure data continuity and stability. Let the historical power data be a sequence P = {p1, p2, ..., p...} T}, where the outlier p1 at a certain moment satisfies the condition ( Using the mean (σ = mean), standard deviation (σ = standard deviation), and a preset threshold (α = α), record the indices of all outliers and remove these outlier data points to form a sequence containing gaps. Based on the remaining normal data points, apply cubic spline interpolation to construct a smooth interpolation curve passing through all normal points, and calculate the interpolation estimate at the outlier locations. Replace the outliers in the original sequence with the interpolation estimate to complete the repair, ensuring the continuity and stability of the data.

[0098] In one alternative implementation, outlier detection can be based on box plot principles for identification and repair using linear interpolation. The specific steps are: calculating the upper quartile Q3, lower quartile Q1, and interquartile range IQR = Q3 - Q1IQR = Q3 - Q1 of the historical power sequence.

[0099] The normal value range is set as [Q1-1.5IQR, Q3+1.5IQR], and data points falling outside this range are judged as outliers.

[0100] After removing outliers, linear interpolation is performed directly between two adjacent normal data points, and the outliers are filled with the values ​​at the corresponding positions on the line connecting the two points.

[0101] In another alternative implementation, outlier detection can also employ an isolated forest unsupervised learning algorithm for identification, and fill the gaps with the mean of preceding and following time windows. The specific steps are as follows:

[0102] The timestamps of the power data are used as features and input along with the power values ​​into the Isolation Forest model for training. The model calculates an anomaly score for each data point, and points with scores higher than a set threshold are identified as outliers. For each outlier, the arithmetic mean of all normal data within a fixed time window before and after it (e.g., 1 hour before and after) is taken and directly replaced with the outlier.

[0103] Furthermore, B2 introduces multivariate statistical analysis to quantitatively analyze and characterize the nonlinear dependency structure among variables in historical meteorological data, including the following steps B21-B22:

[0104] B21. Introduce the copula function to analyze the nonlinear correlation in historical meteorological data and obtain the weights of each variable, weight1-weight5.

[0105] B22. The input sequence is processed using a 3-layer dilated convolution with a fixed kernel size of 3 and a base dilation rate of 2. Using the weights of the k-th layer, the historical data features are represented as W-Feature:

[0106]

[0107] Among them, H (1) [i] represents the value of the first-layer convolutional feature map at position i. The weight parameters of the first convolutional kernel are represented by j1, the subscript j1 represents the index of the first convolutional kernel, the superscript is used to distinguish the layer number, and W[i] represents the value of the input feature map at position i.

[0108] Furthermore, in S3, the spatiotemporal features of the sky image sequence and the high-dimensional features of historical meteorological data are integrated, including aligning and standardizing the sky image sequence with historical photovoltaic data.

[0109] Furthermore, feature extraction of the pre-processed sky image sequence includes the following steps C1-C4:

[0110] C1. Input the preprocessed sky image sequence I into the Modified ViT model to extract features.

[0111] C2, Modified ViT, the first layer extracts spatial features related to the shape, density, and coverage of the cloud layer through multiple Transformer Encoders.

[0112] C3. If the sequence has T images, take the spatial feature vector S-Feature from the output feature vector of the last Encoder:

[0113] S-Feature = Encoder l1-T (I T )

[0114] Among them, Encoder l1-T For the last Transformer Encoder in the first layer, I T Let T be the Tth image, where T represents the total number of frames in the image sequence.

[0115] C4, the second layer of Modified ViT contains a Transformer Encoder, which further processes the features extracted by the T Encoders in the previous layer, and finally outputs a time feature T-Feature that captures the dynamic trend of cloud changes over time.

[0116]

[0117] Among them, Encoder l2 It is the Transformer Encoder for the second layer.

[0118] Furthermore, the historical power data is used to infer single-factor prediction results through an LSTM model, which are represented as follows:

[0119] PV2 = LSTM(P)

[0120] Where P represents historical power data.

[0121] Obtaining the multimodal prediction results includes the following steps D1-D6:

[0122] D1. After dimensional unification, if the input... The dimension is K, and the time step is L.

[0123] D2. Divide x into blocks every s time steps to obtain...

[0124] in, M is x kThe maximum number of blocks after partitioning.

[0125] D3, will Linearly mapped to D-dimensional tokens, we have

[0126] in, It is a weight matrix, where D represents the dimension of the token. Represented as x k The value of the feature at index j after linear mapping.

[0127] D4. For the obtained K×M tokens, perform a scan along the time variable to obtain...

[0128] D5. The tokens are fed into an encoder consisting of N temporal Mamba blocks. To ensure the independence of variables and prevent overfitting, the TMB adds a dropout mechanism after removing the causal convolution before SSM from the original Mamba block. The TMB (Temporal Mamba Block, TMB) is represented as follows:

[0129] h=SSM(Dropout(Linear(z)))+σ(Linear(z))

[0130] Here, SSM stands for State-Space Model Operation, Linear represents Linear Layer Operation, σ represents the activation function, and Dropout represents the regularization operation in deep learning algorithms.

[0131] D6. Perform a linear transformation in the decoder to map the encoder's result h to the prediction result. That is, the multimodal prediction results.

[0132] Furthermore, adaptive fusion is represented as follows:

[0133] PV = PV1 × r + (1 - r) × PV2

[0134] Where r represents the adaptive weight.

[0135] Example 3, referring to Figure 2 This embodiment of the present invention provides a multimodal photovoltaic power prediction system based on an improved ViT, comprising: a data preprocessing module, a sky image spatiotemporal feature extraction module, a historical meteorological data feature extraction module, a single-factor prediction module, a multimodal feature fusion module, and an adaptive result fusion module.

[0136] The data preprocessing module is used to perform serialization preprocessing on the sky image sequence. An encoder structure is used to extract the spatiotemporal features of the preprocessed sky image sequence in order to capture the local dependencies in the sky image sequence.

[0137] The sky image spatiotemporal feature extraction module is used to fuse the spatiotemporal features of sky image sequences and the high-dimensional features of historical meteorological data using cross-modal attention to obtain multimodal features.

[0138] The historical meteorological data feature extraction module is used to extract high-dimensional features from historical meteorological data using deep networks.

[0139] The single-factor prediction module is used to input historical meteorological data into the LSTM memory network to obtain single-factor prediction results.

[0140] The multimodal feature fusion module is used to input multimodal features into the MambaTS network to obtain multimodal prediction results.

[0141] The adaptive result fusion module is used to adaptively fuse single-factor prediction results and multimodal prediction results to obtain the final prediction result.

[0142] This embodiment also provides an electronic device applicable to the multimodal photovoltaic power prediction method based on the improved ViT, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the multimodal photovoltaic power prediction method based on the improved ViT as proposed in the above embodiment.

[0143] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the multimodal photovoltaic power prediction method based on the improved ViT proposed in the above embodiments.

[0144] The storage medium proposed in this embodiment and the multi-mode photovoltaic power prediction method based on improved ViT proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0145] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0146] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multimode photovoltaic power prediction method based on improved ViT, characterized in that: include, The sky image sequence is preprocessed by serialization, and an encoder structure is used to extract the spatiotemporal features of the preprocessed sky image sequence in order to capture the local dependencies in the sky image sequence. High-dimensional features of historical meteorological data are extracted using deep networks, and the historical meteorological data are input into the memory network LSTM to obtain single-factor prediction results. Cross-modal attention is used to fuse the spatiotemporal features of sky image sequences and the high-dimensional features of historical meteorological data to obtain multimodal features. These multimodal features are then input into the MambaTS network to obtain multimodal prediction results. The final prediction result is obtained by adaptively fusing the single-factor prediction results and the multimodal prediction results.

2. The multimode photovoltaic power prediction method based on improved ViT as described in claim 1, characterized in that: The step of performing serialization preprocessing on the sky image sequence includes... Each frame of the sky image sequence and its corresponding natural language description are abstracted into natural language statements. Divide a single frame image in a sky image sequence into image patches; Arrange all the image blocks segmented from each frame into a one-dimensional vector according to the temporal order of the image frames.

3. The multimodal photovoltaic power prediction method based on improved ViT as described in claim 2, characterized in that: The high-dimensional features extracted from historical meteorological data using deep networks include... The raw historical meteorological data is cleaned and standardized to eliminate noise and dimensional effects. Multivariate statistical analysis is introduced to quantitatively analyze and characterize the nonlinear dependency structure among variables in historical meteorological data; A deep dilated convolutional network is constructed to automatically extract high-dimensional spatiotemporal features from preprocessed historical meteorological data.

4. The multimodal photovoltaic power prediction method based on improved ViT as described in claim 3, characterized in that: The standardized preprocessing includes, Outlier detection is performed on historical photovoltaic data sequences in historical meteorological data. For the detected outliers, cubic spline interpolation is used for repair. The same outlier detection and repair methods as those used for photovoltaic historical data were applied to process historical meteorological data. Based on the irradiance values, the photovoltaic power data corresponding to the period from sunset to sunrise is manually set to zero.

5. The multimodal photovoltaic power prediction method based on improved ViT as described in claim 4, characterized in that: The spatiotemporal features of the fused sky image sequence and the high-dimensional features of historical meteorological data include, The sky image sequence was aligned with historical photovoltaic data and standardized. Feature extraction of the pre-processed sky image sequence includes: The preprocessed sky image sequence I is input into the Modified ViT model to extract features; The first layer of Modified ViT extracts spatial features related to the shape, density, and coverage of the cloud layer through multiple Transformer Encoders; If the sequence has T images, the spatial feature is taken from the output feature vector S-Feature of the last Encoder: The second layer of Modified ViT contains a Transformer Encoder, which further processes the features proposed by the T Encoders in the previous layer. The final output captures the dynamic trend of cloud changes over time and is called the time feature T-Feature.

6. The multimodal photovoltaic power prediction method based on improved ViT as described in claim 4, characterized in that: The introduction of multivariate statistical analysis, and the quantitative analysis and characterization of the nonlinear dependency structure among various variables in historical meteorological data, include: The copula function is introduced to analyze the nonlinear correlation in historical meteorological data; The input sequence is processed by dilated convolution to obtain historical data features W-Feature.

7. The multimode photovoltaic power prediction method based on improved ViT as described in claim 4, characterized in that: The historical power data is used to infer single-factor prediction results through an LSTM model, which are represented as follows: PV2 = LSTM(P) Where P represents historical power data; The obtained multimodal prediction results include, After dimensional unification, the input x has a dimension of K and a time step of L. The x is represented as, Divide x into blocks every s time steps to obtain Will Linearly mapped to D-dimensional tokens, we have For the obtained K×M tokens, a scan along the time variable is performed to obtain... The tokens are fed into an encoder consisting of N time-based Mamba blocks, where TMB is represented as: h=SSM(Dropout(Linear(z)))+σ(Linear(z)) A linear transformation is performed in the decoder to map the encoder's result h to the predicted result. That is, multimodal prediction results; The adaptive fusion is expressed as follows: PV = PV1 × r + (1 - r) × PV2 in, M is x k The maximum number of blocks after partitioning It is a weight matrix, where D represents the dimension of the token. Represented as x k The value of the feature after linear mapping at index j, SSM is the state-space model operation, Linear represents the linear layer operation, σ represents the activation function, Dropout represents the regularization operation in the deep learning algorithm, and r represents the adaptive weight.

8. A multimode photovoltaic power prediction system based on improved ViT, using the multimode photovoltaic power prediction method based on improved ViT as described in any one of claims 1 to 7, characterized in that, include: The system includes a data preprocessing module, a sky image spatiotemporal feature extraction module, a historical meteorological data feature extraction module, a single-factor prediction module, a multimodal feature fusion module, and an adaptive result fusion module. The data preprocessing module is used to perform serialization preprocessing on the sky image sequence and to extract the spatiotemporal features of the serialized sky image sequence using an encoder structure in order to capture the local dependencies in the sky image sequence. The sky image spatiotemporal feature extraction module is used to fuse the spatiotemporal features of the sky image sequence and the high-dimensional features of historical meteorological data using cross-modal attention to obtain multimodal features; The historical meteorological data feature extraction module is used to extract high-dimensional features of historical meteorological data using deep networks. The single-factor prediction module is used to input historical meteorological data into the LSTM memory network to obtain single-factor prediction results; The multimodal feature fusion module is used to input multimodal features into the MambaTS network to obtain multimodal prediction results; The adaptive result fusion module is used to adaptively fuse single-factor prediction results and multi-modal prediction results to obtain the final prediction result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the multimodal photovoltaic power prediction method based on the improved ViT as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the multimodal photovoltaic power prediction method based on the improved ViT as described in any one of claims 1 to 7.

Images