A solar power generation power real-time prediction method, system, device and storage medium based on multi-modal deep learning

Abstract

The application provides a solar power generation power real-time prediction method, system, device and storage medium based on multi-modal deep learning. Wherein, the prediction accuracy is improved by fusing the multi-dimensional features of ground surface radiation, equipment temperature and environmental data, the long and short wave components of ground surface solar radiation are subjected to sudden fluctuation decomposition, and the frequency domain sub-modal sequence is extracted; an association model of photovoltaic module temperature field and thermal spot infrared data is constructed to generate a dynamic feature map reflecting the spatial distribution of thermal spot interference; a multi-modal time series prediction model is established to synchronously process the frequency domain features of radiation sub-modal, the spatial correlation of thermal spot feature map and the time series dependence of temperature and humidity data; and the prediction results of each sub-modal are integrated through frequency domain superposition reconstruction technology. The technical scheme provided by the application significantly improves the power prediction accuracy under extreme weather and thermal spot abnormal scene through multi-modal space-time feature joint modeling and dynamic interference decoupling, and simultaneously enhances the dynamic adaptation ability of the prediction system to complex environmental disturbance.

Description

Technical Field

[0001] This application relates to the field of multimodal deep learning technology, and in particular to a method, system, device and storage medium for real-time prediction of solar power generation based on multimodal deep learning. Background Technology

[0002] With the increasing demand for high-precision power forecasting in new energy power systems, real-time forecasting of solar power generation faces multiple challenges: surface solar radiation is affected by sudden cloud changes and aerosol scattering, exhibiting short-term and drastic fluctuations; the hot spot effect of photovoltaic modules leads to localized temperature anomalies, causing nonlinear attenuation of power output; dynamic changes in ambient temperature and humidity are coupled with equipment operating conditions, further increasing the complexity of power fluctuations. It is necessary to construct a real-time forecasting model that takes into account meteorological changes, equipment thermal anomalies, and environmental disturbances to support the coordinated control of grid dispatch and energy storage systems.

[0003] Current mainstream solutions employ time-series forecasting models based on long short-term memory networks, focusing on utilizing historical power generation data and single-modal time-series data such as irradiance and temperature provided by meteorological stations to achieve power prediction through time-series dependency modeling. Some improved solutions introduce convolutional neural networks to extract spatial features from meteorological data, or fuse satellite cloud imagery data at the input layer to enhance the ability to capture radiation abrupt changes.

[0004] The above scheme has three limitations: Modal uniformity: relying on single-point data from meteorological stations or low-resolution satellite cloud images makes it difficult to accurately characterize local abrupt changes in surface radiation components (such as instantaneous fluctuations in shortwave radiation); Lack of thermal anomaly modeling: the lack of integration of equipment-level hotspot monitoring data such as thermal infrared imaging results in the inability to dynamically quantify the power attenuation caused by hotspot effects; Insufficient feature decoupling: the time-series model is insufficient in its ability to model the multimodal correlation between frequency domain abrupt changes (such as high-frequency fluctuations in radiation components), hotspot spatial distribution characteristics, and environmental parameters, leading to a significant increase in prediction errors under complex weather scenarios. Summary of the Invention

[0005] This application provides a method, system, device, and storage medium for real-time prediction of solar power generation based on multimodal deep learning, which solves the problems in the prior art such as inaccurate capture of abrupt changes in surface radiation characteristics due to single modal data, inability to dynamically quantify power attenuation caused by hot spot effect, and significant increase in prediction error in complex scenarios due to insufficient modeling of the correlation between frequency domain and spatial domain multimodal features.

[0006] Firstly, this application provides a real-time prediction method for solar power generation based on multimodal deep learning, including:

[0007] Collect long and short wave component data of solar radiation on the Earth's surface, backsheet temperature data of photovoltaic modules, and ambient temperature and humidity data, and decompose the abrupt fluctuation components in the long and short wave component data into sub-mode sequences.

[0008] Acquire temperature field distribution data of photovoltaic modules, correlate the temperature field distribution data with hot spot area data collected by thermal infrared imager, and generate a hot spot interference feature map by combining the backsheet temperature data.

[0009] The sub-modal sequence, the hot spot interference feature map, and the environmental temperature and humidity data are input into a multi-modal time series prediction model to generate a joint prediction input by fusing the frequency domain features of the sub-modal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multi-modal time series prediction model.

[0010] The multimodal time-series prediction model outputs the prediction results of each sub-mode based on the joint prediction input, and then superimposes the prediction results of each sub-mode in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation prediction curve.

[0011] Optionally, the multimodal time series prediction model outputs prediction results for each submodal based on the joint prediction input, including:

[0012] The frequency domain features of the sub-mode sequence are coupled and mapped with the temporal dependence of the environmental temperature and humidity data to generate a frequency domain-temporal hybrid feature that characterizes the dynamic correlation between irradiance abrupt change and temperature and humidity. The temporal dependence of the temperature and humidity data is quantized by the product of the humidity change rate and the temperature change slope within a sliding time window.

[0013] Based on the spatial correlation distribution of the hot spot interference feature map, the frequency domain-time hybrid feature is divided into transient response features corresponding to the high-frequency sub-mode sequence and steady-state response features corresponding to the low-frequency sub-mode sequence;

[0014] The transient response features are subjected to segmented matching processing based on the duration of irradiance mutation, wherein the start time of each segment is triggered by the moment when the rate of humidity change in the ambient temperature and humidity data exceeds a preset threshold, and the segment length is adjusted synchronously with the update interval of the hot spot interference feature map.

[0015] The steady-state response characteristics are subjected to trend fitting based on the day-night cycle, wherein the time span of the trend fitting is dynamically controlled by the sign flipping frequency of the temperature change slope.

[0016] The transient response features after segmented matching and the steady-state response features after trend fitting are input into a parallel prediction channel. The parallel prediction channel generates high-frequency sub-mode prediction results for the transient response features by waveform morphology matching and generates low-frequency sub-mode prediction results for the steady-state response features by waveform envelope tracking.

[0017] Optionally, the prediction results of each sub-mode are superimposed in the frequency domain according to a preset reconstruction rule to generate a short-term solar power generation prediction curve, including:

[0018] Based on the spatial distribution weight of the temperature gradient values ​​in the hot spot interference feature map, the high-frequency sub-mode prediction results and low-frequency sub-mode prediction results are allocated in a regionalized proportion. The allocation proportion of the high-frequency sub-mode prediction results corresponding to the regions where the temperature gradient values ​​are higher than a preset threshold decreases according to the reciprocal of the humidity change rate.

[0019] The high-frequency submode prediction results and low-frequency submode prediction results are spliced ​​together according to the original frequency band order of the submode sequence. The sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the splicing process to generate a short-term solar power generation prediction curve.

[0020] Optionally, the allocated high-frequency sub-mode prediction results and low-frequency sub-mode prediction results are waveform-stitched according to the original frequency band order of the sub-mode sequence, and the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the stitching process to generate a short-term solar power generation prediction curve, including:

[0021] The high-frequency sub-mode prediction results and the low-frequency sub-mode prediction results are divided into equal-length blocks according to the original arrangement order during sub-mode sequence decomposition, with the high-frequency sub-mode prediction result block first and the low-frequency sub-mode prediction result block second, and the length of each time block is consistent with the update interval of the hot spot interference feature map.

[0022] Within each time block, alignment reference points and matching points are selected based on the sign direction of the temperature change slope to generate position markers for the reference points of the high-frequency submode prediction result block and the matching points of the low-frequency submode prediction result block.

[0023] Align the position marks of the reference point of the high-frequency submode prediction result block and the matching point of the low-frequency submode prediction result block of each time block vertically along the time axis. After alignment, retain all waveform data before the reference point in the high-frequency submode prediction result block and all waveform data after the matching point in the low-frequency submode prediction result block. Directly connect the reference point of the high-frequency submode prediction result block and the matching point of the low-frequency submode prediction result block, with the timestamps at the connection points being continuous and the amplitude values ​​being equal, to generate high-frequency band truncated blocks and low-frequency band truncated blocks.

[0024] By sequentially connecting the high-frequency band cutoff blocks and low-frequency band cutoff blocks of all time blocks in the original frequency band order, a short-term solar power generation prediction curve is generated.

[0025] Optionally, temperature field distribution data of the photovoltaic module is acquired, and the temperature field distribution data is correlated with hot spot area data collected by a thermal infrared imager. A hot spot interference feature map is generated by combining the backsheet temperature data, including:

[0026] Temperature values ​​at various points on the surface of photovoltaic modules are collected by a temperature measuring device to form temperature field distribution data that includes location coordinates and temperature values.

[0027] The hot spot region data on the surface of the photovoltaic module is obtained by a thermal infrared imaging device. Local areas in the hot spot region data that have a significantly higher temperature than the surrounding area and are closed in shape are identified, and the coordinates of the closed boundary and the center coordinates of each local area are recorded.

[0028] The local temperature values ​​that overlap with the closed boundary coordinates of the hot spot region data in the temperature field distribution data are extracted, and the average temperature value and the highest temperature value in each overlapping region are calculated.

[0029] The average temperature value, the highest temperature value, and the backplate temperature data of each overlapping region are arranged in a fixed order to form a temperature feature sequence corresponding to each overlapping region.

[0030] Based on the center coordinates of the hot spot region data, the temperature feature sequence of each overlapping region is mapped to a two-dimensional grid that matches the position coordinates of the temperature field distribution data. Unmapped grid positions are filled with zero values ​​to generate a hot spot interference feature map containing the temperature features of the hot spot region and its position distribution.

[0031] Optionally, a joint prediction input is generated by fusing the frequency domain features of the sub-modal sequences, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multimodal time-series prediction model, including:

[0032] The multimodal time series prediction model decomposes each submodal sequence into components corresponding to multiple frequency intervals, records the frequency interval range of each component and the amplitude value of the frequency interval changing with time, and arranges the frequency interval range and the corresponding amplitude value in a fixed order to form the frequency domain feature data sequence of each submodal.

[0033] Based on the coordinate distribution of non-zero regions in the hot spot interference feature map, the spacing value and relative azimuth angle value between each non-zero region are extracted, and the spacing value and relative azimuth angle value are arranged in a preset order to form the spatial correlation data sequence of the hot spot interference feature map.

[0034] The environmental temperature and humidity data are divided into continuous time windows in chronological order. The direction of change of temperature and humidity values ​​in each time window is statistically analyzed, including whether they are continuously rising, continuously falling, or remaining stable. The direction of temperature and humidity change in each time window is arranged in chronological order to form a time-dependent data sequence of environmental temperature and humidity data.

[0035] The frequency domain feature data sequence, spatial correlation data sequence, and temporal dependence data sequence are sequentially concatenated into a single data sequence as the joint prediction input.

[0036] Optionally, the abrupt fluctuation components in the long and short wave component data are decomposed into sub-mode sequences, including:

[0037] The long and short wave component data are scanned in chronological order to identify target segments. The target segment is a segment whose numerical change exceeds twice the average change of adjacent time periods. The start and end times of the target segment are recorded.

[0038] The data segment located between the start time point and the end time point in the long and short wave component data is extracted as an independent fluctuation component, and multiple independent fluctuation components that are adjacent and have a time interval of less than a preset threshold are merged into a single abrupt fluctuation component.

[0039] Each abrupt fluctuation component is divided into multiple consecutive sub-fluctuation segments according to the time length, and the time span of each sub-fluctuation segment is matched with the duration for which the slope of temperature change in the environmental temperature and humidity data exceeds a preset value.

[0040] The time range of each sub-fluctuation segment is compared with the time period in the environmental temperature and humidity data where the rate of humidity change exceeds a preset threshold. According to the preset overlap ratio, the sub-fluctuation segment is marked as a high-frequency sub-mode sequence and a low-frequency sub-mode sequence.

[0041] All labeled high-frequency sub-mode sequences are arranged in chronological order to form a first set, and low-frequency sub-mode sequences are arranged in chronological order to form a second set. The first set and the second set are then merged into a complete sub-mode sequence.

[0042] Secondly, this application provides a real-time solar power generation prediction system based on multimodal deep learning, comprising:

[0043] The acquisition module collects long and short wave component data of solar radiation on the earth's surface, backsheet temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposes the abrupt fluctuation components in the long and short wave component data into sub-mode sequences.

[0044] The first generation module acquires temperature field distribution data of the photovoltaic module, associates the temperature field distribution data with hot spot area data collected by the thermal infrared imager, and generates a hot spot interference feature map by combining the backsheet temperature data.

[0045] The second generation module inputs the sub-modal sequence, the hot spot interference feature map, and the environmental temperature and humidity data into a multi-modal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the sub-modal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multi-modal time series prediction model.

[0046] The output module outputs the prediction results of each sub-mode based on the joint prediction input through the multi-modal time series prediction model, and performs frequency domain superposition of the prediction results of each sub-mode according to the preset reconstruction rules to generate a short-term solar power generation prediction curve.

[0047] Thirdly, embodiments of this application provide a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a real-time solar power generation prediction method based on multimodal deep learning as described in the first aspect above.

[0048] Fourthly, embodiments of this application provide a computer storage medium storing a computer program, which, when executed by a computer, implements a real-time solar power generation prediction method based on multimodal deep learning as described in the first aspect.

[0049] In this embodiment, by decomposing the abrupt fluctuation components in the long-wave and short-wavelength components of surface solar radiation data, frequency domain sub-mode sequences are extracted to effectively characterize short-term radiation disturbances such as cloud abrupt changes and aerosol scattering, thereby improving the sensitivity and modeling capability for meteorological abrupt changes. A hotspot interference feature map is generated by combining the temperature field distribution of photovoltaic modules with thermal infrared imaging data, dynamically reflecting the spatial distribution of hotspot regions and their nonlinear attenuation effect on power output, avoiding prediction bias caused by local temperature anomalies. Radiation frequency domain characteristics, hotspot spatial correlation, and environmental temperature and humidity temporal dependence are jointly input into the multi-mode prediction model to overcome the limitations of single-mode data and enhance the model's comprehensive characterization capability of the meteorological-equipment-environment coupling effect. By dynamically integrating the prediction results of each sub-mode through frequency domain superposition rules, a coordinated response to multiple time-scale features such as high-frequency radiation abrupt changes, hotspot diffusion, and environmental temperature and humidity evolution is achieved, significantly improving the real-time performance and accuracy of short-term power prediction. By optimizing the entire chain from data decomposition and dynamic feature extraction to multimodal joint modeling, the prediction error problem caused by traditional methods due to single mode, lack of thermal anomaly modeling and insufficient feature decoupling is effectively solved. It has stronger robustness and adaptability in complex meteorological conditions and equipment anomaly scenarios.

[0050] Furthermore, by coupling the frequency domain features of irradiance with the temporal dynamic correlation of temperature and humidity, a hybrid feature distinguishing transient and steady-state responses is generated. Based on the spatial distribution of hot spots, the hybrid feature is divided and differentiated through segmented matching triggered by humidity changes (transient) and day-night trend fitting (steady-state). Finally, parallel prediction channels are used to perform waveform morphology matching on high-frequency transient features and envelope tracking on low-frequency steady-state features, achieving cross-timescale collaborative prediction of radiation abrupt changes and hot spot diffusion. The technical effects are: by decoupling frequency domain-time series coupling modeling with dynamic feature decoupling, the model's ability to synchronously predict short-term radiation abrupt changes (such as cloud cover) and long-period hot spot diffusion is significantly improved; simultaneously, the correlation between dynamic changes in temperature and humidity and equipment thermal anomalies is enhanced, reducing prediction errors in complex meteorological and equipment coupling scenarios.

[0051] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1A flowchart of a real-time solar power generation prediction method based on multimodal deep learning provided in this application is shown;

[0054] Figure 2 A schematic diagram of the structure of a real-time solar power generation prediction system based on multimodal deep learning provided in this application is shown.

[0055] Figure 3 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0057] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0058] Researchers have found significant limitations in existing solar power prediction models: short-term abrupt changes in surface solar radiation (such as sudden changes in shortwave radiation caused by cloud cover) are difficult to capture accurately due to a lack of effective frequency domain decomposition methods; nonlinear power attenuation caused by hot spot effects in photovoltaic modules cannot be dynamically quantified due to the lack of integration with equipment-level thermal infrared monitoring data; and insufficient modeling of the correlation between dynamic changes in environmental temperature and humidity and the multimodal characteristics of radiation and thermal anomalies leads to a sharp increase in prediction errors under complex meteorological scenarios. Based on this, this application provides a real-time solar power prediction method based on multimodal deep learning. This method can significantly improve the prediction accuracy and real-time performance in scenarios where short-term meteorological abrupt changes are coupled with equipment thermal anomalies through frequency domain decomposition of radiation abrupt changes, extraction of dynamic features of hot spot interference, and multimodal spatiotemporal joint modeling. The technical solution of this application is applicable to grid minute-level dispatching and photovoltaic power plant anomaly early warning scenarios.

[0059] The entire R&D process demonstrates the ability to accurately capture short-term radiation disturbances such as cloud cover by decomposing the abrupt fluctuation components in the long and short-wavelength components of solar radiation data at the Earth's surface; to generate dynamic hotspot interference feature maps by combining the temperature field distribution of photovoltaic modules with thermal infrared imaging data, quantifying the nonlinear attenuation effect of local thermal anomalies on power; and to construct a multimodal time-series prediction model that integrates radiation frequency domain features, hotspot spatial correlation, and environmental temperature and humidity time-series dependence, generating high-precision short-term prediction curves through frequency domain superposition and reconstruction. This solution solves the prediction bias problems caused by the single mode, lack of hotspot effect modeling, and insufficient decoupling of multimodal features in traditional models, significantly improving the real-time performance and accuracy of predictions in complex scenarios where meteorological abrupt changes and equipment thermal anomalies are coupled.

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] Figure 1 This application provides a flowchart of a real-time solar power generation prediction method based on multimodal deep learning, as shown in the following embodiment. Figure 1 As shown, the method includes:

[0062] 101. Collect long and short wave component data of solar radiation on the Earth's surface, backsheet temperature data of photovoltaic modules, and ambient temperature and humidity data, and decompose the abrupt fluctuation components in the long and short wave component data into sub-mode sequences.

[0063] Short-wave and long-wave component data refer to real-time monitoring data of short-wave (0.3-3 μm, including visible and ultraviolet light) and long-wave (3-100 μm, mainly infrared) radiation received by the Earth's surface. Abrupt fluctuation components refer to short-term, drastic fluctuations in radiation values ​​caused by meteorological disturbances such as abrupt cloud formations and aerosol scattering. Submode sequences refer to the decomposition of abrupt components in radiation data into multiple independent sub-signals (such as high-frequency, mid-frequency, and low-frequency modes) based on frequency characteristics using signal decomposition algorithms.

[0064] In this embodiment, shortwave and longwave radiometers of specific models are used to collect real-time data on the shortwave and longwave components of solar radiation at the Earth's surface. The shortwave radiometer is a CMP21 model, covering the 0.3 to 3 micrometer band and including visible and ultraviolet radiation monitoring functions; the longwave radiometer is a CGR4 model, covering the 3 to 100 micrometer infrared band. Simultaneously, a PT1000 high-precision sensor collects backsheet temperature data of the photovoltaic modules, while an SHT35 sensor monitors ambient temperature and humidity parameters.

[0065] For abrupt fluctuation components in radiation data, signal decomposition techniques are employed for time-frequency analysis. Specifically, wavelet transform algorithms are used, with Daubechies wavelet basis functions preferred, or the Empirical Mode Decomposition (EMD) algorithm is employed. Power spectral density is calculated using Fast Fourier Transform, and based on the spectral energy distribution characteristics, the non-stationary high-frequency abrupt fluctuation components are decomposed into multiple independent sub-modes. Typical decomposition results include transient sub-modes in the 0.1–1 Hz frequency band and fluctuation sub-modes in the 1–10 Hz frequency band.

[0066] 102. Obtain temperature field distribution data of photovoltaic modules, associate the temperature field distribution data with hot spot area data collected by thermal infrared imager, and generate a hot spot interference feature map by combining the backsheet temperature data.

[0067] Temperature field distribution data refers to the spatial distribution data of the surface temperature of photovoltaic modules, which is acquired by infrared thermal imagers or distributed temperature sensor arrays; hot spot area data refers to the local abnormal high temperature area (temperature difference ≥ 5℃) of the module detected by thermal infrared imagers; hot spot interference feature map refers to a two-dimensional matrix that integrates temperature field distribution, hot spot area coordinates and backsheet temperature, and quantifies the influence weight of hot spots on power.

[0068] In this embodiment, spatial distribution data of the photovoltaic module surface temperature is acquired using a specific type of infrared thermal imaging device. Specifically, a FLIR T840 thermal infrared imager is used, which boasts a high resolution of 640 x 480 pixels and a thermal sensitivity of 0.03 degrees Celsius, enabling precise detection of the module surface temperature distribution. Simultaneously, coordinate mapping technology is used to spatially align the hotspot area image with the temperature field data, constructing a complete temperature distribution matrix.

[0069] Hot spot region is defined as a localized abnormally high temperature area in the module detected by a thermal infrared imager. The criterion for its determination is that the temperature difference between the local area and the surrounding area reaches or exceeds 5 degrees Celsius. Based on temperature field distribution data, hot spot region coordinate information, and backsheet temperature data, a two-dimensional matrix-form hot spot interference feature map is constructed to quantify the impact of the hot spot effect on the output power of the photovoltaic module.

[0070] In the specific implementation process, the temperature difference between the hot spot region and the backplane is first calculated and denoted as ΔT. Simultaneously, the ratio of the hot spot area to the total module area is measured. Based on these two parameters, a dynamic influence weighting formula is used to multiply the hot spot temperature difference by the area ratio, and the backplane temperature change rate is introduced as a correction factor to ultimately generate a hot spot interference feature map. In practical applications, a Kriging space interpolation algorithm is used to fuse discretely distributed temperature sensor data with the hot spot image, generating a temperature field distribution matrix with a resolution of 1 cm. This enables dynamic updating of the weighting coefficients, thereby accurately quantifying the nonlinear attenuation effect of hot spot diffusion on power output.

[0071] 103. Input the sub-modal sequence, the hot spot interference feature map, and the environmental temperature and humidity data into a multi-modal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the sub-modal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multi-modal time series prediction model;

[0072] Frequency domain characteristics refer to the spectral characteristics of submode sequences (such as energy concentration bands and harmonic components); spatial correlation refers to the spatial correlation of weights at different locations in the hot spot interference feature map (such as hot spot diffusion paths); temporal dependence refers to the trend of environmental temperature and humidity data changing over time (such as humidity accumulation effect).

[0073] In this embodiment, the sub-modal sequences processed by signal decomposition, hot spot interference feature maps, and environmental temperature and humidity monitoring data are jointly input into a multimodal time series prediction model. This model achieves feature fusion and joint prediction of multi-source heterogeneous data through deep learning methods.

[0074] First, a short-time Fourier transform was performed on the submode sequence, with an analysis window length of 64 seconds and an overlap rate of 75%, to extract spectral feature parameters. Taking the high-frequency submode obtained in step 101 as an example, this submode has a frequency range of 0.5 to 2 Hz and a fluctuation amplitude of ±50 watts per square meter. After transformation, a frequency domain energy peak of 3.2 kHz was identified, thereby constructing a 64-dimensional feature vector to characterize the spectral characteristics of the radiation fluctuation.

[0075] Secondly, a deep convolutional neural network is used to process the hotspot interference feature map, specifically employing the ResNet-18 network architecture, which has been pre-trained on the ImageNet dataset. In practical applications, the parameters of the first three layers of the network are kept fixed, and only the subsequent layers are fine-tuned during training. For the hotspot feature map with a weight value of 0.58 in step 102, the network outputs a 128-dimensional feature vector, effectively capturing the spatial distribution characteristics of the hotspot spreading southeast at a rate of 0.2 meters per minute.

[0076] Simultaneously, a bidirectional long short-term memory network was constructed to process environmental temperature and humidity data. The network was configured with 32 hidden units, and a 10-minute sliding window was used to analyze time-series changes. For the measured environmental data, the humidity increased from 45% to 60%, and the temperature decreased from 28 degrees Celsius to 25 degrees Celsius. The network output a 32-dimensional feature vector, which included the humidity change rate of 1.5 percentage points per minute and the temperature change trend of 0.3 degrees Celsius per minute.

[0077] Finally, a multi-head attention mechanism with four attention heads was designed, setting the key-value dimension to 64, and dynamically weighting and fusing the three types of features. Specifically, the allocation ratio is 40% for frequency domain features, 35% for spatial features, and 25% for temporal features. Through feature concatenation and dimensionality transformation, a 256-dimensional joint feature vector is generated, fully encoding the combined impact of cloud shading, hot spot diffusion, and drastic temperature and humidity changes on the photovoltaic system. This joint feature vector provides comprehensive and accurate feature input for subsequent power prediction.

[0078] 104. The multimodal time series prediction model outputs the prediction results of each sub-mode based on the joint prediction input, and performs frequency domain superposition of the prediction results of each sub-mode according to the preset reconstruction rules to generate a short-term solar power generation prediction curve.

[0079] Among them, the sub-mode prediction result refers to the predicted value of the future change trend of each frequency band sub-mode; the frequency domain superposition rule refers to reconstructing the sub-mode prediction result into a complete radiation sequence based on inverse wavelet transform or spectrum synthesis algorithm.

[0080] In this embodiment, the multimodal time-series prediction model generates prediction results for each sub-mode based on the joint prediction input, and outputs a short-term solar power generation prediction curve through frequency domain reconstruction technology. The specific implementation process is as follows:

[0081] The prediction model employs a channel-based processing architecture, dividing the 256-dimensional joint feature vector into 128 high-frequency features and 128 low-frequency features via a fully connected layer. The high-frequency prediction channel is processed using a Temporal Convolutional Network (TCN), configured with four layers and dilated kernels of size 3, with dilation coefficients set to 1, 2, 4, and 8 respectively. This branch outputs the predicted radiation fluctuations over the next 15 minutes, specifically a negative peak fluctuation of 80 watts per square meter at the 5th minute and a positive fluctuation of 40 watts per square meter at the 12th minute.

[0082] The low-frequency prediction channel uses the Autoregressive Integral Moving Average (ARIMA) model with parameters set to autoregressive order 2, differencing order 1, and moving average order 1. This model predicts that irradiance will rise to 650 W / m² after 1 hour. The high-frequency and low-frequency sub-mode prediction results are reconstructed using the inverse ensemble mode decomposition algorithm to generate a complete irradiance prediction sequence, with a 5-minute prediction of 480 W / m², a 15-minute prediction of 520 W / m², and a 1-hour prediction of 650 W / m².

[0083] The reconstructed radiation sequence is input into the photovoltaic power conversion model, which is built based on the physical principle of a single diode. The series resistance is set to 0.2 ohms, the parallel resistance to 100 ohms, and the photocurrent to 8.2 amperes. At the same time, a backsheet temperature correction factor of -0.5% per degree Celsius is introduced to compensate for the impact of temperature changes on the output power.

[0084] The final power prediction results show that the current output power is 4.8 MW, which drops to 4.5 MW after 5 minutes due to the combined effects of clouds and hot spots, and recovers to 5.1 MW after 1 hour. The system updates the prediction results every 30 seconds. Comparison with the measured data from the SCADA data acquisition and monitoring system shows that the prediction error is controlled within 2%, meeting the accuracy requirements for engineering applications. This prediction method effectively integrates the multi-dimensional characteristics of radiation fluctuations, hot spot effects, and environmental parameters, achieving high-precision short-term power prediction.

[0085] In some embodiments, the multimodal temporal prediction model outputs prediction results for each submodal based on the joint prediction input, including:

[0086] 201. Couple and map the frequency domain features of the sub-mode sequence with the temporal dependence of the environmental temperature and humidity data to generate a frequency domain-temporal hybrid feature that characterizes the dynamic correlation between irradiance abrupt change and temperature and humidity.

[0087] The temporal dependence of the temperature and humidity data is quantified by multiplying the rate of change of humidity and the slope of change of temperature within a sliding time window.

[0088] Frequency domain features refer to the spectral characteristics extracted from the submode sequence through Fourier transform, including the dominant frequency and harmonic components. Temporal dependence refers to the dynamic correlation of environmental temperature and humidity data over time, such as the rate of humidity change ΔH / Δt and the slope of temperature change ΔT / Δt. Frequency-temporal hybrid features refer to a joint feature vector generated by coupling the frequency domain features of abrupt irradiance changes with the temporal dynamics of temperature and humidity, used to characterize the interaction between meteorological abrupt changes and environmental parameters.

[0089] In this embodiment, the sub-mode sequence is first subjected to a short-time Fourier transform with a window length of 64 seconds and an overlap rate of 75% to extract the spectral energy peak and the main harmonic component. For example, for the 0.5-2Hz fluctuation caused by cloud cover, a peak with 60% energy is detected at 3kHz, and the second harmonic amplitude is identified, thereby quantifying the frequency domain characteristics of the radiation abrupt change.

[0090] Simultaneously, the rate of change of humidity and the slope of change of temperature are calculated within a 10-minute sliding window. For example, the rate of change of humidity is 3% per minute, and the slope of change of temperature is 0.5 degrees Celsius per minute. The time-dependent quantization parameter S is obtained by multiplying the two, and its value is -1.5.

[0091] Subsequently, the 64-dimensional spectral feature vector and the temporal parameter S are input into the fully connected layer and mapped using the ReLU activation function to generate a 128-dimensional hybrid feature vector. When the temporal parameter S is negative, the humidity-related weights in the feature vector are automatically enhanced, making the model more sensitive to the synergistic effect of sudden increases in humidity and sudden drops in temperature in suppressing radiation, thereby generating a frequency-temporal hybrid feature that can characterize the abrupt changes in irradiance and the dynamic relationship between temperature and humidity.

[0092] This process effectively solves the problem of traditional prediction models neglecting multimodal correlations by encoding the interaction between meteorological abrupt changes and environmental parameters.

[0093] 202. Based on the spatial correlation distribution of the hot spot interference feature map, the frequency domain-time hybrid feature is divided into transient response features corresponding to the high-frequency sub-mode sequence and steady-state response features corresponding to the low-frequency sub-mode sequence;

[0094] Spatial correlation distribution refers to the weight of heat diffusion paths in different regions of the hotspot interference feature map; for example, the weight of the hotspot center region is 0.8, and the weight of the edge region is 0.2. Transient response characteristics refer to the short-term disturbance characteristics corresponding to high-frequency submodes, such as fluctuations caused by sudden changes in cloud cover. Steady-state response characteristics refer to the long-term trend characteristics corresponding to low-frequency submodes, such as changes in the daytime radiation baseline.

[0095] In this embodiment, based on the spatial correlation distribution weights of the hotspot interference feature map (e.g., a weight of 0.8 for the central region and 0.2 for the edge region), a specific threshold is set to divide the frequency-time hybrid features into hotspot influence regions associated with high-frequency and low-frequency sub-modes. Specifically, the 128-dimensional hybrid feature vector is dynamically divided into two parts according to the spatial weight distribution: the first 64 dimensions are associated with high-frequency abrupt change regions, such as the hotspot center region, to capture short-term disturbances such as cloud cover, forming transient response features corresponding to the high-frequency sub-mode sequence; the latter 64 dimensions are associated with low-frequency trend regions, such as the hotspot edge region, to model the daytime irradiance baseline change, forming steady-state response features corresponding to the low-frequency sub-mode sequence.

[0096] When a hot spot spreads to a new area, for example, if the weight of a certain area increases from 0.2 to 0.6, the system automatically adjusts the feature partitioning ratio, increasing the transient response feature dimension to 70 dimensions to ensure that feature partitioning remains synchronized with the dynamic spread of the hot spot. This step, through a spatial weight-driven feature decoupling mechanism, effectively enhances the detection sensitivity to local anomalies in photovoltaic modules.

[0097] 203. Perform segmented matching processing on the transient response features based on the duration of irradiance mutation, wherein the start time of each segment is triggered by the moment when the rate of humidity change in the ambient temperature and humidity data exceeds a preset threshold, and the segment length is adjusted synchronously with the update interval of the hot spot interference feature map.

[0098] The duration of irradiance abrupt change characterizes the time range within which events such as cloud cover cause a sudden drop in radiation. The system sets the humidity change rate trigger threshold to 2% per minute; when three consecutive sampling points exceed this threshold, it is determined to be a valid segment start point. The hotspot feature map is updated every 30 seconds, and the segment length is synchronized with it.

[0099] In this embodiment, the system monitors the rate of change of ambient humidity ΔH / Δt in real time. When three consecutive sampling points with a 10-second interval are detected to exceed the threshold of 2% per minute, the system automatically marks the start time of the current segment. Subsequently, using 30-second intervals as the basic segment unit, transient response characteristic data within that time period are extracted, including energy characteristics in the 0.5-2Hz frequency band and hotspot spatial weight data, to ensure that the segmentation processing is synchronized with the spatiotemporal evolution of equipment thermal anomalies.

[0100] The system incorporates a Dynamic Time Warping (DTW) algorithm, which performs similarity matching between the current segmented data and a pre-stored historical waveform library. This library contains templates for typical meteorological events such as cumulus cloud obstruction and dust scattering. The algorithm selects the matching template with the lowest DTW path cost. For example, when a "cumulus cloud obstruction for 4 minutes" template is matched and the path cost does not exceed 0.1, the system predicts the duration of the current irradiance abrupt change. Simultaneously, the system dynamically adjusts the segmentation parameters based on real-time feedback from hotspot diffusion. For instance, when a 10% increase in hotspot area is detected, the segment length is automatically shortened from 30 seconds to 20 seconds, and the system switches to a dedicated "accelerated hotspot diffusion" template.

[0101] 204. Perform trend fitting processing on the steady-state response characteristics based on the day-night cycle, wherein the time span of the trend fitting is dynamically controlled by the sign flip frequency of the temperature change slope.

[0102] The day-night cycle reflects the periodic pattern of rising radiation during the day and returning to zero at night. The frequency of sign reversal of the temperature slope refers to the number of times the direction of temperature change changes, for example, once per hour.

[0103] In this embodiment, the steady-state response characteristics are first decomposed using Fourier series, with a focus on extracting the 24-hour diurnal cycle component and the 12-hour semi-diurnal cycle component. By analyzing these cycle components, the system can accurately resolve the diurnal upward trend of irradiance, such as the typical change pattern of an increase of 50 W / m² per hour, and the natural characteristic of returning to zero at night.

[0104] The system monitors the sign-flipping frequency of the temperature change slope in real time. For example, if two sign-flipping events are detected per hour, the system automatically adjusts the trend fitting time span to a 30-minute window. This dynamic adjustment mechanism ensures that the temperature change trend remains consistent within each analysis window. The steady-state response characteristics are then input into the Prophet prediction model, which comprehensively considers the seasonal periodic component, linear / nonlinear trend terms, and holiday effects such as special weather markers. This model outputs a prediction of baseline irradiance changes within the future time window, such as predicting a linear increase in irradiance from 500 W / m² to 540 W / m² within 30 minutes. The system automatically forces the fitting results to decay to zero after sunset, effectively avoiding erroneous predictions of abnormal irradiance fluctuations at night.

[0105] 205. Input the transient response features after segmented matching and the steady-state response features after trend fitting into the parallel prediction channel, so as to generate high-frequency sub-mode prediction results by waveform morphology matching of the transient response features through the parallel prediction channel, and generate low-frequency sub-mode prediction results by waveform envelope tracking of the steady-state response features.

[0106] Waveform morphology matching predicts high-frequency fluctuation patterns by calculating the similarity between the current waveform and a historical database of abrupt waveform changes, typically using methods such as cosine similarity. Waveform envelope tracking extracts the upper and lower envelopes of low-frequency trends through mathematical transformations to define the possible range of baseline changes, usually achieved using the Hilbert transform.

[0107] In this embodiment, the specific workflow of the parallel prediction channel is as follows: When processing transient response characteristics, the system uses the Dynamic Time Warping (DTW) algorithm to intelligently retrieve the waveform template with the highest matching degree from a pre-built historical meteorological event waveform database. The retrieval process comprehensively considers the current radiation baseline level and dynamically scales the amplitude of the matching template, thereby generating accurate short-term fluctuation prediction results for high-frequency submodes.

[0108] For steady-state response characteristics, the system first performs a Hilbert transform to accurately extract the upper and lower envelopes of the low-frequency trend. These envelopes are then fused with the baseline predictions output by the Prophet model to jointly define the fluctuation range of long-period radiation variations.

[0109] In the final prediction stage, the system scientifically superimposes high-frequency and low-frequency prediction results according to a preset frequency domain energy weighting scheme. A typical weighting is set to 0.6 for the high-frequency component and 0.4 for the low-frequency component. The superimposed comprehensive prediction result is reconstructed using inverse wavelet transform to form a complete radiation variation curve. This curve is then input into the photovoltaic power conversion model, achieving an organic fusion of minute-level fluctuation details and hourly-level trend changes, outputting high-precision multi-timescale power prediction results.

[0110] The multimodal time-series prediction method proposed in this application achieves accurate characterization of the interaction of meteorological parameters by coupling the frequency domain characteristics of irradiance abrupt changes with the time-series dynamics of ambient temperature and humidity. Based on the feature decoupling mechanism of hot spot spatial correlation distribution, it effectively distinguishes between transient disturbances and steady-state trend characteristics. By adopting dynamic segmented matching triggered by humidity changes and trend fitting of temperature slope regulation, it ensures adaptive modeling of irradiance abrupt changes and diurnal cycles. Through waveform morphology matching and envelope tracking technology of parallel prediction channels, it collaboratively generates prediction results at multiple time scales. Finally, the complete radiation curve formed by frequency domain reconstruction significantly improves the response capability of photovoltaic power prediction to complex meteorological conditions and abnormal equipment operating conditions, providing a more reliable decision-making basis for power plant operation and scheduling.

[0111] In some embodiments, the prediction results of each sub-mode are superimposed in the frequency domain according to a preset reconstruction rule to generate a short-term solar power generation prediction curve, including:

[0112] 301. Based on the spatial distribution weight of the temperature gradient values ​​in the hot spot interference feature map, the high-frequency sub-mode prediction results and low-frequency sub-mode prediction results are allocated in a regionalized proportion, wherein the allocation proportion of the high-frequency sub-mode prediction results corresponding to the region where the temperature gradient value is higher than a preset threshold decreases according to the reciprocal of the humidity change rate.

[0113] The spatial distribution weight of temperature gradient values ​​refers to the normalized weight of temperature gradient values ​​(ΔT / Δx) in each region of the hot spot interference feature map, where ΔT represents the temperature difference and Δx represents the spatial distance difference. The temperature gradient value ΔT / Δx is used to characterize the rate of change of the photovoltaic module surface temperature in space, reflecting the spatial intensity of hot spot diffusion. Regionalized proportional allocation refers to the spatial differentiation of high-frequency / low-frequency sub-mode prediction results based on the temperature gradient weights. The proportion of high-frequency predictions in high-temperature gradient regions is reduced to avoid overfitting. The inverse decay of the humidity change rate refers to the dynamic adjustment of the high-frequency sub-mode allocation proportion with the inverse of the humidity change rate (ΔH / Δt), where the high-frequency weight decays when humidity increases sharply.

[0114] In this embodiment, the temperature gradient values ​​(e.g., ΔT / Δx = 0.8℃ / cm) of each region are first extracted from the hot spot interference feature map and normalized into a spatial weight matrix (e.g., 0.9 for high temperature gradient regions and 0.3 for low temperature regions). Regions with gradient values ​​exceeding a preset threshold (e.g., 0.5℃ / cm) are then selected. Subsequently, the reciprocal of the humidity change rate (e.g., ΔH / Δt = 3% / min corresponding to an attenuation coefficient of 0.33) is calculated to obtain the attenuation coefficient. The high-frequency sub-mode prediction results for regions with temperature gradient values ​​higher than the preset threshold are proportionally allocated according to the weight × attenuation coefficient (e.g., 0.9 × 0.33 ≈ 0.3). The low-frequency sub-modes for regions with temperature gradient values ​​lower than the preset threshold are allocated according to the remaining proportion (1 - 0.3 = 0.7). This suppresses the overfitting effect of high-frequency noise on the hot spot region in the scenario of sudden increase in humidity, while preserving the global evolution characteristics of the low-frequency trend.

[0115] 302. The high-frequency sub-mode prediction results and low-frequency sub-mode prediction results after allocation are waveform spliced ​​according to the original frequency band order of the sub-mode sequence, and the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the splicing process to generate a short-term solar power generation prediction curve.

[0116] The original frequency band order refers to the order in which the frequency bands of the sub-mode sequence are arranged (e.g., high frequency, medium frequency, low frequency); waveform phase alignment refers to adjusting the phase offset of the sub-mode waveform according to the sign (positive / negative) of the slope of the temperature change to ensure the physical consistency of the spliced ​​curve.

[0117] In this embodiment, waveform synthesis is achieved by splicing the original frequency bands based on physical constraints: First, the prediction results are arranged in the order of the original frequency bands of the sub-modes (from high frequency to low frequency), and a phase delay (e.g., π / 4 radians) is applied to the high-frequency sub-modes according to the sign of the temperature change slope (e.g., ΔT / Δt=-0.5℃ / min is negative), and the phase of the low-frequency sub-modes is advanced (e.g., π / 8 radians) to match the physical law of "cooling suppresses radiation recovery"; then, the overlap-add algorithm is used to splice the phase-aligned sub-mode waveforms along the time axis. For example, the high-frequency band is superimposed with a -100W / m² fluctuation in the first 5 minutes, and the low-frequency band is superimposed with a +50W / m² trend in the last hour. The discontinuity of the splicing boundary is eliminated by weighted interpolation, and finally a smooth radiation prediction curve that conforms to the laws of hot spot diffusion and temperature and humidity evolution is generated.

[0118] In some embodiments, the allocated high-frequency sub-mode prediction results and low-frequency sub-mode prediction results are waveform-stitched according to the original frequency band order of the sub-mode sequence, and the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the stitching process to generate a short-term solar power generation prediction curve, including:

[0119] 401. Divide the high-frequency sub-mode prediction results and the low-frequency sub-mode prediction results into equal-length blocks according to the original arrangement order when decomposing the sub-mode sequence, wherein the high-frequency sub-mode prediction result block comes first and the low-frequency sub-mode prediction result block comes last, and the length of each time block is consistent with the update interval of the hot spot interference feature map.

[0120] Equal-length blocks refer to dividing the prediction results of high-frequency and low-frequency sub-modes into continuous time periods according to a fixed duration (e.g., 30 seconds); hotspot interference feature map update interval refers to the period of recalculation of hotspot feature maps (e.g., 30 seconds), which determines the length of the time block; original frequency band order arrangement refers to high-frequency sub-mode blocks first and low-frequency blocks last, maintaining the frequency band arrangement rules during decomposition.

[0121] In this embodiment, based on the update interval of the hotspot interference feature map (e.g., 30 seconds), the prediction results of high-frequency sub-modes (e.g., cloud abrupt change waveforms of 0.5-2Hz) and low-frequency sub-modes (e.g., diurnal cycle trends of <0.1Hz) are divided into equal-length blocks of 30 seconds each. High-frequency sub-mode data occupies the first 1 / 3 of each block (0-10 seconds), and low-frequency sub-modes occupy the last 2 / 3 (10-30 seconds). A timestamp calibration algorithm (e.g., sliding window dynamic calibration) ensures strict synchronization of the start times of the high-frequency and low-frequency blocks. If the hotspot update interval changes dynamically (e.g., from 30 seconds to 20 seconds), the time block length is adjusted synchronously to ensure alignment with real-time hotspot data.

[0122] 402. Within each time block, an alignment reference point and a matching point are selected based on the sign direction of the temperature change slope to generate position markers for the reference point of the high-frequency submode prediction result block and the matching point of the low-frequency submode prediction result block.

[0123] The sign of the temperature change slope indicates the trend of temperature change (positive: rising temperature, negative: falling temperature), which determines the selection logic of the alignment reference point; the reference point refers to the reference time point used for alignment within the high-frequency block (such as the start point or peak point of the time block); the matching point is the target time point within the low-frequency block that is aligned with the reference point.

[0124] In this embodiment, the reference point for the high-frequency submode prediction result block is selected based on the sign (positive / negative) of the temperature change slope ΔT / Δt: the starting point (second 0) of the high-frequency submode prediction result block is selected for a positive slope, and the ending point (second 10) is selected for a negative slope. The similarity matrix between the waveform of the low-frequency submode prediction result block and the waveform near the high-frequency reference point (e.g., the 8-10 second drop segment of the high-frequency submode prediction result block) is calculated using the Dynamic Time Warping (DTW) algorithm. The optimal matching point within the low-frequency submode prediction result block (e.g., second 25) is determined using the minimum path cumulative cost (Bellman-Ford algorithm). If the temperature change slope ΔT / Δt exceeds a threshold (e.g., ±0.5℃ / min), the matching point position is dynamically shifted proportionally (e.g., when ΔT / Δt = -1℃ / min, the matching point moves forward by 3 seconds).

[0125] 403. Align the position marks of the reference point of the high-frequency sub-mode prediction result block and the matching point of the low-frequency sub-mode prediction result block of each time block vertically along the time axis. After alignment, retain all waveform data before the reference point in the high-frequency sub-mode prediction result block and all waveform data after the matching point in the low-frequency sub-mode prediction result block. Directly connect the reference point of the high-frequency sub-mode prediction result block and the matching point of the low-frequency sub-mode prediction result block. The timestamps at the connection points are continuous and the amplitude values ​​are equal, thus generating a high-frequency band truncated block and a low-frequency band truncated block.

[0126] Vertical alignment means placing the reference point of the high-frequency submode prediction result block and the matching point of the low-frequency block on the same time axis; the truncation rule means retaining the high-frequency waveform before the reference point and the low-frequency waveform after the matching point, and directly connecting the two points.

[0127] In this embodiment, the reference point (e.g., 10 seconds) of the high-frequency submode prediction result block and the matching point (e.g., 25 seconds) of the low-frequency submode prediction result block are forcibly aligned along the time axis to the same timestamp (25 seconds). The 0-10 seconds data of the high-frequency submode prediction result block are losslessly compressed using a phase vocoder algorithm to maintain the waveform frequency characteristics and generate a high-frequency truncated block. The 10-30 seconds data of the low-frequency submode prediction result block are filled with data in the 25-30 second interval using a linear interpolation algorithm to ensure waveform continuity and generate a low-frequency truncated block. At the connection point (25 seconds), the high-frequency end amplitude (e.g., 350 W / m²) and the low-frequency start amplitude (e.g., 352 W / m²) are linearly interpolated to generate a smooth transition segment (e.g., a 0.4 W / s gradient), meaning that the timestamps at the connection point are continuous and the amplitude values ​​are equal. If the amplitude difference exceeds ±5%, the rematch mechanism is triggered: return to step 402 to reselect the matching point and adjust the alignment.

[0128] 404. Connect the high-frequency band cutoff blocks and low-frequency band cutoff blocks of all time blocks in series according to the original frequency band order to generate a short-term solar power generation prediction curve.

[0129] Original frequency band sequential concatenation refers to splicing all time blocks in the order of high frequency first and low frequency last.

[0130] In this embodiment, within each 30-second time block, the high-frequency band (0-25 seconds) and the low-frequency band (25-30 seconds) are seamlessly stitched together using a sliding window overlap and addition algorithm in their original frequency band order. Multiple blocks are concatenated into a complete curve in chronological order (e.g., a 30-minute prediction consists of 60 blocks), and a timestamp compensation algorithm (e.g., linear interpolation) is used to eliminate minor inter-block errors (e.g., ±0.1 seconds). Finally, an energy conservation check is performed on the complete curve (e.g., total energy integral deviation ≤2%). If the deviation exceeds this limit, local re-stitching is triggered (returning to step 401 to re-segment the blocks), ensuring the prediction result is physically reasonable.

[0131] This solution addresses waveform breakage and phase jump issues in traditional frequency domain superposition through a temperature slope-driven phase alignment and time block truncation splicing mechanism: dynamically matching the reference point and the matching point eliminates waveform shape conflicts caused by temperature changes; vertical alignment and truncation rules ensure time continuity and smooth amplitude transition; it adapts to real-time interference scenarios such as hot spot diffusion and meteorological changes, improving the physical rationality and engineering practicality of the prediction curve, and providing high-reliability power prediction support for photovoltaic power plants.

[0132] In some embodiments, temperature field distribution data of the photovoltaic module is acquired, and the temperature field distribution data is correlated with hot spot area data collected by a thermal infrared imager. A hot spot interference feature map is then generated by combining the backsheet temperature data, including:

[0133] 501. Collect temperature values ​​at various points on the surface of the photovoltaic module using a temperature measuring device to form temperature field distribution data that includes location coordinates and temperature values;

[0134] Temperature measuring devices refer to distributed temperature sensor arrays or infrared thermometers used to collect temperature values ​​at various points on the surface of photovoltaic modules; temperature field distribution data refers to matrix data containing the position coordinates (e.g., x, y) of each measuring point on the surface of the photovoltaic module and the corresponding temperature value.

[0135] In this embodiment, a distributed temperature sensor (such as DS18B20) installed on the backsheet or frame of the photovoltaic module collects surface temperature at a grid density of 10cm×10cm, recording the coordinates (x,y) and temperature value T of each grid point; the collected temperature data is organized into a two-dimensional matrix, with row / column indices corresponding to position coordinates, matrix element values ​​being temperature values, and missing positions being filled by interpolation (such as bilinear interpolation); a complete temperature field distribution data matrix is ​​generated with a resolution of 10cm / pixel and a temperature accuracy of ±0.5℃.

[0136] 502. Acquire hot spot area data on the surface of photovoltaic modules using a thermal infrared imaging device, identify local areas in the hot spot area data that have a significantly higher temperature than the surrounding area and are closed in shape, and record the closed boundary coordinates and center coordinates of each local area.

[0137] Thermal infrared imaging devices refer to infrared thermal imagers (such as FLIR A655sc) used to acquire temperature distribution images of the surface of photovoltaic modules; hot spot area data refers to abnormally high temperature areas identified from infrared images that are significantly higher than the surrounding area (ΔT≥5℃) and have a closed shape.

[0138] In this embodiment, infrared images of the photovoltaic module surface are captured at 30-second intervals using a thermal infrared imaging device (resolution 640×480 pixels, temperature sensitivity 0.05℃). Gaussian filtering is applied to the images for noise reduction. The Canny edge detection algorithm is used to extract local areas in the hot spot region data that are significantly hotter than the surrounding area and have a closed shape. Adjacent high-temperature pixels are merged using a region growing algorithm, and closed regions with an area ≥ 4cm² and ΔT ≥ 5℃ are selected. The coordinates of the vertices of the closed boundary polygon of each local region (such as the four corner points of a rectangular region) and the coordinates of the center point (centroid calculation) are recorded.

[0139] 503. Extract the local temperature values ​​that overlap with the closed boundary coordinates of the hot spot region data in the temperature field distribution data, and calculate the average temperature value and the highest temperature value in each overlapping region.

[0140] Overlapping region: The local region in the temperature field distribution data that overlaps with the boundary coordinates of the hot spot; Temperature characteristic values: the average temperature (T_avg) and the maximum temperature (T_max) within the overlapping region.

[0141] In this embodiment, the coordinates of the closed boundary vertices of the hot spot region data (infrared image coordinate system) are transformed to the grid coordinate system of the temperature field distribution data (e.g., through affine transformation); a sub-matrix (e.g., a 5×5 grid region) overlapping with the hot spot boundary is extracted from the temperature field distribution data matrix; the average value T_avg of the effective temperature points within the sub-matrix is ​​calculated (ignoring 0-value interpolation points), and the maximum value of the sub-matrix is ​​taken as T_max, thus calculating the average temperature value and the highest temperature value within each overlapping region.

[0142] 504. Arrange the average temperature value, the highest temperature value and the back plate temperature data of each overlapping region in a fixed order to form a temperature feature sequence corresponding to each overlapping region;

[0143] Backplane temperature data refers to the average temperature value (T_back) collected by the PT1000 sensor installed on the backplane of the module; the temperature feature sequence refers to a one-dimensional array arranged in a fixed order of [T_avg, T_max, T_back].

[0144] In this embodiment, the average temperature (T_avg) and maximum temperature (T_max) of the hot spot overlap region obtained in step 503 are used, for example, T_avg = 58.1℃ and T_max = 60.5℃; the weighted average value of the backplane temperature sensor network (e.g., 4 PT1000s arranged at the four corners of the component) is taken as the backplane temperature data T_back (e.g., 42.3℃), and the weights are inversely distributed according to the distance between the sensor and the center of the hot spot; a one-dimensional feature sequence is generated in a fixed order of [average temperature of hot spot overlap region T_avg, maximum temperature T_max, backplane temperature data T_back]; and a temperature feature sequence corresponding to each overlap region is formed.

[0145] 505. Based on the center coordinates of the hot spot region data, map the temperature feature sequence of each overlapping region to a two-dimensional grid that matches the position coordinates of the temperature field distribution data. Fill the unmapped grid positions with zero values ​​to generate a hot spot interference feature map containing the temperature features of the hot spot region and its position distribution.

[0146] Two-dimensional grid mapping refers to mapping the temperature feature sequence to a grid with the same resolution as the temperature field distribution data according to the hot spot center coordinates; hot spot interference feature map refers to a three-dimensional matrix, the first two dimensions are position coordinates, and the third dimension is the temperature feature channel ([T_avg, T_max, T_back]).

[0147] In this embodiment, the center coordinates of the hot spot region data are transformed from the infrared image coordinate system to a two-dimensional grid coordinate system consistent with the position coordinates of the temperature field distribution data, and the position of the center grid is determined by rounding. The [T_avg, T_max, T_back] sequence of each hot spot region data is written into channels 1 / 2 / 3 of the corresponding center grid, and the influence radius r is calculated according to the actual coverage range (boundary coordinates) of the hot spot region data. The neighboring grids within the radius are filled with feature values ​​by distance weighting. The grids not covered by the hot spot region data keep the value zero in channel 1 / 2, and channel 3 is filled with the measured back plate temperature value at that position (if there is no data, interpolation is used). The physical rationality of T_max≥T_avg≥T_back is checked, and the isolated non-zero grids are subjected to morphological dilation processing. Finally, a normalized hot spot interference feature map containing the temperature characteristics and position distribution of the hot spot region is output. At the same time, statistical indicators such as the number of hot spots and the maximum temperature rise are recorded to provide structured input for subsequent spatial feature analysis.

[0148] This solution automates the entire process of hot spot detection and feature extraction. Through multi-source temperature data fusion and spatial feature structuring, it can accurately identify and locate abnormal hot spot regions on the module surface, comprehensively characterizing the temperature characteristics of the hot spots and their impact on the module's condition. The generated standardized feature map can be directly used for subsequent analysis and prediction, providing reliable data support for fault diagnosis and operation and maintenance decisions in photovoltaic power plants, significantly improving the efficiency and accuracy of hot spot monitoring.

[0149] In some embodiments, the multimodal temporal prediction model fuses the frequency domain features of the submodal sequences, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data to generate a joint prediction input, including:

[0150] 601. The multimodal time series prediction model decomposes each submodal sequence into components corresponding to multiple frequency intervals, records the frequency interval range of each component and the amplitude value of the frequency interval changing with time, and arranges the frequency interval range and the corresponding amplitude value in a fixed order to form the frequency domain feature data sequence of each submodal.

[0151] Frequency range refers to the submode sequence being decomposed into different frequency bands through Fourier transform; amplitude value: the amplitude intensity of the signal in each frequency band within the time window.

[0152] In this embodiment, the multimodal time-series prediction model first adaptively divides each input sub-mode sequence (e.g., 0.5-2Hz fluctuation data of a high-frequency sub-mode) into frequency intervals. An improved wavelet packet decomposition algorithm (based on the db4 wavelet basis) is then used to decompose the signal into multiple key frequency intervals (e.g., for the 0.5-2Hz high-frequency sub-mode, it can be decomposed into three intervals: 0.5-1Hz, 1-1.5Hz, and 1.5-2Hz; the specific number of intervals and their boundaries are dynamically determined based on the spectral energy distribution). The boundaries of each interval are dynamically adjusted by the model according to the current signal's spectral energy distribution. Then, a Hilbert transform is performed on the signal components within each frequency interval to extract their time-varying amplitude envelopes. The maximum amplitude value within the interval and its corresponding timestamp are recorded (e.g., a peak amplitude of 25W / m² occurs in the 1-1.5Hz interval at t=15s). Finally, the data is organized according to a fixed format of [lower interval limit, upper interval limit, peak amplitude 1, timestamp 1, peak amplitude 2, timestamp 2...] to form a structured frequency domain feature data sequence.

[0153] 602. Based on the coordinate distribution of the non-zero regions in the hot spot interference feature map, extract the spacing value and relative azimuth angle value between each non-zero region, and arrange the spacing value and relative azimuth angle value in a preset order to form the spatial correlation data sequence of the hot spot interference feature map;

[0154] Non-zero regions refer to grid regions where T_avg>0 in the hot spot interference feature map; Spacing value: Euclidean distance between the center points of two hot spots (e.g., grid unit distance 5); Azimuth angle refers to the angle between the line connecting the center points of the hot spots with the lower left corner of the module as the origin and the horizontal axis (e.g., 30°); Relative azimuth angle value refers to the angle between the line connecting the center points of two hot spots and the reference direction with the preset reference direction on the surface of the photovoltaic module (e.g., the horizontal direction or the long side direction of the module) as the reference, used to quantify the spatial orientation relationship between hot spots.

[0155] In this embodiment, all non-zero grid center coordinates (e.g., (18,23), (45,60)) are extracted from the hotspot interference feature map. The Euclidean distance (e.g., √[(45-18)²+(60-23)²]=46) and relative azimuth angle (atan2(37,27)=53.8°) for each pair of hotspots are calculated. The spatial topological relationship of the hotspots is established through Delaunay triangulation. After removing abnormal isolated points, the spatial correlation data sequence of the hotspot interference feature map is generated in the order of [hotspot1x, hotspot1y, hotspot2x, hotspot2y, spacing, angle...], which quantifies the spatial distribution pattern of the hotspot cluster.

[0156] 603. Divide the environmental temperature and humidity data into continuous time windows in chronological order, and statistically analyze the direction of change of temperature and humidity values ​​in each time window, whether they are continuously rising, continuously falling, or remaining stable. Arrange the direction of temperature and humidity change in each time window in chronological order to form a time-dependent data sequence of environmental temperature and humidity data.

[0157] Direction of change: Temperature / humidity exhibits monotonicity (increasing, decreasing, or remaining stable) within a time window; Time window: a fixed duration segment (e.g., 5 minutes).

[0158] In this embodiment, the environmental temperature and humidity data are divided into 5-minute non-overlapping time windows, and the linear regression slope of temperature / humidity within each window is calculated. The direction of temperature change is defined as follows: slope > 0.1℃ / min indicates an increase, slope < -0.1℃ / min indicates a decrease, otherwise it remains unchanged. The direction of humidity change is also defined as follows: slope > 0.1% / min indicates an increase, slope < -0.1% / min indicates a decrease, otherwise it remains unchanged. "Sudden increase" and "Sudden decrease" labels are added for abrupt temperature and humidity changes (e.g., ΔH / Δt > 2% / min). Finally, the time-dependent data sequence of environmental temperature and humidity data is output in [window 1 temperature direction, humidity direction, window 2...] time order, encoding the temporal evolution law of meteorological parameters.

[0159] 604. The frequency domain feature data sequence, spatial correlation data sequence, and temporal dependence data sequence are sequentially concatenated into a single data sequence as the joint prediction input.

[0160] Length alignment refers to padding or truncation of three sequences to make them of the same length (e.g., uniformly 100 dimensions); splicing: merging the [frequency domain sequence, spatial sequence, time series] into a 300-dimensional vector; normalization refers to performing Min-Max normalization on each modality data to eliminate dimensional differences.

[0161] In this embodiment, the frequency domain feature data sequence, spatial correlation data sequence, and temporal dependency data sequence are standardized in length: the frequency domain feature data sequence is padded with zeros to 150 dimensions, the spatial correlation data sequence is truncated to 100 dimensions, and the temporal dependency data sequence is interpolated to 50 dimensions. Weighted concatenation is performed using attention weight allocation (frequency domain:spatial:temporal = 4:3:3) to generate a 300-dimensional joint input vector. Layer normalization (LayerNorm) is used to eliminate intermodal dimensional differences, and the final output is a single data sequence that can be directly input into multimodal prediction models such as LSTM / Transformer as a joint prediction input.

[0162] This solution achieves accurate condition monitoring and power prediction for photovoltaic power generation systems through deep multimodal data fusion technology. The system innovatively combines frequency domain feature analysis, hotspot spatial distribution characteristics, and temporal variation characteristics of environmental parameters to construct a multi-dimensional joint prediction model. At the technical implementation level, the solution accurately captures the frequency domain characteristics of radiation fluctuations through advanced signal processing algorithms, combines spatial correlation analysis of hotspot distribution, and integrates the temporal evolution patterns of environmental parameters to form a complete feature representation system. This solution significantly improves the prediction robustness under complex meteorological conditions and equipment anomalies, effectively supporting refined grid scheduling decisions. Simultaneously, the system employs a high-efficiency computing architecture for real-time data processing, providing an integrated solution for photovoltaic power plant operation and maintenance from anomaly detection to power prediction, demonstrating significant engineering application value.

[0163] In some embodiments, the abrupt fluctuation components in the long and short wave component data are decomposed into sub-mode sequences, including:

[0164] 701. Scan the long and short wave component data in chronological order to identify the target segment, which is a segment whose numerical change exceeds twice the average change of adjacent time periods, and record the start and end time points of the target segment.

[0165] The change amplitude threshold is twice the average change amplitude of adjacent time periods, used to determine data mutations; the segment marker is used to record the start and end time points of the mutation.

[0166] In this embodiment, a sliding window algorithm (30-second window) is used to scan the long and short wave component data in real time. When a change in value at a certain moment exceeds 200% of the average change in adjacent time periods, it is marked as the start time of the mutation. The algorithm continues to track until the change falls below a threshold, which is then determined as the endpoint. Finally, a list containing all mutation start and end times (e.g., 10:15:00-10:15:30) is output. A dynamic threshold mechanism is used to avoid missing weak mutations by a fixed threshold, and the peak amplitude and duration of the mutation are recorded simultaneously.

[0167] 702. Extract the data segment between the start time point and the end time point in the long and short wave component data as an independent fluctuation component, and merge multiple independent fluctuation components that are adjacent and have a time interval of less than a preset threshold into a single abrupt fluctuation component.

[0168] Independent fluctuation components refer to data segments within a single mutation region; the merging threshold refers to merging adjacent mutations into the same component when the interval between them is less than 1 minute.

[0169] In this embodiment, based on the start and end time markers of abrupt changes at the start and end times in the long and short wave component data described in step 701, independent fluctuation components are extracted from the original data, and the interval between adjacent components is checked: if the interval is less than a preset merging threshold (default 1 minute), multiple independent fluctuation components are merged into a single composite component (e.g., 10:15:00-10:15:30 and 10:15:50-10:16:30 are merged into 10:15:00-10:16:30). The merging process preserves the amplitude characteristics of each sub-component and outputs the integrated abrupt change component time series.

[0170] 703. Divide each abrupt fluctuation component into multiple consecutive sub-fluctuation segments according to the time length, and match the time span of each sub-fluctuation segment with the duration for which the temperature change slope in the environmental temperature and humidity data exceeds a preset value.

[0171] A sub-fluctuation segment refers to a segment in which the abrupt change component is divided into equal segments whose duration matches the temperature and humidity changes; the matching rule is that the duration of the sub-segment equals the duration of the significant temperature and humidity changes.

[0172] In this embodiment, based on the duration (e.g., 2 minutes) of a temperature change slope > 0.5℃ / min or a humidity change rate > 1% / min in the environmental temperature and humidity data, each abrupt fluctuation component is divided into several sub-segments according to the time length. An adaptive equal division algorithm is used to ensure that the duration of the sub-fluctuation segment strictly matches the temperature and humidity change time period in the environmental temperature and humidity data (error < ±5 seconds), and a sub-segment segmentation list with timestamps is output.

[0173] 704. Compare the time range of each sub-fluctuation segment with the time period in the environmental temperature and humidity data where the rate of humidity change exceeds a preset threshold, and mark the sub-fluctuation segment as a high-frequency sub-mode sequence and a low-frequency sub-mode sequence according to a preset overlap ratio.

[0174] The overlap ratio threshold refers to marking a sub-segment as high frequency when it overlaps with a period of sudden temperature and humidity changes by ≥60%, otherwise it is marked as low frequency.

[0175] In this embodiment, the overlap ratio between each sub-fluctuation segment and the period of abrupt change in environmental temperature and humidity data is calculated: when the time overlap is ≥60%, it is marked as a high-frequency sub-mode (e.g., 10:15:00-10:15:40); otherwise, it is marked as a low-frequency sub-mode. The reliability of the marking is verified using sliding correlation analysis, and abnormal markings are removed (segments with a correlation coefficient <0.6 need to be re-evaluated). The high-frequency sub-mode sequence and the low-frequency sub-mode sequence are then output with classification labels.

[0176] 705. Arrange all the marked high-frequency sub-mode sequences in chronological order to form a first set, arrange the low-frequency sub-mode sequences in chronological order to form a second set, and merge the first set and the second set into a complete sub-mode sequence.

[0177] Set generation refers to sorting the high-frequency and low-frequency sub-modes by time to form two independent sets; sequence merging refers to merging the two sets by interleaving them along the original time axis while retaining the label information.

[0178] This step first arranges the labeled high-frequency sub-mode sequences (e.g., 10:15:00-10:15:40, 10:18:20-10:19:10) in chronological order to form the first set, and simultaneously arranges the low-frequency sub-mode sequences (e.g., 10:16:20-10:16:30, 10:19:30-10:20:00) in chronological order to form the second set; then, the two sets are merged according to the original data time sequence using a time axis alignment algorithm, retaining all sub-mode labeling information (high-frequency / low-frequency). The system uses frequency and time range markers to apply a high-frequency priority processing rule to overlapping time periods. Finally, it generates a submodal sequence in a unified format (including start and end times and type markers) and outputs statistical information for the two sets (number, total duration, and average amplitude of the high-frequency set, and number, total duration, and average rate of change of the low-frequency set). At the same time, it performs interpolation compensation for discontinuous timestamp intervals and records conflict resolution logs to ensure that the output submodal sequence maintains temporal continuity and clearly distinguishes set attributes, providing structured input for subsequent multimodal analysis.

[0179] This solution achieves three major technological breakthroughs through an innovative five-step decomposition process: First, it employs a dynamic relative threshold (twice the mean) mutation detection mechanism, improving the mutation recognition rate compared to traditional fixed threshold methods. Second, it guides sub-segment division through the physical correlation of temperature and humidity changes, ensuring that the decomposition results conform to actual meteorological change patterns. Finally, the clearly distinguished high / low frequency sub-mode sets provide clearly characterized input data for the prediction model. Practical applications show that this solution improves the accuracy of radiation fluctuation prediction under cloudy weather conditions, while supporting minute-level real-time response capabilities. The decomposition results also intuitively reflect cloud movement characteristics, providing important decision-making basis for cloud shadow prediction of photovoltaic power plants.

[0180] Figure 2 This application provides a schematic diagram of the structure of a real-time solar power generation prediction system based on multimodal deep learning, as an embodiment of the present application. Figure 2 As shown, the system includes:

[0181] The acquisition module 21 acquires long and short wave component data of solar radiation on the earth's surface, backsheet temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposes the abrupt fluctuation components in the long and short wave component data into sub-mode sequences.

[0182] The first generation module 22 acquires temperature field distribution data of the photovoltaic module, associates the temperature field distribution data with hot spot area data collected by the thermal infrared imager, and generates a hot spot interference feature map by combining the backsheet temperature data.

[0183] The second generation module 23 inputs the sub-modal sequence, the hot spot interference feature map, and the environmental temperature and humidity data into the multi-modal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the sub-modal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multi-modal time series prediction model.

[0184] The output module 24 outputs the prediction results of each sub-mode based on the joint prediction input through the multi-mode time series prediction model, and performs frequency domain superposition of the prediction results of each sub-mode according to the preset reconstruction rules to generate a short-term solar power generation prediction curve.

[0185] Figure 2 The aforementioned real-time solar power generation prediction system based on multimodal deep learning can perform... Figure 1 The implementation principle and technical effects of the real-time solar power generation prediction method based on multimodal deep learning described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit performs its operations in the real-time solar power generation prediction system based on multimodal deep learning described in the above embodiments have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0186] In one possible design, Figure 2 The real-time solar power generation prediction system based on multimodal deep learning in the illustrated embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0187] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0188] The processing component 32 is used for the above Figure 1 The embodiment describes a method for real-time prediction of solar power generation based on multimodal deep learning.

[0189] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0190] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0191] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0192] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0193] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0194] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0195] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a method for real-time prediction of solar power generation based on multimodal deep learning.

[0196] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0197] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0198] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A real-time prediction method for solar power generation based on multimodal deep learning, characterized in that, include: Collect long and short wave component data of solar radiation on the Earth's surface, backsheet temperature data of photovoltaic modules, and ambient temperature and humidity data, and decompose the abrupt fluctuation components in the long and short wave component data into sub-mode sequences. Acquire temperature field distribution data of photovoltaic modules, correlate the temperature field distribution data with hot spot area data collected by thermal infrared imager, and generate a hot spot interference feature map by combining the backsheet temperature data. The sub-modal sequence, the hot spot interference feature map, and the environmental temperature and humidity data are input into a multi-modal time series prediction model to generate a joint prediction input by fusing the frequency domain features of the sub-modal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multi-modal time series prediction model. The multimodal time series prediction model outputs the prediction results of each sub-mode based on the joint prediction input, and superimposes the prediction results of each sub-mode in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation prediction curve. The step of generating a joint prediction input by fusing the frequency domain features of the sub-modal sequences, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multimodal temporal prediction model includes: The multimodal time-series prediction model converts the frequency domain features of the submodal sequences, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data into data sequences, and then concatenates them in sequence as a joint prediction input. The process of generating a short-term solar power generation prediction curve by outputting the prediction results of each sub-mode based on the joint prediction input using the multi-modal time-series prediction model, and then superimposing the prediction results of each sub-mode in the frequency domain according to a preset reconstruction rule, includes: The frequency domain features of the sub-mode sequence in the joint prediction input are coupled with the temporal dependence of the environmental temperature and humidity data to generate a frequency-temporal hybrid feature. The temporal dependence of the temperature and humidity data is quantized by the product of the humidity change rate and the temperature change slope within a sliding time window. Based on the spatial correlation of the hot spot interference feature map, the frequency-temporal hybrid feature is divided into transient response features and steady-state response features. After the transient response features and steady-state response features are processed by segmented matching and trend fitting, respectively, high-frequency sub-mode prediction results and low-frequency sub-mode prediction results are generated through parallel prediction channels. Based on the spatial distribution weight of the temperature gradient values ​​in the hot spot interference feature map, the high-frequency sub-mode prediction results and low-frequency sub-mode prediction results are allocated in a regionalized proportion. The allocation proportion of the high-frequency sub-mode prediction results corresponding to the regions where the temperature gradient values ​​are higher than a preset threshold decreases according to the reciprocal of the humidity change rate. The high-frequency submode prediction results and low-frequency submode prediction results are spliced ​​together according to the original frequency band order of the submode sequence. The sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the splicing process to generate a short-term solar power generation prediction curve.

2. The method according to claim 1, characterized in that, The multimodal time-series prediction model outputs prediction results for each submodal based on the joint prediction input, including: The frequency domain features of the sub-mode sequence are coupled and mapped with the temporal dependence of the environmental temperature and humidity data to generate a frequency domain-temporal hybrid feature that characterizes the dynamic correlation between irradiance abrupt change and temperature and humidity. The temporal dependence of the temperature and humidity data is quantized by the product of the humidity change rate and the temperature change slope within a sliding time window. Based on the spatial correlation distribution of the hot spot interference feature map, the frequency domain-time hybrid feature is divided into transient response features corresponding to the high-frequency sub-mode sequence and steady-state response features corresponding to the low-frequency sub-mode sequence; The transient response features are subjected to segmented matching processing based on the duration of irradiance mutation, wherein the start time of each segment is triggered by the moment when the rate of humidity change in the ambient temperature and humidity data exceeds a preset threshold, and the segment length is adjusted synchronously with the update interval of the hot spot interference feature map. The steady-state response characteristics are subjected to trend fitting based on the day-night cycle, wherein the time span of the trend fitting is dynamically controlled by the sign flipping frequency of the temperature change slope. The transient response features after segmented matching and the steady-state response features after trend fitting are input into the parallel prediction channel. The transient response features are then used to generate high-frequency sub-mode prediction results by waveform morphology matching, and the steady-state response features are used to generate low-frequency sub-mode prediction results by waveform envelope tracking.

3. The method according to claim 1, characterized in that, The high-frequency sub-mode prediction results and low-frequency sub-mode prediction results are then stitched together according to the original frequency band order of the sub-mode sequence. During the stitching process, the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment, generating a short-term solar power generation prediction curve, including: The high-frequency sub-mode prediction results and the low-frequency sub-mode prediction results are divided into equal-length blocks according to the original arrangement order when the sub-mode sequence is decomposed, with the high-frequency sub-mode prediction result block first and the low-frequency sub-mode prediction result block second, and the length of each time block is consistent with the update interval of the hot spot interference feature map. Within each time block, alignment reference points and matching points are selected based on the sign direction of the temperature change slope to generate position markers for the reference points of the high-frequency submode prediction result block and the matching points of the low-frequency submode prediction result block. Align the position marks of the reference point of the high-frequency submode prediction result block and the matching point of the low-frequency submode prediction result block of each time block vertically along the time axis. After alignment, retain all waveform data before the reference point in the high-frequency submode prediction result block and all waveform data after the matching point in the low-frequency submode prediction result block. Directly connect the reference point of the high-frequency submode prediction result block and the matching point of the low-frequency submode prediction result block, with the timestamps at the connection points being continuous and the amplitude values ​​being equal, to generate high-frequency band truncated blocks and low-frequency band truncated blocks. By sequentially connecting the high-frequency band cutoff blocks and low-frequency band cutoff blocks of all time blocks in the original frequency band order, a short-term solar power generation prediction curve is generated.

4. The method according to claim 1, characterized in that, Acquire temperature field distribution data of photovoltaic modules, correlate the temperature field distribution data with hot spot area data collected by a thermal infrared imager, and generate a hot spot interference feature map by combining the backsheet temperature data, including: Temperature values ​​at various points on the surface of photovoltaic modules are collected by a temperature measuring device to form temperature field distribution data that includes location coordinates and temperature values. The hot spot region data on the surface of the photovoltaic module is obtained by a thermal infrared imaging device. Local areas in the hot spot region data that have a significantly higher temperature than the surrounding area and are closed in shape are identified, and the coordinates of the closed boundary and the center coordinates of each local area are recorded. The local temperature values ​​that overlap with the closed boundary coordinates of the hot spot region data in the temperature field distribution data are extracted, and the average temperature value and the highest temperature value in each overlapping region are calculated. The average temperature value, the highest temperature value, and the backplate temperature data of each overlapping region are arranged in a fixed order to form a temperature feature sequence corresponding to each overlapping region. Based on the center coordinates of the hot spot region data, the temperature feature sequence of each overlapping region is mapped to a two-dimensional grid that matches the position coordinates of the temperature field distribution data. Unmapped grid positions are filled with zero values ​​to generate a hot spot interference feature map containing the temperature features of the hot spot region and its position distribution.

5. The method according to claim 1, characterized in that, The multimodal temporal prediction model integrates the frequency domain features of the submodal sequences, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data to generate a joint prediction input, including: The multimodal time series prediction model decomposes each submodal sequence into components corresponding to multiple frequency intervals, records the frequency interval range of each component and the amplitude value of the frequency interval changing with time, and arranges the frequency interval range and the corresponding amplitude value in a fixed order to form the frequency domain feature data sequence of each submodal. Based on the coordinate distribution of non-zero regions in the hot spot interference feature map, the spacing value and relative azimuth angle value between each non-zero region are extracted, and the spacing value and relative azimuth angle value are arranged in a preset order to form the spatial correlation data sequence of the hot spot interference feature map. The environmental temperature and humidity data are divided into continuous time windows in chronological order. The direction of change of temperature and humidity values ​​in each time window is statistically analyzed, including whether they are continuously rising, continuously falling, or remaining stable. The direction of temperature and humidity change in each time window is arranged in chronological order to form a time-dependent data sequence of environmental temperature and humidity data. The frequency domain feature data sequence, spatial correlation data sequence, and temporal dependence data sequence are sequentially concatenated into a single data sequence as the joint prediction input.

6. The method according to claim 1, characterized in that, The abrupt fluctuation components in the long and short wave component data are decomposed into sub-mode sequences, including: The long and short wave component data are scanned in chronological order to identify target segments. The target segment is a segment whose numerical change exceeds twice the average change of adjacent time periods. The start and end times of the target segment are recorded. The data segment located between the start time point and the end time point in the long and short wave component data is extracted as an independent fluctuation component, and multiple independent fluctuation components that are adjacent and have a time interval of less than a preset threshold are merged into a single abrupt fluctuation component. Each abrupt fluctuation component is divided into multiple consecutive sub-fluctuation segments according to the time length, and the time span of each sub-fluctuation segment is matched with the duration for which the slope of temperature change in the environmental temperature and humidity data exceeds a preset value. The time range of each sub-fluctuation segment is compared with the time period in the environmental temperature and humidity data where the rate of humidity change exceeds a preset threshold. According to the preset overlap ratio, the sub-fluctuation segments are marked as high-frequency sub-mode sequences and low-frequency sub-mode sequences. All labeled high-frequency sub-mode sequences are arranged in chronological order to form a first set, and low-frequency sub-mode sequences are arranged in chronological order to form a second set. The first set and the second set are then merged into a complete sub-mode sequence.

7. A real-time solar power generation prediction system based on multimodal deep learning, used to execute the real-time solar power generation prediction method based on multimodal deep learning as described in any one of claims 1 to 6, characterized in that, include: The acquisition module collects long and short wave component data of solar radiation on the earth's surface, backsheet temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposes the abrupt fluctuation components in the long and short wave component data into sub-mode sequences. The first generation module acquires temperature field distribution data of the photovoltaic module, associates the temperature field distribution data with hot spot area data collected by the thermal infrared imager, and generates a hot spot interference feature map by combining the backsheet temperature data. The second generation module inputs the sub-modal sequence, the hot spot interference feature map, and the environmental temperature and humidity data into a multi-modal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the sub-modal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependence of the environmental temperature and humidity data through the multi-modal time series prediction model. The output module outputs the prediction results of each sub-mode based on the joint prediction input through the multi-modal time series prediction model, and performs frequency domain superposition of the prediction results of each sub-mode according to the preset reconstruction rules to generate a short-term solar power generation prediction curve.

8. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a real-time solar power generation prediction method based on multimodal deep learning as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a real-time solar power generation prediction method based on multimodal deep learning as described in any one of claims 1 to 6.

Images