Solar power generation power real-time prediction method, system and device based on multi-modal deep learning, and storage medium

Abstract

The invention provides a solar power generation power real-time prediction method, system and device based on multi-mode deep learning and a storage medium. Wherein the prediction precision is improved by fusing multi-dimensional features of earth surface radiation, equipment temperature and environmental data, mutation fluctuation decomposition is performed on long and short wave components of earth surface solar radiation, and a frequency domain sub-modal sequence is extracted; constructing a correlation model of a photovoltaic module temperature field and hot spot infrared data, and generating a dynamic feature map reflecting hot spot interference space distribution; establishing a multi-mode time sequence prediction model, and synchronously processing frequency domain features of radiation sub-modes, spatial correlation of hot spot feature maps and time sequence dependence of temperature and humidity data; and integrating the prediction results of the sub-modals through a frequency domain superposition reconstruction technology. According to the technical scheme provided by the invention, through multi-modal spatial-temporal feature joint modeling and dynamic interference decoupling, the power prediction precision in extreme weather and hot spot abnormal scenes is remarkably improved, and meanwhile, the dynamic adaptive capacity of a prediction system to complex environment disturbance is enhanced.

Description

Technical Field

[0001] The present application relates to the field of multimodal deep learning technology, and in particular to a method, system, device and storage medium for real-time prediction of solar power generation power based on multimodal deep learning. Background Art

[0002] As the demand for high-precision power forecasting in renewable energy power systems increases, real-time prediction of solar power generation faces multiple challenges: Surface solar radiation is affected by sudden cloud changes and aerosol scattering, resulting in short-term and dramatic fluctuations; hot spot effects in photovoltaic modules lead to localized temperature anomalies, resulting in nonlinear power output attenuation; and dynamic changes in ambient temperature and humidity coupled with equipment operating conditions further complicate power fluctuations. A real-time prediction model that accounts for sudden meteorological changes, equipment thermal anomalies, and environmental disturbances is needed to support coordinated control of grid dispatch and energy storage systems.

[0003] The current mainstream approach uses a time series prediction model based on long-short-term memory networks. This model primarily leverages historical power generation data and single-modal time series data such as irradiance and temperature provided by weather stations, achieving power forecasting through time series dependency modeling. Some improved approaches incorporate convolutional neural networks to extract spatial features from meteorological data or incorporate satellite cloud imagery into the input layer to enhance the ability to capture sudden changes in radiation.

[0004] The above scheme has three limitations: modal singleness: relying on single-point data from meteorological stations or low-resolution satellite cloud images, it is difficult to accurately characterize the local mutation components of surface radiation (such as instantaneous fluctuations in shortwave radiation); lack of thermal anomaly modeling: the failure to integrate device-level hot spot monitoring data such as thermal infrared imaging results in the inability to dynamically quantify the power attenuation caused by the hot spot effect; insufficient feature decoupling: the time series model has insufficient modeling capabilities for frequency domain mutation characteristics (such as high-frequency fluctuations in radiation components), hot spot spatial distribution characteristics, and multimodal correlations with environmental parameters, resulting in a significant increase in prediction errors under complex weather scenarios. Summary of the Invention

[0005] The present application provides a real-time prediction method, system, device and storage medium for solar power generation based on multimodal deep learning, which is used to solve the problems in the prior art such as inaccurate capture of surface radiation mutation characteristics due to single modal data, inability to dynamically quantify power attenuation caused by hot spot effects, and insufficient modeling of frequency-domain-spatial domain multimodal feature correlations, which leads to a significant increase in prediction errors in complex scenarios.

[0006] In a first aspect, the present application provides a method for real-time prediction of solar power generation based on multimodal deep learning, comprising: Collecting long- and short-wave component data of surface solar radiation, backplane temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposing the sudden fluctuation components in the long- and short-wave component data into sub-modal sequences; Acquiring temperature field distribution data of the photovoltaic module, correlating the temperature field distribution data with hot spot area data collected by a thermal infrared imager, and generating a hot spot interference feature map in combination with the backplane temperature data; Inputting the submodal sequence, the hot spot interference feature map, and the ambient temperature and humidity data into a multimodal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model; The multimodal time series prediction model outputs the prediction results of each submodal according to the joint prediction input, and superimposes the prediction results of each submodal in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation power prediction curve.

[0007] Optionally, outputting prediction results of each sub-modality according to the joint prediction input by the multi-modal time series prediction model includes: Couple the frequency domain features of the submodal sequence with the temporal dependency of the ambient temperature and humidity data to generate a frequency domain-temporal hybrid feature that characterizes the dynamic association between irradiance mutation and temperature and humidity, wherein the temporal dependency of the temperature and humidity data is quantified by multiplying the humidity change rate and the temperature change slope within a sliding time window; According to the spatial correlation distribution of the hot spot interference feature map, the frequency domain-time series hybrid feature is divided into a transient response feature corresponding to a high-frequency sub-modal sequence and a steady-state response feature corresponding to a low-frequency sub-modal sequence; Performing segmented matching processing on the transient response characteristics based on the duration of the irradiance mutation, wherein the start time of each segment is triggered by the time when the humidity change rate in the ambient temperature and humidity data exceeds a preset threshold, and the segment length is synchronously adjusted with the update interval of the hot spot interference feature map; performing a trend fitting process based on a diurnal cycle on the steady-state response characteristics, wherein a time span of the trend fitting is dynamically controlled by a sign flipping frequency of the temperature change slope; The transient response characteristics after the segmented matching processing and the steady-state response characteristics after the trend fitting processing are input into a parallel prediction channel, wherein the parallel prediction channel uses a waveform morphology matching method to generate a high-frequency sub-modal prediction result for the transient response characteristics, and uses a waveform envelope tracking method to generate a low-frequency sub-modal prediction result for the steady-state response characteristics.

[0008] Optionally, the prediction results of each sub-modal are superimposed in the frequency domain according to a preset reconstruction rule to generate a short-term solar power generation power prediction curve, including: Based on the spatial distribution weight of the temperature gradient value in the hot spot interference feature map, the high-frequency submodal prediction results and the low-frequency submodal prediction results are regionally proportionally allocated, wherein the allocation ratio of the high-frequency submodal prediction results corresponding to the area where the temperature gradient value is higher than the preset threshold is attenuated according to the inverse of the humidity change rate; The allocated high-frequency submodal prediction results and low-frequency submodal prediction results are waveform-spliced according to the original frequency band order of the submodal sequence. During the splicing process, the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment to generate a short-term solar power generation prediction curve.

[0009] Optionally, the allocated high-frequency submodal prediction results and the low-frequency submodal prediction results are waveform spliced according to the original frequency band order of the submodal sequence, and the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the splicing process to generate a short-term solar power generation prediction curve, including: Dividing the high-frequency submodal prediction results and the low-frequency submodal prediction results into equal-time blocks according to the original arrangement order when the submodal sequence is decomposed, wherein the high-frequency submodal prediction result block comes first and the low-frequency submodal prediction result block comes last, and the length of each time block is consistent with the update interval of the hot spot interference feature map; In each time block, an alignment reference point and a matching point are selected according to the sign direction of the temperature change slope to generate position marks of the reference point of the high-frequency submodal prediction result block and the matching point of the low-frequency submodal prediction result block; Aligning the position marks of the reference points of the high-frequency submodal prediction result block and the matching points of the low-frequency submodal prediction result block of each time block vertically along the time axis, retaining all waveform data before the reference points in the high-frequency submodal prediction result block after the alignment, retaining all waveform data after the matching points in the low-frequency submodal prediction result block, and directly connecting the reference points of the high-frequency submodal prediction result block and the matching points of the low-frequency submodal prediction result block, with the timestamps at the connection being continuous and the amplitude values being equal, thereby generating a high-frequency band truncated block and a low-frequency band truncated block; The high-frequency truncated blocks and low-frequency truncated blocks of all time blocks are sequentially connected in series according to the original frequency band order to generate a short-term solar power generation prediction curve.

[0010] Optionally, obtaining temperature field distribution data of a photovoltaic module, correlating the temperature field distribution data with hot spot area data collected by a thermal infrared imager, and generating a hot spot interference feature map in combination with the backplane temperature data, includes: The temperature value of each point on the surface of the photovoltaic module is collected by the temperature measuring device to form temperature field distribution data including position coordinates and temperature values; Acquire hot spot area data on the surface of the photovoltaic module using a thermal infrared imaging device, identify local areas in the hot spot area data that have a temperature significantly higher than the surrounding area and are closed in shape, and record the closed boundary coordinates and center coordinates of each local area; Extracting local temperature values overlapping with the closed boundary coordinates of the hot spot area data from the temperature field distribution data, and calculating the average temperature value and the maximum temperature value of the temperature values in each overlapping area; Arrange the average temperature value, the maximum temperature value and the backplane temperature data of each overlapping area in a fixed order to form a temperature feature sequence corresponding to each overlapping area; According to the center coordinates of the hot spot area data, the temperature feature sequence of each overlapping area is mapped to a two-dimensional grid consistent with the position coordinates of the temperature field distribution data, and the unmapped grid positions are filled with zero values to generate a hot spot interference feature map containing the temperature characteristics of the hot spot area and its position distribution.

[0011] Optionally, generating a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model includes: Decomposing each submodal sequence into components corresponding to a plurality of frequency intervals using the multimodal time series prediction model, recording the frequency interval range of each component and the amplitude value varying over time within the frequency interval, and arranging the frequency interval ranges and the corresponding amplitude values in a fixed order to form a frequency domain feature data sequence for each submodal; Extracting spacing values and relative azimuth angle values between non-zero areas according to the coordinate distribution of non-zero areas in the hot spot interference feature map, and arranging the spacing values and relative azimuth angle values in a preset order to form a spatial correlation data sequence of the hot spot interference feature map; Dividing the ambient temperature and humidity data into continuous time windows in chronological order, counting the direction of continuous increase, continuous decrease, or stable change of the temperature and humidity values in each time window, and arranging the temperature change direction and humidity change direction in each time window in chronological order to form a time-dependent data sequence of the ambient temperature and humidity data; The frequency domain feature data sequence, the spatial correlation data sequence and the temporal dependency data sequence are sequentially spliced into a single data sequence as a joint prediction input.

[0012] Optionally, decomposing the sudden change fluctuation component in the long- and short-wave component data into a sub-modal sequence includes: Scanning the long and short wave component data in chronological order to identify a target segment, wherein the target segment is a segment in which the amplitude of the value change exceeds twice the average amplitude of the value change in adjacent time periods, and recording the start time point and the end time point of the target segment; The data segment between the starting time point and the ending time point in the long and short wave component data is intercepted as an independent fluctuation component, and multiple independent fluctuation components that are adjacent and have a time interval less than a preset threshold are merged into a single sudden change fluctuation component; Divide each sudden fluctuation component into a plurality of continuous sub-fluctuation segments according to the time length, wherein the time span of each sub-fluctuation segment matches the duration of the temperature change slope exceeding the preset value in the ambient temperature and humidity data; Comparing the time range of each sub-fluctuation segment with the period in which the humidity change rate in the ambient temperature and humidity data exceeds a preset threshold, and marking the sub-fluctuation segment as a high-frequency sub-modal sequence and a low-frequency sub-modal sequence according to a preset overlap ratio; All marked high-frequency submodal sequences are arranged in chronological order to form a first set, and all marked low-frequency submodal sequences are arranged in chronological order to form a second set, and the first set and the second set are merged into a complete submodal sequence.

[0013] In a second aspect, the present application provides a real-time prediction system for solar power generation based on multimodal deep learning, comprising: An acquisition module collects long- and short-wave component data of surface solar radiation, backplane temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposes the sudden fluctuation components in the long- and short-wave component data into sub-modal sequences; The first generation module acquires temperature field distribution data of the photovoltaic module, associates the temperature field distribution data with hot spot area data collected by the thermal infrared imager, and generates a hot spot interference feature map in combination with the backplane temperature data; a second generation module, inputting the submodal sequence, the hot spot interference feature map, and the ambient temperature and humidity data into a multimodal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model; The output module outputs the prediction results of each sub-modality according to the joint prediction input through the multi-modal time series prediction model, and superimposes the prediction results of each sub-modality in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation power prediction curve.

[0014] In a third aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a real-time prediction method for solar power generation power based on multimodal deep learning as described in the first aspect above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements a real-time prediction method for solar power generation power based on multimodal deep learning as described in the first aspect.

[0016] In the embodiment of the present application, by decomposing the sudden fluctuation components in the long- and short-wave component data of surface solar radiation, the frequency domain sub-modal sequence is extracted to effectively characterize short-term radiation disturbances such as cloud mutations and aerosol scattering, thereby improving the sensitivity and modeling capability to meteorological mutations; combining the temperature field distribution of photovoltaic modules with thermal infrared imaging data to generate a hot spot interference characteristic map, dynamically reflecting the spatial distribution of the hot spot area and its nonlinear attenuation effect on power output, and avoiding prediction deviations caused by local temperature anomalies; jointly inputting the radiation frequency domain characteristics, hot spot spatial correlation and ambient temperature and humidity temporal dependence into a multi-modal prediction model to overcome the limitations of single modal data and enhance the model's comprehensive characterization capability for meteorological-equipment-environmental coupling; dynamically integrating the prediction results of each sub-modal through the frequency domain superposition rule to achieve a coordinated response to multi-time scale characteristics such as radiation high-frequency mutations, hot spot diffusion and ambient temperature and humidity evolution, thereby significantly improving the real-time and accuracy of short-term power prediction. Through full-link optimization from data decomposition, dynamic feature extraction to multimodal joint modeling, the prediction error problem caused by the traditional method due to the single modality, lack of thermal anomaly modeling and insufficient feature decoupling is effectively solved, and it has stronger robustness and adaptability under complex meteorological conditions and equipment abnormality scenarios.

[0017] Furthermore, by coupling the irradiance frequency domain characteristics with the temperature and humidity time series dynamic correlation, a hybrid feature that distinguishes transient and steady-state responses is generated; the hybrid feature is divided based on the spatial distribution of hot spots, and differentiated by segmented matching (transient) and day and night trend fitting (steady state) triggered by humidity changes; finally, a parallel prediction channel is used to match the high-frequency transient features with waveform morphology and envelope tracking is used for the low-frequency steady-state features to achieve cross-time scale collaborative prediction of radiation mutations and hot spot diffusion. Its technical effect is: through frequency domain-time series coupling modeling and dynamic feature decoupling, the model's ability to simultaneously predict short-term radiation mutations (such as cloud cover) and long-period hot spot diffusion is significantly improved, while the correlation between dynamic changes in temperature and humidity and equipment thermal anomalies is enhanced, reducing prediction errors in complex meteorological and equipment coupling scenarios.

[0018] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of a method for real-time prediction of solar power generation based on multimodal deep learning provided by the present application is shown; Figure 2 A schematic diagram of the structure of a solar power generation real-time prediction system based on multimodal deep learning provided by the present application is shown; Figure 3 A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0022] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0023] Researchers have found that existing solar power generation power prediction models have significant limitations: the short-term mutation component of surface solar radiation (such as sudden changes in shortwave radiation caused by cloud cover) is difficult to accurately capture due to the lack of effective frequency domain decomposition methods; the nonlinear power attenuation caused by the hot spot effect of photovoltaic modules cannot be dynamically quantified due to the lack of integration of device-level thermal infrared monitoring data; the dynamic changes in ambient temperature and humidity and the multimodal characteristics of radiation and thermal anomalies are not modeled sufficiently, resulting in a sharp increase in prediction errors in complex meteorological scenarios. Based on this, the present application provides a real-time prediction method for solar power generation based on multimodal deep learning. This method can significantly improve the prediction accuracy and real-time performance in scenarios where short-term meteorological mutations and equipment thermal anomalies are coupled through frequency domain decomposition of radiation mutation components, dynamic feature extraction of hot spot interference, and multimodal spatiotemporal joint modeling. The technical solution of the present application can be applied to minute-level grid scheduling and photovoltaic power station abnormality warning scenarios.

[0024] The entire research and development process reflects the decomposition of the sudden fluctuation components in the long- and short-wave component data of surface solar radiation to accurately capture short-term radiation disturbances such as cloud cover; combining the temperature field distribution of photovoltaic modules with thermal infrared imaging data to generate a dynamic hot spot interference feature map, and quantify the nonlinear attenuation effect of local thermal anomalies on power; and then constructing a multimodal time series prediction model, integrating the radiation frequency domain characteristics, hot spot spatial correlation and the time series dependence of ambient temperature and humidity, and generating a high-precision short-term prediction curve through frequency domain superposition and reconstruction. This solution solves the prediction deviation problem caused by the traditional model due to the single mode, the lack of hot spot effect modeling and the insufficient decoupling of multimodal features, and significantly improves the real-time and accuracy of prediction in scenarios with complex meteorological mutations and equipment thermal anomalies.

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0026] Figure 1 The present invention provides a flowchart of a method for real-time prediction of solar power generation based on multimodal deep learning, such as Figure 1 As shown, the method includes: 101. Collecting long-wave and short-wave component data of surface solar radiation, backplane temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposing abrupt fluctuation components in the long-wave and short-wave component data into sub-modal sequences; Long- and short-wavelength component data refers to real-time monitoring data of medium- and short-wavelength (0.3-3μm, including visible light and ultraviolet) and long-wavelength (3-100μm, primarily infrared) solar radiation received by the Earth's surface. Sudden fluctuation components refer to short-term, dramatic fluctuations in radiation values caused by meteorological disturbances such as sudden cloud formation and aerosol scattering. Submodal sequences are generated by breaking down sudden fluctuations in radiation data into multiple independent sub-signals (e.g., high-frequency, medium-frequency, and low-frequency modes) based on their frequency characteristics using a signal decomposition algorithm.

[0027] In this embodiment, specific shortwave and longwave radiometers are used to collect real-time data on the shortwave and longwave components of surface solar radiation. The shortwave radiometer uses a CMP21 model, covering the 0.3 to 3 micron band and including visible and ultraviolet radiation monitoring capabilities; the longwave radiometer uses a CGR4 model, covering the 3 to 100 micron infrared band. Simultaneously, a high-precision PT1000 sensor collects photovoltaic module backplane temperature data, while an SHT35 sensor monitors ambient temperature and humidity.

[0028] Signal decomposition techniques are used to perform time-frequency analysis on the sudden fluctuation components in the radiation data. Specifically, wavelet transform algorithms, preferably using Daubechies wavelet basis functions, or empirical mode decomposition (EMD) are used for processing. The power spectral density is calculated using a fast Fourier transform, and the non-stationary high-frequency sudden fluctuation components are decomposed into multiple independent submodes based on the spectral energy distribution characteristics. Typical decomposition results include transient submodes in the 0.1 to 1 Hz frequency band and fluctuating submodes in the 1 to 10 Hz frequency band.

[0029] 102. Acquire temperature field distribution data of the photovoltaic module, associate the temperature field distribution data with hot spot area data collected by a thermal infrared imager, and generate a hot spot interference feature map in combination with the backplane temperature data; Temperature field distribution data refers to the spatial distribution data of the surface temperature of photovoltaic modules, which is obtained by infrared thermal imagers or distributed temperature sensor arrays; hot spot area data refers to the local abnormally high temperature area of the module detected by the thermal infrared imager (temperature difference ≥ 5°C); hot spot interference feature map refers to a two-dimensional matrix that integrates the temperature field distribution, hot spot area coordinates and backplane temperature, and quantifies the influence weight of hot spots on power.

[0030] In this embodiment, the spatial distribution of photovoltaic module surface temperature is acquired using a specific model of infrared thermal imaging equipment. Specifically, a FLIR T840 thermal infrared imager is used. This device boasts a high resolution of 640 x 480 pixels and a thermal sensitivity of 0.03 degrees Celsius, enabling precise detection of module surface temperature distribution. Furthermore, coordinate mapping technology is used to spatially align the hot spot area image with the temperature field data to construct a complete temperature distribution matrix.

[0031] A hot spot is defined as an abnormally high temperature area on a module detected by a thermal infrared imager. The criterion for determining a hot spot is a temperature difference of 5°C or greater between the local area and the surrounding area. Based on temperature field distribution data, hot spot coordinates, and backplane temperature data, a two-dimensional matrix hot spot interference feature map is constructed to quantify the impact of the hot spot effect on the output power of PV modules.

[0032] During the specific implementation process, the temperature difference between the hot spot area and the backplane is first calculated, recorded as ΔT. The ratio of the hot spot area to the total area of the module is also measured. Based on these two parameters, a dynamic impact weight calculation formula is used to multiply the hot spot temperature difference by the area ratio, and the backplane temperature change rate is introduced as a correction factor to ultimately generate a hot spot interference feature map. In actual application, the Kriging spatial interpolation algorithm is used to fuse the discretely distributed temperature sensor data with the hot spot image to generate a temperature field distribution matrix with a resolution of 1 cm. This enables dynamic updating of the weight coefficients, thereby accurately quantifying the nonlinear attenuation effect of hot spot diffusion on power output.

[0033] 103. Input the submodal sequence, the hot spot interference feature map, and the ambient temperature and humidity data into a multimodal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model; Frequency domain features refer to the spectral characteristics of the sub-modal sequence (such as energy concentration frequency bands and harmonic components); spatial correlation refers to the spatial correlation of weights at different positions in the hot spot interference feature map (such as the hot spot diffusion path); and temporal dependence refers to the trend of environmental temperature and humidity data changing over time (such as the humidity accumulation effect).

[0034] In this embodiment of the present application, the submodal sequence processed by signal decomposition, the hot spot interference feature map, and the ambient temperature and humidity monitoring data are input into a multimodal time series prediction model. The model uses deep learning methods to achieve feature fusion and joint prediction of multi-source heterogeneous data.

[0035] First, a short-time Fourier transform (STFT) is performed on the submodal sequence, with an analysis window length of 64 seconds and a 75% overlap ratio. Spectral feature parameters are then extracted. For example, the high-frequency submodal obtained in step 101 has a frequency range of 0.5 to 2 Hz, with a fluctuation amplitude of ±50 watts per square meter. After the transformation, a frequency domain energy peak at 3.2 kHz is identified. From this, a 64-dimensional feature vector is constructed to characterize the spectral characteristics of the radiation fluctuations.

[0036] Next, a deep convolutional neural network (DCNN) was used to process the hotspot interference feature map. Specifically, the ResNet-18 network architecture was pre-trained on the ImageNet dataset. In practice, the parameters of the first three layers of the network were kept fixed, and only subsequent layers were fine-tuned. For the hotspot feature map with a weight of 0.58 in step 102, the network output was a 128-dimensional feature vector, effectively capturing the spatial distribution of the hotspot, which is spreading southeastward at a rate of 0.2 meters per minute.

[0037] A bidirectional long-short-term memory (LSTM) network was constructed to process ambient temperature and humidity data. The network was configured with 32 hidden units and used a 10-minute sliding window to analyze temporal changes. For the measured ambient data, where humidity increased from 45% to 60% and temperature decreased from 28°C to 25°C, the network output a 32-dimensional feature vector containing a humidity rate of change of 1.5 percentage points per minute and a temperature trend of 0.3°C per minute.

[0038] Finally, a multi-head attention mechanism consisting of four attention heads was designed, with a key-value dimension of 64, to dynamically weight the three feature types. The specific allocation ratio was 40% for frequency domain features, 35% for spatial features, and 25% for temporal features. Through feature concatenation and dimensionality transformation, a 256-dimensional joint feature vector was generated, fully encoding the combined impact of cloud obscuration, hot spot diffusion, and drastic temperature and humidity fluctuations on the photovoltaic system. This joint feature vector provides comprehensive and accurate feature input for subsequent power prediction.

[0039] 104. Outputting the prediction results of each sub-modality according to the joint prediction input through the multi-modal time series prediction model, and superimposing the prediction results of each sub-modality in the frequency domain according to a preset reconstruction rule to generate a short-term solar power generation power prediction curve.

[0040] Among them, the submodal prediction results refer to the predicted values of future change trends of submodals in each frequency band; the frequency domain superposition rule refers to reconstructing the submodal prediction results into a complete radiation sequence based on inverse wavelet transform or spectrum synthesis algorithm.

[0041] In the embodiment of the present application, the multimodal time series prediction model generates prediction results for each sub-modality based on the joint prediction input, and outputs a short-term solar power generation prediction curve through frequency domain reconstruction technology. The specific implementation process is as follows: This prediction model uses a channel-wise processing architecture, splitting the 256-dimensional joint feature vector into 128 high-frequency features and 128 low-frequency features via a fully connected layer. The high-frequency prediction channel is processed using a Temporal Convolutional Network (TCN) with four layers, a dilated convolution kernel of size 3, and dilation coefficients set to 1, 2, 4, and 8, respectively. This branch outputs a 15-minute radiation fluctuation forecast, specifically a negative peak fluctuation of 80 watts per square meter in the 5th minute and a positive fluctuation of 40 watts per square meter in the 12th minute.

[0042] The low-frequency prediction channel uses the Autoregressive Integrated Moving Average (ARIMA) model, with parameters set to autoregressive order 2, differencing order 1, and moving average order 1. This model predicts that irradiance will rebound to 650 watts per square meter after one hour. The high- and low-frequency submodal predictions are reconstructed using the inverse ensemble mode decomposition algorithm to generate a complete radiation forecast sequence. The 5-minute forecast is 480 watts per square meter, the 15-minute forecast is 520 watts per square meter, and the 1-hour forecast reaches 650 watts per square meter.

[0043] The reconstructed radiation sequence was fed into a photovoltaic power conversion model based on the physics of a single diode. The model set a series resistor of 0.2 ohms, a parallel resistor of 100 ohms, and a photocurrent of 8.2 amperes. A backplane temperature correction factor of -0.5% per degree Celsius was also introduced to compensate for the effect of temperature changes on output power.

[0044] The final power forecast results show that the current output power is 4.8 MW, which will drop to 4.5 MW after 5 minutes due to the combined effects of cloud cover and hot spots, and then recover to 5.1 MW after 1 hour. The system updates the forecast every 30 seconds. When compared with measured data from the SCADA system, the forecast error is kept within 2%, meeting the accuracy requirements for engineering applications. This prediction method effectively integrates the multi-dimensional characteristics of radiation fluctuations, hot spot effects, and environmental parameters to achieve highly accurate short-term power forecasts.

[0045] In some embodiments, outputting prediction results for each sub-modality based on the joint prediction input by the multi-modal time series prediction model includes: 201. Couple and map the frequency domain features of the submodal sequence with the temporal dependency of the ambient temperature and humidity data to generate a frequency domain-temporal hybrid feature representing the dynamic correlation between the irradiance mutation and the temperature and humidity.

[0046] The temporal dependence of the temperature and humidity data is quantified by multiplying the humidity change rate and the temperature change slope within a sliding time window.

[0047] Frequency domain features refer to the spectral characteristics of the submodal sequence extracted via Fourier transform, including the main frequency and harmonic components. Time series dependency refers to the dynamic correlation of ambient temperature and humidity data over time, such as the humidity change rate ΔH / Δt and the temperature change slope ΔT / Δt. Frequency domain-time series hybrid features are joint feature vectors generated by coupling the frequency domain features of irradiance mutations with the temperature and humidity time series dynamics. They are used to characterize the interaction between meteorological mutations and environmental parameters.

[0048] In this embodiment, the submodal sequence is first subjected to a short-time Fourier transform (STFT) with a window length of 64 seconds and an overlap ratio of 75% to extract the spectral energy peak and primary harmonic components. For example, for the 0.5-2 Hz fluctuations caused by cloud cover, a peak with 60% of the energy is detected at 3 kHz, and the second harmonic amplitude is identified, thereby quantifying the frequency domain characteristics of the radiation mutation.

[0049] At the same time, the humidity change rate and temperature change slope are calculated within a 10-minute sliding window. For example, the humidity change rate is increasing by 3% per minute, and the temperature change slope is decreasing by 0.5 degrees Celsius per minute. By multiplying the two together, the time-dependent quantization parameter S is obtained, which has a value of negative 1.5.

[0050] The 64-dimensional spectral feature vector and the time series parameter S are then input into a fully connected layer and mapped using the ReLU activation function to generate a 128-dimensional mixed feature vector. When the time series parameter S is negative, the humidity-related weights in the feature vector are automatically increased, making the model more sensitive to the synergistic suppression of radiation by sudden humidity increases and sudden temperature drops. This generates a frequency-domain-time series hybrid feature that can characterize the dynamic relationship between sudden irradiance changes and temperature and humidity.

[0051] This process effectively solves the problem of traditional prediction models ignoring multimodal correlations by encoding the interaction between meteorological mutations and environmental parameters.

[0052] 202. Divide the frequency-domain-time series hybrid feature into a transient response feature corresponding to a high-frequency sub-modal sequence and a steady-state response feature corresponding to a low-frequency sub-modal sequence according to the spatial correlation distribution of the hot spot interference feature map; Spatial correlation distribution refers to the weighting of heat diffusion paths in different regions of the hot spot interference feature map, for example, a weight of 0.8 for the center of a hot spot and 0.2 for the edge. Transient response characteristics refer to short-term disturbances corresponding to high-frequency submodes, such as fluctuations caused by sudden changes in cloud cover. Steady-state response characteristics refer to long-term trends corresponding to low-frequency submodes, such as changes in the daytime radiation baseline.

[0053] In an embodiment of the present application, based on the spatial correlation distribution weights of the hot spot interference feature map, for example, a weight of 0.8 for the center area and a weight of 0.2 for the edge area, a specific threshold is set to divide the frequency domain-time series hybrid features into hot spot affected areas associated with high-frequency and low-frequency sub-modals. In specific implementation, the 128-dimensional hybrid feature vector is dynamically divided into two parts according to the spatial weight distribution: the first 64-dimensional features are associated with high-frequency mutation areas, such as the hot spot center area, which is used to capture short-term disturbances such as cloud cover and form transient response features corresponding to the high-frequency sub-modal sequence; the second 64-dimensional features are associated with low-frequency trend areas, such as the hot spot edge area, which are used to model the daytime irradiance baseline changes and form steady-state response features corresponding to the low-frequency sub-modal sequence.

[0054] When a hot spot spreads to a new area, for example, if the weight of a particular region increases from 0.2 to 0.6, the system automatically adjusts the feature partitioning ratio, increasing the transient response feature dimension to 70, ensuring that the feature partitioning keeps pace with the dynamic spread of the hot spot. This step, through a feature decoupling mechanism driven by spatial weights, effectively enhances the detection sensitivity of localized anomalies in PV panels.

[0055] 203. Perform segmented matching processing on the transient response characteristics based on the duration of the irradiance mutation, wherein the start time of each segment is triggered by the time when the humidity change rate in the ambient temperature and humidity data exceeds a preset threshold, and the segment length is synchronously adjusted with the update interval of the hot spot interference feature map; The duration of the sudden irradiance change indicates the time span over which radiation drops suddenly due to events such as cloud cover. The system sets a humidity change rate trigger threshold of 2% per minute. When three consecutive sampling points exceed this threshold, a valid segmentation starts. The hot spot feature map is updated every 30 seconds, and the segment length is synchronized with this.

[0056] In this embodiment, the system monitors the rate of change of ambient humidity, ΔH / Δt, in real time. When three consecutive sampling points, separated by 10 seconds, each exceed a threshold of 2% per minute, the system automatically marks the start of the current segment. Subsequently, using 30-second intervals as the basic unit, the system captures transient response characteristic data within that period, including energy characteristics in the 0.5-2 Hz frequency band and hot spot spatial weight data, ensuring that the segmented processing is synchronized with the spatiotemporal evolution of the device's thermal anomaly.

[0057] The system's built-in dynamic time warping (DTW) algorithm matches the current segmented data against a pre-stored library of historical waveforms. This library contains templates for typical meteorological events, such as cumulus obscuration and dust scattering. The algorithm selects the matching template with the lowest DTW path cost. For example, if it matches the "4-minute cumulus obscuration" template and the path cost does not exceed 0.1, the system uses this to predict the duration of the current irradiance mutation. Simultaneously, the system dynamically adjusts segmentation parameters based on real-time feedback from hot spot expansion. For example, if a 10% expansion of the hot spot area is detected, the segment length is automatically shortened from 30 seconds to 20 seconds, and the dedicated "accelerated hot spot expansion" template is switched.

[0058] 204. Perform trend fitting processing on the steady-state response characteristics based on the day-night cycle, wherein the time span of the trend fitting is dynamically controlled by the sign reversal frequency of the temperature change slope.

[0059] The diurnal cycle reflects the periodic pattern of radiation rising during the day and returning to zero at night. The temperature slope sign reversal frequency refers to the number of times the temperature changes in direction, for example, once per hour.

[0060] In this embodiment, the steady-state response characteristics are first decomposed using Fourier series, focusing on extracting the 24-hour diurnal component and the 12-hour semi-diurnal component. By analyzing these periodic components, the system can accurately interpret the daytime upward trend of irradiance, such as the typical increase of 50W / m² per hour, and the natural characteristic of returning to zero at night.

[0061] The system monitors the sign flip frequency of the temperature slope in real time. For example, if it detects two sign flips per hour, it automatically adjusts the trend fitting time span to a 30-minute window. This dynamic adjustment mechanism ensures consistent temperature trends within each analysis window. The steady-state response characteristics are then input into the Prophet forecasting model, which comprehensively considers seasonal cyclical components, linear and nonlinear trend terms, and holiday effects such as special weather markers. The model outputs a forecast of radiation baseline changes within the future time window, such as a linear increase in irradiance from 500 W / m² to 540 W / m² over 30 minutes. The system automatically forces the fitting results to decay to zero after sunset, effectively avoiding erroneous predictions of abnormal irradiance fluctuations at night.

[0062] 205. Input the transient response characteristics after the segmented matching processing and the steady-state response characteristics after the trend fitting processing into a parallel prediction channel, so as to generate a high-frequency sub-modal prediction result for the transient response characteristics by using a waveform morphology matching method through the parallel prediction channel, and generate a low-frequency sub-modal prediction result for the steady-state response characteristics by using a waveform envelope tracking method.

[0063] Waveform shape matching predicts the shape of high-frequency fluctuations by calculating the similarity between the current waveform and a library of historical mutation waveforms. Typically, this is done using metrics like cosine similarity. Waveform envelope tracking uses mathematical transformations to extract the upper and lower envelopes of low-frequency trends, thereby defining the possible range of baseline changes. This is typically achieved using the Hilbert transform.

[0064] In this embodiment, the parallel prediction channel operates as follows: When processing transient response features, the system uses the dynamic time warping (DTW) algorithm to intelligently retrieve the waveform template with the highest matching accuracy from a pre-built library of historical meteorological event waveforms. This retrieval process comprehensively considers the current baseline radiation level and dynamically scales the amplitude of the matching template, generating accurate predictions of short-term fluctuations in high-frequency submodals.

[0065] For steady-state response characteristics, the system first performs a Hilbert transform to accurately extract the upper and lower envelopes of the low-frequency trend. These envelopes are then integrated with the baseline forecast output by the Prophet model to jointly define the fluctuation range of long-term radiation changes.

[0066] In the final prediction phase, the system scientifically superimposes the high-frequency and low-frequency prediction results according to a preset frequency-domain energy weighting scheme. Typical weightings are set at 0.6 for the high-frequency component and 0.4 for the low-frequency component. The combined predictions are then reconstructed using an inverse wavelet transform to form a complete radiation variation curve. This curve is then input into the photovoltaic power conversion model, seamlessly integrating minute-level fluctuation details with hourly trend changes, resulting in highly accurate multi-timescale power forecasts.

[0067] The multimodal time series prediction method proposed in the embodiment of the present application achieves accurate characterization of the interaction between meteorological parameters by coupling the frequency domain characteristics of irradiance mutations with the time series dynamics of ambient temperature and humidity; based on the characteristic decoupling mechanism of the spatial correlation distribution of hot spots, it effectively distinguishes between transient disturbances and steady-state trend characteristics; adopts dynamic segmented matching triggered by humidity changes and trend fitting of temperature slope regulation to ensure adaptive modeling of irradiance mutations and diurnal cycles; through the waveform morphology matching and envelope tracking technology of parallel prediction channels, multi-time scale prediction results are collaboratively generated; and finally, the complete radiation curve formed by frequency domain reconstruction significantly improves the response capability of photovoltaic power prediction to complex meteorological conditions and abnormal equipment operating conditions, providing a more reliable decision-making basis for power station operation and scheduling.

[0068] In some embodiments, the sub-modal prediction results are superimposed in the frequency domain according to a preset reconstruction rule to generate a short-term solar power generation power prediction curve, including: 301. Based on the spatial distribution weights of the temperature gradient values in the hot spot interference feature map, the high-frequency submodal prediction results and the low-frequency submodal prediction results are distributed proportionally to the regions, wherein the distribution ratio of the high-frequency submodal prediction results corresponding to regions where the temperature gradient values are higher than a preset threshold is attenuated by the inverse of the humidity change rate. The spatial distribution weight of the temperature gradient value refers to the normalized weight of the temperature gradient value (ΔT / Δx) in each region of the hot spot interference feature map. ΔT represents the temperature difference, and Δx represents the spatial distance difference. The temperature gradient value ΔT / Δx is used to characterize the spatial rate of change of the PV module surface temperature. It reflects the spatial intensity of hot spot diffusion. Regionalized proportional allocation refers to the spatial differentiation of the high-frequency and low-frequency sub-modal prediction results based on the temperature gradient weight. The high-frequency prediction ratio is reduced in high-temperature gradient areas to avoid overfitting. The inverse attenuation of the humidity change rate refers to the dynamic adjustment of the high-frequency sub-modal allocation ratio according to the inverse (Δt / ΔH) of the humidity change rate (ΔH / Δt). The high-frequency weight is attenuated when the humidity rises suddenly.

[0069] In an embodiment of the present application, the temperature gradient value of each region is first extracted from the hot spot interference characteristic map (such as ΔT / Δx=0.8℃ / cm), normalized into a spatial weight matrix (such as 0.9 for high temperature gradient regions and 0.3 for low temperature regions), and regions with gradient values exceeding a preset threshold (such as 0.5℃ / cm) are screened out; then the inverse of the humidity change rate is calculated (for example, ΔH / Δt=3% / min corresponds to an attenuation coefficient of 0.33) to obtain the attenuation coefficient, and the high-frequency sub-modal prediction results of the region with a temperature gradient value higher than the preset threshold are proportionally distributed according to the weight × attenuation coefficient (such as 0.9×0.33≈0.3), and the low-frequency sub-modal of the region with a temperature gradient value lower than the preset threshold is distributed according to the remaining proportion (1-0.3=0.7), so as to suppress the overfitting effect of high-frequency noise on the hot spot region in the humidity sudden rise scenario, while retaining the global evolution characteristics of the low-frequency trend.

[0070] 302. The allocated high-frequency sub-modal prediction results and the low-frequency sub-modal prediction results are waveform-joined according to the original frequency band order of the sub-modal sequence, and the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the splicing process to generate a short-term solar power generation prediction curve.

[0071] The original frequency band order refers to the frequency band arrangement order of the sub-modal sequence (such as high frequency, medium frequency, low frequency); waveform phase alignment refers to adjusting the phase offset of the sub-modal waveform according to the slope sign (positive / negative) of the temperature change to ensure the physical consistency of the curve after splicing.

[0072] In an embodiment of the present application, waveform synthesis is achieved by splicing original frequency bands based on physical constraints: first, the prediction results are arranged in the order of the original frequency bands of the sub-modes (high frequency to low frequency), and according to the sign of the temperature change slope (such as ΔT / Δt=-0.5℃ / min is a negative value), a phase delay (such as π / 4 radians) is applied to the high-frequency sub-mode, and the phase of the low-frequency sub-mode is advanced (such as π / 8 radians) to match the physical law of "cooling inhibits radiation recovery"; then, the overlap-addition algorithm (Overlap-Add) is used to splice the phase-aligned sub-mode waveforms along the time axis, for example, the high-frequency band is superimposed with a -100W / m² fluctuation in the first 5 minutes, and the low-frequency band is superimposed with a +50W / m² trend in the next hour, and the discontinuity of the splicing boundary is eliminated by weighted interpolation, and finally a smooth radiation prediction curve that conforms to the laws of hot spot diffusion and temperature and humidity evolution is generated.

[0073] In some embodiments, the allocated high-frequency submodal prediction results and the low-frequency submodal prediction results are waveform-joined according to the original frequency band order of the submodal sequence, and the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the splicing process to generate a short-term solar power generation prediction curve, including: 401. Divide the high-frequency submodal prediction results and the low-frequency submodal prediction results into equal-time blocks according to the original arrangement order when the submodal sequence is decomposed, with the high-frequency submodal prediction result blocks in front and the low-frequency submodal prediction result blocks in the back, and the length of each time block is consistent with the update interval of the hot spot interference feature map; Equal time blocks refer to dividing the high-frequency and low-frequency sub-modal prediction results into continuous time periods according to a fixed time length (such as 30 seconds); the hot spot interference feature map update interval refers to the period of recalculation of the hot spot feature map (such as 30 seconds), which determines the length of the time block; the original frequency band sequence arrangement means that the high-frequency sub-modal block is in front and the low-frequency block is in the back, maintaining the frequency band arrangement rules during decomposition.

[0074] In an embodiment of the present application, based on the update interval of the hot spot interference feature map (e.g., 30 seconds), the high-frequency submodal prediction results (e.g., 0.5-2 Hz cloud layer mutation waveform) and the low-frequency submodal prediction results (e.g., <0.1 Hz daily cycle trend) are divided into equal time blocks of 30 seconds. The high-frequency submodal data occupies the first 1 / 3 of the time length of each block (0-10 seconds), and the low-frequency submodal data occupies the last 2 / 3 of the time length (10-30 seconds). The timestamp calibration algorithm (e.g., sliding window dynamic calibration) is used to ensure that the start times of the high-frequency and low-frequency blocks are strictly synchronized. If the hot spot update interval changes dynamically (e.g., from 30 seconds to 20 seconds), the time block length is adjusted synchronously to ensure alignment with the real-time hot spot data.

[0075] 402. In each time block, select alignment reference points and matching points according to the sign direction of the temperature change slope to generate position marks of the reference points of the high-frequency submodal prediction result block and the matching points of the low-frequency submodal prediction result block; The sign and direction of the temperature change slope refers to the temperature change trend (positive: warming, negative: cooling), which determines the selection logic of the alignment reference point. The reference point refers to the reference time point used for alignment within the high-frequency block (such as the starting point or peak point of the time block). The matching point is the target time point in the low-frequency block that is aligned with the reference point.

[0076] In this embodiment of the present application, the reference point of the high-frequency submodal prediction result block is selected based on the sign direction (positive / negative) of the temperature change slope ΔT / Δt: the starting point of the high-frequency submodal prediction result block (0 seconds) is selected for a positive slope, and the end point (10 seconds) is selected for a negative slope. The dynamic time warping (DTW) algorithm is used to calculate the similarity matrix between the waveform of the low-frequency submodal prediction result block and the waveform near the high-frequency reference point (such as the 8-10 second drop segment of the high-frequency submodal prediction result block). The minimum path cumulative cost (Bellman-Ford algorithm) is then used to determine the best matching point within the low-frequency submodal prediction result block (such as the 25th second of the low-frequency submodal prediction result block). If the temperature change slope ΔT / Δt exceeds a threshold (such as ±0.5°C / min), the matching point position is dynamically shifted according to the rate ratio (for example, when ΔT / Δt = -1°C / min, the matching point is moved forward by 3 seconds).

[0077] 403. Align the position marks of the reference points of the high-frequency submodal prediction result block and the matching points of the low-frequency submodal prediction result block in each time block vertically along the time axis. After the alignment, retain all waveform data before the reference points in the high-frequency submodal prediction result block, retain all waveform data after the matching points in the low-frequency submodal prediction result block, and directly connect the reference points of the high-frequency submodal prediction result block and the matching points of the low-frequency submodal prediction result block. The timestamps at the connection points are continuous and the amplitude values are equal, thereby generating a high-frequency truncated block and a low-frequency truncated block. Vertical alignment refers to placing the reference point of the high-frequency submodal prediction result block and the matching point of the low-frequency block at the same time axis position; the truncation rule refers to retaining the high-frequency waveform before the reference point and the low-frequency waveform after the matching point, and directly connecting the two points.

[0078] In this embodiment of the present application, the reference point of the high-frequency submodal prediction result block (e.g., the 10th second) and the matching point of the low-frequency submodal prediction result block (e.g., the 25th second) are forcibly aligned along the time axis to the same timestamp (the 25th second). The 0-10 second data of the high-frequency submodal prediction result block is losslessly compressed using a phase vocoder algorithm to maintain the waveform frequency characteristics, thereby generating a high-frequency truncated block. The 10-30 second data of the low-frequency submodal prediction result block is padded with data within the 25-30 second interval using a linear interpolation algorithm to ensure waveform continuity, thereby generating a low-frequency truncated block. At the connection point (the 25th second), the high-frequency end amplitude (e.g., 350W / m²) and the low-frequency starting amplitude (e.g., 352W / m²) are linearly interpolated to generate a smooth transition segment (e.g., a 0.4W / s gradient), that is, the timestamps at the connection are continuous and the amplitude values are equal. If the amplitude difference exceeds ±5%, the rematching mechanism is triggered: return to step 402 to reselect the matching points and adjust the alignment.

[0079] 404. Connect the high-frequency truncation blocks and the low-frequency truncation blocks of all time blocks in series in the original frequency band order to generate a short-term solar power generation power prediction curve.

[0080] The original frequency band sequential concatenation refers to splicing all time blocks in the order of high frequency first and low frequency last.

[0081] In the embodiment of the present application, within each 30-second time block, the high-frequency band (0-25 seconds) and the low-frequency band (25-30 seconds) are seamlessly spliced together in the original frequency band order through a sliding window overlap-addition algorithm. Multiple blocks are connected in chronological order to form a complete curve (e.g., a 30-minute prediction consists of 60 blocks), and a timestamp compensation algorithm (e.g., linear interpolation) is used to eliminate small errors between blocks (e.g., ±0.1 seconds). Finally, the complete curve is checked for energy conservation (e.g., total energy integral deviation ≤ 2%). If the error exceeds the limit, local re-splicing is triggered (returning to step 401 for re-blocking) to ensure that the prediction result is physically reasonable.

[0082] This solution solves the problems of waveform breakage and phase jumps in traditional frequency domain superposition through a temperature slope-driven phase alignment and time block truncation and splicing mechanism: dynamically matching reference points and matching points eliminates waveform morphology conflicts caused by temperature changes; vertical alignment and truncation rules ensure time continuity and smooth amplitude transition; adapting to real-time interference scenarios such as hot spot diffusion and sudden meteorological changes, it improves the physical rationality and engineering practicality of the prediction curve, providing high-reliability power prediction support for photovoltaic power stations.

[0083] In some embodiments, obtaining temperature field distribution data of a photovoltaic module, correlating the temperature field distribution data with hot spot area data collected by a thermal infrared imager, and generating a hot spot interference feature map in combination with the backplane temperature data includes: 501. Collect temperature values of various points on the surface of the photovoltaic module using a temperature measuring device to form temperature field distribution data including position coordinates and temperature values; The temperature measuring device refers to a distributed temperature sensor array or an infrared thermometer, which is used to collect the temperature values of various points on the surface of the photovoltaic module; the temperature field distribution data refers to the matrix data containing the position coordinates (such as x, y) and corresponding temperature values of each measurement point on the surface of the photovoltaic module.

[0084] In an embodiment of the present application, a distributed temperature sensor (such as DS18B20) is installed on the back panel or frame of the photovoltaic module to collect the surface temperature with a grid density of 10cm×10cm, and the coordinates (x, y) and temperature value T of each grid point are recorded; the collected temperature data is organized into a two-dimensional matrix, the row / column index corresponds to the position coordinate, the matrix element value is the temperature value, and the vacant position is filled by interpolation (such as bilinear interpolation); a complete temperature field distribution data matrix is generated with a resolution of 10cm / pixel and a temperature accuracy of ±0.5℃.

[0085] 502. Acquire hot spot area data on the surface of the photovoltaic module using a thermal infrared imaging device, identify local areas in the hot spot area data that have a temperature significantly higher than the surrounding area and are closed in shape, and record the closed boundary coordinates and center coordinates of each local area; A thermal infrared imaging device refers to an infrared thermal imager (such as the FLIR A655sc), which is used to obtain temperature distribution images on the surface of photovoltaic modules. Hot spot area data refers to abnormally high temperature areas identified from infrared images, where the temperature is significantly higher than the surrounding area (ΔT ≥ 5°C) and the shape is closed.

[0086] In an embodiment of the present application, an infrared image of the surface of a photovoltaic module is captured at intervals of 30 seconds by a thermal infrared imaging device (resolution 640×480 pixels, temperature sensitivity 0.05°C); the image is subjected to Gaussian filtering for noise reduction, and the Canny edge detection algorithm is used to extract local areas in the hot spot area data where the temperature is significantly higher than the surrounding area and the shape is closed. The adjacent high-temperature pixels are merged through the region growing algorithm, and closed areas with an area ≥4cm² and ΔT ≥5°C are screened; the coordinates of the vertices of the closed boundary polygon (such as the four corner points of a rectangular area) and the coordinates of the center point (calculated as the center of mass) of each local area are recorded.

[0087] 503. Extract local temperature values overlapping with the closed boundary coordinates of the hot spot area data from the temperature field distribution data, and calculate the average temperature value and the maximum temperature value of the temperature values in each overlapping area; Overlapping area: the local area in the temperature field distribution data that overlaps with the hot spot boundary coordinates; temperature characteristic values: the average temperature (T_avg) and maximum temperature (T_max) in the overlapping area.

[0088] In an embodiment of the present application, the closed boundary vertex coordinates of the hot spot area data (infrared image coordinate system) are converted to the grid coordinate system of the temperature field distribution data (such as through affine transformation); a sub-matrix (such as a 5×5 grid area) overlapping with the hot spot boundary is intercepted in the temperature field distribution data matrix; the average value T_avg of the valid temperature points in the sub-matrix is calculated (ignoring the 0-value interpolation points), and the maximum value of the sub-matrix is taken as T_max, and the average temperature value and the maximum temperature value of the temperature values in each overlapping area are calculated.

[0089] 504. Arrange the average temperature value, the maximum temperature value, and the backplane temperature data of each overlapping region in a fixed order to form a temperature feature sequence corresponding to each overlapping region; Backplane temperature data refers to the average temperature value (T_back) collected by the PT1000 sensor installed on the component backplane; the temperature feature sequence refers to a one-dimensional array arranged in a fixed order of [T_avg, T_max, T_back].

[0090] In an embodiment of the present application, the average temperature (T_avg) and the maximum temperature (T_max) of the hot spot overlapping area obtained from step 503 are, for example, T_avg=58.1°C and T_max=60.5°C; the weighted average value of the backplane temperature sensor network (such as 4 PT1000s arranged at the four corners of the component) is taken as the backplane temperature data T_back (such as 42.3°C), and the weight is distributed in inverse proportion to the distance between the sensor and the center of the hot spot; a one-dimensional feature sequence is generated in a fixed order of [average temperature T_avg of the hot spot overlapping area, maximum temperature T_max, backplane temperature data T_back]; and a temperature feature sequence corresponding to each overlapping area is formed.

[0091] 505. Based on the center coordinates of the hot spot area data, the temperature feature sequence of each overlapping area is mapped to a two-dimensional grid consistent with the position coordinates of the temperature field distribution data, and the unmapped grid positions are filled with zero values to generate a hot spot interference feature map containing the temperature characteristics of the hot spot area and its position distribution.

[0092] Two-dimensional grid mapping refers to mapping the temperature feature sequence according to the coordinates of the hot spot center to a grid with the same resolution as the temperature field distribution data; the hot spot interference feature map refers to a three-dimensional matrix, the first two dimensions are the position coordinates, and the third dimension is the temperature feature channel ([T_avg, T_max, T_back]).

[0093] In an embodiment of the present application, the center coordinates of the hot spot area data are converted from the infrared image coordinate system to a two-dimensional grid coordinate system consistent with the position coordinates of the temperature field distribution data, and the center grid position is determined by rounding; the [T_avg, T_max, T_back] sequence of each hot spot area data is written into channel 1 / 2 / 3 of the corresponding center grid, and the influence radius r is calculated according to the actual coverage range (boundary coordinates) of the hot spot area data, and the characteristic values of the neighboring grids within the radius are filled in according to the distance weighting; the grids not covered by the hot spot area data are kept at zero value in channel 1 / 2, and channel 3 is filled with the measured value of the backplane temperature at that position (if there is no data, it is interpolated); the physical rationality of T_max≥T_avg≥T_back is checked, and the isolated non-zero grids are morphologically expanded, and finally a normalized hot spot interference feature map containing the temperature characteristics of the hot spot area and its position distribution is output, and statistical indicators such as the number of hot spots and the maximum temperature rise are recorded to provide structured input for subsequent spatial feature analysis.

[0094] This solution fully automates the hot spot detection and feature extraction process. By fusing multi-source temperature data and structuring spatial features, it accurately identifies and locates abnormal hot spot areas on the module surface, comprehensively characterizing the temperature characteristics of hot spots and their impact on module status. The resulting standardized feature maps can be directly used for subsequent analysis and prediction, providing reliable data support for fault diagnosis and operation and maintenance decision-making in photovoltaic power plants, significantly improving the efficiency and accuracy of hot spot monitoring.

[0095] In some embodiments, generating a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model includes: 601. Decompose each submodal sequence into components corresponding to multiple frequency intervals using the multimodal time series prediction model, record the frequency interval range of each component and the amplitude value that changes over time within the frequency interval, and arrange the frequency interval ranges and corresponding amplitude values in a fixed order to form a frequency domain feature data sequence for each submodal. Frequency interval refers to the decomposition of the submodal sequence into different frequency bands through Fourier transform; amplitude value: the amplitude intensity of the signal in each frequency band within the time window.

[0096] In this embodiment, the multimodal time series prediction model first adaptively divides each input submodal sequence (e.g., the 0.5-2 Hz fluctuation data of the high-frequency submodal) into frequency intervals. An improved wavelet packet decomposition algorithm (based on the db4 wavelet basis) is then used to decompose the signal into multiple key frequency intervals (e.g., a 0.5-2 Hz high-frequency submodal can be decomposed into three intervals: 0.5-1 Hz, 1-1.5 Hz, and 1.5-2 Hz. The specific number of intervals and their boundaries are dynamically determined based on the spectral energy distribution). The boundaries of each interval are dynamically adjusted by the model based on the spectral energy distribution of the current signal. Then, a Hilbert transform is performed on the signal components within each frequency interval to extract their time-varying amplitude envelopes. The maximum amplitude within the interval and its corresponding timestamp are recorded (e.g., the 1-1.5 Hz interval has a peak amplitude of 25 W / m² at t = 15 s). Finally, the data is organized according to a fixed format of [interval lower limit, interval upper limit, peak amplitude 1, timestamp 1, peak amplitude 2, timestamp 2, ...] to form a structured frequency domain feature data sequence.

[0097] 602. Extracting spacing values and relative azimuth angle values between non-zero regions in the hot spot interference feature map based on the coordinate distribution of the non-zero regions, and arranging the spacing values and relative azimuth angle values in a preset order to form a spatial correlation data sequence of the hot spot interference feature map; The non-zero area refers to the grid area where T_avg>0 in the hot spot interference characteristic map; the spacing value is the Euclidean distance between the center points of the two hot spots (such as the grid unit distance 5); the azimuth angle refers to the angle between the line connecting the centers of the hot spots and the horizontal axis with the lower left corner of the component as the origin (such as 30°); the relative azimuth angle value refers to the angle between the line connecting the center points of the two hot spots and the reference direction based on the preset reference direction of the photovoltaic component surface (such as the horizontal direction or the long side direction of the component), which is used to quantify the spatial orientation relationship between the hot spots.

[0098] In the embodiment of the present application, all non-zero grid center coordinates (e.g., (18, 23), (45, 60)) are extracted from the hot spot interference feature map, and the Euclidean distance (e.g., √[(45-18)²+(60-23)²]=46) and relative azimuth (atan²(37, 27)=53.8°) of each pair of hot spots are calculated. The hot spot spatial topology is established through Delaunay triangulation. After removing abnormal isolated points, the spatial correlation data sequence of the hot spot interference feature map is generated in the order of [hot spot 1x, hot spot 1y, hot spot 2x, hot spot 2y, spacing, angle...] to quantify the spatial distribution pattern of hot spot clusters.

[0099] 603. Divide the ambient temperature and humidity data into continuous time windows in chronological order, count the direction of continuous increase, continuous decrease, or stable change of the temperature and humidity values in each time window, and arrange the temperature change direction and humidity change direction in each time window in chronological order to form a time-dependent data sequence of the ambient temperature and humidity data; Change direction: Temperature / humidity is monotonic within the time window (increasing, decreasing, or stable); Time window: a fixed duration segment (such as 5 minutes).

[0100] In an embodiment of the present application, the ambient temperature and humidity data are divided into 5-minute non-overlapping windows in time order, and the linear regression slope of the temperature / humidity in each window is calculated. The direction of temperature change is defined as follows: a slope > 0.1°C / min indicates an increase, a slope < -0.1°C / min indicates a decrease, and otherwise it is unchanged. The direction of humidity change is defined as follows: a slope > 0.1% / min indicates an increase, a slope < -0.1% / min indicates a decrease, and otherwise it is unchanged. "Sudden rise" and "sudden drop" labels are added for sudden changes in temperature and humidity (such as ΔH / Δt > 2% / min). Finally, the temporal dependency data sequence of the ambient temperature and humidity data is arranged and output in the time order of [window 1 temperature direction, humidity direction, window 2...] to encode the temporal evolution law of the meteorological parameters.

[0101] 604. Sequentially concatenate the frequency domain feature data sequence, the spatial correlation data sequence, and the temporal dependency data sequence into a single data sequence as a joint prediction input.

[0102] Length alignment refers to padding or truncating the three sequences to make them consistent in length (for example, unified to 100 dimensions); splicing: merging them into a 300-dimensional vector in the order of [frequency domain sequence, spatial sequence, time series sequence]; normalization refers to performing Min-Max normalization on each modal data to eliminate dimensional differences.

[0103] In this embodiment of the present application, the frequency domain feature data sequence, spatial correlation data sequence, and temporal dependency data sequence are length-normalized: the frequency domain feature data sequence is padded to 150 dimensions, the spatial correlation data sequence is truncated to 100 dimensions, and the temporal dependency data sequence is interpolated to 50 dimensions. Weighted concatenation is performed using attention weight allocation (frequency domain: space: temporal = 4:3:3) to generate a 300-dimensional joint input vector. Layer normalization (LayerNorm) is used to eliminate dimensional differences between modalities, and the final output is a single data sequence that can be directly input into multimodal prediction models such as LSTM / Transformer as a joint prediction input.

[0104] This solution achieves accurate status monitoring and power prediction of photovoltaic power generation systems through deep fusion technology of multimodal data. The system innovatively combines frequency domain feature analysis, hot spot spatial distribution characteristics and environmental parameter time series change characteristics to build a multi-dimensional joint prediction model. At the technical implementation level, the solution accurately captures the frequency domain characteristics of radiation fluctuations through advanced signal processing algorithms, combines the spatial correlation analysis of hot spot distribution, and integrates the time series evolution laws of environmental parameters to form a complete feature characterization system. This solution significantly improves the robustness of predictions under complex meteorological conditions and equipment abnormal conditions, and can effectively support the refined scheduling decisions of the power grid. At the same time, the system adopts an efficient computing architecture to realize real-time data processing, providing an integrated solution from anomaly detection to power prediction for photovoltaic power station operation and maintenance, which has important engineering application value.

[0105] In some embodiments, decomposing the sudden fluctuation component in the long and short wave component data into a sub-modal sequence includes: 701. Scan the long and short wave component data in chronological order to identify a target segment, where the target segment is a segment where the amplitude of the value change exceeds twice the average amplitude of the value change in adjacent time periods, and record the start and end time points of the target segment; The change amplitude threshold refers to twice the average change amplitude of adjacent time periods and is used to determine data mutations; the segment mark refers to the start and end time points of recording the mutation.

[0106] In this embodiment, a sliding window algorithm (with a 30-second window) is used to scan the long-wave and short-wave component data in real time. When a change in value at a certain moment exceeds 200% of the average change in the adjacent time period, it is marked as the start time of the mutation. Tracking is continued until the change falls below a threshold, which is then determined as the end time. Finally, a list containing the start and end times of all mutations (e.g., 10:15:00-10:15:30) is output. A dynamic threshold mechanism is used to prevent weak mutations from being missed by fixed thresholds, and the peak amplitude and duration of the mutation are simultaneously recorded.

[0107] 702. Extracting the data segment between the starting time point and the ending time point in the long and short wave component data as an independent fluctuation component, and merging multiple adjacent independent fluctuation components whose time interval is less than a preset threshold into a single sudden change fluctuation component; Independent fluctuation components refer to data segments within a single mutation segment; the merging threshold refers to merging adjacent mutations into the same component when the interval between them is less than 1 minute.

[0108] In this embodiment of the present application, based on the sudden change start and end time marks at the start and end time points in the long and short wave component data described in step 701, independent fluctuation components are intercepted from the original data, and the interval between adjacent components is checked. If the interval is less than a preset merging threshold (default 1 minute), the multiple independent fluctuation components are merged into a single composite component (e.g., merging 10:15:00-10:15:30 and 10:15:50-10:16:30 into 10:15:00-10:16:30). The merging process retains the amplitude characteristics of each sub-component and outputs the integrated sudden change component time series.

[0109] 703. Divide each sudden change fluctuation component into a plurality of continuous sub-fluctuation segments according to the time length, wherein the time span of each sub-fluctuation segment matches the duration when the temperature change slope in the ambient temperature and humidity data exceeds a preset value; Sub-fluctuation segments refer to dividing the mutation component into segments whose duration matches the temperature and humidity changes; the matching rule is that the sub-segment duration = the duration of significant changes in temperature and humidity.

[0110] In this embodiment, each sudden fluctuation component is divided into several sub-segments based on the duration of the temperature change slope > 0.5°C / min or the humidity change rate > 1% / min in the ambient temperature and humidity data (e.g., 2 minutes). An adaptive equal division algorithm is used to ensure that the duration of the sub-fluctuation segment strictly matches the temperature and humidity change period in the ambient temperature and humidity data (with an error of <±5 seconds), and a sub-segment segmentation list with timestamps is output.

[0111] 704. Compare the time range of each sub-fluctuation segment with the time period in which the humidity change rate in the ambient temperature and humidity data exceeds a preset threshold, and mark the sub-fluctuation segment as a high-frequency sub-modal sequence and a low-frequency sub-modal sequence according to a preset overlap ratio; The overlap ratio threshold means that when the sub-segment overlaps with the temperature and humidity mutation period by ≥60%, it is marked as high frequency, otherwise it is marked as low frequency.

[0112] In this embodiment, the overlap ratio between each sub-fluctuation segment and the period of sudden change in ambient temperature and humidity data is calculated. When the time overlap is ≥60%, it is labeled as a high-frequency sub-mode (e.g., 10:15:00-10:15:40); otherwise, it is labeled as a low-frequency sub-mode. Sliding correlation analysis is used to verify the reliability of the labeling, removing abnormal labels (segments with correlation coefficients < 0.6 require re-evaluation), and outputting high-frequency and low-frequency sub-mode sequences with classification labels.

[0113] 705. Arrange all marked high-frequency submodal sequences in chronological order to form a first set, arrange all marked low-frequency submodal sequences in chronological order to form a second set, and merge the first set and the second set into a complete submodal sequence.

[0114] Set generation refers to sorting the high-frequency / low-frequency sub-modal sequences by time to form two independent sets; sequence merging refers to interleaving and merging the two sets along the original time axis while retaining the tag information.

[0115] In this step, the marked high-frequency submodal sequences (such as 10:15:00-10:15:40, 10:18:20-10:19:10) are arranged in chronological order to form the first set, and the low-frequency submodal sequences (such as 10:16:20-10:16:30, 10:19:30-10:20:00) are arranged in chronological order to form the second set; then the two sets are merged according to the original data time sequence through the time axis alignment algorithm, retaining all submodal marking information (high frequency / low frequency). Frequency) and time range tags, and high-frequency priority processing rules are implemented for overlapping time periods; finally, a submodal sequence in a unified format (including start and end time and type tags) is generated and statistical information of the two sets is output (the number, total duration and average amplitude of high-frequency sets, and the number, total duration and average change rate of low-frequency sets). At the same time, interpolation compensation is performed for discontinuous time stamp intervals and conflict resolution logs are recorded to ensure that the output submodal sequence maintains temporal continuity and clearly distinguishes set attributes, providing structured input for subsequent multimodal analysis.

[0116] This solution achieves three major technological breakthroughs through an innovative five-step decomposition process: First, a mutation detection mechanism using a dynamic relative threshold (twice the mean) improves mutation recognition compared to traditional fixed-threshold methods; second, the physical correlation between temperature and humidity changes guides sub-segment division, ensuring that the decomposition results conform to actual meteorological variability; and finally, clearly distinguished high- and low-frequency submodal sets provide distinct input data for the prediction model. Practical applications have demonstrated that this solution improves the accuracy of radiation fluctuation forecasts under cloudy conditions while supporting real-time response capabilities at the minute level. The decomposition results also intuitively reflect cloud movement characteristics, providing important decision-making basis for cloud shadow prediction in photovoltaic power plants.

[0117] Figure 2 The present invention provides a schematic diagram of a solar power generation real-time prediction system based on multimodal deep learning, as shown in FIG. Figure 2 As shown, the system includes: The acquisition module 21 collects long-wave and short-wave component data of surface solar radiation, backplane temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposes the sudden fluctuation component in the long-wave and short-wave component data into a sub-modal sequence; The first generating module 22 acquires temperature field distribution data of the photovoltaic module, associates the temperature field distribution data with hot spot area data collected by the thermal infrared imager, and generates a hot spot interference feature map in combination with the backplane temperature data; The second generation module 23 inputs the submodal sequence, the hot spot interference feature map, and the ambient temperature and humidity data into a multimodal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model; The output module 24 outputs the prediction results of each sub-modality according to the joint prediction input through the multi-modal time series prediction model, and superimposes the prediction results of each sub-modality in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation power prediction curve.

[0118] Figure 2 The solar power generation real-time prediction system based on multimodal deep learning can be executed Figure 1 The implementation principles and technical effects of the multimodal deep learning-based real-time solar power prediction method described in the illustrated embodiment are not further elaborated. The specific manner in which each module and unit performs operations in the multimodal deep learning-based real-time solar power prediction system described in the above embodiment has been described in detail in the relevant embodiments of the method and will not be further elaborated here.

[0119] In one possible design, Figure 2 The solar power generation real-time prediction system based on multimodal deep learning of the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32; The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0120] The processing component 32 is used for the above Figure 1 The embodiment provides a real-time prediction method for solar power generation based on multimodal deep learning.

[0121] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0122] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0123] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0124] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0125] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0126] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0127] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment provides a method for real-time prediction of solar power generation power based on multimodal deep learning.

[0128] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0130] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A real-time prediction method for solar power generation based on multimodal deep learning, characterized in that: include: Collecting long- and short-wave component data of surface solar radiation, backplane temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposing the sudden fluctuation components in the long- and short-wave component data into sub-modal sequences; Acquiring temperature field distribution data of the photovoltaic module, correlating the temperature field distribution data with hot spot area data collected by a thermal infrared imager, and generating a hot spot interference feature map in combination with the backplane temperature data; Inputting the submodal sequence, the hot spot interference feature map, and the ambient temperature and humidity data into a multimodal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model; The multimodal time series prediction model outputs the prediction results of each submodal according to the joint prediction input, and superimposes the prediction results of each submodal in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation power prediction curve.

2. The method according to claim 1, characterized in that Outputting prediction results of each sub-modality according to the joint prediction input by the multi-modal time series prediction model includes: Couple the frequency domain features of the submodal sequence with the temporal dependency of the ambient temperature and humidity data to generate a frequency domain-temporal hybrid feature that characterizes the dynamic association between irradiance mutation and temperature and humidity, wherein the temporal dependency of the temperature and humidity data is quantified by multiplying the humidity change rate and the temperature change slope within a sliding time window; According to the spatial correlation distribution of the hot spot interference feature map, the frequency domain-time series hybrid feature is divided into a transient response feature corresponding to a high-frequency sub-modal sequence and a steady-state response feature corresponding to a low-frequency sub-modal sequence; Performing segmented matching processing on the transient response characteristics based on the duration of the irradiance mutation, wherein the start time of each segment is triggered by the time when the humidity change rate in the ambient temperature and humidity data exceeds a preset threshold, and the segment length is synchronously adjusted with the update interval of the hot spot interference feature map; performing a trend fitting process based on a diurnal cycle on the steady-state response characteristics, wherein a time span of the trend fitting is dynamically controlled by a sign flipping frequency of the temperature change slope; The transient response characteristics after the segmented matching processing and the steady-state response characteristics after the trend fitting processing are input into the parallel prediction channel, so that the transient response characteristics are subjected to waveform morphology matching through the parallel prediction channel to generate high-frequency submodal prediction results, and the steady-state response characteristics are subjected to waveform envelope tracking to generate low-frequency submodal prediction results.

3. The method according to claim 2, characterized in that The prediction results of each sub-modal are superimposed in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation prediction curve, including: Based on the spatial distribution weight of the temperature gradient value in the hot spot interference feature map, the high-frequency submodal prediction results and the low-frequency submodal prediction results are regionally proportionally allocated, wherein the allocation ratio of the high-frequency submodal prediction results corresponding to the area where the temperature gradient value is higher than the preset threshold is attenuated according to the inverse of the humidity change rate; The allocated high-frequency submodal prediction results and low-frequency submodal prediction results are waveform-spliced according to the original frequency band order of the submodal sequence. During the splicing process, the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment to generate a short-term solar power generation prediction curve.

4. The method according to claim 3, characterized in that The allocated high-frequency submodal prediction results and low-frequency submodal prediction results are waveform-joined according to the original frequency band order of the submodal sequence, and the sign direction of the temperature change slope is introduced as a constraint condition for waveform phase alignment during the joining process to generate a short-term solar power generation prediction curve, including: Dividing the high-frequency submodal prediction results and the low-frequency submodal prediction results into equal-time blocks according to the original arrangement order when the submodal sequence is decomposed, wherein the high-frequency submodal prediction result block is in front and the low-frequency submodal prediction result block is in the back, and the length of each time block is consistent with the update interval of the hot spot interference feature map; In each time block, an alignment reference point and a matching point are selected according to the sign direction of the temperature change slope to generate position marks of the reference point of the high-frequency submodal prediction result block and the matching point of the low-frequency submodal prediction result block; Aligning the position marks of the reference points of the high-frequency submodal prediction result block and the matching points of the low-frequency submodal prediction result block of each time block vertically along the time axis, retaining all waveform data before the reference points in the high-frequency submodal prediction result block after the alignment, retaining all waveform data after the matching points in the low-frequency submodal prediction result block, and directly connecting the reference points of the high-frequency submodal prediction result block and the matching points of the low-frequency submodal prediction result block, with the timestamps at the connection being continuous and the amplitude values being equal, thereby generating a high-frequency band truncated block and a low-frequency band truncated block; The high-frequency truncated blocks and low-frequency truncated blocks of all time blocks are sequentially connected in series according to the original frequency band order to generate a short-term solar power generation prediction curve.

5. The method according to claim 1, wherein Acquiring temperature field distribution data of the photovoltaic module, correlating the temperature field distribution data with hot spot area data collected by a thermal infrared imager, and generating a hot spot interference feature map in combination with the backplane temperature data, including: The temperature value of each point on the surface of the photovoltaic module is collected by the temperature measuring device to form temperature field distribution data including position coordinates and temperature values; Acquire hot spot area data on the surface of the photovoltaic module using a thermal infrared imaging device, identify local areas in the hot spot area data that have a temperature significantly higher than the surrounding area and are closed in shape, and record the closed boundary coordinates and center coordinates of each local area; Extracting local temperature values overlapping with the closed boundary coordinates of the hot spot area data from the temperature field distribution data, and calculating the average temperature value and the maximum temperature value of the temperature values in each overlapping area; Arrange the average temperature value, the maximum temperature value and the backplane temperature data of each overlapping area in a fixed order to form a temperature feature sequence corresponding to each overlapping area; According to the center coordinates of the hot spot area data, the temperature feature sequence of each overlapping area is mapped to a two-dimensional grid consistent with the position coordinates of the temperature field distribution data, and the unmapped grid positions are filled with zero values to generate a hot spot interference feature map containing the temperature characteristics of the hot spot area and its position distribution.

6. The method according to claim 1, characterized in that The multimodal time series prediction model is used to fuse the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the time series dependency of the ambient temperature and humidity data to generate a joint prediction input, including: Decomposing each submodal sequence into components corresponding to a plurality of frequency intervals using the multimodal time series prediction model, recording the frequency interval range of each component and the amplitude value varying over time within the frequency interval, and arranging the frequency interval ranges and the corresponding amplitude values in a fixed order to form a frequency domain feature data sequence for each submodal; Extracting spacing values and relative azimuth angle values between non-zero areas according to the coordinate distribution of non-zero areas in the hot spot interference feature map, and arranging the spacing values and relative azimuth angle values in a preset order to form a spatial correlation data sequence of the hot spot interference feature map; Dividing the ambient temperature and humidity data into continuous time windows in chronological order, counting the direction of continuous increase, continuous decrease, or stable change of the temperature and humidity values in each time window, and arranging the temperature change direction and humidity change direction in each time window in chronological order to form a time-dependent data sequence of the ambient temperature and humidity data; The frequency domain feature data sequence, the spatial correlation data sequence and the temporal dependency data sequence are sequentially spliced into a single data sequence as a joint prediction input.

7. The method according to claim 1, characterized in that: Decomposing the sudden change fluctuation component in the long and short wave component data into a sub-modal sequence includes: Scanning the long and short wave component data in chronological order to identify a target segment, wherein the target segment is a segment in which the amplitude of the value change exceeds twice the average amplitude of the value change in adjacent time periods, and recording the start time point and the end time point of the target segment; The data segment between the starting time point and the ending time point in the long and short wave component data is intercepted as an independent fluctuation component, and multiple independent fluctuation components that are adjacent and have a time interval less than a preset threshold are merged into a single sudden change fluctuation component; Divide each sudden fluctuation component into a plurality of continuous sub-fluctuation segments according to the time length, wherein the time span of each sub-fluctuation segment matches the duration of the temperature change slope exceeding the preset value in the ambient temperature and humidity data; Comparing the time range of each sub-fluctuation segment with the period in which the humidity change rate in the ambient temperature and humidity data exceeds a preset threshold, and marking the sub-fluctuation segment as a high-frequency sub-modal sequence and a low-frequency sub-modal sequence according to a preset overlap ratio; All marked high-frequency submodal sequences are arranged in chronological order to form a first set, and all marked low-frequency submodal sequences are arranged in chronological order to form a second set, and the first set and the second set are merged into a complete submodal sequence.

8. A solar power generation real-time prediction system based on multimodal deep learning, characterized in that: include: An acquisition module collects long- and short-wave component data of surface solar radiation, backplane temperature data of photovoltaic modules, and ambient temperature and humidity data, and decomposes the sudden fluctuation components in the long- and short-wave component data into sub-modal sequences; The first generation module acquires temperature field distribution data of the photovoltaic module, associates the temperature field distribution data with hot spot area data collected by the thermal infrared imager, and generates a hot spot interference feature map in combination with the backplane temperature data; a second generation module, inputting the submodal sequence, the hot spot interference feature map, and the ambient temperature and humidity data into a multimodal time series prediction model, so as to generate a joint prediction input by fusing the frequency domain features of the submodal sequence, the spatial correlation of the hot spot interference feature map, and the temporal dependency of the ambient temperature and humidity data through the multimodal time series prediction model; The output module outputs the prediction results of each sub-modality according to the joint prediction input through the multi-modal time series prediction model, and superimposes the prediction results of each sub-modality in the frequency domain according to the preset reconstruction rules to generate a short-term solar power generation power prediction curve.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a real-time prediction method for solar power generation based on multimodal deep learning as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, a real-time prediction method for solar power generation based on multimodal deep learning as described in any one of claims 1 to 7 is implemented.

Images