A DNI prediction method based on dense optical flow method and deep learning network

Abstract

The application relates to the field of DNI prediction, and discloses a DNI prediction method based on a dense optical flow method and a deep learning network, which comprises the following steps: performing distortion correction on images collected by a full-sky imager; according to images at two time points before and after distortion correction, the speed of each pixel point of a current image is calculated by using a dense optical flow method; when the maximum pixel speed is less than a specified speed threshold, a static model is used to predict the DNI at a future time point, otherwise, a motion model is used to predict the DNI at the future time point; when the motion model is used, a trained deep learning network model is used to predict the clear sky index at the future time point, and then the clear sky index at the future time point is multiplied by the clear sky DNI at the future time point, so that the predicted DNI at the future time point can be obtained. The method disclosed by the application can more accurately reflect the change of the actual DNI, has better prediction accuracy, and is highly adaptable.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of DNI prediction, and in particular to a DNI prediction method based on dense optical flow method and deep learning network. BACKGROUND

[0002] In recent decades, solar energy has emerged as an important renewable and environmentally friendly energy source, with its ecological benefits and cost advantages attracting widespread attention. As the scale of solar energy utilization continues to expand, particularly in the field of large-scale solar power plants, the importance of solar radiation prediction accuracy has become increasingly prominent. Among the various components of a solar energy system, the heliostat field system is particularly unique, consisting of a large number of mirrors (heliostats) that focus sunlight onto a central heat absorber for power generation. However, the application of concentrated solar power (CSP) technology still faces many technological challenges, with effective control of CSP operation being particularly critical. When cloud cover causes significant fluctuations in irradiance, accurate short-term DNI (direct normal irradiance) prediction is particularly important. Because such fluctuations can cause significant thermal shock to the heat absorber, leading to rapid temperature changes that affect system stability and operational efficiency. Therefore, developing advanced short-term DNI prediction models is of decisive significance for improving the efficiency, stability, and overall competitiveness of CSP systems.

[0003] Traditional DNI prediction methods mainly rely on statistical methods, such as time series analysis and regression analysis, to predict DNI by mining the correlation between historical data and solar irradiance. However, these methods struggle to handle the complex nonlinear relationships between various meteorological variables and solar irradiance, limiting further improvements in prediction accuracy.

[0004] All-weather imagers are commonly used in concentrated solar power plants (CSP) to monitor and collect cloud image data. DNI prediction methods based on all-weather images have also been developed. There are some methods in the prior art that use convolutional neural networks or self-attention mechanism network models based on all-sky imaging. For example, patent CN114021442A uses a red-to-blue ratio for cloud detection and then uses a VGG-16 convolutional neural network for DNI prediction. In patent CN118296499A, the fusion model uses a ViT model and a time series analysis model to predict DNI, but does not consider the impact of clear sky DNI fitting. These methods all use the same model to predict DNI in different weather conditions and fail to effectively judge and explicitly process cloud-covered areas in the model. Therefore, these methods are still not accurate enough for predicting cloudy weather (i.e., where DNI mutations are likely to occur). SUMMARY

[0005] To solve the above technical problems, the application provides a DNI prediction method based on a dense optical flow method and a deep learning network, which can more accurately reflect the change of actual DNI, has better prediction accuracy and strong adaptability.

[0006] To achieve the above purpose, the technical scheme of the application is as follows:

[0007] A DNI prediction method based on a dense optical flow method and a deep learning network comprises the following steps:

[0008] Step one, converting the image collected by the fisheye camera into pinhole camera imaging to realize distortion correction;

[0009] Step two, according to the images before and after distortion correction, the speed of each pixel point in the current image is calculated by using the dense optical flow method;

[0010] Step three, when the maximum pixel speed is less than the specified speed threshold, a static model is used to predict the DNI at the future time, otherwise a motion model is used to predict the DNI at the future time;

[0011] When the static model is used, when the current DNI is greater than or equal to the specified DNI threshold, the clear sky DNI function is used to predict the DNI at the future time, otherwise it is considered that the DNI at the future time is equal to the DNI at the current time;

[0012] When the motion model is used, the trained deep learning network model is used to predict the clear sky index at the future time, and then the clear sky index at the future time is multiplied by the clear sky DNI at the future time to obtain the predicted DNI at the future time.

[0013] In the above scheme, the specific method of step one is as follows:

[0014] The relationship between fisheye camera imaging and pinhole camera imaging is as follows:

[0015] For a point on the pinhole camera imaging , the distance from the point to the image center is:

[0016] ;

[0017] The tangent value of the angle between the corresponding point in space and the pinhole camera imaging plane is represented as:

[0018] ;

[0019] Wherein, represents the focal length of the pinhole camera model corresponding to the distortion-corrected all-sky imaging; and the angle is:​

[0020] ;

[0021] If the pinhole camera is directed upward, the angle is the pitch angle; the azimuth angle of the point is denoted as A, and the sine and cosine of the angle are respectively:

[0022] ;

[0023] ;

[0024] The distance of the point to the image center in fisheye camera imaging is:

[0025] ;

[0026] wherein, and are the first-order and zero-order parameters in the fisheye camera model respectively;

[0027] The coordinates of the point in the fisheye camera image are:

[0028] ;

[0029] ;

[0030] wherein, and represent the horizontal and vertical coordinates of the point in the fisheye camera image respectively;

[0031] If the parameters and are known, the all-sky imaging can be converted from the fisheye camera imaging to the pinhole camera imaging with a focal length by the above formulas to realize distortion correction; if the parameters and are unknown, the parameters can be solved by marking two or more points in space, and then distortion correction is performed.

[0032] In the above scheme, the specific method of step two is as follows:

[0033] First, the images at the two time points after distortion correction are grayed; then the speed of each pixel is solved by using the Farnebäck algorithm.

[0034] In the above scheme, when the static model is used in step three, the method for predicting the DNI at the future time point by using the clear sky DNI function is as follows:

[0035] (1) The Linke turbidity coefficient is calculated:

[0036] ;

[0037] wherein, is the current time, is the DNI at the current time, is the clear sky DNI function:

[0038] ;

[0039] is the inverse function of the clear sky DNI function ; denotes the altitude, denotes the extraterrestrial irradiance, denotes the optical air mass at time t;

[0040] (2) calculating the DNI at a future time:

[0041] ;

[0042] wherein, denotes the predicted DNI after H minutes.

[0043] In the above scheme, in step three, the deep learning network model constructed adopts a convolutional neural network model, the input of the model includes the full-sky image after distortion correction and the ROI image; the model first performs down-sampling on the full-sky image after distortion correction, so that the pixel size is equal to that of the ROI image, and the two inputs are fused into a unified six-channel tensor at the pixel level through alignment operation, the first three channels are derived from the ROI image, and the last three channels correspond to the adjusted full-sky image;

[0044] The composite tensor is input into a sequence composed of 4 convolutional layers, each convolutional layer extracts hierarchical features through a learned filter; a rectified linear unit ReLU activation function is connected after each convolutional layer to introduce non-linear characteristics; then a 2x2 window maximum pooling operation is adopted to perform spatial down-sampling on the feature map;

[0045] After the convolutional block processing, the extracted spatial features are flattened into a one-dimensional vector and processed through two fully connected layers, and the last layer outputs the DNI clear sky index.

[0046] In the above scheme, in step three, the training of the deep learning network model includes the following steps:

[0047] Step 1: pre-processing the full-sky imaging data collected historically to obtain the full-sky imaging after distortion correction and the ROI image;

[0048] Step 2, selecting clear sky DNI from DNI historical data and fitting to obtain clear sky DNI at each time, then converting clear sky DNI at each time into clear sky index, which is expressed as:

[0049] ;

[0050] wherein, represents clear sky index at time t, is the measured DNI value at time t, represents clear sky DNI at time t;

[0051] Step 3, using the obtained distortion-corrected full-sky imaging and ROI image and the corresponding clear sky index at the time to construct a data set, and dividing the training set and the validation set;

[0052] Step 4, inputting the distortion-corrected full-sky imaging and ROI image in the training set into the deep learning network model, using the corresponding clear sky index at the time in the training set as the model output, training the model; verifying the model using the validation set, calculating the normalized mean absolute error and the normalized root mean square error to calculate the prediction accuracy of the model, saving all models at different training rounds, and taking the model with the minimum normalized mean absolute error and normalized root mean square error on the validation set as the finally trained model.

[0053] In the further technical solution, the specific method of step 1 is as follows:

[0054] (1) performing distortion correction on the historically collected full-sky imaging to obtain the distortion-corrected full-sky imaging;

[0055] (2) using Farnebäck algorithm to calculate the motion speed of each point of the distortion-corrected full-sky imaging at the previous and subsequent two times, and taking the average of the speeds of all points greater than the specified speed threshold to obtain the average cloud speed ;

[0056] (3) using the solar position algorithm to calculate the pitch angle and the azimuth angle of the sun, and the distance from the center coordinate of the sun in the distortion-corrected full-sky imaging to the image center is expressed as:

[0057] ;

[0058] wherein, represents the focal length of the pinhole camera model corresponding to the distortion-corrected full-sky imaging;

[0059] When the image center is , the center coordinate of the sun in the distortion-corrected full-sky imaging is expressed as:

[0060] ;

[0061] ;

[0062] wherein, and respectively represent the horizontal and vertical coordinates of the center of the sun in the distortion-corrected all-sky image;

[0063] (4) determining the ROI region according to the average cloud speed , the prediction length H and the center coordinates of the sun , and the center coordinates of the ROI region are represented as:

[0064] ;

[0065] ;

[0066] wherein, and respectively represent the horizontal and vertical coordinates of the center of the ROI region, and respectively represent the components of the average cloud speed in the x and y directions;

[0067] The ROI region is a square with a side length of 2 times the radius of the sun in the distortion-corrected all-sky image.

[0068] In a further technical solution, the specific method for selecting the clear-sky DNI from the DNI historical data and then fitting to obtain the clear-sky DNI at each time in step 2 is as follows:

[0069] The clear-sky DNI model is as follows:

[0070] ;

[0071] wherein, represents the clear-sky DNI, represents the altitude, represents the extraterrestrial radiation intensity, represents the optical air mass at the time t, represents the turbidity at the time t, which is a parameter to be fitted; and the extraterrestrial solar radiation intensity is represented as:

[0072] ;

[0073] wherein, represents the reference extraterrestrial solar radiation intensity, ​indicates the ordinal number of the day in the year, i.e. this day is the xth day of the year; optical air mass is expressed as:

[0074] ;

[0075] wherein, indicates the solar zenith angle at time t.

[0076] The specific method for fitting the clear sky DNI model is as follows:

[0077] (1) Divide the DNI data of each day into data segments of a certain time length;

[0078] (2) Initialize the turbidity, a parameter of the clear sky DNI model to be fitted;

[0079] Before the entire fitting begins, the initial value of turbidity is taken as 8, and the initial value of turbidity fitted in a time period in the first day is taken as the turbidity fitted in the previous time period; starting from the second day, the initial value of turbidity in a time period is taken as the turbidity fitted in the same time period of the previous day;

[0080] (3) Select the first clear sky DNI value and other clear sky DNI values in a time period according to the solar zenith angle to obtain a list of clear sky DNI and solar zenith angle;

[0081] (4) Use the obtained list to fit the turbidity, a parameter of the clear sky DNI model to be fitted, in a time period by using the least square method;

[0082] (5) Repeat steps (2)-(4) until the fitting of the clear sky DNI model in each time period of each day is completed.

[0083] In a further technical solution, in step three, when using the trained deep learning network model to predict DNI, the full-sky imaging data collected in step one is preprocessed by using the method of step one to obtain the distortion-corrected full-sky imaging and ROI image; then the distortion-corrected full-sky imaging and ROI image are input into the trained deep learning network model to obtain the clear sky index at the future time, and the clear sky index at the future time is multiplied by the clear sky DNI at the future time to obtain the predicted DNI at the future time.

[0084] In a further technical solution, the calculation method of the clear sky DNI at the future time is as follows:

[0085] The DNI value at the current time t is taken as the clear sky DNI, the Linke turbidity coefficient is calculated through the inverse function of the clear sky DNI function , and then is compared with 8, and the minimum value of the two is taken to calculate the clear sky DNI at the future time t+H through the clear sky DNI function.

[0086] Through the technical scheme, the DNI prediction method based on the dense optical flow method and the deep learning network has the following beneficial effects:

[0087] 1. The method for determining whether the pixel speed obtained by the dense optical flow method is greater than a specified speed threshold value is used for DNI prediction in different cases (sunny day / cloudy day and cloudy day), thereby improving the accuracy of DNI prediction.

[0088] 2. The region of interest (ROI region) that may block the sun and the all-sky image after distortion correction are fused into the same tensor, thereby improving the evaluation of the influence of cloud blocking, and the introduction of the all-sky image reduces the influence of cloud cluster uncertainty (sudden disappearance or appearance).

[0089] 3. The accuracy of the overall model is higher than the prediction accuracy reported in the literature. BRIEF DESCRIPTION OF DRAWINGS

[0090] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows.

[0091] Figure 1 The flowchart of the DNI prediction method based on the dense optical flow method and the deep learning network disclosed in the embodiments of the present application;

[0092] Figure 2 The flowchart of the static model;

[0093] Figure 3 The deep learning network structure of the motion model;

[0094] Figure 4 The changes of nMAE and nRMSE with the prediction length H when the maximum cloud pixel speed is equal to or greater than 5 pixels per minute, (a) is the change of nMAE with the prediction length H, and (b) is the change of nRMSE with the prediction length H;

[0095] Figure 5 The changes of nMAE and nRMSE with the prediction length H in different cases, (a) is the change of nMAE with the prediction length H in different cases, and (b) is the change of nRMSE with the prediction length H in different cases. DETAILED DESCRIPTION

[0096] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application.

[0097] The present application provides a DNI prediction method based on a dense optical flow method and a deep learning network, as shown in Figure 1As shown, comprising the following steps:

[0098] Step one, the image collected by the all-sky imager is converted from fisheye camera imaging to pinhole camera imaging to realize distortion correction.

[0099] The all-sky imager is a fisheye camera, and the picture collected (i.e. all-sky imaging) has serious distortion. In order to effectively calculate the dense optical flow method, the all-sky imaging needs to be corrected first. The specific method is as follows:

[0100] The relationship between fisheye camera imaging and pinhole camera imaging is as follows:

[0101] For a point on the pinhole camera imaging , the distance from the image center is:

[0102] ;

[0103] The tangent value of the angle between the corresponding point in space and the pinhole camera imaging plane is expressed as:

[0104] ;

[0105] Wherein, represents the focal length of the pinhole camera model corresponding to the distortion-corrected all-sky imaging; then the angle is:

[0106] ;

[0107] If this pinhole camera is vertical to the horizontal plane and points upwards, then the angle is the pitch angle; the azimuth angle corresponding to the point is denoted as A, and its sine and cosine are respectively:

[0108] ;

[0109] ;

[0110] The distance of the point in the fisheye camera imaging to the image center is:

[0111] ;

[0112] Wherein, and are the first-order and zero-order parameters in the fisheye camera model, respectively;

[0113] The coordinates of the point in the fisheye camera image are:

[0114] ;

[0115] ;

[0116] wherein, and denote the horizontal and vertical coordinates of the point in fisheye camera image respectively;

[0117] If the parameters and are known, the all-sky imaging can be converted from fisheye camera imaging to pinhole camera imaging with focal length by the above formulas, realizing distortion correction; if the parameters and are unknown, the parameters can be solved by spatial calibration of two or more points, and then distortion correction is performed.

[0118] In addition, other distortion correction methods mentioned in the literature [A Toolbox for Easily Calibrating Omnidirectional Cameras. in 2006 IEEE / RSJ International Conference on Intelligent Robots and Systems. 2006; Solar Energy, 2015. 119: p. 1-17] can also be used.

[0119] Step two, according to the images after distortion correction at the previous and subsequent two time points, the speed of each pixel point in the current image is calculated by using the dense optical flow method, which is used to evaluate the cloud movement speed. The specific method is as follows:

[0120] First, the images after distortion correction at the previous and subsequent two time points are grayed; then the speed of each pixel is solved by using the Farnebäck algorithm [Image Analysis, Proceedings, 2003. p. 363-370].

[0121] Step three, when the maximum pixel speed is less than the specified speed threshold, a static model is used to predict the DNI at the future time point, otherwise a motion model is used to predict the DNI at the future time point.

[0122] In the present application, different models are used according to different conditions, and the specific basis is whether the maximum pixel speed calculated by the dense optical flow method is less than the speed threshold or not ( , for example 5 pixels / minute, i.e. , as shown in Figure 1 ). When When the DNI is predicted, a rule-based "stationary model" is used to predict the DNI at the future moment; otherwise, a convolutional neural network (CNN) model called "motion model" is used. Indicates clear or cloudy conditions, while It corresponds to cloudy weather. Speed ​​threshold The choice of is related to the size of the image after distortion correction and the characteristics of local cloud motion. As the image becomes larger, a larger threshold should be selected.

[0123] (1) Stationary model:

[0124] like Figure 2 As shown, if the DNI at the current time t ( ) is less than the threshold , for example 100W / m², then the predicted DNI value at the moment ( ) will be set to , where H represents the prediction duration. On the contrary, when ≥ When , use the clear sky DNI function to predict the DNI at the future time:

[0125] (1) Calculation of Link turbidity coefficient , Link turbidity coefficient By substituting the current DNI into the clear sky DNI function [Applied Mathematical Modelling, 2016. 40(15-16): p. 7245-7264] And the calculation is:

[0126] ;

[0127] in, For the current moment, is the DNI at the current moment, For the clear sky DNI function:

[0128] ;

[0129] For the clear sky DNI function The inverse function of Indicates altitude, represents the intensity of extraterrestrial radiation, represents the optical air quality at time t;

[0130] (2) Calculate the DNI at the future time and predict the DNI (i.e. ) is derived by Substitute into the clear sky DNI function At , the calculation yields that:

[0131] ;

[0132] where, represents the predicted DNI after H minutes.

[0133] (B) Motion model:

[0134] The motion model adopts a convolutional neural network (CNN) architecture to process all-sky imaging data (e.g., as shown in Figure 3 ) to predict the DNI clear sky index.

[0135] When using the motion model for DNI prediction, first, the data needs to be preprocessed, then the motion model is trained, and finally the trained motion model is used to predict the DNI clear sky index, and the predicted DNI is calculated by combining the clear sky DNI.

[0136] 1. Motion model construction

[0137] As shown in Figure 3 , the motion model integrates two complementary visual inputs: a region of interest (ROI) image of a specific sky area, and a distortion-corrected all-sky image. The model first downsamples the distortion-corrected all-sky image (size ) to make its pixel size exactly match the ROI image (size ). This alignment operation fuses the two inputs at the pixel level into a unified six-channel tensor (size ). The first three channels are derived from the ROI image (typically RGB), and the last three channels correspond to the adjusted all-sky image.

[0138] This composite tensor is input into a sequence composed of 4 convolutional layers, each performing hierarchical feature extraction through learned filters. The rectified linear unit (ReLU) activation function following each convolutional layer introduces nonlinearity by outputting the positive component of the input (i.e., ), helping the model capture the complex nonlinear relationship between cloud dynamics and solar irradiance. Subsequently, a 2x2 window max-pooling operation is applied to spatially downsample the feature maps, reducing the dimensionality while preserving key features. The output tensor size after 4 convolution + ReLU + pooling operations is , , , . This stepwise dimensionality reduction method effectively enhances translation invariance and improves computational efficiency.

[0139] After the convolutional blocks, the extracted spatial features are flattened into a one-dimensional vector (size 640) and processed through two fully connected (dense) layers with output vector sizes (or number of layer neurons) of 500 and 1, respectively. These layers perform high-level feature integration and regression analysis, progressively refining the data representation to extract predictive patterns of solar irradiance modulation. The final layer outputs a scalar value representing the DNI (Dark Sky Index)—a standardized metric that quantifies the attenuation of direct sunlight by cloud cover compared to ideal clear-sky conditions. When combined with the Clear Sky Directional Index (DNI), DNI can be calculated.

[0140] 2. Motion model training

[0141] 2.1 Data Preprocessing

[0142] Data preprocessing includes the preprocessing of all-sky imaging data and DNI data.

[0143] 2.1.1 All-sky imaging data preprocessing

[0144] The purpose of preprocessing the full sky imaging data is to obtain the distortion-corrected full sky imaging and ROI image.

[0145] (1) First, perform distortion correction on the full sky imaging, using the method in step 1 to obtain a value of Distortion-corrected full-sky imaging.

[0146] (2) Then, the Farnebäck algorithm in step 2 is used to calculate the motion speed of each point in the distortion-corrected full-sky image at the two moments before and after. The average speed of all points greater than a specified threshold (e.g., 5 pixels / minute) is taken to obtain the average cloud speed. .

[0147] (3) Next, calculate the sun's pitch angle using the solar position algorithm [Solar Energy, 2004. 76(5): p. 577-589] and azimuth , the distance from the center coordinate of the sun to the center of the image in the full sky imaging after distortion correction It is expressed as:

[0148] ;

[0149] in, Represents the focal length parameter of the pinhole camera model corresponding to the full sky imaging after distortion correction;

[0150] The center of the image is When , the central coordinates of the sun in the distortion-corrected full-sky imaging are expressed as:

[0151] ;

[0152] ;

[0153] wherein, and denote the horizontal and vertical coordinates of the center of the sun in the distortion-corrected all-sky image, respectively.

[0154] (4) Finally, the ROI region is determined according to the average cloud speed , the prediction length H and the center coordinates of the sun . The center coordinates of the ROI region are expressed as:

[0155] ;

[0156] ;

[0157] wherein, and denote the horizontal and vertical coordinates of the center of the ROI region, and denote the components of the average cloud speed in the x and y directions, respectively.

[0158] The ROI region is a square with a side length of twice the radius of the sun in the distortion-corrected all-sky image. For a distortion-corrected all-sky image with a size of , the radius is 44, and then the radius of the ROI region is 88.

[0159] 2.1.2 DNI data preprocessing

[0160] The training set and the validation set of the motion model need to convert the DNI into the clear sky index. The clear sky index is expressed as:

[0161] ;

[0162] wherein, denotes the clear sky index at time t, is the measured DNI value at time t, denotes the clear sky DNI at time t.

[0163] 2.2 Construction of data set

[0164] The distortion-corrected all-sky image and the ROI image obtained and the corresponding clear sky index at the time are used to construct the data set, and the training set and the validation set are divided.

[0165] The training / validation set is preferably composed of at least one year of full-day imaging data and DNI data. In the model evaluation, a multi-fold cross-validation method is adopted. Taking a five-fold as an example, this technique divides the original dataset into five mutually exclusive and equal time subsets by randomizing by date. One of the subsets is designated as the validation set, and the remaining four subsets are combined to form the training set. The entire process is systematically repeated five times, ensuring that each subset is used as the validation set exactly once.

[0166] Due to the influence of cloudy or overcast days, only a small part of the DNI historical data is clear-sky DNI data, so the clear-sky DNI at each time used in the construction of the training set and the validation set is obtained by first selecting clear-sky DNI from the DNI historical data and then fitting to obtain the clear-sky DNI at each time. Specifically as follows:

[0167] The clear-sky DNI model is as follows:

[0168] ;

[0169] wherein, represents the clear-sky DNI, represents the altitude, represents the extraterrestrial solar radiation intensity, represents the optical air mass at time t, represents the turbidity at time t, which is a parameter to be fitted; the extraterrestrial solar radiation intensity is represented as:

[0170] ;

[0171] wherein, represents the reference extraterrestrial solar radiation intensity, represents the day number in the year, that is, the day of the year; the optical air mass is represented as:

[0172] ;

[0173] wherein, represents the solar zenith angle at time t.

[0174] The specific method for fitting the clear-sky DNI model is as follows:

[0175] (1) Divide the DNI data of each day into data segments of a certain time length.

[0176] (2) Initialize the turbidity, a parameter to be fitted in the clear-sky DNI model;

[0177] Before the entire fitting begins, the initial turbidity value is taken as 8. The initial turbidity value of a time period on the first day is taken as the turbidity fitting value of the previous time period; starting from the second day, the initial turbidity value of a time period is taken as the turbidity fitting value of the same time period on the previous day.

[0178] (3) Select the first clear sky DNI value and other clear sky DNI values ​​within a time period according to the solar zenith angle, and obtain a list of clear sky DNI and solar zenith angle;

[0179] The method for selecting the first clear sky DNI value in a time period is as follows:

[0180] First, a DNI data point The corresponding time is subtracted by 1 minute and determined based on the time 1 minute ago and the solar zenith angle 1 minute ago , combined with the initial turbidity value, calculate the estimated clear sky DNI value 1 minute ago ;

[0181] Then, based on the DNI data points The corresponding time determines the solar zenith angle at the current moment ,if , and satisfies When Confirm it as the first clear sky DNI value in this time period and record the corresponding ;in, is a threshold value between 0 and 1;

[0182] if , and satisfies When Confirm it as the first clear sky DNI value in this time period and record the corresponding ,in, is a threshold value between 0 and 1;

[0183] If the DNI data point does not meet the above requirements, repeat the above steps for the next data point in the time period.

[0184] The method for selecting other clear sky DNI values ​​within a time period is as follows:

[0185] Assume that the last clear sky DNI and solar zenith angle selected in the current time period are and , the DNI and solar zenith angle of the current data point are and ;if , and satisfies When The clear sky DNI value is added to the list, while the corresponding ;

[0186] If , and the condition is met, the clear sky DNI value is added to the list, while the corresponding ; wherein, represents the time of the last point in the selected list;

[0187] If the DNI data point does not meet the above requirements, the clear sky DNI value is not added to the list.

[0188] The list of clear sky DNI and solar zenith angle obtained is as follows:

[0189] ;

[0190] wherein, represent the first, second, …, and last time points corresponding to the data in the selected list of clear sky DNI and solar zenith angle, respectively, and ; represents the clear sky DNI at the time point, represents the solar zenith angle at the time point, and so on.

[0191] (4) Using the obtained list, a clear sky DNI model in a time period is fitted using the least squares method. The parameter to be fitted is the turbidity.

[0192] (5) Repeat steps (2)-(4) until the clear sky DNI model for each time period of each day is fitted.

[0193] 2.3 Training the motion model

[0194] The distortion-corrected full-sky image and ROI image in the training set are input into the deep learning network model, and the clear sky index at the corresponding time in the training set is used as the model output to train the model. The model is verified using the validation set, and the prediction accuracy of the model is calculated by calculating the normalized mean absolute error and normalized root mean square error.

[0195] The calculation of the normalized mean absolute error (nMAE) and the normalized root mean square error (nRMSE) is as follows:

[0196] ;

[0197] ;

[0198] wherein, DNI value at time t is predicted, n represents the number of data points.

[0199] Save all models trained in each fold for different training rounds, and take the model with the minimum nMAE or nRMSE on the validation set as the final trained model.

[0200] In the embodiment, all daytime measurement data are collected from a Hami pilot photovoltaic power station. The station is located in Hami City, Xinjiang, China, with geographic coordinates of 42.890 degrees north latitude and 93.575 degrees east longitude, and an altitude of 848 meters. Meteorological station data is obtained at a 1-minute sampling frequency.

[0201] The all-sky images used are in RGB format taken by an all-sky camera (SONA 202U) with a resolution of 1158x1172 pixels, also collected at a 1-minute sampling frequency. When the solar zenith angle exceeds 80 degrees, the system will automatically exclude such images to avoid including hazy skies or potential obstructions.

[0202] The training / validation set contains all-day imaging data and DNI data from August 1, 2023 to August 20, 2024, and the test set covers corresponding data from August 21, 2024 to June 30, 2025.

[0203] 3. DNI prediction using motion model

[0204] When using the trained deep learning network model to predict DNI, the method of 2.2.1.1 is used to pre-process the all-sky imaging data collected in step one to obtain the distortion-corrected all-sky imaging and ROI image; then the distortion-corrected all-sky imaging and ROI image are input into the trained deep learning network model to obtain the future time clear sky index, and then the future time clear sky DNI is multiplied by the future time clear sky index to obtain the future time predicted DNI.

[0205] When pre-processing the training data, the parameter fitting of the clear sky DNI model is performed hourly, and then the clear sky DNI at each time point is calculated. When performing real-time DNI prediction, the future DNI value within an hour cannot be predicted, so parameter fitting cannot be performed. Here, the DNI value at the current time t is taken as the clear sky DNI to solve the turbidity parameter of the clear sky DNI model , and then and 8 are compared, and the minimum value is taken to calculate the clear sky DNI at time t+H through the clear sky DNI function. The clear sky DNI is multiplied by the clear sky index predicted by the motion model at time t+H to obtain the predicted DNI value at time t+H.

[0206] In this embodiment, according to the maximum cloud pixel speed size (Vmax) ) exceeds 5 pixels per minute, the weather is divided into two categories. When , it usually corresponds to cloudless (clear sky) or cloudy (uniform cloud cover). This type of weather state has homogeneous characteristics, and its irradiance changes are mainly dominated by the position of the sun rather than the complex cloud edge interaction, so it is highly predictable. When the cloud layer is 100% or higher, it indicates the presence of fast-moving clouds with significant dynamic changes. This type of weather usually poses greater challenges to forecasting due to the increased complexity and uncertainty of cloud flow movement and evolution. In previous studies [Solar Energy, 2023. 249: pp. 541-558], the following criteria were used to classify weather on a daily basis:

[0207] Sunny conditions: The measured daily index of incidence (DNI) remains within 10% of the clear sky DNI value for 90% of the day;

[0208] Cloudy conditions: The measured daily intensity (DNI) is less than 60% of the clear sky DNI value for 90% of the day, and the average DNI variation is less than 20 W / m²;

[0209] Mixed conditions: All other conditions; note that this type of weather may include periods of both clear skies and overcast skies.

[0210] Figure 4 In , "cross validation" corresponds to the best result obtained during cross validation, and "test" corresponds to the result obtained from real-time testing, from Figure 4 As can be seen in (a) and (b), the performance of the models on the test set (real-time prediction) is very close to their performance on the cross-validation set, which indicates that the models do not exhibit significant overfitting or underfitting.

[0211] Figure 5 In , “all cases” means all cases corresponding to the test data; "Indicates a scenario where the maximum cloud pixel velocity amplitude is less than 5 pixels per minute;" " indicates a scene with a maximum cloud pixel velocity amplitude equal to or exceeding 5 pixels per minute; "Sunny or cloudy" indicates sunny or cloudy conditions; and "Mixed" indicates mixed conditions. Figure 5As can be seen from (a) and (b), the prediction index of the DNI prediction model of the present application is significantly higher than the results reported in the current literature, such as the 10-minute prediction nMAE of the model in the literature [Zhu, T.; Guo, Y.; Li, Z.; Wang, C., Solar Radiation Prediction Based on Convolution Neural Network and Long Short-Term Memory. Energies, 2021. 14(24): p. 8498] is 13.75% (10.83% in the present application), the nRMSE of 5, 10, 15 minutes prediction in the literature [Karout, Y.; Thil, S.; Eynard, J.; Guillot, E.; Grieu, S., Hybrid intrahour DNI forecast model based on DNI measurements and sky-imaging data. Solar Energy, 2023. 249: p. 541-558] is 46.10%, 51.40%, 56.30% (26.61%, 30.04, 32.18% in the present application) in mixed conditions, and 10.00%, 12.80%, 15.00% (3.12%, 3.51, 3.89% in the present application) in sunny or cloudy conditions.

[0212] The above description of disclosed embodiments enables one skilled in the art to make or use the application. Numerous modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not to be limited to the implementations illustrated herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A DNI prediction method based on dense optical flow and deep learning network, characterized in that: The steps include: Step 1: Convert the image collected by the all-sky imager from fisheye camera imaging to pinhole camera imaging to achieve distortion correction; Step 2: Based on the distortion-corrected images at the previous and next moments, the velocity of each pixel in the current image is calculated using the dense optical flow method. Step 3: When the maximum pixel speed is less than the specified speed threshold, the static model is used to predict the DNI at the future time; otherwise, the motion model is used to predict the DNI at the future time; When using the stationary model, if the current DNI is greater than or equal to the specified DNI threshold, the clear-sky DNI function is used to predict the DNI at the future time. Otherwise, the DNI at the future time is considered equal to the DNI at the current time. When using the motion model, the trained deep learning network model is used to predict the clear sky index at the future moment. Then, the predicted DNI at the future moment is obtained by multiplying the clear sky index at the future moment by the clear sky DNI at the future moment. In step three, a convolutional neural network model is constructed. The inputs to the model include the distortion-corrected full-sky image and the ROI image. The model first downsamples the distortion-corrected full-sky image to the same pixel size as the ROI image. Then, through an alignment operation, the two inputs are fused at the pixel level into a unified six-channel tensor. The first three channels are derived from the ROI image, and the last three channels correspond to the adjusted full-sky image. This six-channel tensor is fed into a sequence of four convolutional layers, each of which performs hierarchical feature extraction using learned filters. Each convolutional layer is followed by a rectified linear unit (ReLU) activation function to introduce nonlinearity. The feature maps are then spatially downsampled using a 2×2 windowed max pooling operation. After the convolution block, the extracted spatial features are flattened into a one-dimensional vector and processed through two fully connected layers. The last layer outputs the DNI clear sky index. The method for obtaining the distortion-corrected full sky image and ROI image is as follows: (1) Perform distortion correction on the historically collected full-sky imaging to obtain the distortion-corrected full-sky imaging; (2) The Farnebäck algorithm is used to calculate the motion speed of each point in the full sky image after distortion correction at two moments before and after, and the average speed of all points greater than the specified speed threshold is taken to obtain the average cloud speed. ; (3) Calculate the sun's pitch angle using the sun position algorithm and azimuth , the distance from the center coordinate of the sun to the center of the image in the full sky imaging after distortion correction Expressed as: ； in, represents the focal length of the pinhole camera model corresponding to the full-sky imaging after distortion correction; The center of the image is When , the central coordinates of the sun in the distortion-corrected full-sky imaging are expressed as: ； ； in, and They represent the horizontal and vertical coordinates of the center of the sun in the full-sky imaging after distortion correction; (4) Based on average cloud speed , prediction duration H and the center coordinates of the sun To determine the ROI area, the center coordinates of the ROI area are expressed as: ； ； in, and Respectively represent the horizontal and vertical coordinates of the center of the ROI area, and Represents the average cloud speed Components in the x and y directions; The ROI area is a square with a side length of twice the radius of the sun in the distortion-corrected full-sky imaging.

2. The DNI prediction method based on dense optical flow and deep learning network according to claim 1, characterized in that: The specific method of step one is as follows: The relationship between fisheye camera imaging and pinhole camera imaging is as follows: For a point on the pinhole camera imaging , to the center of the image distance for: ； The tangent value of the angle between the corresponding point in space and the imaging plane of the pinhole camera Expressed as: ； in, represents the focal length of the pinhole camera model corresponding to the full-sky imaging after distortion correction; then the angle is: ； If the pinhole camera is pointing upwards perpendicular to the horizontal plane, the included angle is the pitch angle; the azimuth angle corresponding to this point is recorded as A, and its sine and cosine are: ； ； The distance from this point to the center of the image in the fisheye camera imaging for: ； in, and are the first-order and zero-order parameters in the fisheye camera model respectively; The coordinates of this point in the fisheye camera image are: ； ； in, and Respectively represent the horizontal and vertical coordinates of the point in the fisheye camera image; If the parameters are known and , then the above formulas can be used to convert the full sky imaging from fisheye camera imaging to a certain focal length Pinhole camera imaging to achieve distortion correction; if the unknown parameters and , the distortion correction can be performed by solving the parameters by calibrating more than two points in space.

3. The DNI prediction method based on dense optical flow method and deep learning network according to claim 1, characterized in that: The specific method of step 2 is as follows: First, the distortion-corrected images at the two moments before and after are converted to grayscale; then the Farnebäck algorithm is used to calculate the velocity of each pixel.

4. The DNI prediction method based on dense optical flow method and deep learning network according to claim 1, characterized in that: In step 3, when the stationary model is used, the method for predicting the DNI at future times using the clear-sky DNI function is as follows: (1) Calculation of Link turbidity coefficient : ； in, For the current moment, is the DNI at the current moment, For the clear sky DNI function: ； For the clear sky DNI function The inverse function of Indicates altitude, represents the intensity of extraterrestrial radiation, represents the optical air quality at time t; (2) Calculate the DNI at the future time: ； in, Indicates the predicted DNI after a prediction duration of H minutes.

5. The DNI prediction method based on dense optical flow method and deep learning network according to claim 1, characterized in that: In step 3, the training of the deep learning network model includes the following steps: Step 1: preprocess the historically collected full-sky imaging data to obtain the distortion-corrected full-sky imaging and ROI image; Step 2: Select the clear sky DNI from the historical DNI data and perform fitting to obtain the clear sky DNI at each moment. Then, convert the clear sky DNI at each moment into a clear sky index. The clear sky index is expressed as: ； in, represents the clear sky index at time t, is the DNI value measured at time t, represents the clear sky DNI at time t; Step 3: Use the obtained distortion-corrected full-sky imaging and ROI images and the clear sky index at the corresponding time to construct a dataset, and divide it into a training set and a validation set; In step 4, the distortion-corrected all-sky imaging and ROI images in the training set are input into the deep learning network model, and the clear sky index at the corresponding time in the training set is used as the model output to train the model. The model is verified using the validation set, and the model prediction accuracy is calculated by calculating the normalized mean absolute error and normalized root mean square error. All models from different training rounds are saved, and the model with the smallest normalized mean absolute error and normalized root mean square error on the validation set is taken as the final trained model.

6. The DNI prediction method based on dense optical flow method and deep learning network according to claim 5, characterized in that: In step 2, the specific method for selecting clear-sky DNI from historical DNI data and then fitting to obtain the clear-sky DNI at each moment is as follows: The clear sky DNI model is as follows: ； in, Indicates clear sky DNI, Indicates altitude, represents the intensity of extraterrestrial radiation, express Optical air quality at all times, Represents the Link turbidity coefficient at time t, which is the parameter to be fitted; the extraterrestrial solar radiation intensity Expressed as: ； in, represents the benchmark extraterrestrial solar radiation intensity, Indicates the day number of the year, that is, the day is the number of the year; Optical air quality Expressed as: ； in, represents the solar zenith angle at time t; The specific method for fitting the clear sky DNI model is as follows: (1) Divide the daily DNI data into data segments of a certain length; (2) Initialize the Link turbidity coefficient, a parameter to be fitted in the clear sky DNI model; Before the entire fitting begins, the initial value of the Link turbidity coefficient is taken as 8. The initial value of the Link turbidity coefficient for a time period on the first day is taken as the Link turbidity coefficient fitting value of the previous time period; starting from the second day, the initial value of the Link turbidity coefficient for a time period is taken as the Link turbidity coefficient fitting value of the same time period on the previous day; (3) Select the first clear sky DNI value and other clear sky DNI values ​​within a time period according to the solar zenith angle, and obtain a list of clear sky DNI and solar zenith angle; (4) Using the obtained list, the Link turbidity coefficient of the clear sky DNI model to be fitted within a time period is fitted using the least squares method; (5) Repeat steps (2) to (4) until the clear sky DNI model fitting for each day and time period is completed.

7. The DNI prediction method based on dense optical flow method and deep learning network according to claim 6, characterized in that: In step three, when using the trained deep learning network model to predict DNI, the method of step 1 is used to preprocess the full-sky imaging data collected in step 1 to obtain distortion-corrected full-sky imaging and ROI images. The distortion-corrected full-sky imaging and ROI images are then input into the trained deep learning network model to obtain the clear sky index at the future time. The clear sky index at the future time is then multiplied by the clear sky DNI at the future time to obtain the predicted DNI at the future time.

8. The DNI prediction method based on dense optical flow method and deep learning network according to claim 7, characterized in that: The calculation method of clear sky DNI at a future time is as follows: The DNI value at the current time t is regarded as the clear sky DNI, and the Link turbidity coefficient is calculated by the inverse function of the clear sky DNI function. , then Compare with 8 and take the minimum value of the two to calculate the clear sky DNI at the future time t+H using the clear sky DNI function.

Images