Regional horizontal irradiance inversion method and system based on satellite data and deep learning

Abstract

The invention discloses a regional horizontal irradiance inversion method and system based on satellite data and deep learning, and belongs to the technical field of solar energy meteorology, and the method comprises the steps: obtaining FY4A satellite data; obtaining GHI data of a measurement station and deleting a GHI missing measurement value to obtain GHI preprocessing data; enabling the satellite data to correspond to the GHI preprocessing data, and removing missing data in the satellite data; calculating the sun-earth distance, the solar zenith angle and the solar altitude angle of each station at a certain moment according to the geographical time information of each station; calculating extraterrestrial solar radiation based on the sun-earth distance and the solar zenith angle, and performing quality control on the GHI preprocessing data based on the extraterrestrial solar radiation to obtain a processed GHI training set; performing improved training on the initial TabNet model according to the processed GHI training set to obtain an irradiance inversion model; and inputting the reprocessed satellite data feature set into the irradiance inversion model to obtain a high-precision inversion graph.

Description

Technical Field

[0001] The present invention belongs to the technical field of solar meteorology, and particularly relates to a method and system for retrieving regional horizontal irradiance based on satellite data and deep learning. Background Art

[0002] Global horizontal irradiance (GHI) is one of the most important parameters in solar resource assessment. Currently, the retrieval of GHI is mainly completed through two methods - namely, physical methods and statistical methods (including machine learning). In physical methods, the theory of radiation transfer is quite mature, but it has obvious limitations because physical inversion algorithms are essentially a kind of local fitting function. For different regions, the atmospheric parameters required to drive the radiation transfer model are also different, which means that even though physical methods are very mature, they face bottlenecks. Under the increasing requirements for high-precision and high-accuracy solar resource maps, compared with physical methods, statistical inversion methods (machine learning) are a highly competitive alternative. Machine learning can make full use of a large amount of existing high-quality observational data, and it can fit any complex non-linear function to ensure its applicability after training. Although training such a model requires big data and computers, once trained, the network can quickly generate retrieval results. Summary of the Invention

[0003] The present invention aims to solve the deficiencies of the prior art and provides the following solutions:

[0004] A method for retrieving regional horizontal irradiance based on satellite data and deep learning, comprising the following steps:

[0005] Obtain FY4A full-disk L1-level satellite data with a resolution of 4KM; obtain GHI data from radiation measurement stations and delete the missing GHI values to obtain preprocessed GHI data;

[0006] Correspond the satellite data with the preprocessed GHI data and eliminate the missing data in the satellite data;

[0007] According to the geographical time information of each radiation measurement station, calculate the Earth-Sun distance, solar zenith angle, and solar altitude angle of each station at a certain moment;

[0008] Calculate the extraterrestrial solar radiation based on the Earth-Sun distance and the solar zenith angle, and perform quality control on the preprocessed GHI data based on the extraterrestrial solar radiation to obtain a processed GHI training set;

[0009] Improve the initial TabNet model according to the non-zero characteristics of the processed GHI training set data, and train the improved model based on the solar-earth distance, the solar zenith angle, the solar altitude angle, and the processed GHI training set to obtain an irradiance inversion model;

[0010] Input the reprocessed satellite data feature set into the irradiance inversion model to obtain a high-precision inversion map.

[0011] Preferably, the method for removing the missing measurement data in the satellite data includes:

[0012] Remove the data where all channels in the satellite data are 0;

[0013] Remove the data with missing measurements in any channel of the satellite data.

[0014] Preferably, the method for obtaining the processed GHI training set includes:

[0015] Calculate the extraterrestrial solar radiation based on the solar-earth distance and the solar zenith angle:

[0016]

[0017] Among them, R0 represents the extraterrestrial solar radiation, S0 represents the solar constant, r0 represents the average solar-earth distance, r represents the solar-earth distance on a certain day, represents the solar zenith angle;

[0018] Set the expected minimum value of GHI, and remove the data in the preprocessed data that is greater than the extraterrestrial solar radiation and less than the expected minimum value of GHI to obtain the processed GHI training set.

[0019] Preferably, the method for improving the initial TabNet model includes: according to the non-zero characteristics of the GHI data, add a ReLU activation function after the last fully connected layer in the initial TabNet model.

[0020] The present invention also provides a regional horizontal irradiance inversion system based on satellite data and deep learning. The inversion system applies the inversion method described in any one of the above, and includes: a data acquisition module, a satellite data processing module, a geographical data calculation module, a GHI data processing module, a model training module, and an inversion module;

[0021] The data acquisition module is used to acquire satellite data with a resolution of 4KM at the L1 level of the full disk of FY4A; acquire the GHI data of the radiation measurement station and delete the GHI missing values to obtain the GHI preprocessed data;

[0022] The satellite data processing module is used to correspond the satellite data with the GHI preprocessed data and eliminate the missing measurement data in the satellite data;

[0023] The geographic data calculation module calculates the solar-earth distance, solar zenith angle, and solar altitude angle of each site at a certain moment according to the geographic time information of each radiation measurement station;

[0024] The GHI data processing module calculates the extraterrestrial solar radiation based on the solar-earth distance and the solar zenith angle, and performs quality control on the GHI preprocessed data based on the extraterrestrial solar radiation to obtain the processed GHI training set;

[0025] The model training module improves the initial TabNet model according to the characteristic that the data in the processed GHI training set is non-zero, and trains the improved model based on the solar-earth distance, the solar zenith angle, the solar altitude angle, and the processed GHI training set to obtain an irradiance inversion model;

[0026] The inversion module is used to input the reprocessed satellite data feature set into the irradiance inversion model to obtain a high-precision inversion map.

[0027] Preferably, in the satellite data processing module, the process of eliminating the missing measurement data in the satellite data includes:

[0028] Eliminating the data in the satellite data where all channels are 0;

[0029] Eliminating the data in the satellite data where there are missing measurements in any channel.

[0030] Preferably, the working process of the GHI data processing module includes:

[0031] Calculating the extraterrestrial solar radiation based on the solar-earth distance and the solar zenith angle:

[0032]

[0033] Among them, R0 represents the extraterrestrial solar radiation, S0 represents the solar constant, r0 represents the average solar-earth distance, r represents the solar-earth distance on a certain day, represents the solar zenith angle;

[0034] Setting the expected minimum value of GHI, and eliminating the data in the preprocessed data that is greater than the extraterrestrial solar radiation and less than the expected minimum value of GHI to obtain the processed GHI training set.

[0035] Preferably, in the model training module, the process of improving the initial TabNet model includes: according to the characteristic that the GHI data is non-zero, adding a ReLU activation function after the last fully connected layer in the initial TabNet model.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] The present invention proposes to optimize the inversion reliability compared with the ordinary TabNet model, so that the inverted GHI will not show abnormal negative values. The FY4A satellite data is used to achieve full coverage of the region, solving the "boundary problem" of the previous GHI inversion. At the same time, compared with other deep learning models, the improved TabNet model has interpretability and can output the global importance and local importance of features, improving the inversion effect of GHI. Comparing the evaluation results of the GHI inversion method proposed by the present invention with the random forest model, the accuracy is improved to a certain extent, and good results are obtained. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions of the present invention, the following briefly introduces the drawings required in the embodiments. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0039] Figure 1 It is a schematic flowchart of the method of the embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of dividing 101 stations across the country into a training set and a test set in the embodiment of the present invention;

[0041] Figure 3 It is a scatter plot of the actual values and inversion values of each station in the test set of the embodiment of the present invention. Among them, (a) is the scatter plot of the actual values and inversion values of the improved TabNet model, and (b) is the scatter plot of the actual values and inversion values of the random forest model;

[0042] Figure 4 It is a spatial distribution map of the root mean square error RMSE, mean bias error MBE and correlation coefficient R of each station in the test set of the embodiment of the present invention. Among them, (a) is the root mean square error RMSE of each station in the test set, (b) is the mean bias error MBE of each station in the test set, and (c) is the correlation coefficient R of each station in the test set;

[0043] Figure 5: This is an inversion rendering of the GHI from 00:00 to 11:00 on July 1, 2018, Universal Coordinated Time (UTC) according to an embodiment of the present invention, wherein a is the inversion rendering at 00:00, b is the inversion rendering at 01:00, c is the inversion rendering at 02:00, d is the inversion rendering at 03:00, e is the inversion rendering at 04:00, f is the inversion rendering at 05:00, g is the inversion rendering at 06:00, g is the inversion rendering at 07:00, i is the inversion rendering at 08:00, j is the inversion rendering at 09:00, k is the inversion rendering at 10:00, and l is the inversion rendering at 11:00. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Example 1

[0047] In this embodiment, if Figure 1 As shown in the figure, a method for retrieving regional horizontal irradiance based on satellite data and deep learning includes the following steps:

[0048] S1. Obtain FY4A full-disk L1-level 4km resolution satellite data; obtain the GHI data of the radiometric measurement station and delete the missing GHI values to obtain the GHI preprocessed data.

[0049] In this example, FY4A full-disk L1-level 4km resolution data covering 14 spectral bands was downloaded from March 12, 2018, to December 13, 2018. The download period was from 00:00 to 23:00 daily, for a total of 24 satellite data points per day. Hourly GHI ground-based data from 2018, provided by 101 radiometric measurement stations of the China Meteorological Administration, was preprocessed, and the satellite data at each time point was mapped to the GHI data. The mapping rule was to average the satellite data within 0.04° of the station's latitude and longitude to represent the satellite data for the 14 spectral bands at that time.

[0050] S2. Match the satellite data with the GHI pre-processed data and remove the missing data in the satellite data.

[0051] The method for eliminating missing data in satellite data includes: eliminating data where all channels in the satellite data are 0; eliminating data with missing measurements in any channel of the satellite data.

[0052] S3. According to the geographical time information (including longitude, latitude, terrain, and time) of each radiation measurement station, calculate the solar-earth distance, solar zenith angle, and solar altitude angle of each station at a certain moment as the input features of the model.

[0053] S4. Calculate the extraterrestrial solar radiation based on the solar-earth distance and solar zenith angle, and perform quality control on the preprocessed data based on the extraterrestrial solar radiation to obtain the processed GHI training set.

[0054] The method for obtaining the processed GHI training set includes: calculating the extraterrestrial solar radiation based on the solar-earth distance and solar zenith angle:

[0055]

[0056] where, R0 represents the extraterrestrial solar radiation, S0 represents the solar constant, r0 represents the average solar-earth distance, r represents the solar-earth distance on a certain day, represents the solar zenith angle; set the expected minimum value of GHI, and eliminate the data in the preprocessed data that is greater than the extraterrestrial solar radiation and less than the expected minimum value of GHI to obtain the processed GHI data. In this embodiment, the expected minimum value of GHI is set to 10W / m 2 .

[0057] S5. Improve the initial TabNet model according to the non-zero characteristics of the data in the processed GHI training set, and train the improved model based on the solar-earth distance, solar zenith angle, solar altitude angle, and the processed GHI data to obtain the irradiance inversion model.

[0058] The method for improving the initial TabNet model includes: according to the non-zero characteristics of the GHI data, add a ReLU activation function after the last fully connected layer in the initial TabNet model.

[0059] The training method includes: dividing 101 stations across the country into a training set and a test set at a ratio of 8:2. The training set consists of 81 stations, and the test set has 20 stations. Use the solar-earth distance, solar zenith angle, solar altitude angle, the longitude, latitude, terrain of the station, and 14 spectral channels of FY4A, a total of 20 features, as the input of the improved TabNet model, and use the preprocessed GHI data as the output to train the improved model; during the process of training the improved TabNet model, the specific training method of the improved TabNet model is to use the Adam optimizer for training, with a learning rate of 0.01, and the hyperparameters of the model are set to the width n of the decision prediction layer ais 16, the attention embedding width of each mask is n d is 16, the number of decision steps in the structure n steps The default settings are 3, and the number of early stopping rounds is 20. After training, the model is compared with the random forest model to evaluate the model. The statistics used are the root mean square error (RMSE), the correlation coefficient (R), and the mean bias error (MBE). RMSE and MBE are defined as:

[0060]

[0061] Among them, y i Indicates the measured GHI value, Indicates the predicted GHI value, cov indicates the covariance, represents the standard deviation of the measured GHI values, represents the standard deviation of the predicted GHI values.

[0062] S6. Input the reprocessed satellite data feature set into the irradiance inversion model to obtain a high-precision inversion map.

[0063] In this example, all 101 stations were input into the evaluated model. Specifically, an input feature dataset was constructed by flattening the data of each channel in the processed satellite data into one dimension and matching the corresponding latitude, longitude, and altitude. The Sun-Earth distance, solar zenith angle, and solar azimuth were calculated to obtain an input feature set with 4 km resolution. This input feature set was then fed into the irradiance inversion model to obtain the inverted GHI, which was then visualized, resulting in a high-precision 4 km resolution inversion map of the GHI.

[0064] Embodiment 2

[0065] In this embodiment, a regional horizontal irradiance inversion system based on satellite data and deep learning includes: a data acquisition module, a satellite data processing module, a geographic data calculation module, a GHI data processing module, a model training module and an inversion module.

[0066] The data acquisition module is used to obtain FY4A full-disk L1-level 4KM resolution satellite data; obtain the GHI data of the radiation measurement station and delete the GHI missing values to obtain GHI preprocessed data.

[0067] The satellite data processing module is used to match satellite data with GHI preprocessed data and remove missing data from the satellite data. The process of removing missing data from the satellite data processing module includes: removing data where all channels are zero; and removing data where any channel is missing.

[0068] The geographic data calculation module calculates the sun-earth distance, solar zenith angle and solar altitude angle of each station at a certain moment according to the geographic time information of each radiation measurement station.

[0069] The workflow of the GHI data processing module includes: Calculating extraterrestrial solar radiation based on the distance between the Sun and the Earth and the solar zenith angle:

[0070]

[0071] Among them, R0 represents extraterrestrial solar radiation, S0 represents the solar constant, r0 represents the average distance between the sun and the earth, and r represents the distance between the sun and the earth on a certain day. Represents the solar zenith angle; set the expected minimum value of GHI, eliminate the data with a value greater than the extraterrestrial solar radiation and less than the expected minimum value of GHI in the preprocessed data, and obtain the processed GHI training set.

[0072] The GHI data processing module calculates the extraterrestrial solar radiation based on the distance between the Sun and the Earth and the solar zenith angle, and performs quality control on the GHI preprocessed data based on the extraterrestrial solar radiation to obtain the processed GHI training set.

[0073] The model training module improves the initial TabNet model based on the non-zero nature of the processed GHI training set. The improved model is trained based on the Sun-Earth distance, solar zenith angle, solar altitude angle, and the processed GHI training set to produce an irradiance inversion model. The model training module improves the initial TabNet model by adding a ReLU activation function after the final fully connected layer in the initial TabNet model based on the non-zero nature of the GHI data.

[0074] The inversion module is used to input the reprocessed satellite data feature set into the irradiance inversion model to obtain a high-precision inversion map.

[0075] Embodiment 3

[0076] In this embodiment, a practical case is used to verify the feasibility and accuracy of the GHI inversion method of the present invention.

[0077] The GHI inversion method in this paper can invert all time periods for which data is available, and is entirely based on domestically produced data and performed on a domestically produced platform. Here, we use the results of a single partition as an example, and demonstrate the high-precision inversion of the GHI 4KM from 00:00 to 11:00 UTC on July 1, 2018, to illustrate the effectiveness of this method in inverting the GHI.

[0078] The steps for obtaining the GHI distribution on July 1, 2018 using the GHI inversion method of the present invention are as follows:

[0079] 1. IfFigure 2 As shown, 101 stations across the country were divided into a training set and a test set at a ratio of 8:2. The training set consisted of 81 stations, and the test set consisted of 20 stations, so as to test the spatial generalization ability of the improved TabNet model for GHI inversion (the GHI data of all the above stations have been made into corresponding input features through S1 to S4 in the specific implementation steps).

[0080] 2. The training set and the test set were put into the model according to the hyperparameter settings mentioned in the specific implementation step S5, and verification was carried out.

[0081] Figure 3 The scatter plot distribution of the GHI values inverted by this method and the observed GHI values is given in, where Figure 3 (a) in is the scatter plot of the actual values and the inverted values of the improved TabNet model, Figure 3 (b) in is the scatter plot of the actual values and the inverted values of the random forest model. The solid line is the standard line of 100% inversion, and the dotted line is the unary linear regression of the actual values and the inverted values. The unary linear regression function established using the predicted values and the observed values shows that the improved TabNet model can better invert the GHI values, and the difference from the actual values of the observed values is not large. The inversion trend of the random forest model for the hourly GHI is similar to that of the improved TabNet model, but the inversion effect of the random forest model is significantly not as good as that of the improved TabNet model.

[0082] Table 1 gives the score comparison between the improved TabNet model and the random forest model. The improved TabNet model is superior to the random forest model in both RMSE and R, but the MBE is larger than that of the random forest model. This may be because the overestimated and underestimated hourly GHI values of the random forest model cancel each other out. Combining Table 1 and Figure 3 the results, it can be concluded that the improved TabNet model has a better effect in inverting the hourly GHI than the random forest model.

[0083] Table 1

[0084]

[0085] Figure 4 The spatial distribution maps of the root mean square error RMSE, correlation coefficient R, and mean bias error MBE of each station on the test set are given in. Figure 4 (a) in shows that in East China, Central China, and North China with more training stations, the RMSE is smaller, while in the Western and Northeast regions, the RMSE is larger. In addition, 70% of the test stations have better accuracy, and the RMSE is below 86 W / m2. Figure 4Figure (b) shows that the model tends to overestimate GHI at the test stations in North China, East China, and Yunnan Province. While in Qinghai, northern Xinjiang, and southern Hainan Island of China, it tends to underestimate GHI. In addition, the MBE of 50% of the stations is between -7 W / m² and 9 W / m², which means the model has good accuracy. Figure 4 Figure (c) shows that the correlation coefficient R of all stations is above 0.92, which proves that the model can well learn the temporal variation relationship of GHI. All in all, the model also has good spatial generalization ability in space. The model can not only learn the correct temporal variation of GHI, but also has good accuracy on unknown data.

[0086] 3. The high-precision inversion of GHI4KM from 00:00 to 11:00 UTC on July 1, 2018, was obtained according to the method mentioned in Example 1 and is shown in Figure 5 Figure.

[0087] It can be seen from Figure 5 Figure that in the early morning at 8 o'clock, the eastern part of China is brighter and the western part is darker. As time goes by, the value of GHI reaches the maximum at 12 noon and remains bright until 2 pm. After 2 pm, the value of GHI slowly decreases and all becomes dark at 7 pm. This means that the improved TabNet model can learn the temporal law of GHI change through characteristic quantities with time information such as the solar altitude angle. In addition, the improved TabNet model can also learn the high-precision characteristics of clouds, water vapor, and aerosols from FY4A satellite data, so as to give the numerical values of GHI in different regions according to these characteristics, and has high fineness, enabling the power department to conduct photovoltaic resource scheduling through the distribution of regional GHI and making contributions to the energy transformation of our country.

[0088] The embodiments described above are only descriptions of the preferred embodiments of the present invention and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for retrieving regional horizontal irradiance based on satellite data and deep learning, characterized in that The following steps are involved: Obtain FY4A full-disk L1-level 4km resolution satellite data; obtain GHI data from the radiometric measurement station and delete GHI missing values to obtain GHI preprocessed data; Matching the satellite data with the GHI pre-processed data, and removing missing data from the satellite data; Calculating the Earth-Sun distance, solar zenith angle, and solar altitude angle at each station at a certain moment based on the geographic time information of each radiation measurement station; calculating extraterrestrial solar radiation based on the sun-earth distance and the solar zenith angle, and performing quality control on the GHI preprocessed data based on the extraterrestrial solar radiation to obtain a processed GHI training set; The initial TabNet model is improved according to the characteristic that the data of the processed GHI training set is non-zero, and the improved model is trained based on the sun-earth distance, the solar zenith angle, the solar altitude angle, and the processed GHI training set to obtain an irradiance inversion model. The reprocessed satellite data feature set is input into the irradiance inversion model to obtain a high-precision inversion map.

2. The method for inverting the regional horizontal irradiance based on satellite data and deep learning according to claim 1, wherein The method for eliminating missing data in the satellite data includes: Eliminate the data in which all channels of the satellite data are all 0; Eliminate the data with missing data in any channel of the satellite data.

3. The method for inverting the regional horizontal irradiance based on satellite data and deep learning according to claim 1, wherein The method for obtaining the processed GHI training set includes: Calculate the extraterrestrial solar radiation based on the distance between the sun and the earth and the solar zenith angle: Among them, R0 represents extraterrestrial solar radiation, S0 represents the solar constant, r0 represents the average sun-earth distance, and r represents the sun-earth distance on a certain day. represents the solar zenith angle; A minimum expected GHI value is set, and data in the preprocessed data that is greater than the extraterrestrial solar radiation and less than the minimum expected GHI value is eliminated to obtain the processed GHI training set.

4. The method for inverting the regional horizontal irradiance based on satellite data and deep learning according to claim 1, wherein, The method for improving the initial TabNet model includes: adding a ReLU activation function after the last fully connected layer in the initial TabNet model according to the non-zero characteristic of GHI data.

5. A regional horizontal irradiance inversion system based on satellite data and deep learning, wherein the inversion system applies the inversion method according to any one of claims 1-4, characterized in that include: Data acquisition module, satellite data processing module, geographic data calculation module, GHI data processing module, model training module and inversion module; The data acquisition module is used to acquire FY4A full-disk L1-level 4KM resolution satellite data; acquire GHI data of the radiation measurement station and delete GHI missing values to obtain GHI pre-processed data; The satellite data processing module is used to match the satellite data with the GHI pre-processed data and remove missing data in the satellite data; The geographic data calculation module calculates the sun-earth distance, solar zenith angle and solar altitude angle of each station at a certain moment according to the geographic time information of each radiation measurement station; The GHI data processing module calculates extraterrestrial solar radiation based on the sun-earth distance and the solar zenith angle, and performs quality control on the GHI preprocessed data based on the extraterrestrial solar radiation to obtain a processed GHI training set; The model training module improves the initial TabNet model according to the non-zero data of the processed GHI training set, and trains the improved model based on the sun-earth distance, the solar zenith angle, the solar altitude angle, and the processed GHI training set to obtain an irradiance inversion model; The inversion module is used to input the reprocessed satellite data feature set into the irradiance inversion model to obtain a high-precision inversion map.

6. The regional horizontal irradiance inversion system based on satellite data and deep learning according to claim 5, wherein In the satellite data processing module, the process of removing the missing measurement data in the satellite data includes: Removing the data where all channels in the satellite data are 0; Removing the data with missing measurements in any channel of the satellite data.

7. The regional horizontal irradiance inversion system based on satellite data and deep learning according to claim 5, wherein The workflow of the GHI data processing module includes: Calculating the extraterrestrial solar radiation based on the Earth-Sun distance and the solar zenith angle: Among them, R0 represents extraterrestrial solar radiation, S0 represents the solar constant, r0 represents the average sun-earth distance, and r represents the sun-earth distance on a certain day. represents the solar zenith angle; Setting the expected minimum value of GHI, and removing the data in the preprocessed data that is greater than the extraterrestrial solar radiation and less than the expected minimum value of GHI to obtain the processed GHI training set.

8. The regional horizontal irradiance inversion system based on satellite data and deep learning according to claim 5, characterized in that, In the model training module, the process of improving the initial TabNet model includes: according to the characteristic that the GHI data is non-zero, adding a ReLU activation function after the last fully connected layer in the initial TabNet model.

Images