Large-range high-frequency all-sky surface solar irradiance machine learning estimation method and system suitable for cloudy area

Abstract

The invention discloses a large-range high-frequency all-sky earth surface solar irradiance machine learning estimation method and system suitable for a cloudy area, and relates to the technical field of irradiance estimation, and the method comprises the steps: obtaining earth surface radiation observation data, generating a station position table, and constructing a training data set; constructing a ground solar radiation degree model, and dividing the training data set into a training set and a test set according to a proportion of 80% to 20%; determining hyper-parameters of the Xgboost algorithm through parameter search, training the model on the training set, and checking the model by using the test set; new satellite reflectivity is used for observing data and meteorological elements, data cleaning is carried out, and the model is input to estimate the space-time distribution of the total solar radiation irradiance of the earth surface. According to the method, the space and time precision of radiation inversion is improved, the complex nonlinear characteristics and data interaction relation are accurately captured, efficient inversion of the earth surface solar radiation degree is achieved, and the prediction performance and generalization ability of the model are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of irradiance estimation, and in particular to a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method and system suitable for cloudy areas. Background Art

[0002] Solar radiation, one of Earth's most important natural energy sources, has a crucial impact on climate, ecology, agriculture, and the use of renewable energy. Solar irradiance is a key indicator of solar radiation intensity. It represents the sum of direct solar radiation and atmospheric diffuse radiation received per unit area on a horizontal plane perpendicular to the ground. Its variations are influenced by a variety of factors, including atmospheric composition, cloud cover, topography, and seasonal variations, with cloud parameter variables having a significant impact. These factors collectively determine the intensity and distribution of radiation in different regions. Accurate solar irradiance data is crucial for the development of renewable energy, particularly solar power generation. It helps decision-makers optimize the siting and design of solar facilities and improve energy efficiency.

[0003] However, traditional ground-based solar radiation monitoring methods are limited by the spatial coverage of measurement sites, making it difficult to provide comprehensive, real-time regional and global solar radiation data. In recent years, with the rapid development of satellite and remote sensing technologies, satellite-based solar radiation estimation methods have gradually emerged. Geostationary satellites, due to their stationary nature relative to the Earth, can continuously observe specific areas, offering advantages such as wide coverage, real-time monitoring, and good data continuity. Therefore, using geostationary satellites to estimate solar radiation across the entire sky has become a new research hotspot and technological trend.

[0004] Currently, integrating satellite data and advanced modeling techniques is crucial for generating more accurate and high-resolution assessments of solar radiation distribution. However, how to effectively integrate these data sources and construct models that can capture complex spatial variations while ensuring high accuracy remains a key challenge in existing research. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the problem to be solved by the present invention is: how to solve the problem that traditional ground-based solar radiation monitoring methods are limited by the spatial coverage of measurement sites and are difficult to provide comprehensive and real-time regional and global solar radiation data.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas, comprising: obtaining surface radiation observation data, generating a site location table, and constructing a training data set; constructing a ground solar irradiance model, and dividing the training data set into a training set and a test set at a ratio of 80% to 20%; determining the hyperparameters of the Xgboost algorithm through parameter search, training the model on the training set, and using the test set to test the model; using new satellite reflectivity observation data and meteorological elements, and performing data cleaning, and inputting the data into the model to estimate the spatiotemporal distribution of the surface total solar irradiance.

[0008] As a preferred solution of the large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas described in the present invention, the acquisition of surface radiation observation data includes obtaining radiation stations with representative spatial distribution, a total radiation irradiance historical data of a time length of 1 year, a time resolution of 1 hour, preliminary quality control of the data, removal of outliers, and UTC time conversion of the preprocessed samples.

[0009] As a preferred solution of the large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas described in the present invention, the construction of the training data set includes using surface radiation observation data as labels, and merging reanalysis data and satellite reflectivity observation data as feature values into the training data set, and performing training data set cleaning, including solar angle screening and cosine transformation.

[0010] As a preferred solution of the large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas described in the present invention, the reanalysis data includes obtaining reanalysis data within the same time period with a time resolution of 1 hour, and extracting corresponding reanalysis data using a time-space matching algorithm based on a station location table; the satellite reflectivity observation data includes obtaining satellite observation data within the same time period with a time resolution of 1 hour, performing preprocessing including radiation calibration, brightness temperature conversion, and geometric correction and clipping, and extracting corresponding reflectivity data using a time-space matching algorithm based on a station location table.

[0011] As a preferred solution of the large-scale high-frequency full-sky surface solar irradiance machine learning estimation method applicable to cloudy areas described in the present invention, the ground solar irradiance model includes adopting an XGBoost machine learning model; calculating the Gain index, and selecting the split with the largest Gain index as the optimal split point for the current node, which is expressed as:

[0012] Gain m =i m -(wleft ·i left +w right ·i right )

[0013] Among them, Gain m To measure the extent to which the loss function is reduced after splitting m nodes, i m is the impurity of node m, w left is the weight ratio of the parent node sample in the left child node, w right is the weight ratio of the parent node sample in the right child node, i left is the impurity of the left child node of node m, i right is the impurity of the right child node of node m; the total importance of the feature is calculated after the importance of feature f of all trees t in the random forest containing a total of T trees. The total importance of feature f is expressed as:

[0014]

[0015] Among them, Importance f is the total importance of feature f, T is the total number of trees, t is the t-th tree, is the importance of a given feature f in tree t.

[0016] As a preferred embodiment of the large-scale high-frequency full-sky surface solar irradiance machine learning estimation method applicable to cloudy areas of the present invention, the importance of a given feature f in the tree t is expressed as:

[0017]

[0018] in, is the total number of nodes in the t-th tree.

[0019] As a preferred solution of the large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas described in the present invention, the verification model includes using a 10-fold cross-validation method to fix the model's hyperparameters to ensure the model's generalization ability, dividing the data set into a training set and a test set at a ratio of 80% and 20%, and using the root mean square error and determination coefficient to evaluate the model accuracy.

[0020] Another object of the present invention is to provide a system for a large-scale high-frequency full-sky surface solar irradiance machine learning estimation method suitable for cloudy areas. The system can solve the problem of large-scale high-frequency full-sky surface solar irradiance machine learning estimation suitable for cloudy areas by constructing a large-scale high-frequency full-sky surface solar irradiance machine learning estimation system suitable for cloudy areas.

[0021] To solve the above technical problems, the present invention provides the following technical solutions: a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation system suitable for cloudy areas, comprising a data acquisition module, a model construction module, a training verification module and an irradiance estimation module; the data acquisition module is used to obtain surface radiation observation data, generate a site location table, and construct a training data set; the model construction module is used to construct a ground solar irradiance model, and divide the training data set into a training set and a test set at a ratio of 80% to 20%; the training verification module is used to determine the hyperparameters of the Xgboost algorithm through parameter search, train the model on the training set, and use the test set to test the model; the irradiance estimation module is used to use new satellite reflectivity observation data and meteorological elements, perform data cleaning, and input the data into the model to estimate the spatiotemporal distribution of the surface total solar radiation irradiance.

[0022] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above-mentioned large-scale high-frequency full-sky surface solar irradiance machine learning estimation method suitable for cloudy areas.

[0023] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas.

[0024] The beneficial effects of the present invention are as follows: a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas provides a method that takes into account the key influence of cloud parameters on solar radiation, combines high-temporal and spatial resolution observation data from geostationary meteorological satellites, constructs a method for estimating all-sky solar radiation, improves the spatial and temporal accuracy of radiation inversion, introduces a machine learning algorithm, integrates multi-source meteorological, surface and satellite observation data, accurately captures complex nonlinear characteristics and data interaction relationships, realizes efficient inversion of surface solar irradiance, and significantly improves the prediction performance and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 A flowchart of a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas is provided for the first embodiment of the present invention.

[0027] Figure 2 A technical roadmap for a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas is provided for the second embodiment of the present invention.

[0028] Figure 3 This is a structural diagram of a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation system suitable for cloudy areas, provided in the third embodiment of the present invention.

[0029] Figure 4 A density scatter plot of the model fitting verification of a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas, provided in the fourth embodiment of the present invention.

[0030] Figure 5 A scatter plot of the validation results of three-station data for a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas, provided in the fourth embodiment of the present invention.

[0031] Figure 6 This is a solar irradiance map of the FY-4A, which is suitable for a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method in cloudy areas, provided in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0034] Example 1, with reference to Figure 1 , which is the first embodiment of the present invention, provides a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas, including: obtaining surface radiation observation data, generating a site location table, and constructing a training data set; constructing a ground solar irradiance model, and dividing the training data set into a training set and a test set at a ratio of 80% to 20%; determining the hyperparameters of the Xgboost algorithm through parameter search, training the model on the training set, and using the test set to verify the model; using new satellite reflectivity observation data and meteorological elements, and performing data cleaning, and inputting them into the model to estimate the spatiotemporal distribution of the surface total solar irradiance.

[0035] Based on a thorough analysis of the physical mechanism of solar irradiance formation, this invention fully considers cloud parameters, a key influencing variable, and utilizes high-resolution observation data from geostationary satellites and advanced machine learning algorithms to propose a full-sky solar radiation estimation method based on geostationary satellites. This method provides more accurate and comprehensive data support for the development and utilization of solar energy resources, climate change research, and environmental protection, and has important scientific significance and application value.

[0036] S1. Obtain surface radiation observation data, generate site location tables, and construct training datasets.

[0037] Obtain historical data of total radiation irradiance for a period of one year at radiation stations with representative spatial distribution, with a time resolution of one hour. Perform preliminary quality control on the data, remove outliers, and convert the preprocessed samples to UTC time.

[0038] The surface radiation observation data are used as labels, and the reanalysis data and satellite reflectivity observation data are merged into a training dataset as feature values. The training dataset is then cleaned, including solar angle screening and cosine transformation.

[0039] The reanalysis data include obtaining reanalysis data within the same time period with a time resolution of 1 hour. According to the station location table, a time-space matching algorithm is used to extract the corresponding reanalysis data.

[0040] Satellite reflectivity observation data include obtaining satellite observation data within the same time period with a time resolution of 1 hour, performing preprocessing, including radiometric calibration, brightness temperature conversion, and geometric correction and cropping, and extracting the corresponding reflectivity data using a time-space matching algorithm based on the station location table.

[0041] S2. Build a ground solar radiation model and divide the training data set into a training set and a test set in a ratio of 80% to 20%.

[0042] The ground solar radiation model uses the XGBoost machine learning model; the hyperparameters include the number of trees and the maximum tree depth; the Gain index is calculated, and the split with the largest Gain index is selected as the optimal split point for the current node, which is expressed as:

[0043] Gain m =i m -(w left ·i left +w right ·i right )

[0044] Among them, Gain m To measure the extent to which the loss function is reduced after splitting m nodes, i mis the impurity of node m, w left is the weight ratio of the parent node sample in the left child node, w right is the weight ratio of the parent node sample in the right child node, i left is the impurity of the left child node of node m, i right is the impurity of the right child of node m.

[0045] The total importance of a feature is calculated by calculating the importance of feature f for all trees t in a random forest containing a total number of T trees. The total importance of feature f is expressed as:

[0046]

[0047] Among them, Importance f is the total importance of feature f, T is the total number of trees, t is the t-th tree, is the importance of a given feature f in tree t.

[0048] The importance of a given feature f in a tree t is expressed as:

[0049]

[0050] in, is the total number of nodes in the t-th tree.

[0051] S3. Determine the hyperparameters of the Xgboost algorithm through parameter search, train the model on the training set, and use the test set to test the model.

[0052] The 10-fold cross-validation method was used to fix the model's hyperparameters to ensure the model's generalization ability. The dataset was divided into a training set and a test set at a ratio of 80% and 20%, respectively. The root mean square error and coefficient of determination were used to evaluate the model's accuracy.

[0053] S4. Use new satellite reflectivity observation data and meteorological elements, perform data cleaning, and input them into the model to estimate the spatiotemporal distribution of total surface solar radiation irradiance.

[0054] Example 2, reference Figure 2 , which is the second embodiment of the present invention, provides a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas, and further includes:

[0055] This paper proposes a high-frequency, large-scale surface solar radiation inversion method based on machine learning. This method integrates meteorological data, spectral characteristics, and cloud parameters, and directly estimates surface solar radiation intensity using observational data from a new generation of geostationary meteorological satellites. This method fully considers the significant influence of clouds on solar radiation variations and, by combining ERA5 meteorological data with high-temporal and spatial resolution continuous observations from geostationary satellites, effectively improves the accuracy of surface solar radiation estimation in cloudy conditions. By introducing satellite spectral characteristics and time series constraints, as well as cloud optical parameter information, a full-sky solar radiation estimation model applicable to both cloudy and clear-sky conditions is established. This model provides more accurate and comprehensive data support for fields such as solar energy development, climate change analysis, and environmental monitoring, and has significant application potential and research value.

[0056] The technical solution of the present invention is: taking the Fengyun-4 meteorological satellite as an example to obtain the ground solar irradiance in the cloudy Guangxi Zhuang Autonomous Region, the main technical solution is explained. The relevant technical method steps are as follows: Figure 2 :

[0057] Obtain surface radiation observation data. Obtain historical global irradiance (GHI) data for representative radiation stations in the Guangxi Zhuang Autonomous Region, covering a period of one year and with a temporal resolution of one hour. Perform preliminary quality control on the station data, remove outliers, and convert the preprocessed samples to UTC.

[0058] Generate a site location table for subsequent data matching.

[0059] Obtain reanalysis data. Obtain reanalysis data for the same time period with a time resolution of 1 hour. Based on the station location table in the previous step, use the space-time matching algorithm to extract the corresponding reanalysis data.

[0060] Obtain satellite reflectivity observation data. Obtain satellite observation data for the same time period with a 1-hour temporal resolution and perform preprocessing, including radiometric calibration, brightness temperature conversion, and geometric correction and cropping. Based on the station location table from the previous step, use a spatiotemporal matching algorithm to extract the corresponding reflectivity data.

[0061] Construct a training dataset. Surface radiation observation data is used as labels, and reanalysis data and satellite reflectivity observation data are combined as features to form the training dataset. The training dataset is then cleaned, including sun angle screening and cosine transformation.

[0062] Estimation model training. Split the training dataset into a training set and a test set at an 80%:20% ratio. Determine the hyperparameters of the Xgboost algorithm through parameter search. Train the model on the training set and test it on the test set to check for overfitting, underfitting, and other issues to ensure model generalization.

[0063] Solar radiation distribution estimation: Using new satellite reflectance observations and meteorological elements (consistent with those in the reanalysis dataset), and after data cleaning, the model can be used to estimate the spatiotemporal distribution of the global solar irradiance (GHI) at the surface.

[0064] Furthermore, a feature dataset was constructed using data such as reflectance in 14 bands, solar zenith angle, land use and surface elevation, and cloud optical thickness from the FY-4A geostationary meteorological satellite. ERA5 reanalysis data were also introduced, including key meteorological parameters such as surface atmospheric pressure (P), air temperature (T), planetary boundary layer height (PBLH), relative humidity (RH), u wind (U10) and v wind (V10) components at 10 meters, total water vapor column (TCW), and total ozone column. These data were resampled to match the spatial resolution of the FY-4A. Data from ground-based radiation measurement sites were used for model training and validation.

[0065] Furthermore, a ground solar irradiance inversion model is constructed. The present invention adopts the XGBoost machine learning model to construct a statistical model of ground solar irradiance inversion from satellite remote sensing, and performs model improvement and optimization training according to the training target requirements. Similar to the random forest (RF) algorithm, XGBoost is an integration method based on multiple weak learners. XGBoost uses a boosting strategy, while random forest uses a bagging strategy. There are significant differences in model construction between these two methods: the weak learners in random forest run in parallel and share the same data distribution; while the weak learners of XGBoost run serially, focusing on samples that were predicted incorrectly by the previous round.

[0066] The XGBoost algorithm is computationally very efficient. Under the same hyperparameter settings, its training time is only 1 / 7 of that of the random forest model. Its main hyperparameters include the number of trees and the maximum tree depth.

[0067] Gain m =i m -(w left ·i left +w right ·i right )

[0068] Among them, Gain mTo measure the extent to which the loss function is reduced after splitting m nodes, it is used to measure the extent to which the loss function is reduced after splitting a certain node. Simply put, it represents the contribution of a candidate split to the improvement of the model prediction error. The larger the Gain, the more accurate the prediction, and therefore the more valuable it is. When constructing a decision tree, XGBoost will traverse all possible splitting schemes, calculate their respective Gains, and select the split with the largest Gain as the best split point for the current node. The calculation of Gain takes into account both model complexity and prediction error. Only when the candidate split can bring about sufficient loss reduction will the split be executed, which helps prevent overfitting. As a measurement indicator, Gain not only helps determine which split can maximize loss reduction, but also provides a basis for the pruning process. After the tree is constructed, Gain can be further used to determine whether some branches need to be pruned to improve the generalization ability of the model. i m 、i left and i right They represent the impurities of node m, its left child node, and its right child node respectively. w represents the weight, which is defined as the proportion of parent node samples in the child node. left is the weight ratio of the parent node sample in the left child node, w right is the weight ratio of the parent node sample in the right child node. In order to derive the total impurity reduction of a given feature f in the tree t, we need to calculate all the nodes that are split on the feature f. The impurity reductions of the nodes are summed and then divided by the total impurity reduction of all nodes in the tree. Finally, the total importance of feature f is calculated after the importance of feature f of all trees t in the random forest containing a total number of trees T, and is expressed as:

[0069]

[0070] Among them, Importance f is the total importance of feature f, T is the total number of trees, t is the t-th tree, The importance of a given feature f in tree t reflects the contribution of a feature to the model, helping us understand which features have the greatest impact on predictions. This metric automatically handles feature interactions and considers the global role of features in the model. Compared to traditional methods, XGBoost's feature importance is more consistent with the model's internal logic and supports visual analysis. It can be expressed as:

[0071]

[0072] in, is the total number of nodes in the t-th tree.

[0073] To capture the complex nonlinear relationship between solar radiation and environmental characteristics, the model incorporates 32 multi-source heterogeneous features, including radiation observation data from the FY-4A satellite at 14 wavelengths, two observation angles, land cover type, altitude, cloud mask, cloud parameters, and 14 meteorological variables (such as temperature, humidity, and wind speed) from the ERA-5 reanalysis data. This feature integration significantly enhances the model's ability to depict the atmospheric and geographical environment, helping to improve the accuracy and applicability of solar radiation estimates.

[0074] Furthermore, the accuracy of the ground solar radiation inversion model was verified. In the present invention, data from 162 ground radiation measurement stations in China provided by the China Meteorological Administration (CMA) in 2018 were used for model training and verification, of which 3 stations in Guangxi were used as independent validation sets, and the remaining 159 stations were used for model training. A 10-fold cross-validation (CV) method was used to fix the hyperparameters of the model to ensure the generalization ability of the model. The dataset was then divided into a training set and a test set at a ratio of 80% (355,146 samples) and 20% (88,787 samples). Finally, the root mean square error (RMSE) and the coefficient of determination (R 2 ) to evaluate the model accuracy.

[0075] Example 3, reference Figure 3 , which is the third embodiment of the present invention, is different from the previous embodiment in that it provides a large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation system suitable for cloudy areas, including: a data acquisition module, a model building module, a training and verification module, and an irradiance estimation module.

[0076] The data acquisition module is used to obtain surface radiation observation data, generate site location tables, and construct training data sets.

[0077] The model building module is used to build a ground solar radiation model and divide the training data set into a training set and a test set in a ratio of 80% to 20%.

[0078] The training and validation module is used to determine the hyperparameters of the Xgboost algorithm through parameter search, train the model on the training set, and verify the model using the test set.

[0079] The irradiance estimation module is used to use new satellite reflectance observation data and meteorological elements, perform data cleaning, and input the model to estimate the spatiotemporal distribution of the total solar radiation irradiance on the surface.

[0080] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0081] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0082] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0083] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0084] Example 4, with reference to Figures 4 to 6 , which is the fourth embodiment of the present invention. It is different from the previous two embodiments in that it is used to verify the technical effects adopted in the present invention in order to verify the real effects of this method.

[0085] The fitting of the full sky solar radiation estimation model on the training set and test set in 2018 is shown as follows: Figure 4 As shown, Figure 2 The left side shows the training set, and the right side shows the density scatter plot of the model fit validation on the test set. The color of the dots represents the data. The predicted results are highly consistent with the actual solar radiation values in the test set, with R² greater than 0.70, indicating that the model not only has strong fitting ability but also exhibits good generalization performance.

[0086] The solar radiation estimation results under clear and cloudy conditions were verified using observation data from Guangxi that were not involved in the model training. Figure 5 As shown in the figure, the scatter plot of the verification results of the data from three stations in Guangxi in 2018. The upper part shows the results under sunny conditions, while the lower part represents cloudy conditions. The classification of sunny and cloudy days is based on the cloud mask product of FY-4A. The black dotted line represents the 1:1 line, and the red line is the fitting line between the estimated value and the observed value. The results show that the estimated results of the three stations of Guilin, Beihai and Nanning are similar to the performance of the training set and the test set, indicating that the model has good generalization ability. Among them, the estimated value of Guilin station is closest to the actual observation value, followed by Beihai and Nanning. Under sunny conditions, the determination coefficient (R 2 ) and root mean square error (RMSE) performed better under cloudy conditions, while Guilin station performed better under cloudy conditions. Beihai station’s RMSE was lower on sunny days, but its R 2 The above results indicate that the model is able to estimate solar radiation for the entire sky. Figure 6 is the solar irradiance of FY-4A on April 1, 2018.

[0087] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A machine learning method for estimating large-scale, high-frequency, all-sky surface solar irradiance suitable for cloudy areas, characterized by: include, Obtain surface radiation observation data, generate site location tables, and construct training datasets; Build a ground solar radiation model and divide the training data set into a training set and a test set at a ratio of 80% to 20%; Determine the hyperparameters of the Xgboost algorithm through parameter search, train the model on the training set, and use the test set to test the model; New satellite reflectivity observation data and meteorological elements are used, and data cleaning is performed to input the model to estimate the temporal and spatial distribution of the total solar radiation irradiance on the surface.

2. The large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas according to claim 1, characterized in that: The acquisition of surface radiation observation data includes obtaining radiation stations with representative spatial distribution, historical data of total radiation irradiance for a period of 1 year, a time resolution of 1 hour, performing preliminary quality control on the data, removing outliers, and converting the pre-processed samples to UTC time.

3. The large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas according to claim 2, characterized in that: The construction of the training data set includes combining surface radiation observation data as labels, reanalysis data and satellite reflectivity observation data as feature values into the training data set, and performing training data set cleaning, including sun angle screening and cosine transformation.

4. The large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas according to claim 3, characterized in that: The reanalysis data includes obtaining reanalysis data within the same time period with a time resolution of 1 hour, and extracting corresponding reanalysis data using a time-space matching algorithm based on a station location table; The satellite reflectivity observation data includes obtaining satellite observation data within the same time period with a time resolution of 1 hour, performing preprocessing, including radiation calibration, brightness temperature conversion, geometric correction and clipping, and extracting the corresponding reflectivity data based on the site location table using a time-space matching algorithm.

5. The large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas according to claim 4, characterized in that: The ground solar radiation model includes adopting an XGBoost machine learning model; Calculate the Gain index and select the split with the largest Gain index as the best split point for the current node, expressed as, Gain m =i m -(w left ·i left +w right ·i right ) Among them, Gain m To measure the extent to which the loss function is reduced after splitting m nodes, i m is the impurity of node m, w left is the weight ratio of the parent node sample in the left child node, w right is the weight ratio of the parent node sample in the right child node, i left is the impurity of the left child node of node m, i right is the impurity of the right child node of node m; The total importance of a feature is calculated by calculating the importance of feature f for all trees t in a random forest containing a total number of trees T. The total importance of feature f is expressed as, Among them, Importance f is the total importance of feature f, T is the total number of trees, t is the t-th tree, is the importance of a given feature f in tree t.

6. The large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas according to claim 5, characterized in that: The importance of a given feature f in the tree t is expressed as, in, is the total number of nodes in the t-th tree.

7. The large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method applicable to cloudy areas according to claim 6, characterized in that: The test model includes fixing the hyperparameters of the model using a 10-fold cross-validation method to ensure the generalization ability of the model, dividing the data set into a training set and a test set in a ratio of 80% and 20%, and using the root mean square error and determination coefficient to evaluate the model accuracy.

8. A system using the large-scale, high-frequency, all-sky surface solar irradiance machine learning estimation method for cloudy areas according to any one of claims 1 to 7, characterized in that: Including data acquisition module, model building module, training and verification module and irradiance estimation module; The data acquisition module is used to obtain surface radiation observation data, generate a station location table, and construct a training data set; The model building module is used to build a ground solar radiation model, dividing the training data set into a training set and a test set in a ratio of 80% to 20%; The training and verification module is used to determine the hyperparameters of the Xgboost algorithm through parameter search, train the model on the training set, and use the test set to verify the model; The irradiance estimation module is used to use new satellite reflectivity observation data and meteorological elements, perform data cleaning, and input the data into a model to estimate the spatiotemporal distribution of the total solar radiation irradiance on the surface.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a large-scale high-frequency all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a large-scale high-frequency all-sky surface solar irradiance machine learning estimation method suitable for cloudy areas are implemented according to any one of claims 1 to 7.

Images