Scattered radiation prediction correction method based on ANN neural network algorithm

Abstract

The invention discloses a scattered radiation prediction correction method based on an ANN neural network algorithm, and belongs to the field of atmospheric environment monitoring and ecological meteorological modeling, and the method comprises the following steps: S1, data collection and preprocessing; s2, constructing a model training set; s3, constructing and training an ANN model; s4, model verification and performance evaluation; s5, calculation and correction of scattered radiation fractions; the method aims at solving the problems that scattered radiation fraction (Kd) data in an existing observation network is missing and a traditional empirical model is large in prediction error. Through constructing a trained neural network model, scattered radiation reconstruction of an unobserved site is realized, and high-quality input is further provided for an ecological model.

Description

Technical Field

[0001] This invention relates to the field of atmospheric environment monitoring and ecological meteorological modeling, and in particular to a method for predicting and correcting scattered radiation based on an artificial neural network (ANN) algorithm, which belongs to the intelligent modeling technology for solar radiation parameter inversion and its ecological applications. Background Technology

[0002] Solar radiation is the primary energy source driving the Earth's surface energy balance and plant photosynthesis. Based on its transmission path, it can be divided into direct radiation and diffuse radiation. Numerous studies have shown that diffuse radiation (i.e., diffused radiation) is more effective than direct radiation in promoting photosynthetic efficiency at different levels of leaves within the plant canopy, thus playing a crucial role in ecosystem gross primary productivity (GPP) simulations.

[0003] However, direct observational data on diffuse radiation (usually expressed as the diffuse radiation fraction Kd) is extremely scarce globally. For example, among the more than 200 stations in the global FLUXNET flux observation network, fewer than 30 stations provide direct Kd observational data. To fill this observational gap, researchers typically use empirical models based on the sky transparency index (Kt) (such as the Liu and Jordan, Erbs, and Reindl models) to invert Kd. These empirical models vary considerably in structure, and their parameter fitting is limited to specific regions, resulting in poor generalization ability across different geographical regions or climatic backgrounds, large prediction errors, and thus restricting their application in global-scale ecological modeling.

[0004] To address the aforementioned issues, the development of artificial intelligence technology in recent years, especially the application of artificial neural network (ANN) algorithms, has provided a new approach for high-precision modeling of scattered radiation. ANN models do not rely on prior physical mechanisms for modeling; instead, they establish a complex mapping relationship between input factors (such as the cosine of the solar zenith angle, shortwave radiation, and Kd in reanalysis data) and the target output (Kd) through nonlinear learning of large-scale observational data. Compared with traditional empirical models, ANN models exhibit higher correlation and lower root mean square error in scattered radiation prediction, demonstrating stronger adaptability and greater potential for accuracy improvement.

[0005] Nevertheless, the application of ANNs in Kd prediction still faces challenges in areas such as model generalization, training sample dependence, and extrapolation ability. Therefore, developing a method for correcting scattered radiation predictions that combines reanalysis data (such as MERRA radiation data), FLUXNET measured data, and ANN models is of great significance for improving the accuracy of scattered radiation inversion and revealing the response mechanisms of global ecosystems to scattered radiation. Summary of the Invention

[0006] The present invention aims to provide a method for predicting and correcting scattered radiation based on an ANN neural network algorithm to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for correcting scattered radiation prediction based on an ANN neural network algorithm includes the following steps:

[0009] Step S1: Data acquisition and preprocessing;

[0010] Step S2: Constructing the model training set;

[0011] Step S3: ANN model construction and training;

[0012] Step S4: Model validation and performance evaluation;

[0013] Step S5: Calculation and correction of the scattered radiation fraction.

[0014] Preferably, step S1 specifically includes the following steps:

[0015] The S101 and LUXNET carbon flux observation stations acquired raw observation data including total shortwave radiation (SW), solar altitude angle cosine (cosφ), air temperature, and humidity, with a data time resolution interval of 0.5h-1h.

[0016] Shortwave radiation and scattered radiation fraction data, i.e. Kd data, were obtained from S102 and NASA's MERRA reanalysis data for the corresponding time periods and locations, with the time intervals consistent with the FLUXNET data.

[0017] S103. Perform data quality control on the above data, including outlier removal, time alignment, missing value imputation, unit conversion and standardization, to obtain a unified dataset that can be used for modeling.

[0018] Preferably, step S2 specifically includes the following steps:

[0019] A training sample set was constructed by selecting several representative stations with more than 4 consecutive years of measured scattered radiation fraction (Kd) observation records from S201 and FLUXNET.

[0020] S202. Construct the training sample input vector, which contains four variables: cosφ, SW observed by FLUXNET, and SW and Kd values ​​provided by MERRA.

[0021] S203. Use the measured Kd value of FLUXNET at the corresponding time point as the output target value of the model, construct input-output paired data samples, and use them to train the ANN model.

[0022] Preferably, step S3 specifically includes the following steps:

[0023] The design employs a feedforward structure for the artificial neural network model, which includes: an input layer with four neurons corresponding to four input variables; a hidden layer with the number of nodes selected through cross-validation; and an output layer that outputs a single value, namely the predicted Kd.

[0024] The model is trained using the backpropagation algorithm, BP, with the root mean square error (RMSE) as the loss function and a certain proportion of the data used as a validation set for early stopping mechanism control.

[0025] During training, the learning rate and batch size are dynamically adjusted to prevent overfitting, improve generalization ability, and train until the model performance is stable or reaches the preset number of rounds.

[0026] Preferably, step S4 specifically includes the following steps:

[0027] S401. Use observation data from FLUXNET sites that were not used for training to independently validate the model, calculate the correlation coefficient between the predicted and measured values, i.e., R, the normalized mean deviation, i.e., NMB, and the root mean square error, i.e., RMSE index.

[0028] S402. Compare the predictive performance of the ANN model with that of the traditional Kt-Kd empirical model to verify its adaptability and accuracy under different ecological function types and climate zones.

[0029] Preferably, step S5 specifically includes the following steps:

[0030] S501. Using the trained ANN model, input the variables cosφ, FLUXNET's SW, MERRA's SW, and MERRA's Kd at the FLUXNET site where there is no measured Kd data, and calculate and output the predicted Kd value.

[0031] S502. If it is necessary to correct the MERRA reanalysis data, the difference between the model output and the original MERRA-Kd can be used as the bias correction value to generate corrected Kd time series data.

[0032] S503. Perform continuity and smoothness checks on the obtained Kd data to ensure that it is accurate for subsequent ecological modeling.

[0033] Preferably, the hidden layer is a multi-layer design.

[0034] Preferably, the traditional Kt-Kd empirical model includes the Reindl model and the Erbs model.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] This invention innovatively solves the technical problems of severe data gaps in scattered radiation, systematic biases in reanalysis data, and poor applicability of traditional empirical models in ecological flux observation networks by introducing artificial intelligence modeling technology, and has the following significant beneficial effects:

[0037] (1) Improved accuracy of scattered radiation prediction: Compared with existing Kt-Kd type empirical models, the ANN neural network model used in this invention can fully explore the nonlinear coupling relationship between solar geometric features, measured radiation and reanalysis data, and significantly improve the prediction accuracy of the scattered radiation fraction (Kd). The model shows higher correlation coefficient, smaller root mean square error and lower systematic bias on the validation dataset, with an average accuracy improvement of more than 10%, effectively ensuring the reliability requirements of subsequent ecological models for input radiation data.

[0038] (2) It has the ability to generalize across regions and climate zones: Traditional scattering radiation estimation models mostly rely on regional parameter fitting, which is difficult to apply to global flux observation networks. However, the method proposed in this invention introduces MERRA reanalysis data as auxiliary input and combines training samples from multiple regions and vegetation function types, which significantly enhances the spatial generalization ability of the model. It can be widely applied to stations in different climate zones and different underlying surface types, and realize the estimation and correction of scattering radiation on a global scale.

[0039] (3) Completing Key Observational Elements of the Ecological Flux Network: Due to the high cost of observation equipment for scatter radiation data, most FLUXNET sites worldwide lack measured Kd data, limiting its in-depth application in ecological simulation. This invention uses a neural network model to derive high-precision Kd data from existing observation and reanalysis data, achieving temporally continuous and spatially consistent scatter radiation completion for more than 200 sites worldwide, greatly enriching the FLUXNET data system and improving the comprehensive utilization efficiency of observational data.

[0040] (4) Improve the accuracy of terrestrial carbon sink response research: The high-quality Kd data output by this invention can be used to calculate key parameters of plant response to scattered light, such as diffuse fertilization efficiency (DFE), thereby revealing the impact of changes in scattered light on the carbon absorption capacity of global terrestrial ecosystems. This technology can provide high-value data support for terrestrial carbon sink assessment, ecological model calibration, and vegetation-climate feedback research. Attached Figure Description

[0041] Figure 1 This is a flowchart of a method for correcting scattered radiation prediction based on an ANN neural network algorithm;

[0042] Figure 2A flowchart for an ANN;

[0043] Figure 3 Comparison of Kd at 5 validation sites between ANN and other methods. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments:

[0045] A method for predicting and correcting scattered radiation based on an artificial neural network (ANN) algorithm is proposed to address the issues of missing scattered radiation data and poor applicability of existing empirical models in the global ecological flux observation network. This method utilizes a combination of reanalysis data and observational data to construct an ANN model, enabling high-precision inversion and correction of the fractional scattered radiation (Kd).

[0046] I. The main body of a method for predicting and correcting scattered radiation based on an artificial neural network (ANN) algorithm;

[0047] like Figure 1 As shown, it mainly includes the following steps:

[0048] Step S1: Data Acquisition and Preprocessing

[0049] First, the FLUXNET carbon flux observation station acquires raw observation data including total shortwave radiation (SW), cosine of solar altitude angle (cosφ), temperature, and humidity, with a data time resolution interval of half an hour or one hour.

[0050] Meanwhile, NASA’s MERRA reanalysis data obtained shortwave radiation (SW) and diffuse radiation fraction (Kd) data for the corresponding time periods and locations, with time intervals consistent with FLUXNET data;

[0051] Finally, data quality control was performed on the above data, including outlier removal, time alignment, missing value imputation, unit conversion, and standardization, to obtain a unified dataset that can be used for modeling.

[0052] Step S2: Model training set construction

[0053] First, a training sample set was constructed by selecting several representative stations in FLUXNET that have continuous observation records of measured fractional scattered radiation (Kd) for more than 4 years.

[0054] Secondly, construct the training sample input vector, which contains four variables: cosφ, SW observed by FLUXNET, and SW and Kd values ​​provided by MERRA;

[0055] Finally, the measured Kd values ​​of FLUXNET at the corresponding time points are used as the output target values ​​of the model to construct input-output paired data samples for training the ANN model.

[0056] Step S3: ANN Model Construction and Training

[0057] The design employs a feedforward structure for the artificial neural network model, whose structure includes:

[0058] The input layer contains four neurons, corresponding to four input variables;

[0059] At least one (preferably multiple) hidden layer, with the number of nodes selected through cross-validation;

[0060] The output layer outputs a single value, namely the predicted Kd.

[0061] The model is trained using the backpropagation (BP) algorithm, with the root mean square error (RMSE) as the loss function. A certain proportion (e.g., 20%) of the data is used as a validation set for early stopping. During training, the learning rate and batch size are dynamically adjusted to prevent overfitting and improve generalization ability. Training continues until the model performance stabilizes or reaches the preset number of epochs.

[0062] Step S4: Model Validation and Performance Evaluation

[0063] First, the model was independently validated using observation data from FLUXNET sites that were not involved in the training, and the correlation coefficient (R), normalized mean bias (NMB), and root mean square error (RMSE) between the predicted and measured values ​​were calculated.

[0064] Then, the predictive performance of the ANN model was compared with that of traditional Kt-Kd empirical models (such as the Reindl model and the Erbs model) to verify its adaptability and accuracy under different ecological function types and climate zones.

[0065] Step S5: Calculation and correction of the scattered radiation fraction

[0066] First, using the trained ANN model, input variables such as cosφ, SW (FLUXNET), SW (MERRA), and Kd (MERRA) at the FLUXNET site where there is no measured Kd data, and calculate and output the predicted Kd value.

[0067] Secondly, if it is necessary to correct the MERRA reanalysis data, the difference between the model output and the original MERRA-Kd can be used as the bias correction value to generate corrected Kd time series data.

[0068] Finally, the obtained Kd data is checked for continuity and smoothness to ensure that it is accurate for subsequent ecological modeling.

[0069] II. Further Explanation

[0070] To better understand the scattered radiation prediction and correction method based on the artificial neural network (ANN) algorithm provided by this invention, several key steps and technical points of the present invention are further explained below:

[0071] 1. The basis and significance of selecting input variables

[0072] The cosine of the solar zenith angle (cosφ) reflects the sun's altitude in the sky and is closely related to the intensity of radiation received by the ground and the length of the radiation path. It is a fundamental astronomical factor affecting the degree of radiation scattering. The smaller the cosφ, the closer the sun is to the horizon, resulting in a longer scattering path and an increased proportion of scattered radiation. Incorporating cosφ as an input variable into a neural network helps the model capture the modulation effect of the diurnal variation cycle on Kd.

[0073] Shortwave radiation (SW) observed by FLUXNET represents the total solar radiation intensity measured on the ground and is a direct component of the diffuse radiation fraction. Because the FLUXNET network covers a variety of ecosystems, its SW data is highly spatially representative and can effectively support regionalized model training.

[0074] MERRA's SW and Kd data, a global reanalysis product released by NASA, offers SW and Kd data characterized by good continuity and wide spatial coverage. Although some bias exists, the background illumination information it provides can serve as a reference for neural network input, helping to improve model generalization ability when measured data is insufficient. Especially when no measured Kd data is available at the target site, MERRA-Kd, as an input variable, can convey optical state information at the regional scale, providing a foundation for extrapolation.

[0075] 2. Regarding the structure and training methods of artificial neural networks

[0076] The ANN model used in this invention is a typical feedforward backpropagation neural network structure, which can effectively model nonlinear function relationships. The number of network layers and nodes can be flexibly adjusted according to the amount of training data and model complexity. The preferred configuration is 1 input layer, 1 to 3 hidden layers, and 1 output layer, wherein the number of hidden layer nodes is generally set between 10 and 100.

[0077] During training, the backpropagation (BP) algorithm is used for weight adjustment, and the root mean square error (RMSE) is used as the loss function to measure the model's prediction bias. A validation set is used to monitor the model's training performance and prevent overfitting. The training process can combine early stopping and cross-validation strategies to improve the model's robustness and generalization ability.

[0078] 3. The role and significance of the scattered radiation correction method

[0079] In traditional scattering radiation inversion, Kd values ​​are often estimated using Kt-Kd empirical models (such as the Liu-Jordan, Erbs, and Reindl models). These models are usually based on empirical fitting under specific regional or climatic conditions, with fixed model structures and unadjustable parameters, making it difficult to adapt to different ecological regions and dynamic climatic conditions.

[0080] In contrast, the ANN model used in this invention captures complex, nonlinear, and regionally adaptable relationships through data-driven learning. Especially when there are biases or missing data in MERRA, ANN can automatically "learn" from historical data and correct these biases, thereby achieving effective correction of reanalysis data and improving the reliability of the results.

[0081] 4. Calculation principle and application value of diffused fertilization efficiency (DFE)

[0082] The Kd prediction results provided by this invention can be used to further evaluate the impact of scattered radiation on plant photosynthetic efficiency. By categorizing Kd into segments (such as high scattering and low scattering conditions) and analyzing the rate of change in GPP caused by a unit increase in scattered radiation, the scattering fertilization efficiency (DFE) index can be obtained.

[0083] This index can be used to assess the sensitivity of different vegetation types to scattered light; it helps to explain changes in carbon absorption in ecosystems such as forests, grasslands, and farmland under cloudy weather or high aerosol concentration conditions; and it can provide a basis for parameter optimization for land surface process models (such as CLM, ORCHIDEE, etc.) to improve the accuracy of global carbon cycle simulation.

[0084] The specific implementation process is as follows:

[0085] like Figure 2-3As shown, this invention proposes a method for predicting and correcting scattered radiation based on an artificial neural network (ANN) algorithm, aiming to solve the problems of missing scattered radiation fraction (Kd) data and large prediction errors of traditional empirical models in existing observation networks. By constructing a trained neural network model, the scattered radiation of unobserved sites can be reconstructed, and high-quality input can be provided for ecological models. The following detailed description of the technical solution of this invention is provided in conjunction with the specific implementation process.

[0086] First, 20 observation stations with at least four years of measured fractional diffuse radiance (Kd) records were selected from the Global Carbon Flux Observation Network (FLUXNET2015) to serve as the data foundation for model development. These stations cover a variety of ecosystem types, from tropical rainforests and temperate forests to grasslands and farmlands, and are representative of typical ecosystems. Simultaneously, NASA's MERRA-2 reanalysis data were acquired as auxiliary input, including total shortwave radiation (SW) and fractional diffuse radiance (Kd) information from the reanalysis product at the corresponding spatiotemporal resolution.

[0087] Subsequently, all raw data underwent preprocessing. First, the data recorded at the FLUXNET site was matched spatially and temporally with the MERRA product based on the site's latitude, longitude, and timestamp. Second, linear interpolation was performed on data with inconsistent time resolutions to unify all input data to a half-hour resolution. For missing values, imputation using the mean of neighboring time periods was used. To improve model training performance, all numerical input data were standardized to the range [0,1] to avoid learning bias caused by different units of measurement.

[0088] During the model building phase, 15 sites were selected as training samples, and 5 sites were used as independent validation sets. The model adopts a typical feedforward multilayer neural network structure. The input layer has 4 neurons, corresponding to: the cosine of the solar zenith angle (cosφ), the shortwave radiation (SW) observed by FLUXNET, the SW provided by MERRA, and Kd provided by MERRA. The hidden layer consists of two layers, each containing 12 neurons, and the ReLU activation function is used to enhance nonlinear expression. The output layer has a single neuron, used to output the predicted Kd value. The model uses the root mean square error (RMSE) as the loss function, and the Adam training optimization algorithm is used, with an initial learning rate of 0.001, a batch size of 256, and network weights updated in each iteration. The maximum number of training epochs is set to 500. To prevent overfitting, 20% of the validation data is introduced in each iteration for real-time evaluation. If the error on the validation set no longer decreases after 15 consecutive epochs, training is stopped early.

[0089] After training, the model's prediction accuracy was tested on the validation set. The results showed that the model had an average correlation coefficient of 0.81 and an average RMSE of 0.12 across the five sites, significantly outperforming the traditional empirical model Reindl (correlation coefficient of 0.72 and RMSE of 0.17). This demonstrates that the ANN model constructed in this invention possesses good generalization ability and stability.

[0090] Furthermore, the trained neural network model was extended to the remaining 180 FLUXNET sites that did not have actual scattered radiation sensors. At each time step, the cosφ value, measured SW value, and SW and Kd provided by MERRA for that site were input, and the corresponding scattered radiation fraction Kd was predicted by the model output. To ensure the physical consistency and continuity of the reanalysis product, the systematic deviation between the model prediction and the original MERRA-Kd value could be calculated and used to correct the reanalysis data, thereby correcting and completing the scattered radiation field.

[0091] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solutions of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for scatter radiation prediction correction based on ANN neural network algorithm, characterized in that: The method comprises the following steps: Step S1: data acquisition and preprocessing; Step S2: model training set construction; Step S3: ANN model construction and training; Step S4: model verification and performance evaluation; Step S5: calculation and correction of scattered radiation fraction.

2. The method of claim 1, wherein the ANN neural network algorithm is based on, The specific steps of the step S1 comprise: S101, obtaining total shortwave radiation, i.e. SW, solar height angle cosine, i.e. cosφ, air temperature, humidity original observation data from LUXNET carbon flux observation site, and the data time resolution interval is 0.5h-1h; S102, obtaining shortwave radiation and scattered radiation fraction data, i.e. Kd data, of the corresponding period and place from MERRA reanalysis data of NASA, and the time interval is consistent with the FLUXNET data time; S103, performing data quality control on the above data, including outlier rejection, time alignment, missing value interpolation, unit conversion and standardization processing, to obtain a unified data set that can be used for modeling.

3. The method of claim 1, wherein the ANN neural network algorithm is based on, The specific steps of the step S2 comprise: S201, selecting several representative sites with continuous observation records of measured scattered radiation fraction (Kd) for more than 4 years in FLUXNET to construct a training sample set; S202, constructing a training sample input vector, which contains four variables: cosφ, SW observed by FLUXNET, SW and Kd values provided by MERRA; S203, taking the measured Kd value of FLUXNET at the corresponding time as the output target value of the model, constructing input-output paired data samples for training the ANN model.

4. The method of claim 1, wherein the ANN neural network algorithm is based on, The specific steps of the step S3 comprise: An artificial neural network model with a feedforward structure is designed, and the model structure comprises: an input layer containing 4 neurons corresponding to four input variables; a hidden layer is set, and the number of nodes is selected through cross-validation; an output layer outputs a single value, i.e. the predicted Kd; The back propagation algorithm, i.e. BP, is used to train the model, the root mean square error, i.e. RMSE, is used as the loss function, and a certain proportion of data is used as the validation set for early stopping mechanism control; During the training process, the learning rate and batch size are dynamically adjusted to prevent overfitting, improve the generalization ability, and train until the model performance is stable or the preset number of rounds is reached.

5. The method of claim 1, wherein the ANN neural network algorithm is based on a feed-forward neural network. The specific steps of the step S4 comprise: S401, using the observation data of the FLUXNET sites not participating in the training to independently verify the model, and calculating the correlation coefficient, i.e. R, between the predicted value and the measured value, the normalized mean bias, i.e. NMB, and the root mean square error, i.e. RMSE, indicators; S402, comparing the prediction performance of the ANN model with that of the traditional Kt-Kd empirical model to verify its adaptability and accuracy under different ecological function types and climate zones.

6. The method of claim 1, wherein the ANN neural network algorithm is based on, The specific steps of the step S5 comprise: S501, using the trained ANN model to input the variables of cosφ, SW of FLUXNET, SW of MERRA, and Kd of MERRA at FLUXNET sites without measured Kd data to calculate and output the predicted Kd value; S502, if the MERRA reanalysis data needs to be corrected, the difference between the model output and the original MERRA-Kd can be used as a bias correction value to generate corrected Kd time series data; S503, the obtained Kd data is checked for continuity and smoothness to ensure that it is correct for subsequent ecological modeling.

7. The method of claim 4, wherein the ANN neural network algorithm is based on a feed-forward neural network. The setting hidden layer is a multi-layer design.

8. The method of claim 5, wherein the ANN neural network algorithm is based on a back propagation algorithm. The traditional Kt-Kd empirical model includes Reindl model and Erbs model.

Images