Automatic station soil humidity observation data layer-by-layer correction method based on meteorological environment factors and multi-model integration

Abstract

The invention discloses an automatic station soil humidity observation data layer-by-layer correction method based on meteorological environment factors and multi-model integration, which considers that the influence of evaporation and rainfall on soil humidity has a hysteresis effect and the change of humidity of different layers of soil is influenced by the change of humidity of an upper layer at the same time. The method comprises the following steps of: performing layer-by-layer deviation correction on automatic station soil humidity observation data by taking manually observed soil humidity as a reference, combining meteorological and environmental factors such as air temperature, wind speed, rainfall and vegetation index and comprehensively adopting a generalized additive model (GAM) to integrate various machine learning algorithms such as Cubist, random forest, XGBoost and CatBoost; the observation precision of the corrected soil humidity data is remarkably improved, particularly, more obvious improvement is shown in deep soil, construction of a high-precision and long-term stable soil humidity observation network is facilitated, and a reliable data basis is provided for verification of remote sensing and numerical simulation soil humidity products.

Description

Technical Field

[0001] This invention relates to the field of soil moisture monitoring technology, specifically to a method for layer-by-layer correction of soil moisture observation data from automatic weather stations based on meteorological environmental factors and multi-model integration. Background Technology

[0002] Soil moisture, as a key climate variable in the Earth system, plays a vital role in research on the hydrological cycle, ecosystems, agricultural production, carbon cycle, and climate change. Accurate soil moisture monitoring data is of great significance for drought monitoring, farmland water management, and climate change research, especially in ecologically vulnerable areas.

[0003] Currently, the main methods for obtaining soil moisture data include ground observation, numerical simulation, and remote sensing inversion. Numerical simulation relies on inputs such as soil properties and meteorological drivers, and its accuracy is significantly affected by the quality of the input data and the model structure. Remote sensing methods can provide information on soil moisture distribution over a large area, but suffer from low spatial resolution, limited temporal resolution, and decreased accuracy under vegetation cover or complex terrain conditions. In contrast, ground observation provides high-precision point-based soil moisture data, serving as a crucial basis for verifying remote sensing inversion products and numerical simulation results. Ground observation methods include manual measurement and automatic station monitoring: manual measurement offers high accuracy but is labor-intensive and has a limited observation frequency; automatic stations enable continuous and automated monitoring and have been widely used for accuracy verification of remote sensing and simulation products, but their observation results may exhibit systematic biases due to differences in sensor type, burial depth, and installation conditions, affecting data reliability and subsequent applications.

[0004] In existing studies, some scholars have used quality control and threshold testing methods to screen soil moisture observation data from automatic weather stations. For example, they have used methods such as comparing the relationship between precipitation and soil temperature changes to remove outliers, or using autocorrelation features to assess the quality of time series data. However, there are very few studies on the systematic correction of soil moisture data from automatic weather stations, and there is a lack of systematic correction schemes that combine manual observation data with multi-model algorithms.

[0005] The invention disclosed in CN120275612B is a GNSS-R soil moisture inversion method for mountainous areas that integrates topographic moisture index. It uses a Stacking ensemble model to assign weights to the soil moisture prediction results output by random forest, XGBoost, and support vector machine models, and outputs the final soil moisture prediction result. However, this invention is based on remote sensing data and cannot accurately obtain moisture data from deeper soil layers.

[0006] The invention disclosed in CN103645295A presents a multi-layer soil moisture simulation method and system. It constructs a soil water stratification equilibrium model, adjusts the model structure using remote sensing technology, acquires model parameters using remote sensing technology, and constructs a watershed hydrological spatial information database. Using the adjusted soil water stratification equilibrium model and the watershed hydrological spatial information database, it conducts multi-layer soil moisture process numerical simulations. This invention is also based on remote sensing inversion methods. However, although it performs stratification of soil water layers, the calculations are quite complex, focusing more on the impact of vegetation on precipitation and evaporation, and therefore lacks universality.

[0007] The invention disclosed in CN119986726A is a GNSS-R deep soil moisture inversion method that considers precipitation information. It employs a multi-step correction method, considering factors such as the rate of change and precipitation to correct the obtained surface soil moisture. Finally, statistical results obtained from a large amount of data are used to further obtain deeper soil moisture levels. However, this invention divides the correction interval based on soil moisture values ​​and corrects soil moisture through the rate of change, without considering the correction requirements of automatic weather station sensor data. Summary of the Invention

[0008] The purpose of this invention is to provide a layer-by-layer correction method for soil moisture observation data from automatic weather stations based on meteorological environmental factors and multi-model integration. This method considers the lag effect of evaporation and precipitation on soil moisture, as well as the influence of upper-layer moisture changes on moisture variations in different soil layers. By using manually observed soil moisture as a benchmark, and combining meteorological and environmental factors such as air temperature, wind speed, precipitation, and vegetation index, a generalized additive model (GAM) is employed, integrating multiple machine learning algorithms including Cubist, random forest, XGBoost, and CatBoost, to perform layer-by-layer bias correction on the soil moisture observation data from automatic weather stations. The corrected soil moisture data significantly improves observation accuracy, especially in deep soil layers, contributing to the construction of a high-precision, long-term stable soil moisture observation network and providing a reliable data foundation for the validation of remote sensing and numerical simulation soil moisture products.

[0009] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0010] A method for layer-by-layer correction of soil moisture observation data from automatic weather stations based on meteorological environmental factors and multi-model ensemble, the method comprising the following steps:

[0011] S1. Divide the soil layer into N layers and number them according to the burial depth of the sensor probes deployed at the automatic station, let n=1,2,...,N, where n=1 represents the soil surface layer;

[0012] S2. The obtained manually measured soil moisture and automatically observed soil moisture are preprocessed to remove outliers from the automatically observed soil moisture data. The preprocessed manually observed and automatically observed soil moisture are matched from two dimensions: soil depth and observation time, to obtain a matching dataset. This matching dataset includes several sets of manually measured and automatically observed soil moisture data for all soil depths corresponding to different dates. The manually observed and automatically observed soil moisture are compared, and outliers are removed based on the 3σ principle according to the residuals between the two.

[0013] S3. Based on the matching dataset, hourly meteorological observation data from automatic weather stations, including near-surface air temperature, wind speed, relative humidity, evaporation, and precipitation, are obtained. Daily-scale near-surface air temperature, daily-scale wind speed, and daily-scale relative humidity are calculated. Simultaneously, based on the soil moisture lag days m, the soil moisture lag weighting coefficient and the weighted cumulative evaporation and weighted cumulative precipitation for the previous m days are calculated. Based on the location and date of the automatic weather station, the Normalized Difference Vegetation Index (NDVI) for the corresponding pixels and date is extracted from the MODIS remote sensing data.

[0014] S4, let n=1, take the surface soil moisture observed manually as the dependent variable, and take the surface soil moisture observed by the automatic station, the daily near-surface air temperature, the daily wind speed, the daily relative humidity, the weighted cumulative precipitation, the weighted cumulative evaporation, and the normalized vegetation index NDVI as independent variables. Four machine learning algorithms, including Cubist, RF, XGBoost and CatBoost, are used to construct soil correction sub-models. Then, the correction results obtained by the four machine learning algorithms are used as independent variables, and the generalized additive model GAM algorithm is used to construct the surface soil moisture correction model to output the corrected surface soil moisture.

[0015] S5, let n=n+1, take the soil moisture of the i-th layer observed by manual observation as the dependent variable, and take the soil moisture of the n-th layer, the soil corrected moisture of the (n-1)-th layer, the daily near-surface air temperature, the daily wind speed, the daily relative humidity, the weighted cumulative precipitation, the weighted cumulative evaporation, and the normalized vegetation index NDVI observed by automatic stations as independent variables. Four machine learning algorithms, including Cubist, RF, XGBoost and CatBoost, are used to construct soil correction sub-models. Then, the correction results obtained by the four machine learning algorithms are used as independent variables, and the generalized additive model GAM algorithm is used to construct the soil moisture correction model of the n-th layer to output the soil corrected moisture of the n-th layer.

[0016] S6. Repeat step S5 until n=N. Combine the soil moisture correction models of all soil layers to construct the layer-by-layer soil moisture correction model.

[0017] S7 uses a matching dataset to train the layer-by-layer soil correction model, outputs the trained layer-by-layer soil correction model, verifies the accuracy of the correction results through cross-validation, and outputs the soil correction moisture of each layer.

[0018] Step S2 further includes:

[0019] The following formula converts manually measured soil moisture from mass water content to volumetric water content to ensure consistency with the units of soil moisture observed by automatic weather stations. The conversion formula is as follows:

[0020] ;

[0021] in, Indicates soil volumetric water content. Indicates soil moisture content. Indicates soil bulk density;

[0022] Based on the soil moisture threshold range observed by automatic weather stations, obviously erroneous or invalid observations are removed from the soil moisture data observed by automatic weather stations. Data sequences that show no change for several consecutive days are identified as invalid data and then removed.

[0023] Soil moisture measured by manual observation and automatic station observation is matched by soil depth, so that the soil depth corresponding to manual observation matches the soil depth and observation time of automatic station observation.

[0024] Furthermore, step S2 further includes:

[0025] The matched manual observations are used as the true values ​​and compared with the soil moisture observed by the automatic weather station. The accuracy evaluation index of the soil moisture observed by the automatic weather station is calculated. If the accuracy evaluation index is less than the preset accuracy threshold, it means that there are some observations in the automatic weather station data that deviate significantly from the manual observations. These outliers are usually caused by instrument noise, poor probe contact, data transmission signal interference, power supply voltage fluctuations, etc., which will affect the correction results. Outliers are identified according to the 3σ principle based on the residuals between the two. After removing outliers, a matched dataset is generated, and the accuracy evaluation index is recalculated.

[0026] Furthermore, the accuracy evaluation indicators include multiple or all of the following: correlation coefficient R, mean absolute error MAE, root mean square error RMSE, and mean deviation MB.

[0027] Step S3 further includes:

[0028] The near-surface air temperature, wind speed, and relative humidity were averaged on a daily scale to obtain the daily average near-surface air temperature, daily average wind speed, and daily average relative humidity.

[0029] Evaporation and precipitation are summed on a daily scale to obtain daily evaporation and daily precipitation.

[0030] For soil layers 1 to N, the weighted cumulative evaporation and weighted cumulative precipitation over the previous m days were calculated, where m = 1, 2, ..., 15. The m-value with the highest correlation coefficient was taken as the lag day for soil moisture, and the corresponding weighted cumulative evaporation and weighted cumulative precipitation were calculated. The weighted cumulative evaporation and weighted cumulative precipitation over the previous m days were calculated using the following formula:

[0031] ;

[0032] In the formula, This represents the weighted cumulative value of evaporation or precipitation corresponding to date t. This represents the daily precipitation or evaporation on the i-th day before date t, where m is the number of days the soil moisture lags. Let lag weighting coefficient be the soil moisture content for day i.

[0033] ;

[0034] Based on the location and date of the observation point, the Normalized Difference Vegetation Index (NDVI) for the corresponding pixel and date is extracted from the MODIS remote sensing data; if there is cloud cover on the observation date, the average value of the cloudless phases of the preceding and following dates is used as the replacement.

[0035] Further, in step S6, based on the following formula, the generalized additive model (GAM) algorithm is used to construct the soil moisture correction model for the i-th layer to output the corrected soil moisture for the i-th layer:

[0036]

[0037] In the formula, M represents soil moisture; The intercept; The model function representing each independent variable. Cubist, RF, XGBoost, and CatBoost are the correction results output by the Cubist, RF, XGBoost, and CatBoost models, respectively. This is the error term.

[0038] Furthermore, the method also includes:

[0039] The effective soil moisture values ​​after correction for soil layers 1 to N were used as dependent variables, and the effective soil moisture values ​​after correction for other soil layers were used as independent variables. Interlayer fitting models were constructed using Cubist, RF, XGBoost and CatBoost models respectively. The models were then integrated based on the generalized additive model GAM to construct an integrated interlayer interpolation model.

[0040] Obtain the soil layer numbers corresponding to the outliers and isolated values ​​in the soil moisture observation data of the automatic stations removed in step S2. Based on the integrated interpolation model, estimate and interpolate the outliers and isolated values ​​according to the corrected effective moisture values ​​of other soil layers.

[0041] Furthermore, when soil moisture data for all soil layers is missing at a certain moment, the soil moisture at the current moment is estimated using the corrected soil moisture from adjacent time phases as a basis, and then interpolated using a time series linear interpolation method to generate a corrected soil moisture dataset that is continuous in both time and depth.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] First, the automatic station soil moisture observation data correction method based on meteorological environmental factors and multi-model integration of the present invention addresses the systematic error problems caused by insufficient sensor accuracy, different installation conditions and environmental interference in existing automatic soil moisture observations. It uses manually measured volumetric water content as a benchmark and combines various machine learning methods to correct the deviation of automatic station data. While ensuring the continuity of observation, it effectively improves the accuracy and stability of observation data at different soil depths.

[0044] Second, the automatic station soil moisture observation data layer-by-layer correction method based on meteorological environmental factors and multi-model integration of the present invention addresses the technical problem that existing studies are mostly limited to single factors or simple models. It constructs a comprehensive integrated model that integrates multiple machine learning algorithms based on meteorological factors such as temperature, wind speed, relative humidity, precipitation, and evaporation, as well as environmental factors such as vegetation index. It also considers the lag effect of evaporation and precipitation on soil moisture to calculate the lag days m, and uses the weighted cumulative evaporation and precipitation of the previous m days instead of the evaporation and precipitation of the current day in the modeling, thereby improving the prediction accuracy and robustness of the overall model.

[0045] Third, the layer-by-layer correction method for automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration in this invention considers that the upper soil layer is greatly affected by precipitation and evaporation, while the moisture changes in deeper soil layers lag behind. Deep soil layers are also more susceptible to sensor response delays and soil heterogeneity, resulting in more significant deviations. Through layered correction and layered modeling, the method considers the correlation between adjacent layers during the correction of deep soil moisture, using the corrected upper soil moisture as an auxiliary independent variable for the correction of lower soil moisture, thereby optimizing soil moisture at different depths of automatic stations. This invention improves the accuracy and reliability of automatic station soil moisture monitoring data, providing high-quality ground reference data for the verification of remote sensing inversion soil moisture products and numerical simulation results. It also provides more accurate and stable fundamental support for agricultural drought and flood monitoring, hydrological process simulation, ecological environment research, and climate change assessment.

[0046] Fourth, to address the problem of missing soil moisture data caused by outliers and anomalies resulting from automatic weather station sensor malfunctions, communication interruptions, or quality control anomalies, this invention proposes a soil moisture interpolation and correction method based on multi-model ensemble. This method utilizes the correlation of moisture changes between soil layers to dynamically estimate and correct the soil moisture of missing layers. When all soil layers are missing simultaneously, this invention further introduces a time-series linear interpolation method to reconstruct the data using corrected moisture from adjacent time phases. This method effectively solves the problem of abnormal changes and data loss caused by anomalies and outliers in automatic weather station observation data, significantly improving the completeness, spatiotemporal continuity, and usability of soil moisture data. Attached Figure Description

[0047] Figure 1 This is a flowchart of the automatic station soil moisture observation data layer-by-layer correction method based on meteorological environmental factors and multi-model integration, according to the present invention.

[0048] Figure 2 A scatter plot of soil moisture measured by the DZN2 sensor of the automatic soil moisture monitoring station and the value observed manually;

[0049] Figure 3 This is a scatter plot showing the soil moisture content of the calibrated automatic soil moisture meter versus that measured manually. Detailed Implementation

[0050] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0051] A method for layer-by-layer correction of soil moisture observation data from automatic weather stations based on meteorological environmental factors and multi-model ensemble, the method comprising the following steps:

[0052] S1. Divide the soil layer into N layers and number them according to the burial depth of the sensor probes deployed at the automatic station, let n=1,2,...,N, where n=1 represents the soil surface layer;

[0053] S2. The obtained manually measured soil moisture and automatically observed soil moisture are preprocessed to remove outliers from the automatically observed soil moisture data. The preprocessed manually observed and automatically observed soil moisture are matched from two dimensions: soil depth and observation time, to obtain a matching dataset. This matching dataset includes several sets of manually measured and automatically observed soil moisture data for all soil depths corresponding to different dates. The manually observed and automatically observed soil moisture are compared, and outliers are removed based on the 3σ principle according to the residuals between the two.

[0054] S3. Based on the matching dataset, hourly meteorological observation data from automatic weather stations, including near-surface air temperature, wind speed, relative humidity, evaporation, and precipitation, are obtained. Daily-scale near-surface air temperature, daily-scale wind speed, and daily-scale relative humidity are calculated. Simultaneously, based on the soil moisture lag days m, the soil moisture lag weighting coefficient and the weighted cumulative evaporation and weighted cumulative precipitation for the previous m days are calculated. Based on the location and date of the automatic weather station, the Normalized Difference Vegetation Index (NDVI) for the corresponding pixels and date is extracted from the MODIS remote sensing data.

[0055] S4, let n=1, take the surface soil moisture observed manually as the dependent variable, and take the surface soil moisture observed by the automatic station, the daily near-surface air temperature, the daily wind speed, the daily relative humidity, the weighted cumulative precipitation, the weighted cumulative evaporation, and the normalized vegetation index NDVI as independent variables. Four machine learning algorithms, including Cubist, RF, XGBoost and CatBoost, are used to construct soil correction sub-models. Then, the correction results obtained by the four machine learning algorithms are used as independent variables, and the generalized additive model GAM algorithm is used to construct the surface soil moisture correction model to output the corrected surface soil moisture.

[0056] S5, let n=n+1, take the soil moisture of the i-th layer observed by manual observation as the dependent variable, and take the soil moisture of the n-th layer, the soil corrected moisture of the (n-1)-th layer, the daily near-surface air temperature, the daily wind speed, the daily relative humidity, the weighted cumulative precipitation, the weighted cumulative evaporation, and the normalized vegetation index NDVI observed by automatic stations as independent variables. Four machine learning algorithms, including Cubist, RF, XGBoost and CatBoost, are used to construct soil correction sub-models. Then, the correction results obtained by the four machine learning algorithms are used as independent variables, and the generalized additive model GAM algorithm is used to construct the soil moisture correction model of the n-th layer to output the soil corrected moisture of the n-th layer.

[0057] S6. Repeat step S5 until n=N. Combine the soil moisture correction models of all soil layers to construct the layer-by-layer soil moisture correction model.

[0058] S7 uses a matching dataset to train the layer-by-layer soil correction model, outputs the trained layer-by-layer soil correction model, verifies the accuracy of the correction results through cross-validation, and outputs the soil correction moisture of each layer.

[0059] 1) Soil moisture data processing

[0060] First, the manually measured soil moisture content is converted from mass water content to volumetric water content to be consistent with the unit of soil moisture in the automatic monitoring system. The conversion formula is as follows:

[0061]

[0062] in, This indicates the soil volumetric water content (%). This indicates the soil moisture content (%). This indicates the soil bulk density (g / cm³).

[0063] Secondly, obviously erroneous or invalid observations were removed from the soil moisture threshold range observed by automatic weather stations. Data sequences showing no change for 10 consecutive days were also deemed invalid.

[0064] Soil moisture measured manually and automatically is matched by soil depth to ensure a strict correspondence between the soil depth measured manually and automatically. Time matching is also performed to ensure consistency between the two observation times.

[0065] 2) Accuracy Verification

[0066] The matched manual observations are used as the true values ​​and compared with the soil moisture observed by the automatic weather station. The accuracy evaluation index of the soil moisture observed by the automatic weather station is calculated, such as correlation coefficient (R), mean absolute error (MAE), root mean square error (RMSE), and mean deviation (MB). If the accuracy evaluation index is less than the preset accuracy threshold, it means that there are some observations in the automatic weather station data that deviate significantly from the manual observations. These outliers are usually caused by instrument noise, poor probe contact, data transmission signal interference, power supply voltage fluctuations, etc., which will affect the correction results. Outliers are identified according to the 3σ principle based on the residuals between the two, and after removing outliers, a matched dataset is generated. The accuracy evaluation index is then recalculated. This step is used to further screen and optimize the data in the matched dataset.

[0067] 3) Meteorological and Environmental Data Processing

[0068] Based on hourly meteorological observation data from the observation points, including near-surface air temperature, wind speed, relative humidity, evaporation, and precipitation.

[0069] The near-surface air temperature, wind speed, and relative humidity are averaged on a daily scale to obtain the daily average near-surface air temperature, daily average wind speed, and daily average relative humidity.

[0070] Evaporation and precipitation are summed on a daily scale to obtain daily evaporation and daily precipitation. Considering the lag effect of evaporation and precipitation on soil moisture, this invention introduces a soil moisture lag day number. Based on the soil moisture lag day number *m*, the soil moisture lag weighting coefficient and the weighted cumulative evaporation and weighted cumulative precipitation for the previous *m* days are calculated. The selection of the soil moisture lag day number is crucial; in this embodiment, the following selection strategy is proposed:

[0071] For soil layers 1 to N, the weighted cumulative evaporation and weighted cumulative precipitation of manually observed soil moisture over the previous m days were calculated, where m = 1, 2, ..., 15. The m value with the highest multiple correlation coefficient was taken as the lag day for soil moisture. The weighted cumulative evaporation and weighted cumulative precipitation corresponding to this lag day were calculated to ensure that the weighted cumulative evaporation and weighted cumulative precipitation as input variables are sufficiently representative and can effectively reflect the influence characteristics of evaporation and precipitation on soil moisture of different soil types.

[0072] The formula for calculating the weighted cumulative evaporation (or weighted cumulative precipitation) over the first m days for evaporation and precipitation is as follows:

[0073] (2);

[0074] In the formula, This represents the weighted cumulative value of evaporation (or precipitation) over the preceding m days corresponding to date t; m is the number of days the soil moisture lags. This represents the daily precipitation (or evaporation) on the i-th day before date t. The weighting coefficient for day i:

[0075] (3);

[0076] For soil layers 1 to N, calculate the multiple correlation coefficient between artificially observed soil moisture and the weighted cumulative evaporation and weighted cumulative precipitation for the previous 1 to 15 days (m=1,2,…,15). Take the m value with the highest correlation as the lag day and calculate the weighted cumulative evaporation and weighted cumulative precipitation corresponding to that day.

[0077] For soil layers 1 to N, the weighted cumulative evaporation and weighted cumulative precipitation over the previous m days were calculated, where m = 1, 2, ..., 15. The m-value with the highest correlation coefficient was taken as the lag day for soil moisture, and the corresponding weighted cumulative evaporation and weighted cumulative precipitation were calculated. The weighted cumulative evaporation and weighted cumulative precipitation over the previous m days were calculated using the following formula:

[0078] ;

[0079] In the formula, This represents the weighted cumulative value of evaporation or precipitation corresponding to date t. This represents the daily precipitation or evaporation on the i-th day before date t, where m is the number of days the soil moisture lags. Let lag weighting coefficient be the soil moisture content for day i.

[0080] ;

[0081] Based on the observation point location and date, the Normalized Difference Vegetation Index (NDVI) for the corresponding pixel and date is extracted from MODIS remote sensing data. If there is cloud cover on the observation date, the average value of the cloudless phases of the preceding and following dates is used as the replacement.

[0082] 4) Surface soil moisture correction

[0083] Using manually observed surface soil moisture as the dependent variable, and automatic station-observed surface soil moisture, near-surface air temperature, wind speed, relative humidity, weighted cumulative precipitation over the previous m days, weighted cumulative evaporation over the previous m days, and NDVI as independent variables, a calibration model was constructed using machine learning algorithms such as Cubist, RF, XGBoost, and CatBoost.

[0084] Then, using the surface soil moisture observed manually as the dependent variable and the surface soil moisture correction results obtained from the base models constructed by four machine learning algorithms as the independent variables, an ensemble model (Formula 4) was constructed using the generalized additive model (GAM) algorithm to achieve secondary optimization of the soil moisture correction results, and the final corrected surface soil moisture was obtained. The accuracy of the correction results was verified by cross-validation.

[0085]

[0086] In the formula, M represents soil moisture; The intercept; The model function representing each independent variable. Cubist, RF, XGBoost, and CatBoost are the model results for Cubist, RF, XGBoost, and CatBoost, respectively. This is the error term.

[0087] 5) Layer-by-layer correction of deep soil moisture

[0088] Considering that changes in soil moisture at different layers are influenced by changes in moisture in the upper layers, for deep soil moisture correction, the corrected moisture of the adjacent upper soil layer is added as an auxiliary variable in addition to meteorological and environmental factors.

[0089] Taking the soil moisture at the second depth as an example, the manually observed second-layer soil moisture was used as the dependent variable, and the corrected surface soil moisture, the second-layer soil moisture observed by automatic weather stations, near-surface air temperature, wind speed, relative humidity, weighted cumulative precipitation over the previous m days, weighted cumulative evaporation over the previous m days, and NDVI were used as independent variables. Cubist, RF, XGBoost, and CatBoost machine learning algorithms were used to construct correction models. Then, using the correction results obtained from the four machine learning algorithms as independent variables, an ensemble model was constructed using the Generalized Additive Model (GAM) algorithm to obtain the final corrected second-layer soil moisture.

[0090] This process is repeated stepwise to achieve progressive correction of soil moisture at different soil layers. During the correction process, the soil moisture of the corrected previous layer, the soil moisture of the current layer observed by an automatic weather station, near-surface air temperature, wind speed, relative humidity, weighted cumulative precipitation over the previous m days, weighted cumulative evaporation over the previous m days, and NDVI are used as independent variables. Machine learning algorithms such as Cubist, RF, XGBoost, and CatBoost are employed to construct correction models. Then, using the correction results obtained from the four machine learning algorithms as independent variables, an ensemble model is constructed using the Generalized Additive Model (GAM) algorithm to obtain the final corrected soil moisture of the current layer. Cross-validation is then used to verify the accuracy of the correction results.

[0091] 6) Missing value interpolation

[0092] As a preferred example, the present invention can also imputate data missing caused by outliers and abnormal values ​​in the soil moisture observation data of automatic stations removed in the preceding quality control stage. The specific steps are as follows:

[0093] Using the effective values ​​of corrected soil moisture in layers 1 through N as dependent variables and the corrected effective moisture in other soil layers as independent variables, interlayer fitting models were constructed using Cubist, RF, XGBoost, and CatBoost, respectively. These models were then integrated using a generalized additive model (GAM) to obtain an inter-layer correlation-based interpolation model. For any soil layer with missing values ​​in layers 1 through N, the constructed integrated interpolation model was applied to estimate and interpolate based on the corrected effective moisture values ​​of other soil layers. When all soil layers are missing at a given time, the soil moisture at the current time was interpolated using time-series linear interpolation based on the corrected soil moisture values ​​of adjacent time phases, ultimately generating a continuous corrected soil moisture dataset in both time and depth.

[0094] This invention utilizes manually measured data, automatic soil moisture observation data, and multi-source environmental factors to construct a deviation correction model for soil layers at different depths, thereby improving the accuracy and stability of soil moisture monitoring data from automatic stations.

[0095] Example

[0096] This example uses manually observed soil moisture data, combined with multiple auxiliary factors such as near-surface air temperature, wind speed, relative humidity, precipitation, evaporation, and vegetation index, to obtain a more realistic estimate of soil moisture content. The technical process is detailed below. Figure 1 .

[0097] 1) Taking a certain study area as an example, the soil moisture of artificial observations at five layers (0-10 cm, 10-20 cm, 20-30 cm, 30-40 cm, 40-50 cm) of the local climate observatory was collected. According to formula (1), the soil moisture of artificial observations was converted from soil mass water content to volume water content.

[0098] 2) Collect soil moisture data from the DZN2 automatic soil moisture sensor at the local climate observatory, and remove outliers and data that have remained unchanged for 10 consecutive days.

[0099] 3) Processed manually observed and automatically observed soil moisture were paired at the same date and soil depth. The differences between the two were compared, and R, MAE, RMSE, and MB were calculated as accuracy indicators. Outliers were identified based on the residuals according to the 3σ principle, and after removing outliers, a matched dataset was generated. The soil moisture comparison results for the five soil layers are shown below. Figure 2 Automatic soil moisture monitoring stations have the highest accuracy in the surface layer (0-10 cm) (R=0.64, RMSE=6.76%, MAE=5.37%), but the accuracy decreases significantly with increasing depth.

[0100] 4) Obtain hourly near-surface air temperature, wind speed, relative humidity, evaporation and precipitation, and then calculate the daily average surface air temperature, daily average wind speed, daily average relative humidity, daily evaporation and daily precipitation.

[0101] 5) Calculate the multiple correlation coefficient using artificially observed soil moisture from layers 1 to 5 as the dependent variable and weighted cumulative precipitation and evaporation from the previous 1 to 15 days as the independent variables. Take the day with the highest multiple correlation coefficient value as the lag day for the current soil moisture. Based on this, determine the weighted cumulative precipitation and evaporation corresponding to the lag day.

[0102] 6) Extract the NDVI value of the pixel containing the Xilinhot National Climate Observatory from MODIS remote sensing data using the Google Earth Engine platform. If cloud cover on the observation day causes missing NDVI data, iterate and use the average value of the cloudless phases of the preceding and following dates as a replacement.

[0103] 7) Match the daily average temperature, daily average wind speed, daily average relative humidity, weighted cumulative precipitation of the previous ten days, weighted cumulative evaporation of the previous ten days, and NDVI with soil moisture observation data to construct a model training set.

[0104] 8) Using manually observed surface soil moisture as the dependent variable, and automatic station-observed surface soil moisture, daily average temperature, daily average wind speed, daily average relative humidity, weighted cumulative precipitation, weighted cumulative evaporation, and NDVI as independent variables, a surface soil moisture correction baseline model was constructed using machine learning algorithms such as Cubist, RF, XGBoost, and CatBoost.

[0105] 9) Using the surface soil moisture correction results obtained from the four base learners as independent variables and the surface soil moisture observed manually as dependent variables, an ensemble model is constructed using the generalized additive model (GAM) algorithm for secondary optimization to obtain the final corrected surface soil moisture.

[0106] 10) Perform layer-by-layer correction for soil moisture in layers 2-5. Using manually observed soil moisture in the current layer as the dependent variable, and the corrected soil moisture in the previous layer, the soil moisture in the current layer observed by automatic weather stations, air temperature, wind speed, relative humidity, precipitation, evaporation, and NDVI as independent variables, construct a soil moisture correction model. Then, use the correction results of the four base learners as new input features, input them into the GAM ensemble model for secondary optimization, and obtain the final corrected soil moisture in the current layer.

[0107] 11) Using the effective soil moisture values ​​after correction for soil layers 1-5 as the dependent variable and the effective soil moisture values ​​after correction for other soil layers as the independent variables, construct an inter-layer fitting model. Then, use the correction results of the four base learners as new input features and input them into the GAM ensemble model to construct an ensemble imputation model based on the correlation between soil layers. For any soil layer with missing values ​​in layers 1-5, use the constructed ensemble model and the effective soil moisture values ​​of other layers to impute the missing values.

[0108] 12) When soil moisture is missing for all soil layers at a certain moment, the soil moisture at the missing moment is estimated by linear interpolation based on the corrected effective soil moisture data of adjacent moments, according to the time series. By performing inter-layer correlation-based interpolation and time-dimensional linear interpolation on all moments and soil layers in sequence, a corrected soil moisture dataset that is continuous in both depth and time is finally generated.

[0109] Figure 3 Cross-validation results of soil moisture measurements from five automatic soil moisture monitoring stations are presented. After correction, the correlation coefficients (R) between the soil moisture measured by the automatic soil moisture monitoring stations and the manually observed values ​​were 0.83, 0.71, 0.62, 0.66, and 0.54 for the 0–10 cm, 10–20 cm, 20–30 cm, 30–40 cm, and 40–50 cm soil layers, respectively. Compared with before correction, the correlation coefficients (R) for the five soil layers increased by 0.19, 0.16, 0.19, 0.29, and 0.14, respectively. Overall, the correction significantly improves the accuracy and stability of soil moisture measurements from the automatic stations and enhances consistency with manual observations.

[0110] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0111] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for layer-by-layer correction of soil moisture observation data from automatic weather stations based on meteorological environmental factors and multi-model ensemble, characterized in that, The method Includes the following steps: S1. Divide the soil layer into N layers and number them according to the burial depth of the sensor probes deployed at the automatic station, let n=1,2,...,N, where n=1 represents the soil surface layer; S2. The obtained manually measured soil moisture and automatically observed soil moisture are preprocessed to remove outliers from the automatically observed soil moisture data. The preprocessed manually observed and automatically observed soil moisture are matched from two dimensions: soil depth and observation time, to obtain a matching dataset. This matching dataset includes several sets of manually measured and automatically observed soil moisture data for all soil depths corresponding to different dates. The manually observed and automatically observed soil moisture are compared, and outliers are removed based on the 3σ principle according to the residuals between the two. S3. Based on the matching dataset, hourly meteorological observation data from automatic weather stations, including near-surface air temperature, wind speed, relative humidity, evaporation, and precipitation, are obtained. Daily-scale near-surface air temperature, daily-scale wind speed, and daily-scale relative humidity are calculated. Simultaneously, based on the soil moisture lag days m, the soil moisture lag weighting coefficient and the weighted cumulative evaporation and weighted cumulative precipitation for the previous m days are calculated. Based on the location and date of the automatic weather station, the Normalized Difference Vegetation Index (NDVI) for the corresponding pixels and date is extracted from the MODIS remote sensing data. S4, let n=1, take the surface soil moisture observed manually as the dependent variable, and take the surface soil moisture observed by the automatic station, the daily near-surface air temperature, the daily wind speed, the daily relative humidity, the weighted cumulative precipitation, the weighted cumulative evaporation, and the normalized vegetation index NDVI as independent variables. Four machine learning algorithms, including Cubist, RF, XGBoost and CatBoost, are used to construct soil correction sub-models. Then, the correction results obtained by the four machine learning algorithms are used as independent variables, and the generalized additive model GAM algorithm is used to construct the surface soil moisture correction model to output the corrected surface soil moisture. S5, let n=n+1, take the soil moisture of the i-th layer observed by manual observation as the dependent variable, and take the soil moisture of the n-th layer, the soil corrected moisture of the (n-1)-th layer, the daily near-surface air temperature, the daily wind speed, the daily relative humidity, the weighted cumulative precipitation, the weighted cumulative evaporation, and the normalized vegetation index NDVI observed by automatic stations as independent variables. Four machine learning algorithms, including Cubist, RF, XGBoost and CatBoost, are used to construct soil correction sub-models. Then, the correction results obtained by the four machine learning algorithms are used as independent variables, and the generalized additive model GAM algorithm is used to construct the soil moisture correction model of the n-th layer to output the soil corrected moisture of the n-th layer. S6. Repeat step S5 until n=N. Combine the soil moisture correction models of all soil layers to construct the layer-by-layer soil moisture correction model. S7 uses a matching dataset to train the layer-by-layer soil correction model, outputs the trained layer-by-layer soil correction model, verifies the accuracy of the correction results through cross-validation, and outputs the soil correction moisture of each layer.

2. The method for layer-by-layer correction of automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration as described in claim 1, characterized in that, Step S2 further includes: The following formula converts manually measured soil moisture from mass water content to volumetric water content to ensure consistency with the units of soil moisture observed by automatic weather stations. The conversion formula is as follows: ； in, Indicates soil volumetric water content. Indicates soil moisture content. Indicates soil bulk density; Based on the soil moisture threshold range observed by automatic weather stations, obviously erroneous or invalid observations are removed from the soil moisture data observed by automatic weather stations. Data sequences that show no change for several consecutive days are identified as invalid data and then removed. Soil moisture measured by manual observation and automatic station observation is matched by soil depth, so that the soil depth corresponding to manual observation matches the soil depth and observation time of automatic station observation.

3. The method for layer-by-layer correction of automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration as described in claim 1, characterized in that, Step S2 further includes: The matched manual observations are used as the true values ​​and compared with the soil moisture observed by the automatic weather station. The accuracy evaluation index of the soil moisture observed by the automatic weather station is calculated. If the accuracy evaluation index is less than the preset accuracy threshold, outliers are identified according to the 3σ principle based on the residual between the two. After removing outliers, a matching dataset is generated, and the accuracy evaluation index is recalculated.

4. The method for layer-by-layer correction of automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration as described in claim 3, is characterized in that, The accuracy evaluation indicators include multiple or all of the following: correlation coefficient R, mean absolute error MAE, root mean square error RMSE, and mean deviation MB.

5. The method for layer-by-layer correction of automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration as described in claim 1, characterized in that, Step S3 further includes: The near-surface air temperature, wind speed, and relative humidity were averaged on a daily scale to obtain the daily average near-surface air temperature, daily average wind speed, and daily average relative humidity. Evaporation and precipitation are summed on a daily scale to obtain daily evaporation and daily precipitation. For soil layers 1 to N, the weighted cumulative evaporation and weighted cumulative precipitation over the previous m days were calculated, where m = 1, 2, ..., 15. The m-value with the highest correlation coefficient was taken as the lag day for soil moisture, and the corresponding weighted cumulative evaporation and weighted cumulative precipitation were calculated. The weighted cumulative evaporation and weighted cumulative precipitation over the previous m days were calculated using the following formula: ； In the formula, This represents the weighted cumulative value of evaporation or precipitation corresponding to date t. This represents the daily precipitation or evaporation on the i-th day before date t, where m is the number of days the soil moisture lags. Let lag weighting coefficient be the soil moisture content for day i. ； Based on the location and date of the observation point, the Normalized Difference Vegetation Index (NDVI) for the corresponding pixel and date is extracted from the MODIS remote sensing data; if there is cloud cover on the observation date, the average value of the cloudless phases of the preceding and following dates is used as the replacement.

6. The method for layer-by-layer correction of automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration as described in claim 1, characterized in that, In step S6, based on the following formula, the generalized additive model (GAM) algorithm is used to construct the soil moisture correction model for the i-th layer to output the corrected soil moisture for the i-th layer: ； In the formula, M represents soil moisture; The intercept; The model function representing each independent variable. Cubist, RF, XGBoost, and CatBoost are the correction results output by the Cubist, RF, XGBoost, and CatBoost models, respectively. This is the error term.

7. The method for layer-by-layer correction of automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration as described in claim 1, characterized in that, The method further includes: The effective soil moisture values ​​after correction for soil layers 1 to N were used as dependent variables, and the effective soil moisture values ​​after correction for other soil layers were used as independent variables. Interlayer fitting models were constructed using Cubist, RF, XGBoost and CatBoost models respectively. The models were then integrated based on the generalized additive model GAM to construct an integrated interlayer interpolation model. Obtain the soil layer numbers corresponding to the outliers and isolated values ​​in the soil moisture observation data of the automatic stations removed in step S2. Based on the integrated interpolation model, estimate and interpolate the outliers and isolated values ​​according to the corrected effective moisture values ​​of other soil layers.

8. The method for layer-by-layer correction of automatic station soil moisture observation data based on meteorological environmental factors and multi-model integration as described in claim 7, is characterized in that, When soil moisture data for all soil layers is missing at a certain moment, the soil moisture at the current moment is estimated using the corrected soil moisture of adjacent time phases as a basis, and then interpolated using the time series linear interpolation method to generate a corrected soil moisture dataset that is continuous in both time and depth.

Images