Water body nutrient salt remote sensing inversion method based on multi-source fusion and machine learning model

Abstract

The application discloses a water body nutrient salt remote sensing inversion method based on multi-source fusion and a machine learning model. The application unifies and formats non-optical environmental factors such as optical remote sensing features, meteorological features and land use data, and performs structured processing, so that the multi-source data is input into the inversion model in the form of a unified feature vector. An adaptive screening strategy based on feature importance is adopted, the contribution of each feature can be dynamically calculated in the model training process, and variables with low contribution or large noise can be automatically deleted. The temperature attenuation, Metropolis criterion and disturbance generation are introduced to automatically find the global optimal model structure and super parameter combination. Through the joint design of data fusion, feature screening and optimization algorithm, the application constructs a stable, generalizable and easy-to-deploy nutrient salt inversion system, can effectively solve the problems that the existing methods rely on a single spectrum, have poor generalization and have difficulty in parameter optimization, and can directly invert the water body nutrient salt concentration.

Description

Technical Field

[0001] This invention belongs to the field of water remote sensing detection technology, specifically relating to a method for remote sensing inversion of water nutrients based on multi-source fusion and machine learning models. Background Technology

[0002] Water nutrients (especially total nitrogen (TN) and total phosphorus (TP)) are key indicators for assessing the water quality of estuaries, lakes, and nearshore waters. Changes in their concentrations directly affect eutrophication processes, the risk of algal blooms, and ecosystem health. Against the backdrop of increased human activity and climate change in watersheds, nutrient output exhibits more significant spatiotemporal fluctuations, necessitating the establishment of efficient, wide-area, and continuous monitoring methods to support water environment management, pollution source tracing, and ecological restoration. Remote sensing, due to its wide coverage and high update frequency, has become an important technical approach for large-scale water environment monitoring in recent years.

[0003] Current nutrient remote sensing inversion studies mainly rely on the indirect relationship between spectral reflectance changes and nutrient concentrations, using spectral features for fitting through single-band indices, empirical relationships, or machine learning models. However, since TN / TP are non-optically active indicators, their concentration changes do not directly affect the spectral signal (the relationship between concentration changes and the spectrum is only indirect). The correlation between the spectrum and nutrients is easily affected by multiple factors such as turbidity, suspended matter, algae, and lighting conditions, which limits the accuracy, stability, and cross-seasonal / cross-regional generalization capabilities of inversion methods based on pure spectral features.

[0004] Furthermore, existing studies do not adequately utilize environmental background factors such as meteorology and land use, making it difficult for models to reflect the combined effects of climate, hydrodynamics, and watershed pollution pressure on nutrient changes. Current research still primarily uses spectral information as input in nutrient retrieval, with limited utilization of auxiliary environmental factors. Most existing remote sensing nutrient (TN / TP) retrieval methods still employ a "single-input spectral" model, lacking supplementary environmental background information such as meteorology and land use, making it difficult for models to reflect the true changes in nutrient concentration driven by both climate-hydrological processes and watershed pollution pressure. In addition, some multi-source fusion methods rely on high-frequency hydrological and meteorological data, which are costly to acquire and suffer from significant data gaps, making them difficult to promote in practical scenarios. Therefore, current technology lacks a nutrient retrieval method with low data requirements, direct utilization of monthly meteorological data, and effective integration with remote sensing imagery.

[0005] This invention aims to address the common problems of insufficient accuracy and weak stability in existing nutrient remote sensing inversion methods. It proposes a nutrient remote sensing inversion method that integrates remote sensing spectral features, monthly meteorological data, and land use information to enhance the environmental characterization capability of nutrient (non-optical features) inversion, thereby improving the accuracy, stability, and applicability of water nutrient monitoring. Summary of the Invention

[0006] This invention provides a remote sensing inversion method for water nutrient concentrations based on multi-source fusion and machine learning models. Through the joint design of data fusion, feature selection and optimization algorithms, a stable, generalizable and easy-to-deploy nutrient concentration inversion system is constructed. It can effectively solve the problems of existing methods relying on a single spectrum, poor generalization and difficulty in parameter optimization. It can directly invert the concentrations (non-optical features) of water nutrients such as total nitrogen (TN) and total phosphorus (TP).

[0007] The present invention specifically includes the following steps:

[0008] Step 1: Data Collection

[0009] Historical remote sensing images of the water body being measured, historical meteorological data of the area, land use data of the area, and historical monitoring data of nutrient concentrations in the water body within the area are collected.

[0010] Step 2: Data Preprocessing and Data Fusion

[0011] Multiband spectral data is extracted from remote sensing images, and reflectance features are constructed based on the multiband spectral data; meteorological data is normalized, and land use data is structured.

[0012] The preprocessed feature data are matched according to the sampling time and location, and the meteorological data, land use data and reflectance features collected at the same time and location are used to form a feature matrix that integrates multiple features.

[0013] Step 3: Select a machine learning tree model as the basic inversion generator and use the feature matrix to perform initial training on the model. During training, embed a feature importance evaluation mechanism to perform adaptive feature selection on the input feature matrix. Automatically remove noisy variables or redundant features based on the feature importance score to obtain the optimal features. Use the optimal features to optimize and train the basic inversion generator to build a stable inversion model.

[0014] Step 4, Hyperparameter Optimization: Adaptive temperature decay function and dynamic perturbation step size are used to generate candidate combinations of hyperparameters, and a global search is performed based on the Metropolis criterion, so that the inversion model trained in Step 3 can continuously approach the optimal parameters under limited sample conditions.

[0015] Step 5: Use the optimal machine learning model obtained in Step 4 to invert the remote sensing image of the water body being measured in real time, and predict the real-time nutrient concentration of the water body being measured.

[0016] Furthermore, in step one, meteorological data is selected monthly, including the average temperature and humidity of the area where the water body is located, and the total evaporation of the watershed.

[0017] Furthermore, in step two, the land use information is structured using a One-Hot or proportional method.

[0018] Furthermore, in step three, the machine learning tree model is either a random forest or an XGBoost model.

[0019] Furthermore, the adaptive feature selection process in step three is as follows:

[0020] First, the machine learning tree model is initially trained using the input feature matrix to obtain the feature importance scores of each feature.

[0021] Then, game theory interpretation value and gain are used as feature importance indicators to evaluate the contribution of each feature. Based on the feature importance scores, the feature importance indicators are integrated through a weighted scoring function to calculate the comprehensive score of each feature and remove features with a comprehensive score less than 0, thus completing the contribution evaluation of all features.

[0022] Then, the correlation coefficients of the features are calculated, and noisy or redundant features are removed to form a candidate feature set. The candidate feature set is used to train the machine learning tree model, and the model performance is evaluated.

[0023] If performance degrades, a rollback strategy is executed. Based on the importance ranking of each feature in the feature importance score, some deleted but significantly affected features are restored and re-evaluated. This process continues until the set number of iterations is reached or the performance improvement reaches the set threshold, at which point the filtering process terminates and the optimal feature set is output.

[0024] Furthermore, the specific process of hyperparameter optimization in step four is as follows:

[0025] Initialize the parameter space to obtain the current parameter combination.

[0026] An adaptive temperature decay function is constructed, and the dynamic perturbation step size of the current parameter combination is calculated based on the current temperature and the initial temperature. New candidate parameters are generated based on the dynamic perturbation step size of the current parameter combination.

[0027] A global search is performed based on the Metropolis criterion. Whether to accept new candidate parameters is determined based on performance differences. If performance is improved, the current parameter combination is updated. If performance is reduced, it is determined whether to escape the local optimum based on the set acceptance probability.

[0028] The adaptive temperature decay function updates the temperature and repeatedly generates new candidate parameters until the temperature drops to the set minimum value or the number of iterations reaches the set maximum value, at which point the optimal parameters and model are output.

[0029] This invention unifies and structures non-optical environmental factors such as optical remote sensing features, meteorological features (soil temperature, air temperature, humidity, rainfall, total evaporation in the watershed, etc.), and land use data, allowing multi-source data to be input into the inversion model in the form of a unified feature vector. Remote sensing data is expressed as continuous variables such as reflectance and indices; meteorological data is input as monthly statistics; and land use data is represented by a scaled feature vector. By unifying the format and standardizing the dimensions of heterogeneous data at the model input, non-optical features can effectively participate in nutrient inversion, improving the model's interpretability and cross-temporal stability. This solves the problem of instability in spectral inversion under variable environments and the difficulty of directly involving non-spectral factors in machine learning inversion, and improves the accuracy of water nutrient inversion models.

[0030] An adaptive selection strategy based on feature importance is adopted, which dynamically calculates the contribution of each feature during model training and automatically removes variables with low contribution or high noise. This strategy can automatically generate the optimal feature set, solve the feature redundancy problem caused by multi-source fusion, and thus improve model training efficiency, prediction accuracy, and generalization ability.

[0031] By introducing simulated annealing (SA) temperature decay, the Metropolis criterion, and perturbation generation, the model automatically seeks the globally optimal model structure and hyperparameter combination, improving its generalization performance and robustness. An adaptive temperature decay function, a variable step-size perturbation mechanism, and a performance fluctuation tolerance function (Δf threshold) are employed. A self-optimization mechanism of "automatic hyperparameter search—evaluation—update" is constructed for the nutrient inversion model, which can retain local fluctuations at low temperatures and expand the exploration space at high temperatures, achieving a global search of model hyperparameters. This mechanism possesses the ability to automatically adapt to different environmental conditions while maintaining high accuracy and stability. This effectively avoids the problem of tree models such as Random Forest and XGBoost, which heavily rely on manual experience for hyperparameter tuning and are prone to getting trapped in local optima. Ultimately, a nutrient inversion tree model with superior accuracy and stability is obtained.

[0032] In terms of simplicity: This invention abandons the use of daily or hourly high-frequency hydrological and meteorological data in the traditional inversion process, and instead uses monthly meteorological characteristics and stable land use characteristics as environmental inputs. This significantly reduces data acquisition costs and enables the method to operate reliably in areas with incomplete monitoring data and limited environmental observation capabilities, demonstrating good engineering adaptability.

[0033] Regarding stability: This invention employs an adaptive feature selection strategy based on feature importance to evaluate the contribution of multi-source input features and automatically remove low-importance or noisy features, enabling the model to focus on the key factors most relevant to nutrient changes. This measure reduces unnecessary dimensionality caused by redundant features, improves training efficiency, reduces the risk of model overfitting, and makes the model more stable when expanding across seasons and regions.

[0034] In terms of scalability: The overall structure and algorithm flow of this invention are modular. The components for multi-source data fusion, feature selection and model self-optimization can be flexibly replaced or expanded as needed. It is applicable to different types of water bodies, different nutrient indices and various machine learning models, and has strong potential for promotion. Attached Figure Description

[0035] Figure 1 This is a flowchart of the present invention;

[0036] Figure 2 A flowchart for data acquisition and multi-source fusion;

[0037] Figure 3 Flowchart for adaptive feature selection;

[0038] Figure 4 This is a flowchart of the model's self-optimization process. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, a remote sensing inversion method for water nutrient levels based on multi-source fusion and machine learning models specifically includes the following steps:

[0041] Step 1: Data Collection

[0042] In this embodiment, Sentinel-2 Level-2A (L2A) was used to extract the reflectance of bands B2-B8A, B11, and B12 at a resolution of 10m.

[0043] Sentinel-2 Level-2A (L2A) is an advanced multispectral remote sensing data product provided by the Sentinel-2 satellite of the European Space Agency's (ESA) Copernicus program. L2A data is the result of Level-1C (L1C) data after atmospheric correction. L1C data only includes radiometric and geometric corrections and does not eliminate atmospheric effects; while L2A data is corrected using algorithms (such as ESA's Sen2Cor plugin) and is suitable for applications such as vegetation index calculation and land cover classification.

[0044] Simultaneously, monthly meteorological data (including temperature, humidity, and total evaporation in the watershed) and land use raster data were collected. Meteorological data: Monthly average temperature, relative humidity, and total evaporation in the watershed were selected from the National Earth System Science Data Center, with a resolution of approximately 1 km. Land use data: The CLCD land use dataset with a resolution of 30 m, publicly available from Wuhan University, was used to extract the land type for each sample point. Measured water quality samples: Concentrations of TN and TP from automatic monitoring stations were collected and used as label data for the model.

[0045] Step Two, as follows Figure 2 As shown, data preprocessing and data fusion:

[0046] Multi-band spectral data is extracted from remote sensing images, and reflectance features are constructed based on the multi-band spectral data. Atmospheric correction is performed on Sentinel-2 remote sensing images, and NDWI (Normalized Difference Water Index) is calculated based on the near-infrared and green bands in the Sentinel-2 images. NDWI = (Green band - Near-infrared band) / (Green band + Near-infrared band). Combined with local water quality conditions and NDWI thresholds (usually greater than 0 indicates water bodies), water body mask extraction is performed, and reflectance features are constructed from the multi-spectral bands.

[0047] Meteorological variables are normalized, and land use information is structured using One-Hot or proportional methods to form a unified feature matrix with spectral variables.

[0048] The preprocessed feature data are matched according to the sampling time and location, and meteorological data, land use information and reflectance features collected at the same time and location are used to form a feature matrix that integrates multiple features.

[0049] In this embodiment, the spectral characteristics are: Sentinel-2 reflectance values ​​are used directly; and 0–1 normalization is applied.

[0050] Meteorological characteristics: The monthly average temperature, humidity, and total evaporation of the watershed were normalized by Z-score; each sample was directly matched with the monthly average value corresponding to its date and aligned with Sentinel-2 imagery.

[0051] Land use features: Based on the CLCD dataset, the proportion of land use types within a 300M buffer zone centered on each pixel is calculated, and the land use categories are converted into proportion vectors (e.g., arable land 0.35, water bodies 0.25, etc.) to form structured learnable features.

[0052] Ultimately, each sample is transformed into a unified input feature matrix X: X = {spectral features + meteorological features + land use features}

[0053] Step 3: Select a machine learning tree model as the basic inversion generator and use the feature matrix to initially train the model. During training, in order to enhance the generalization ability of the model and avoid overfitting caused by redundant features, a feature importance evaluation mechanism is embedded to adaptively filter the input feature matrix. Noise variables or redundant features are automatically removed according to the feature importance score, thereby obtaining the optimal features. The optimal features are used to optimize the basic inversion generator and build a stable inversion model.

[0054] In this embodiment, Random Forest / XGBoost is selected as the basic inversion engine. The input feature matrix is ​​adaptively filtered by embedding a feature importance evaluation mechanism. Noise variables or redundant features are automatically removed according to the feature contribution, thereby obtaining the optimal feature set and building a stable inversion model.

[0055] like Figure 3 As shown, the adaptive feature selection process is as follows:

[0056] First, the machine learning tree model is initially trained using the input feature matrix to obtain the feature importance scores of each feature, thus obtaining the baseline feature contribution matrix I0.

[0057] Then, the game theory interpretation value (SHAP value) and gain are used as feature importance indicators to evaluate the contribution of each feature. Based on the baseline feature contribution value I0, the feature importance indicators are integrated through a weighted scoring function to calculate the comprehensive score Score of each feature and remove features with a comprehensive score Score less than 0. After completing the contribution evaluation of all features, a comprehensive importance matrix M is generated.

[0058] ;

[0059] in w is the feature number. 1,2,3 I0 represents the corresponding weighted value, and I0 represents the contribution of the baseline feature.

[0060] Then, the correlation coefficient R of the features is calculated, and noisy variables or redundant features are removed (features with R < 0.6 are removed in this embodiment) to form a candidate feature set F'.

[0061] ;

[0062] Where X is the feature value, Y is the predicted TN / TP concentration, and n is the total number of samples. The average eigenvalue, This represents the average of the predicted TN / TP concentrations.

[0063] Then, the machine learning tree model is trained using the candidate feature set F', and the coefficient of determination R is used. 2The root mean square error (RMSE) and mean relative error (MAE) are used to evaluate model performance, and the coefficient of determination (R²) is defined. 2 The improvement value is the performance improvement value.

[0064] , , ; This is the actual value. These are predicted values.

[0065] If performance degrades, a rollback strategy is executed. Based on the importance ranking of each feature in the baseline feature contribution matrix I0, some deleted but significantly affected features are restored and re-evaluated. The process continues until the set number of iterations is reached or the performance improvement reaches the set threshold ε, at which point the screening process terminates and the optimal feature set F* is output.

[0066] Step 4, Hyperparameter Optimization: To further improve the model's performance in complex environments, an improved simulated annealing optimization algorithm is introduced. An adaptive temperature decay function and dynamic perturbation step size are used to generate candidate combinations of hyperparameters, and a global search is performed based on the Metropolis criterion, so that the inversion model trained in Step 3 can continuously approach the optimal parameters under limited sample conditions.

[0067] like Figure 4 As shown, this embodiment uses the random forest model as an example, and the specific process of hyperparameter optimization is as follows:

[0068] Initialize the parameter space to obtain the current parameter combination P, which includes the tree depth (max_depth), the number of trees (n_estimators), the maximum number of features (max_features), and the minimum number of leaf node samples (min_samples_leaf).

[0069] Construct an adaptive temperature decay function T k Let T0 be the temperature of the k-th iteration, and α be the initial temperature. k Let α be the decay coefficient for the k-th iteration. The initial temperature T0 is a set value, and the decay coefficient α is... k The model is adaptively adjusted based on recent performance changes to avoid the annealing process converging to a local optimum too quickly.

[0070] If the current acceptance rate is higher than the target value, α k A value greater than 1 slows down the temperature drop to enhance exploration; if the acceptance rate is lower than the target value, α k If the temperature decay coefficient is less than 1, the temperature decreases more rapidly to accelerate convergence; if the acceptance rate is close to the target value, the temperature decay coefficient α is maintained. k constant.

[0071] Generate new candidate parameters P′ based on the current parameter combination P. ,in Let be the dynamic perturbation step size for the k-th iteration, and Random(-1, 1) be a random number between -1 and 1. , This is the maximum perturbation step size. The perturbation step size is dynamically adjusted according to the iteration progress and the current convergence state of the model, enabling the search process to have both local refinement and global exploration capabilities.

[0072] A global search is performed based on the Metropolis criterion, and the acceptance of new parameters is determined based on the performance difference Δf: Δf = Loss(P′) - Loss(P); if Δf ≤ 0 (performance improvement), P′ is accepted; if Δf > 0 (performance degradation), the acceptance probability exp(-Δf / T) is used to determine whether the system has escaped the local optimum. Loss() represents the root mean square error (RMSE) or mean relative error (MAE) of the model under the new parameter combination P′. When the performance improvement is insufficient, an acceptance probability is set to ensure the convergence trend towards lower energy states while allowing the system to escape the local optimum with a controllable probability.

[0073] Adaptive temperature decay function updates temperature T K The process continues to generate new candidate parameters until the temperature T drops to Tmin or the number of iterations reaches the set maximum value Kmax, at which point the optimal parameters and model are output.

[0074] After parameter optimization, the optimized model is validated: the optimal feature set F* is input into the optimized model, and R is used. 2 The RMSE and MAE are used to evaluate the model.

[0075] Step 5: Use the optimal machine learning model obtained in Step 4 to invert the remote sensing image of the water body being measured in real time, and predict the real-time nutrient concentration of the water body being measured.

[0076] To verify the effectiveness of the above methods, the multi-source fusion, feature selection, and parameter optimization methods were applied to random forest and XGBoost models to invert the total nitrogen (TN) and total phosphorus (TP) concentrations in the water body, and the coefficient of determination R was used. 2 The root mean square error (RMSE) and mean relative error (MAE) were used to evaluate the model performance. The evaluation results are shown in Tables 1 and 2.

[0077] Table 1. Accuracy of the original model

[0078]

[0079] Table 2. Model accuracy after using the multi-source fusion, feature selection, and parameter optimization methods described in this invention.

[0080]

[0081] Comparing Tables 1 and 2, it can be seen that the prediction accuracy of the tree model after adopting the multi-source fusion, feature selection and parameter optimization described in this invention is improved compared with the original model.

[0082] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A remote sensing inversion method for water nutrient levels based on multi-source fusion and machine learning models, characterized in that: Specifically, the steps include the following: Step 1: Data Collection Collect historical remote sensing images of the water body being measured, historical meteorological data of the area, land use data of the area, and historical monitoring data of nutrient concentration in the water body within the area. Step 2: Data Preprocessing and Data Fusion Multiband spectral data is extracted from remote sensing images, and reflectance features are constructed based on the multiband spectral data; meteorological data is normalized, and land use data is structured. The preprocessed feature data are matched according to the sampling time and location, and the meteorological data, land use data and reflectance features collected at the same time and location are used to form a feature matrix that integrates multiple features. Step 3: Select a machine learning tree model as the basic inversion generator and use the feature matrix to perform initial training on the model. During training, embed a feature importance evaluation mechanism to perform adaptive feature selection on the input feature matrix. Automatically remove noisy variables or redundant features based on the feature importance score to obtain the optimal features. Use the optimal features to optimize and train the basic inversion generator to build a stable inversion model. Step 4, Hyperparameter Optimization: Adaptive temperature decay function and dynamic perturbation step size are used to generate candidate combinations of hyperparameters, and a global search is performed based on the Metropolis criterion, so that the inversion model trained in Step 3 can continuously approach the optimal parameters under limited sample conditions. Step 5: Use the optimal machine learning model obtained in Step 4 to invert the remote sensing image of the water body being measured in real time, and predict the real-time nutrient concentration of the water body being measured.

2. The method for remote sensing inversion of water nutrients based on multi-source fusion and machine learning models as described in claim 1, characterized in that: In step one, meteorological data is selected monthly, including the average temperature and humidity of the area where the water body is located, and the total evaporation of the watershed.

3. The method for remote sensing inversion of water nutrients based on multi-source fusion and machine learning models as described in claim 1, characterized in that: In step two, the land use information is structured using a One-Hot or proportional method.

4. The method for remote sensing inversion of water nutrients based on multi-source fusion and machine learning models as described in claim 1, characterized in that: In step three, the machine learning tree model is either a random forest or an XGBoost model.

5. The method for remote sensing inversion of water nutrients based on multi-source fusion and machine learning models as described in claim 1, characterized in that: The adaptive feature selection process in step three is as follows: First, the machine learning tree model is initially trained using the input feature matrix to obtain the feature importance scores of each feature; Then, game theory explanatory values ​​and gains are used as feature importance indicators to evaluate the contribution of each feature; Based on the feature importance scores, the feature importance index is integrated through a weighted scoring function to calculate the comprehensive score of each feature and remove features with a comprehensive score less than 0, thus completing the contribution evaluation of all features. Then, the correlation coefficients of the features are calculated, noisy variables or redundant features are removed, and a candidate feature set is formed; the candidate feature set is used to train the machine learning tree model, and the model performance is evaluated. If performance degrades, a rollback strategy is executed. Based on the importance ranking of each feature in the feature importance score, some deleted but significantly affected features are restored and re-evaluated. This process continues until the set number of iterations is reached or the performance improvement reaches the set threshold, at which point the filtering process terminates and the optimal feature set is output.

6. The method for remote sensing inversion of water nutrients based on multi-source fusion and machine learning models as described in claim 1, characterized in that: The specific process of hyperparameter optimization in step four is as follows: Initialize the parameter space to obtain the current parameter combination; Construct an adaptive temperature decay function, calculate the dynamic perturbation step size of the current parameter combination based on the current temperature and the initial temperature, and generate new candidate parameters based on the dynamic perturbation step size of the current parameter combination; A global search is performed based on the Metropolis criterion. Whether to accept new candidate parameters is determined based on poor performance. If the performance is improved, the current parameter combination is updated. If performance degrades, determine whether to exit the local optimum based on the set acceptance probability; The adaptive temperature decay function updates the temperature and repeatedly generates new candidate parameters until the temperature drops to the set minimum value or the number of iterations reaches the set maximum value, at which point the optimal parameters and model are output.

Images