Machine learning based remote sensing method and system for retrieving concentration of suspended matter

Abstract

This invention belongs to the field of water quality parameter inversion technology, specifically a machine learning-based remote sensing inversion method and system for suspended particulate matter (SPM) concentration. The method includes: acquiring satellite remote sensing imagery and synchronously measured SPM concentration data; obtaining a spatially gridded water body remote sensing reflectance matrix through radiometric calibration, atmospheric correction, and water body masking; denoising through wavelet decomposition reconstruction; and obtaining a spectral numerical sequence through spectral normalization and standardization. Based on this sequence, a multi-scale band combination and differential features are constructed, and sensitive features are selected using XGBoost. Physical constraints are constructed by combining sensitive features with water body radiative transfer laws, and intermediate inversion results are obtained through numerical iteration. Deep learning residuals are used to compensate for biases in the intermediate results, and finally, through spatial smoothing, consistency verification, and high-concentration saturation optimization, a high-precision, spatially continuous spatial distribution result of SPM concentration is obtained. This invention achieves efficient feature mining and deep integration of physical mechanisms, significantly improving the model's interpretability and generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of water quality parameter inversion technology, specifically a remote sensing inversion method and system for suspended solids concentration based on machine learning. Background Technology

[0002] Total suspended matter (TSM) is one of the three core elements of the optical properties of water bodies, playing an irreplaceable role in aquatic ecosystems and serving as an important indicator for assessing water quality. Its concentration directly affects the transparency and turbidity characteristics of water bodies, significantly regulates the spatial distribution of underwater light fields, and is also a key adsorbent carrier for various nutrients and pollutants.

[0003] Traditional remote sensing methods for retrieving suspended particulate matter concentration have significant limitations: they often rely on single-band or simple band combinations to construct empirical models, failing to fully exploit multidimensional spectral features and neglecting sensor noise and atmospheric residual interference, resulting in poor spectral data quality. The models lack physical constraints, establishing the correlation between spectrum and concentration solely through statistical fitting, leading to weak generalization capabilities and difficulty adapting to different lake areas and meteorological conditions. They do not consider the spectral saturation characteristics of high-concentration water bodies, easily overestimating concentrations in high-concentration areas. Furthermore, the lack of residual correction and spatial optimization results in inversion results with local noise, outliers, and spatial discontinuities, leading to significant numerical biases and limiting their engineering application value, making them unsuitable for large-scale, high-precision water environment monitoring. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention proposes a machine learning-based remote sensing inversion method for suspended matter concentration. This invention primarily addresses the problems of low accuracy, poor generalization, physical omissions, and spatial discontinuities in traditional inversion methods.

[0005] The machine learning-based remote sensing inversion method for suspended matter concentration provided by this invention includes:

[0006] S1: Collect satellite remote sensing apparent reflectance dataset and synchronously measured suspended matter concentration data, perform radiometric calibration, atmospheric correction and water body masking calculation on the image to obtain a spatially gridded water body remote sensing reflectance numerical matrix.

[0007] S2: Wavelet decomposition and reconstruction are performed on the spatially gridded water remote sensing reflectance numerical matrix. High-frequency noise and abnormal pixel values ​​are removed by adaptive thresholding. Then, the spectrum is normalized and standardized to obtain the spectral numerical sequence.

[0008] S3: Construct multi-scale band combinations and spectral differential features based on the spectral numerical sequence, use the XGBoost algorithm to evaluate and rank the contribution of various features, and select the feature subset that is most sensitive to changes in suspended matter to obtain sensitive features.

[0009] S4: Based on the sensitive characteristics and the simplified law of water body radiation transfer, physical constraint terms are constructed, and preliminary suspended solids concentration values ​​are obtained through numerical iteration calculations to form intermediate inversion results.

[0010] S5: Input the intermediate inversion results and sensitive features, use a deep learning model to learn the residual correction relationship, and only perform bias compensation on the preliminary inversion results to obtain the corrected concentration results.

[0011] S6: Spatial smoothing and numerical consistency verification are performed on the concentration results after deep learning correction. The optimization adjustment is completed by combining the high concentration asymptotic saturation rule to obtain the spatial distribution results of suspended matter concentration.

[0012] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps for obtaining the spatially gridded water body remote sensing reflectance numerical matrix in step S1 are as follows:

[0013] S11: Collect satellite remote sensing images of the target area and synchronous field measured suspended particulate matter concentration data to construct an initial dataset containing original DN value images and corresponding concentration samples.

[0014] S12: Based on the original DN value images and satellite sensor calibration parameters in the initial dataset, perform radiometric calibration calculations to convert the DN values ​​into radiance data, thereby obtaining radiance images.

[0015] S13: The atmospheric correction model is used to calculate the surface reflectance based on the radiance image, eliminating the effects of atmospheric absorption and scattering, and obtaining the true surface reflectance image.

[0016] S14: The water index method is used to construct discrimination rules for the acquired surface reflectance images. Non-water areas such as land, clouds, and shadows are removed by masking, and finally a spatially gridded water remote sensing reflectance numerical matrix is ​​obtained.

[0017] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps for obtaining the spectral numerical sequence in step S2 are as follows:

[0018] S21: Using the spatially gridded water body remote sensing reflectance numerical matrix, wavelet basis functions adapted to the spectral characteristics of the water body are selected to carry out pixel-by-pixel and band-by-band wavelet multi-scale decomposition to obtain low-frequency and high-frequency wavelet decomposition coefficients.

[0019] S22: An adaptive thresholding method is used to perform threshold shrinking on the noisy high-frequency wavelet decomposition coefficients, remove the coefficient values ​​corresponding to noise, and identify and correct abnormal pixels in the numerical matrix to obtain a denoised wavelet coefficient set.

[0020] S23: The denoised wavelet coefficient set is reconstructed by wavelet decomposition to obtain the denoised water remote sensing reflectance numerical matrix, which eliminates high-frequency noise and outlier interference, preserves the true characteristic information of the water body spectrum, and forms the basis of spectral data.

[0021] S24: Normalize the spectral data of each band according to the reflectance numerical matrix to eliminate the differences in dimensions and values ​​between bands, and obtain the spectral numerical sequence.

[0022] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps for obtaining sensitive features in step S3 are as follows:

[0023] S31: Construct multi-scale band ratio and band difference combination features based on the spectral numerical sequence, calculate the first-order differential features of the spectrum, and fuse them to form a multi-dimensional spectral feature set.

[0024] S32: Based on the multi-dimensional spectral feature set and measured suspended matter concentration data, construct training samples for the XGBoost algorithm, using the feature set as input and the measured concentration as label, to complete the initial training and feature mapping of the model, and obtain the trained XGBoost model.

[0025] S33: Based on the feature importance assessment of the trained XGBoost model, calculate the contribution score of various spectral features to the inversion of suspended matter concentration and sort them from high to low scores to quantify the correlation between features and concentration and obtain the contribution ranking.

[0026] S34: Based on the contribution ranking, redundant features with low contribution are eliminated, and the features with the highest scores are selected to form a feature subset. Collinear features within the subset are eliminated through correlation test to obtain the sensitive features most sensitive to changes in suspended matter.

[0027] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps in step S31 for fusing to form a multi-dimensional spectral feature set are as follows:

[0028] Multi-scale band ratio calculations are performed on characteristic bands using spectral numerical sequences. Ratio features are constructed by dividing the reflectance of different bands, and then band difference calculations are performed. Difference features are constructed by subtracting the reflectance of bands, forming a combined feature set.

[0029] Based on the combined feature set and the original spectral numerical sequence, the first-order differential features of each band are calculated using the spectral differential formula to capture subtle spectral changes. The differential features are then added to the combined feature set to form a preliminary multidimensional feature set.

[0030] The preliminary multidimensional feature set is fused with the original band features in the spectral numerical sequence, integrating three types of information: original bands, combined features, and first-order differential features, to form a multidimensional spectral feature set.

[0031] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps for forming the intermediate inversion result in step S4 are as follows:

[0032] S41: Extract core optical parameters based on sensitive features and simplified laws of radiative transfer in inland lakes, and construct physical constraint terms.

[0033] S42: Substitute the spectral values ​​of the sensitive features into the physical constraint terms, establish a numerical iterative calculation equation for the concentration of suspended matter, set initial values ​​and convergence thresholds, perform multiple rounds of iterative calculations until the results converge, and obtain the numerical iterative convergence result.

[0034] S43: Based on the numerical iteration convergence results and the spatial gridded distribution attributes of sensitive features, the preliminary suspended matter concentration values ​​corresponding to the whole space are calculated to form intermediate inversion results.

[0035] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps for constructing the physical constraint term in step S41 are as follows:

[0036] Based on the spectral response characteristics of sensitive features and the simplified law of radiative transfer in inland lakes, the physical correlation between reflectivity and suspended matter concentration is sorted out, and core optical parameters are extracted.

[0037] Using sensitive features as the core variable and combining them with core optical parameters, a quantitative correlation expression between optical parameters and suspended solids concentration is established.

[0038] The constructed quantitative correlation expression and the physical constraints of water body radiation transmission are used to correct the coupling relationship between variables and form physical constraint terms.

[0039] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps for obtaining the corrected concentration result in step S5 are as follows:

[0040] S51: Divide the training set and test set according to the intermediate inversion results and sensitive features. Use the intermediate inversion results and sensitive features as inputs, match the measured suspended solids concentration data as labels, and construct training samples for the deep learning model adapted to the residual correction task.

[0041] S52: Initialize the deep learning model parameters based on the training samples of the deep learning model, set the loss function and number of iterations for model training, train the model to learn the residual correction relationship between the intermediate inversion results and the true concentration, and obtain the trained deep learning model.

[0042] S53: Train the deep learning model by inputting all intermediate inversion results and sensitive features. Then, use the residual correction amount output by the model to compensate for the pixel-by-pixel bias of the intermediate inversion results to obtain the corrected concentration result.

[0043] According to the machine learning-based remote sensing inversion method for suspended matter concentration provided by the present invention, the specific steps for obtaining the trained deep learning model in step S52 are as follows:

[0044] Based on the training samples of the deep learning model, select a network structure suitable for residual fitting and initialize the model weights and bias parameters, and set reasonable learning rates and other basic training hyperparameters.

[0045] Based on the initialized model and training samples, a loss function is defined with the error between the predicted residual and the true residual as the core. The number of iterations and the early stopping strategy are set, and the model is iteratively trained using the training set data to continuously optimize the parameters.

[0046] The accuracy of the model during iterative training is verified using a test set. Training is stopped when the loss function converges and the verification error tends to stabilize, and the optimal parameter combination is retained to obtain the trained deep learning model.

[0047] The present invention also provides a machine learning-based remote sensing inversion system for suspended matter concentration, comprising: a data preprocessing module for acquiring satellite remote sensing apparent reflectance datasets and synchronously measured suspended matter concentration data, performing radiometric calibration, atmospheric correction and water body masking operations on the images, and obtaining a spatially gridded water body remote sensing reflectance numerical matrix.

[0048] The spectral denoising module is used to perform wavelet decomposition and reconstruction on the spatially gridded water remote sensing reflectance numerical matrix. It removes high-frequency noise and abnormal pixel values ​​through adaptive thresholding, and then normalizes and standardizes the spectrum to obtain the spectral numerical sequence.

[0049] The feature optimization module is used to construct multi-scale band combinations and spectral differential features based on the spectral numerical sequence, and to use the XGBoost algorithm to evaluate and rank the contribution of various features, and to select the feature subset that is most sensitive to changes in suspended matter, thus obtaining sensitive features.

[0050] The physical inversion module is used to construct physical constraint terms based on sensitive characteristics and simplified laws of water body radiation transfer, and obtain preliminary suspended solids concentration values ​​through numerical iteration calculations to form intermediate inversion results.

[0051] The residual correction module is used to input intermediate inversion results and sensitive features, and use a deep learning model to learn the residual correction relationship. It only performs bias compensation on the preliminary inversion results to obtain the corrected concentration results.

[0052] The results optimization module is used to perform spatial smoothing and numerical consistency verification on the concentration results corrected by deep learning. It combines the high-concentration asymptotic saturation rule to complete the optimization adjustment and obtain the spatial distribution results of suspended matter concentration.

[0053] The machine learning-based remote sensing inversion method for suspended matter concentration provided by this invention has the following beneficial effects:

[0054] 1. This invention ensures data quality through multi-step refined preprocessing. It employs radiometric calibration, FLAASH atmospheric correction, and MNDWI water body extraction to achieve accurate conversion from raw imagery to a water body remote sensing reflectance matrix, effectively eliminating interference factors such as atmospheric, land, and cloud shadows. The introduction of wavelet decomposition and adaptive threshold denoising separates effective spectral signals from high-frequency noise, corrects anomalous pixels, and eliminates dimensional differences through normalization and standardization. This provides a clean, stable, and high-quality spectral data foundation for subsequent inversion, reducing error accumulation at the source and solving the problems of high spectral noise and poor data consistency in traditional methods.

[0055] 2. This invention enriches the feature dimensions by constructing multi-dimensional features such as band ratios, differences, and first-order derivatives, and incorporating meteorological factors and empirical formulas. The XGBoost algorithm is used to evaluate feature contribution and eliminate collinearity, automatically selecting the feature subset most sensitive to suspended matter and avoiding redundant information interference. Physical constraints are constructed based on the radiative transfer laws of water bodies, and intermediate inversion results are obtained through numerical iteration. This process combines data-driven fitting capabilities with the rationality of a physical model, overcoming the shortcomings of weak generalization of traditional empirical models and insufficient accuracy of purely physical models. It is applicable to typical inland lakes such as Taihu Lake, Chaohu Lake, and Hongze Lake.

[0056] 3. This invention achieves high-precision and high-reliability spatial distribution results through residual correction and spatial optimization. Deep learning is employed to specifically learn the residuals of intermediate inversion results, performing only bias compensation without directly predicting concentrations. This preserves the rationality of physical constraints while compensating for errors introduced by model simplification, significantly improving inversion accuracy. Subsequently, spatial smoothing suppresses local noise, numerical consistency checks eliminate outliers, and high-concentration asymptotic saturation rules correct for overestimation of high concentrations. This ensures spatial continuity, numerical rationality, and conformity to the optical laws of water bodies. The resulting spatial distribution product of suspended solids concentration has higher accuracy and a more natural spatial structure, meeting the needs of large-scale, long-term, high-precision water environment monitoring and possessing strong engineering application value. Attached Figure Description

[0057] The invention will now be further described with reference to the accompanying drawings.

[0058] Figure 1 This is a flowchart illustrating the steps of the machine learning-based remote sensing inversion method for suspended matter concentration provided in this embodiment of the invention.

[0059] Figure 2 This is a flowchart of the machine learning-based remote sensing inversion method for suspended matter concentration provided in this embodiment of the invention;

[0060] Figure 3 This is a block diagram of the machine learning-based remote sensing inversion system for suspended matter concentration provided in this embodiment of the invention. Detailed Implementation

[0061] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below according to specific embodiments.

[0062] like Figures 1 to 3 As shown in the embodiment of the present invention, the method for remote sensing inversion of suspended matter concentration based on machine learning includes:

[0063] S1: Collect satellite remote sensing apparent reflectance dataset and synchronously measured suspended matter concentration data, perform radiometric calibration, atmospheric correction and water body masking calculation on the image to obtain a spatially gridded water body remote sensing reflectance numerical matrix.

[0064] S11: Collect domestically produced high-resolution remote sensing images covering typical inland lakes such as Taihu Lake, Chaohu Lake, and Hongze Lake, along with simultaneous field-measured TSM concentration data. The measured data were acquired using laboratory weighing / spectrophotometry, and GPS sampling coordinates were recorded. Simultaneously, temperature and wind speed data from meteorological stations during the image transit period were collected. The remote sensing images, measured TSM concentrations at sampling points in each lake area, and meteorological data were precisely matched spatiotemporally (time error ≤ 1 hour, spatial matching to the image pixel scale) to construct an initial dataset containing original DN value images, measured concentration samples, and meteorological data.

[0065] S12: The original DN value images in the constructed initial dataset are used to retrieve the radiometric calibration coefficients officially released by the domestic high-resolution satellite. The core radiometric calibration formula is used to calculate the values ​​pixel by pixel in the images. The formula is expressed as: In the formula, Gain is the gain and Bias is the bias. The meaningless DN value is converted into the radiance L with physical meaning, so as to obtain the global standardized radiance image and provide standardized radiance input for atmospheric correction.

[0066] S13: Using radiance imagery as the core input, the FLAASH atmospheric correction model, adapted for inland lake water body retrieval, is selected. Combining geographic parameters (latitude, longitude, altitude) and atmospheric parameters (aerosol optical thickness, water vapor content, ozone concentration) from the imagery capture, the atmospheric absorption, scattering, and path radiation interference processes of the spectral signal are simulated using the atmospheric radiative transfer equation. Atmospheric interference factors are eliminated band by band, converting radiance into true surface reflectance. The core conversion formula is: Where ρ is the true reflectance of the Earth's surface, and d is the Earth-Sun distance. Here, θ represents the solar irradiance in the corresponding band of the top of the atmosphere, and θ is the solar zenith angle. Geometric correction is performed on the rectified image to eliminate spatial geometric distortions caused by topographic undulations and sensor attitude deviations, resulting in a spatially accurate and spectrally accurate surface reflectance image, ensuring the accuracy of subsequent water body extraction and feature analysis.

[0067] S14: Based on the true surface reflectance image, the improved normalized water index (MNDWI), adapted for inland lake water body identification, is first selected to construct water body discrimination rules. The core calculation formula is: ,in, This represents the true surface reflectance in the green band. This represents the true surface reflectance in the mid-infrared band. Combining the water body characteristics of the study area, the optimal threshold for MNDWI was determined through visual verification and repeated experiments. Threshold segmentation was used to initially distinguish water and non-water pixels. Then, through spatial filtering and connectivity analysis, non-water areas such as land, clouds, cloud shadows, and noise were further masked and removed, retaining only pure lake water pixels. Subsequently, based on the results of suspended matter inversion band sensitivity analysis, the water image was band-selected, retaining only the RGB and near-infrared bands, which are sensitive to TSM concentration. Finally, a spatially continuous, gridded water remote sensing reflectance numerical matrix was obtained, with all pixels attributed to water and simplified band information.

[0068] S2: Wavelet decomposition and reconstruction are performed on the spatially gridded water remote sensing reflectance numerical matrix. High-frequency noise and abnormal pixel values ​​are removed by adaptive thresholding. Then, the spectrum is normalized and standardized to obtain the spectral numerical sequence.

[0069] S21: Using the spatially gridded water remote sensing reflectance numerical matrix obtained from the previous preprocessing and retaining the four TSM sensitive bands of RGB and near-infrared, wavelet basis functions adapted to the low-frequency spectral characteristics of inland lake water bodies are selected. Wavelet multi-scale decomposition is carried out sequentially for each sensitive band of each pixel in the matrix. The spectral signal of each band is decomposed into low-frequency wavelet decomposition coefficients that reflect the core characteristics of the spectral response of TSM concentration in lake water bodies, and high-frequency wavelet decomposition coefficients that characterize sensor random noise and atmospheric correction residual interference. The coefficient information at different scales is completely preserved, providing a basic coefficient set for subsequent spectral denoising.

[0070] S22: Based on the low-frequency and high-frequency wavelet decomposition coefficients across the entire band, an adaptive thresholding method is used to shrink the noisy high-frequency wavelet decomposition coefficients. The adaptive threshold is calculated using the formula T=σ²lnN, where σ is the standard deviation of image noise, obtained statistically from pure noise areas without water, and N is the total number of water pixels. The optimal threshold for each band is determined, and high-frequency coefficients below the threshold are directly set to 0 to remove noise-corresponding coefficient values. Simultaneously, by combining the spectral characteristic threshold ranges of TSM measured sampling points in each lake area, abnormal pixels deviating from the normal spectral response range in the numerical matrix are identified. The wavelet coefficients corresponding to these abnormal pixels are corrected using the neighborhood pixel mean method, ultimately resulting in a denoised wavelet coefficient set that has undergone dual processing of noise and outliers.

[0071] S23: The denoised wavelet coefficient set is rigorously reconstructed using inverse wavelet transform, adhering strictly to the scale hierarchy, wavelet basis function type, and decomposition rules of wavelet multi-scale decomposition. The reconstructed coefficients for each band and pixel are calculated and restored point-by-point to obtain the denoised water remote sensing reflectance numerical matrix. This matrix completely eliminates high-frequency noise and outlier effects caused by sensor noise and atmospheric residual interference, fully preserving the true spectral characteristics of lake water in response to TSM concentration changes. This forms a well-organized, noise-free, and high-quality spectral data foundation, providing accurate and reliable spectral input for subsequent feature construction and XGBoost band selection.

[0072] S24: Based on the denoised water body remote sensing reflectance numerical matrix, the spectral data of the four sensitive bands (RGB and near-infrared) are subjected to dual processing: min-max normalization and Z-score standardization. Min-max normalization maps all band values ​​to the 0-1 interval. Z-score standardization further optimizes the data, completely eliminating dimensional differences, numerical magnitude deviations, and systematic errors between different bands, resulting in a standardized and homogeneous spectral numerical sequence.

[0073] S3: Construct multi-scale band combinations and spectral differential features based on the spectral numerical sequence, use the XGBoost algorithm to evaluate and rank the contribution of various features, and select the feature subset that is most sensitive to changes in suspended matter to obtain sensitive features.

[0074] S31: Based on the standardized spectral numerical sequence, for the four bands (RGB and near-infrared) that are sensitive to TSM concentration, construct multi-scale band ratio (e.g., red band / near-infrared band) and band difference (e.g., near-infrared band-green band) combination features, and simultaneously calculate the first-order differential spectral features using a first-order differential formula. In the formula The reflectivity after differentiation The center wavelength of the spectral band. The original band reflectivity, To define the band intervals, the original band features, multi-scale combined features, and spectral differential features are fused in multiple dimensions to form a multi-dimensional spectral feature set covering both basic and derived spectral information. This set incorporates meteorological data on temperature and wind speed in the study area, as well as empirical TSM formula features. Red represents the independent variable related to the spectral characteristics of water bodies, enriching the dimensions of the feature set.

[0075] S32: Based on the multi-dimensional spectral feature set and combined with the measured suspended particulate matter (TSM) concentration data of each lake sampling point, the training set and test set are divided proportionally to construct the training samples of the XGBoost algorithm. The multi-dimensional spectral feature set is used as the input features of the model and the measured TSM concentration value is used as the output label of the model. The core parameters of XGBoost are set, the model is trained by the forward addition strategy, and the model is optimized by the loss function and regularization term. The initial training of the model and the nonlinear mapping of features-concentration are completed to obtain the trained XGBoost model.

[0076] S33: Based on the inherent feature importance assessment function of the trained XGBoost model, and based on the contribution of each feature to the loss function during model training, calculate the contribution scores of various spectral features such as original bands, combined features, differential features, meteorological features, and empirical formula features to the inversion of suspended solids concentration. Sort them from high to low scores to quantify the correlation between various features and TSM concentration. At the same time, refer to the feature importance analysis results to mark the contribution ratio of core features, and obtain the complete feature contribution ranking results.

[0077] S34: Based on the feature contribution ranking results, a contribution threshold is set to eliminate redundant features with low contributions, and the core features with the highest scores are selected to form a preliminary feature subset. Then, the linear correlation between features within the subset is tested using the Pearson correlation coefficient formula, and collinear features with excessively high correlation coefficients are eliminated. Finally, the most significant sensitive features that respond to changes in suspended matter concentration, without redundancy or collinearity, are obtained, including core remote sensing bands, optimal combination features, key meteorological features, and empirical formula features.

[0078] S4: Based on the sensitive characteristics and the simplified law of water body radiation transfer, physical constraint terms are constructed, and preliminary suspended solids concentration values ​​are obtained through numerical iteration calculations, forming intermediate inversion results that combine spectral information and physical meaning.

[0079] S41: Based on the selected suspended matter sensitive characteristics (including RGB, near-infrared sensitive bands, and meteorological coupling characteristics) and the simplified law of radiative transfer in inland lakes, combined with the TSM spectral response characteristics of each lake, the water reflectance is extracted. Water diffuse attenuation coefficient Backscattering coefficient Core optical parameters were incorporated, along with the quantitative relationship of TSM inversion empirical formulas for the study area. Using sensitive features as the core independent variables, a quantitative correlation between optical parameters and TSM concentration was constructed. The coupling relationship between variables was corrected by combining the physical mechanism of water radiative transfer, forming a physical constraint term that uses sensitive features as variables and conforms to the optical characteristics of inland lakes. In the formula, a, b, and c are the lake area adaptability weight coefficients.

[0080] S42: Substitute the spectral values ​​of the sensitive features (including reflectance of each sensitive band and values ​​calculated by empirical formulas) into the physical constraint terms one by one, and establish a numerical iterative calculation equation for suspended solids concentration based on the nonlinear response relationship between TSM concentration and spectral features in inland lakes. In the formula, The TSM concentration value is the value of the nth iteration. This is the iterative correction term. The initial value TSM0 is set based on the TSM concentration range of the measured sampling points in each lake. The root mean square error (RMSE) < 0.05 mg / L is used as the convergence threshold. The concentration value is gradually corrected through multiple rounds of iterative calculations until the calculation result meets the convergence threshold, thus obtaining the numerical iterative convergence result.

[0081] S43: Based on the numerical iteration convergence results and combined with the spatial gridded distribution attributes of sensitive features, the pixel-by-pixel iterative convergence value is precisely matched with the spatial grid coordinates of the remote sensing image. Spatial interpolation is used to complete the concentration values ​​of a small number of non-converged pixels. At the same time, the vector boundary of the lake area in the study area is combined to remove invalid values ​​outside the lake area. The preliminary suspended solids concentration values ​​corresponding to the spatial grid of each lake area are calculated. These values ​​have both physical mechanism support and spectral characteristic response characteristics, and finally form the spatial gridded intermediate inversion results of TSM concentration.

[0082] S5: Input the intermediate inversion results and sensitive features, and use a deep learning model to learn the residual correction relationship. Only perform bias compensation on the preliminary inversion results, rather than directly predicting the final concentration from the spectrum.

[0083] S51: Based on the intermediate inversion results and sensitive features, the training and test sets are divided. The intermediate inversion results and sensitive features are used as input, and measured suspended particulate matter (TSM) concentration data are matched as labels to construct training samples for a deep learning model adapted to the residual correction task. The spatially gridded intermediate inversion results and sensitive features fused with remote sensing bands and meteorological factors are used as model input features. Measured suspended particulate matter (TSM) concentration data from sampling points are accurately matched as labels, and calculated values ​​supplement the feature dimensions. This constructs training samples for a deep learning model adapted to the residual correction task, ensuring spatiotemporal consistency between the sample input and labels.

[0084] S52: Initialize the deep learning model parameters based on the training samples, set the loss function and number of iterations for model training, and train the model to learn the residual correction relationship between the intermediate inversion results and the true concentration, thus obtaining the trained deep learning model. Based on the training samples, initialize the core parameters of the deep learning model, such as weights and biases, set hyperparameters such as the learning rate and the number of hidden layer nodes, and select the root mean square error as the loss function. , Root mean square error (RMSE) measures the average deviation between model predictions and actual values; a smaller value indicates higher model fitting accuracy. i represents the sample index, indicating the i-th training sample (i=1,2,…,n). n represents the total number of samples, indicating the total number of training samples used in the loss calculation. This represents the measured concentration value of the i-th sample, i.e., the actual concentration data collected. This is the model inversion result for the i-th sample, i.e., the intermediate inversion concentration value output by the deep learning model.

[0085] Combining five-fold cross-validation to determine the number of model iterations and an early stopping strategy, the residuals between intermediate inversion results and measured concentrations are used. With the learning objective as the goal, the model is trained to learn the nonlinear residual correction relationship between the two, and the model parameters are iteratively optimized to obtain the trained deep learning model.

[0086] S53: Train the deep learning model by inputting all intermediate inversion results and sensitive features. Then, use the residual correction amount output by the model to compensate for the pixel-by-pixel bias of the intermediate inversion results to obtain the corrected concentration result.

[0087] All intermediate inversion results and sensitive features of the entire spatial grid of each lake are standardized according to the sample construction rules and input into the trained deep learning model. The model outputs the corresponding residual correction amount pixel by pixel. The correction amount is superimposed and compensated with the original intermediate inversion results pixel by pixel. Outliers that exceed the reasonable range of measured concentration after compensation are removed to obtain the suspended matter concentration result after residual correction, thereby improving the fit between the inversion result and the measured value.

[0088] S6: Spatial smoothing and numerical consistency verification are performed on the concentration results after deep learning correction. The optimization adjustment is completed by combining the high concentration asymptotic saturation rule, and finally the spatial distribution results of suspended matter concentration with spatial continuity and reliable accuracy are obtained.

[0089] For the concentration results corrected by deep learning, spatial smoothing is first performed using moving window Gaussian smoothing or mean filtering. Neighborhood pixel weighting is then used to suppress noise and isolated outliers generated during pixel-by-pixel inversion, preserving the large-scale spatial distribution trend of suspended matter concentration and ensuring the results conform to the natural continuity of water substance diffusion. Based on this, numerical consistency verification is conducted. Outliers exceeding reasonable ranges are removed according to the historical measured range of the study area. Simultaneously, the correlation between the concentration gradient of adjacent pixels and spectral sensitivity features is examined, removing abrupt pixels that do not conform to physical laws and repairing them using neighborhood interpolation. Subsequently, optimization is performed using the asymptotic saturation rule for high concentrations. Considering the water's optical properties where spectral reflectance tends to stabilize under high concentration conditions, an asymptotic saturation function is used to constrain reasonable concentration values, preventing overestimation of high-concentration regions and ensuring consistency between the inversion results and the optical-physical mechanisms of water bodies. Finally, spatially continuous, numerically reasonable, and reliable spatial distribution results of suspended matter concentration are obtained.

[0090] like Figure 3 As shown, the present invention also provides a machine learning-based remote sensing inversion system for suspended matter concentration, comprising:

[0091] The data preprocessing module is used to collect satellite remote sensing apparent reflectance datasets and synchronously measured suspended matter concentration data, perform radiometric calibration, atmospheric correction, and water body masking operations on the images, and obtain a spatially gridded water body remote sensing reflectance numerical matrix.

[0092] The spectral denoising module is used to perform wavelet decomposition and reconstruction on the spatially gridded water remote sensing reflectance numerical matrix. It removes high-frequency noise and abnormal pixel values ​​through adaptive thresholding, and then normalizes and standardizes the spectrum to obtain the spectral numerical sequence.

[0093] The feature optimization module is used to construct multi-scale band combinations and spectral differential features based on the spectral numerical sequence, and to use the XGBoost algorithm to evaluate and rank the contribution of various features, and to select the feature subset that is most sensitive to changes in suspended matter, thus obtaining sensitive features.

[0094] The physical inversion module is used to construct physical constraint terms based on sensitive characteristics and simplified laws of water body radiation transfer, and obtain preliminary suspended solids concentration values ​​through numerical iteration calculations to form intermediate inversion results.

[0095] The residual correction module is used to input intermediate inversion results and sensitive features, and use a deep learning model to learn the residual correction relationship. It only performs bias compensation on the preliminary inversion results to obtain the corrected concentration results.

[0096] The results optimization module is used to perform spatial smoothing and numerical consistency verification on the concentration results corrected by deep learning. It combines the high-concentration asymptotic saturation rule to complete the optimization adjustment and obtain the spatial distribution results of suspended matter concentration.

[0097] In summary, this embodiment provides a machine learning-based remote sensing inversion method and system for suspended solids concentration. Through residual correction and spatial optimization, it achieves high-precision and high-reliability spatial distribution results. Deep learning is employed to specifically learn the residuals of intermediate inversion results, performing only bias compensation without directly predicting concentrations. This preserves the rationality of physical constraints while compensating for errors introduced by model simplification, significantly improving inversion accuracy. Subsequently, spatial smoothing suppresses local noise, numerical consistency checks eliminate outliers, and the high-concentration asymptotic saturation rule corrects for overestimation of high concentrations. This ensures spatial continuity, numerical rationality, and conformity to the optical laws of water bodies. The resulting spatial distribution product of suspended solids concentration has higher accuracy and a more natural spatial structure, meeting the needs of large-scale, long-term, high-precision water environment monitoring and possessing strong engineering application value.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing inversion method for suspended matter concentration based on machine learning, characterized in that, include: S1: Collect satellite remote sensing apparent reflectance dataset and synchronous measured suspended matter concentration data, perform radiometric calibration, atmospheric correction and water body masking calculation on the image to obtain a spatially gridded water body remote sensing reflectance numerical matrix. S2: Perform wavelet decomposition and reconstruction on the spatial gridded water body remote sensing reflectance numerical matrix, remove high-frequency noise and abnormal pixel values ​​through adaptive thresholding, and then normalize and standardize the spectrum to obtain the spectral numerical sequence. S3: Construct multi-scale band combinations and spectral differential features based on the spectral numerical sequence, use the XGBoost algorithm to evaluate and rank the contribution of various features, and select the feature subset that is most sensitive to changes in suspended matter to obtain sensitive features. S4: Based on the aforementioned sensitive characteristics and the simplified law of water body radiation transfer, physical constraint terms are constructed, and preliminary suspended solids concentration values ​​are obtained through numerical iteration calculations to form intermediate inversion results; S41: Extract core optical parameters based on the aforementioned sensitive features and the simplified law of radiative transmission of inland lake water bodies, and construct physical constraint terms; Based on the spectral response characteristics of the aforementioned sensitive features and the simplified law of radiative transfer in inland lake water bodies, the physical correlation logic between reflectivity and suspended matter concentration is sorted out, and core optical parameters are extracted. Using the aforementioned sensitive features as core variables and combining them with the core optical parameters, a quantitative correlation expression between optical parameters and suspended matter concentration is established. The constructed quantitative correlation expression and the physical constraints of water body radiation transfer are used to correct the coupling relationship between variables and form physical constraint terms; S42: Substitute the spectral values ​​of the sensitive features into the physical constraint terms to establish a numerical iterative calculation equation for the concentration of suspended matter. Set initial values ​​and convergence thresholds, perform multiple rounds of iterative calculations until the results converge, and obtain the numerical iterative convergence result. S43: Based on the numerical iteration convergence results and the spatial gridded distribution attributes of the sensitive features, the preliminary suspended solids concentration values ​​corresponding to the entire spatial domain are calculated to form intermediate inversion results; S5: Input the intermediate inversion results and sensitive features, use a deep learning model to learn the residual correction relationship, and only perform bias compensation on the preliminary inversion results to obtain the corrected concentration results; S6: Spatial smoothing and numerical consistency verification are performed on the concentration results after deep learning correction. The optimization adjustment is completed by combining the high concentration asymptotic saturation rule to obtain the spatial distribution results of suspended matter concentration.

2. The machine learning-based remote sensing inversion method for suspended matter concentration according to claim 1, characterized in that: In step S1, the specific steps for obtaining the spatially gridded water remote sensing reflectance numerical matrix are as follows: S11: Collect satellite remote sensing images of the target area and synchronous field measured suspended particulate matter concentration data to construct an initial dataset containing original DN value images and corresponding concentration samples; S12: Based on the original DN value image and satellite sensor calibration parameters in the initial dataset, perform radiometric calibration calculations to convert the DN values ​​into radiance data and obtain a radiance image. S13: Based on the radiance image, an atmospheric correction model is used to perform calculations to eliminate the effects of atmospheric absorption and scattering, thereby obtaining the true surface reflectance image. S14: The water index method is used to construct discrimination rules for the acquired surface reflectance images. Non-water areas such as land, clouds, and shadows are removed by masking, and finally a spatially gridded water remote sensing reflectance numerical matrix is ​​obtained.

3. The machine learning-based remote sensing inversion method for suspended matter concentration according to claim 1, characterized in that: In step S2, the specific steps for obtaining the spectral numerical sequence are as follows: S21: Using the spatially gridded water body remote sensing reflectance numerical matrix, wavelet basis functions adapted to the spectral characteristics of the water body are selected to carry out pixel-by-pixel and band-by-band wavelet multi-scale decomposition to obtain low-frequency and high-frequency wavelet decomposition coefficients. S22: An adaptive thresholding method is used to perform threshold shrinking on the noisy high-frequency wavelet decomposition coefficients, remove the coefficient values ​​corresponding to noise, and identify and correct abnormal pixels in the numerical matrix to obtain a set of denoised wavelet coefficients. S23: The denoised wavelet coefficient set is reconstructed by wavelet decomposition scale to obtain the denoised water body remote sensing reflectance numerical matrix, which eliminates high-frequency noise and outlier interference, retains the true characteristic information of the water body spectrum, and forms the basis of spectral data. S24: Normalize the spectral data of each band according to the reflectance numerical matrix to eliminate the differences in dimensions and values ​​between bands, and obtain the spectral numerical sequence.

4. The machine learning-based remote sensing inversion method for suspended matter concentration according to claim 1, characterized in that: In step S3, the specific steps for obtaining the sensitive features are as follows: S31: Construct multi-scale band ratio and band difference combination features based on the spectral numerical sequence, calculate the first-order differential features of the spectrum, and fuse them to form a multi-dimensional spectral feature set; S32: Based on the multi-dimensional spectral feature set and the measured suspended matter concentration data, construct training samples for the XGBoost algorithm, using the feature set as input and the measured concentration as label, to complete the initial training and feature mapping of the model, and obtain the trained XGBoost model. S33: Based on the training, complete the feature importance evaluation of the XGBoost model, calculate the contribution score of various spectral features to the inversion of suspended matter concentration, sort them from high to low according to the score, quantify the correlation between features and concentration, and obtain the contribution ranking. S34: Based on the contribution ranking, redundant features with low contribution are eliminated, and features with high scores are selected to form a feature subset. Collinear features within the subset are eliminated through correlation test to obtain the most sensitive features to changes in suspended matter.

5. The machine learning-based remote sensing inversion method for suspended matter concentration according to claim 4, characterized in that: In step S31, the specific steps for fusing to form a multi-dimensional spectral feature set are as follows: The spectral numerical sequence is used to perform multi-scale band ratio calculations on the characteristic bands. The ratio features are constructed by dividing the reflectance of different bands, and then the band difference calculation is performed. The difference features are constructed by subtracting the reflectance of the bands, thus forming a combined feature set. Based on the combined feature set and the original spectral numerical sequence, the first-order differential features of each band are calculated using the spectral differential formula to capture subtle spectral changes and supplement the differential features into the combined feature set to form a preliminary multidimensional feature set. The preliminary multidimensional feature set is fused with the original band features in the spectral numerical sequence, integrating the three types of information: original bands, combined features, and first-order differential features, to form a multidimensional spectral feature set.

6. The machine learning-based remote sensing inversion method for suspended matter concentration according to claim 1, characterized in that: In step S5, the specific steps to obtain the corrected concentration result are as follows: S51: Divide the training set and the test set according to the intermediate inversion results and sensitive features, take the intermediate inversion results and sensitive features as input, match the measured suspended solids concentration data as labels, and construct training samples for a deep learning model adapted to the residual correction task. S52: Initialize the deep learning model parameters according to the deep learning model training samples, set the loss function and number of iterations for model training, train the model to learn the residual correction relationship between intermediate inversion results and true concentration, and obtain the trained deep learning model. S53: Train the deep learning model by inputting all intermediate inversion results and sensitive features. Then, use the residual correction amount output by the model to compensate for the pixel-by-pixel bias of the intermediate inversion results to obtain the corrected concentration result.

7. The machine learning-based remote sensing inversion method for suspended matter concentration according to claim 6, characterized in that: In step S52, the specific steps to obtain the trained deep learning model are as follows: Based on the training samples of the deep learning model, select a network structure suitable for residual fitting and initialize the model weights and bias parameters, and set a reasonable learning rate; Based on the initialized model and training samples, a loss function with the error between the predicted residual and the actual residual as the core is defined. The number of iterations and the early stopping strategy are set. The model is iteratively trained using the training set data to continuously optimize the parameters. The accuracy of the model during iterative training is verified using a test set. Training is stopped when the loss function converges and the verification error tends to stabilize, and the optimal parameter combination is retained to obtain the trained deep learning model.

8. A machine learning-based remote sensing inversion system for suspended particulate matter concentration, which employs the machine learning-based remote sensing inversion method for suspended particulate matter concentration as described in any one of claims 1 to 7, characterized in that, The inversion system includes: The data preprocessing module is used to collect satellite remote sensing apparent reflectance datasets and synchronously measured suspended matter concentration data, perform radiometric calibration, atmospheric correction and water body masking calculations on the images, and obtain a spatially gridded water body remote sensing reflectance numerical matrix. The spectral denoising module is used to perform wavelet decomposition and reconstruction on the spatial gridded water body remote sensing reflectance numerical matrix, remove high-frequency noise and abnormal pixel values ​​through adaptive thresholding, and then normalize and standardize the spectrum to obtain a spectral numerical sequence. The feature optimization module is used to construct multi-scale band combination and spectral differential features based on the spectral numerical sequence, use the XGBoost algorithm to evaluate and rank the contribution of various features, and select the feature subset that is most sensitive to changes in suspended matter to obtain sensitive features. The physical inversion module is used to construct physical constraint terms based on the aforementioned sensitive features and the simplified law of water body radiation transfer, obtain preliminary suspended solids concentration values ​​through numerical iteration calculations, and form intermediate inversion results. The residual correction module is used to input the intermediate inversion results and sensitive features, use a deep learning model to learn the residual correction relationship, and only perform bias compensation on the preliminary inversion results to obtain the corrected concentration results; The results optimization module is used to perform spatial smoothing and numerical consistency verification on the concentration results corrected by deep learning. It combines the high-concentration asymptotic saturation rule to complete the optimization adjustment and obtain the spatial distribution results of suspended matter concentration.

Images