A river water body CODMn in-situ hyperspectral inversion method, system, medium and product

Abstract

The present disclosure belongs to the technical field of water body monitoring, and particularly relates to a river water body CODMn in-situ hyperspectral inversion method, system, medium and product, the method comprising: collecting in-situ hyperspectral data and CODMn measured label data of river water bodies in a study area through a plurality of in-situ hyperspectral monitoring devices arranged in the study area; constructing a verification framework comprising a plurality of model verification methods; training a preset empirical model and a machine learning model in parallel based on the in-situ hyperspectral data and the CODMn measured label data, comparing the performance of the trained models based on the verification framework, and selecting the model with the best performance as a target model; and performing river water body CODMn in-situ hyperspectral inversion according to the target model. The present disclosure adopts a parallel training and comparison optimization method of empirical models and machine learning models, can automatically select the target model with the best performance, and balances the applicability, stability and prediction accuracy of the final inversion model.

Description

Technical Field

[0001] This disclosure belongs to the field of water body monitoring technology, and specifically relates to a method, system, medium and product for in-situ hyperspectral inversion of CODMn in river water. Background Technology

[0002] The permanganate index (CODMn) is a core indicator characterizing the degree of organic pollution, the load of received pollutants, and the health status of the aquatic environment in river water. Current methods for measuring CODMn in river water typically employ laboratory chemical analysis or online equipment, which offer high accuracy but suffer from limited sampling frequency and insufficient spatiotemporal continuity, making it difficult to reflect short-term fluctuations in water quality. Specifically, the shortcomings of existing technologies are as follows: (1) The validation framework is lenient, overestimating the model's real deployment capability: Existing studies often employ random partitioning or leave-one-out cross-validation at the sample level to evaluate model performance. In multi-device online monitoring scenarios, this validation method allows the training and test sets to share samples from the same devices, failing to reflect the systematic differences in observation background and sampling repeatability among devices. Therefore, the reported model accuracy is artificially inflated and cannot truly reflect the model's cross-device generalization ability when deployed on new devices and locations.

[0003] (2) Ignoring the effect of time drift, the model has poor intertemporal stability: Existing modeling studies often use data collected at different times in combination, failing to adequately consider the issue of time drift. The distribution of target values, spectral response structure, and band-parameter relationships in water bodies can all systematically change with time factors such as seasons and hydrological conditions. This leads to a significant decrease in predictive performance, or even failure, of models trained on historical data when applied to data from future periods.

[0004] (3) The model structure is complex and lacks robustness under conditions of small sample size and multiple devices: Existing research often employs complex machine learning models with high degrees of freedom to achieve high fitting accuracy. However, for weakly spectrally active parameters like CODMn, their inversion depends on indirect covariance with variables such as colored dissolved organic matter and turbidity, resulting in weak and volatile signals. Under conditions of small samples and multiple devices, complex models are prone to overfitting the device background noise and local states in the training data, leading to a sharp decline in their generalization ability under rigorous cross-device validation, rendering them unusable in engineering applications.

[0005] In summary, existing technologies treat CODMn inversion as a simple high-dimensional fitting problem, pursuing optimal accuracy on the training set while neglecting the core requirement of online monitoring engineering deployment—the model must maintain stable and reliable predictive capabilities on unseen equipment and under time-varying water conditions. Inappropriate validation methods, lack of drift identification, and an imbalance between model complexity and robustness in existing technologies collectively hinder the practical application of in-situ hyperspectral monitoring technology for river water bodies. Summary of the Invention

[0006] To address the aforementioned problems, this disclosure provides an in-situ hyperspectral inversion method for CODMn in river water, which has the advantage of high reliability of monitoring results. The method includes: In-situ hyperspectral data and CODMn measured label data of the river water in the study area were collected by several in-situ hyperspectral monitoring devices set up in the study area. Construct a validation framework that includes multiple model validation methods; Based on the in-situ hyperspectral data and CODMn measured label data, the preset empirical model and machine learning model are trained in parallel, and the performance of the trained models is compared based on the verification framework, and the model with the best performance is selected as the target model. In-situ hyperspectral inversion of CODMn in river water was performed based on the target model.

[0007] Furthermore, the model validation methods corresponding to the validation framework include: random partitioning validation, sample-level leave-one-out cross-validation, device-level leave-one-out cross-validation, and inter-period migration validation.

[0008] Furthermore, the empirical model includes at least one of the following: a single-band linear model, an NDSI linear model, and an exponential quadratic empirical model; the machine learning model includes at least one of the following: a Lasso model, a Ridge model, a PLSR model, a random forest model, and an SVR model.

[0009] Furthermore, before performing in-situ hyperspectral inversion of CODMn in river water based on the target model, the process also includes: dividing the in-situ hyperspectral data and the measured CODMn label data into time periods and identifying time drift, and correcting the target model based on the time drift identification results.

[0010] Furthermore, the time drift identification includes: identifying the differences in data distribution between different periods, identifying the differences in the curve structure of the corresponding spectral curves of data between different periods, and identifying the impact of cross-period stability of the model caused by cross-period migration of data between different periods.

[0011] Furthermore, before performing parallel training of the preset empirical model and machine learning model based on the in-situ hyperspectral data and CODMn measured label data, the following steps are also included: The in-situ hyperspectral data is subjected to robust band data screening; the robust band data are the data corresponding to bands that are stable across devices and across periods.

[0012] Furthermore, robust band data screening is performed on the in-situ hyperspectral data, including: robust band data screening of the in-situ hyperspectral data based on the Lasso cross-fold frequency statistical method.

[0013] This disclosure also proposes an in-situ hyperspectral inversion system for CODMn in river water, including: The data acquisition module is used to collect in-situ hyperspectral data of the river water in the study area and CODMn measured label data through several in-situ hyperspectral monitoring devices set up in the study area; The framework building module is used to build a validation framework that includes multiple model validation methods; The model training module is used to train the preset empirical model and machine learning model in parallel based on the in-situ hyperspectral data and CODMn measured label data, and to compare the performance of the trained models based on the verification framework, and select the model with the best performance as the target model. The data inversion module is used to perform in-situ hyperspectral inversion of CODMn in river water based on the target model.

[0014] This disclosure also proposes a computer-readable storage medium storing computer instructions that, when executed by a processor, are used to implement at least the above-described in-situ hyperspectral inversion method for CODMn in river water bodies.

[0015] This disclosure also proposes a computer program product stored in a computer-readable storage medium, which, when executed by a processor, is used to at least implement the above-described in-situ hyperspectral inversion method for CODMn in river water bodies.

[0016] Compared with the prior art, this disclosure has the following advantages: (1) Real-time monitoring of CODMn in river water: Water spectral data can be directly collected by in-situ hyperspectral equipment without manual sampling and laboratory analysis. This breaks through the limitations of traditional monitoring with low spatiotemporal resolution and inability to reflect short-term fluctuations in water quality, and can obtain information on the dynamic changes of pollution in river water in real time.

[0017] (2) Improve the authenticity and reliability of model generalization ability assessment: Construct a comprehensive verification framework that includes multiple model verification methods to avoid overestimation of model performance caused by a single lenient verification method. This will enable a more objective and comprehensive evaluation of the model's predictive ability in real monitoring scenarios and improve the scientific nature of model selection.

[0018] (3) Balancing model prediction accuracy and engineering applicability: By adopting the parallel training and comparison of empirical models and machine learning models, the optimal target model can be automatically selected, so that the final inversion model achieves a balance in applicability, stability and prediction accuracy.

[0019] (4) Improve the credibility of CODMn inversion results: The target model obtained by multi-method verification and multi-model optimization is used for inversion, which reduces the risk of model overfitting and local adaptation, and makes the inversion results more in line with the actual on-site monitoring environment, which can directly support the online monitoring and management decision of river water environment.

[0020] Other features and advantages of this disclosure will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 An in-situ hyperspectral inversion method for CODMn in river water bodies according to an embodiment is shown; Figure 2 The observation-prediction comparison between the empirical three-index model and the Lasso model according to the embodiments is shown; Figure 3 The correlation coefficient curves between a single band and CODMn according to an embodiment are shown; Figure 4 The results of R² comparison under different verification methods according to the embodiments are shown; Figure 5 The average spectral curves of C1 and C2 according to the embodiment are shown; Figure 6 A comparison of LOGO RMSE under different preprocessing methods according to the embodiments is shown; Figure 7 The robust band frequency distribution according to the embodiment is shown; Figure 8 The in-situ hyperspectral inversion implementation process according to the embodiment is shown; Figure 9 An in-situ hyperspectral inversion system for CODMn in river water is shown according to an embodiment; Figure 10 A computer-readable storage medium according to an embodiment is shown. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0024] Figure 1 The in-situ hyperspectral inversion method for CODMn in river water bodies proposed in this disclosure has the advantage of high reliability of monitoring results. The method includes: In-situ hyperspectral data and CODMn measured label data of the river water in the study area were collected by several in-situ hyperspectral monitoring devices set up in the study area. Construct a validation framework that includes multiple model validation methods; Based on the in-situ hyperspectral data and CODMn measured label data, the preset empirical model and machine learning model are trained in parallel, and the performance of the trained models is compared based on the verification framework, and the model with the best performance is selected as the target model. In-situ hyperspectral inversion of CODMn in river water was performed based on the target model.

[0025] According to some embodiments of this disclosure, the technical concept of the in-situ hyperspectral inversion method for CODMn in river water bodies proposed in this disclosure is as follows: In-situ hyperspectral data and measured CODMn label data of river water bodies in the study area are collected by in-situ hyperspectral monitoring equipment set up in the study area. These data are used to train multiple models constructed based on different modeling routes. When training the models, the model performance is determined based on a verification framework that includes multiple model verification methods, and the performance of different models is compared. The model with the best performance is selected for in-situ hyperspectral inversion of CODMn in river water bodies in the study area.

[0026] Furthermore, the model verification methods corresponding to the verification framework include: random partitioning verification, sample-level leave-one-out verification, device-level leave-one-out cross-validation, and inter-period migration verification.

[0027] According to some embodiments of this disclosure, the methods used to construct the verification framework can be random partitioning verification, sample-level leave-one-out verification, device-level leave-one-out cross-validation, and inter-period transfer verification. The principle of random partitioning verification is to randomly divide all samples into training set and test set according to a certain proportion, and repeat this process multiple times and take the average. The principle of sample-level leave-one-out verification is to leave one sample as the test set each time. The principle of device-level leave-one-out cross-validation is to use all samples of one device as the test set each time, and use the samples of the remaining devices as the training set, and repeat this process until all devices have been tested once. The principle of inter-period transfer verification is to train the model with data from one period and test it on data from another period, and vice versa.

[0028] Furthermore, the verification framework uses device-level leave-one-out cross-validation as the main model verification method.

[0029] According to some embodiments of this disclosure, in-situ hyperspectral data and CODMn measured label data come from multiple monitoring devices and there is repeated observation of the same device. Random partitioning or leave-one-out cross-validation at the sample level will cause the training set and test set to share samples from the same device, introducing intra-device similarity dependency and causing the system to overestimate the cross-device generalization ability. A better choice is to adopt device-level leave-one-out cross-validation (LOGO) as the main validation method: each time, all samples from one device are used as the test set, and the remaining M-1 samples constitute the training set. The evaluation index is summarized on M folds; M is the total number of monitoring devices. The essence of LOGO is to sever the device association between the test set and the training set, making the evaluation conditions closer to the real scenario when the model is deployed on new devices or at new locations. To quantify the differences in results from different validation methods, three validation strategies are designed and implemented to form an evaluation system (i.e., validation framework). The significance of the model validation methods in this evaluation system is as follows: Device-level leave-one-out cross-validation: The core validation strategy, which simulates real-world scenarios of deploying the model on new devices, rigorously evaluates its cross-device generalization ability, and serves as a necessary standard for judging whether the model can be engineered.

[0030] Random partitioning verification: as a relaxed control strategy, it reveals that conventional verification methods may lead to performance overestimation; Leave-one-out-of-sample validation: also used as a lenient control strategy to evaluate the model’s ability to fit the same devices.

[0031] Intertemporal transfer validation: Directly and quantitatively assess the impact of time drift on the stability of model intertemporal predictions.

[0032] Furthermore, the empirical model includes at least one of the following: a single-band linear model, an NDSI linear model, and an exponential quadratic empirical model; the machine learning model includes at least one of the following: a Lasso model, a Ridge model, a PLSR model, a random forest model, and an SVR model.

[0033] Furthermore, the exponential quadratic empirical model is an empirical tri-exponential model.

[0034] According to some embodiments of this disclosure, the proposed method employs both empirical and machine learning models in its modeling process. The empirical model approach involves constructing structurally constrained models, such as single-band linear models, dual-band Normalized Difference Index (NDSI) linear models, and triple-exponential quadratic empirical models. For weak spectral activity indicators like CODMn, multi-band combined empirical models can effectively utilize indirect covariance relationships under interpretable conditions. The machine learning model approach involves constructing various models of different complexities as controls, including Lasso, Ridge, PLSR, Random Forest (RF), and Support Vector Machine (SVR). A comprehensive evaluation of the performance of different modeling methods under a rigorous validation framework reveals the performance differences of high-complexity models under relaxed and rigorous validation.

[0035] Furthermore, before performing in-situ hyperspectral inversion of CODMn in river water based on the target model, the process also includes: dividing the in-situ hyperspectral data and the measured CODMn label data into time periods and identifying time drift, and correcting the target model based on the time drift identification results.

[0036] Furthermore, the time drift identification includes: identifying the differences in data distribution between different periods, identifying the differences in the curve structure of the corresponding spectral curves of data between different periods, and identifying the impact of cross-period stability of the model caused by cross-period migration of data between different periods.

[0037] Furthermore, before performing parallel training of the preset empirical model and machine learning model based on the in-situ hyperspectral data and CODMn measured label data, the following steps are also included: The in-situ hyperspectral data is subjected to robust band data screening; the robust band data are the data corresponding to bands that are stable across devices and across periods.

[0038] Furthermore, robust band data screening is performed on the in-situ hyperspectral data, including: robust band data screening of the in-situ hyperspectral data based on the Lasso cross-fold frequency statistical method.

[0039] According to some embodiments of this disclosure, in order to eliminate the failure of model intertemporal prediction caused by the systematic changes in in-situ hyperspectral data and CODMn measured label data over time, when applying the target model to perform in-situ hyperspectral inversion of CODMn in river water, it is also necessary to correct the time drift of the target model. The approach is to identify the time drift from three levels: data distribution, spectral structure and intertemporal migration. Methods for identifying time drift at the data distribution level include: comparing statistical characteristics such as the mean and standard deviation of CODMn across different periods to characterize whether the data distribution has shifted overall, i.e., identifying differences in data distribution across different periods; methods for identifying time drift at the spectral structure level include: calculating the mean spectrum of data from two periods by device and statistically analyzing the spectral response RMSE to determine whether the spectral response structure has changed, i.e., identifying differences in the curve structure of the corresponding spectral curves for data from different periods; methods for identifying time drift at the inter-period migration level include: verifying the inter-period stability of the model by migrating data from the first period to the second period and from the second period to the first period, i.e., identifying the impact of inter-period migration on the model's inter-period stability. The approach to modifying the target model based on the time drift identification results is as follows: using the Lasso inter-fold frequency statistical method, robust band data that are stable across devices and periods are selected from in-situ hyperspectral data and measured CODMn label data, and the target model is retrained.

[0040] The above solution will be further explained below with reference to the embodiments.

[0041] S101. Study Area Determination and Data Acquisition This embodiment uses the river network system of a city in a province as the study area. Monitoring points are systematically set up on major rivers and their tributaries, forming a monitoring network covering multiple types of river channels and hydrodynamic conditions. The region has a dense river network, and the water condition is simultaneously affected by upstream inflows, dam and sluice gate operations, riparian human activities, and seasonal hydrological processes, resulting in significant spatiotemporal heterogeneity and dynamic variability in water quality parameters and spectral characteristics. This complex natural-anthropogenic coupling environment provides an ideal testing ground for verifying the robustness and generalization ability of the water quality hyperspectral inversion model in real-world scenarios. The dataset for this embodiment originates from 20 in-situ hyperspectral online monitoring devices deployed in the aforementioned river network.

[0042] S102. Data Preprocessing In-situ hyperspectral data from multiple online monitoring devices in the target watershed and synchronously measured CODMn tag data were acquired. The data was cleaned, and invalid records were removed. Based on the sampling time, the data was explicitly divided into different periods, laying the foundation for subsequent time-drift analysis. Significance: Clean, structured modeling data was obtained, and the time factor was incorporated into the research design through explicit periodization.

[0043] To delve into the impact of time drift on the model, this study clearly divided the 240 samples into two independent periods based on sampling time: Period C1: collected from August to September, with 120 samples; Period C2: collected in October, with 120 samples. This phased design simulates observations of the same watershed under different seasonal (late summer to early autumn) hydrological conditions, reflecting real-world hydrological contexts. Crucially, the same equipment produced samples in both Period C1 and Period C2, creating a natural "before-and-after control" experimental design. This provides a solid data foundation for subsequent quantitative identification of spectral drift and assessment of the model's intertemporal transferability.

[0044] S103. Construction of the Verification Framework The current samples come from 20 online devices, and there are duplicate observations on the same devices. Random partitioning or leave-one-out-of-sample validation would cause the training and test sets to share samples from the same devices, introducing intra-device similarity dependencies and causing the system to overestimate its cross-device generalization ability. Therefore, this paper uses device-level leave-one-out-of-sample cross-validation (LOGO) as the primary validation method, while also employing random partitioning validation, leave-one-out-of-sample validation, and inter-period transfer validation as lenient controls or to assess the impact of time drift on the model's inter-period prediction stability. The principle of device-level leave-one-out-of-sample cross-validation (LOGO) is as follows: each time, all samples from one device are used as the test set, and the samples from the remaining 19 devices constitute the training set. Evaluation metrics are summarized over 20 folds. The essence of LOGO is to sever the device association between the test set and the training set, making the evaluation conditions closer to the real-world scenario when the model is deployed on new devices or at new locations.

[0045] S104. Construction of different route models Empirical modeling approach: Constructing structurally constrained models, such as single-band linear models, dual-band normalized difference exponential linear models, and triple-exponential quadratic empirical models. For weakly spectrally active indices like CODMn, multi-band combined empirical models can effectively utilize indirect covariance relationships under interpretable conditions.

[0046] Machine learning model roadmap: Constructing machine learning models including Lasso, Ridge, PLSR, Random Forest, and Support Vector Machine. A comprehensive evaluation of the performance of different modeling methods under a rigorous validation framework is conducted, revealing the performance differences of high-complexity models under lenient and rigorous validation.

[0047] S105. Model Training within the Validation Framework As shown in Table 1, under the strict validation conditions of LOGO, the performance of various models shows obvious stratification. The single-band linear model has an R² of only 0.014, indicating that CODMn is difficult to be stably characterized by a single wavelength. The optimal NDSI combination improves R² to 0.237, but this is still limited. The triple exponential quadratic empirical model further improves R² to 0.446 and reduces RMSE to 1.101 mg / L, becoming the best result among all models. It is significantly better than the optimal Lasso model in machine learning (R²=0.253, RMSE=1.278 mg / L) and the PLSR baseline (R²=0.152).

[0048] Table 1

[0049] Figure 2 The results show a comparison of observation-prediction scatter plots between the empirical three-exponential model and the Lasso model. The three-exponential model outperforms Lasso under the LOGO condition. This does not imply that machine learning methods are unsuitable for water quality inversion, but rather reflects the following: Under the combined conditions of small sample size (240 data points), multiple devices (20 units), and weak spectral response, Lasso's sparse constraints are insufficient to completely resist interference from device background differences. The three-exponential model, through explicit band combination, compresses the problem to at least the minimum feature space, actively limiting the expansion of degrees of freedom, thus preserving a more stable device-level inversion structure. The scatter plot distribution of the three-exponential empirical model is closer to the 1:1 reference line, while the Lasso model's predicted values ​​are more concentrated in the middle range, with more significant compression of the response to high-value samples, indicating that the former has better dynamic range preservation capabilities under device-level validation.

[0050] Figure 3 The correlation coefficient curves between a single band and CODMn are shown. The overall correlation coefficient between the single band and CODMn is relatively low, indicating that CODMn is difficult to establish a stable response relationship based on a single wavelength. This is consistent with the results of the single-band linear model showing limited performance under LOGO conditions.

[0051] As shown in Table 2, the R² values ​​for the empirical three-index model are 0.553 and 0.534 for random partitioning and leave-one-out-of-sample, respectively, with limited differences compared to 0.446 for LOGO. The differences widen for Lasso (0.387, 0.400 vs. 0.253). The RF_RFE20 model exhibits the most extreme differences: R² = 0.825 for random partitioning, R² = 0.846 for leave-one-out-of-sample, and drops sharply to 0.011 for device-level validation. These results indicate that the high accuracy of complex models under lenient validation primarily stems from the utilization of sample similarity structures within the device, rather than from the cross-device transferable band-CODMn relationship.

[0052] Table 2

[0053] Figure 4 The results show a comparison of R² under different validation methods. The empirical model exhibits relatively limited fluctuations across the three validation calibers, while RF_RFE20 shows a precipitous drop from random partitioning and leave-one-out-of-sample to LOGO, intuitively demonstrating that lenient validation significantly overestimates the cross-device generalization ability of complex models. The fundamental mechanism of this bias lies in the fact that adjacent samples from the same device share similar observational backgrounds and local water quality conditions. Complex models excel at fitting such high-dimensional local structures, thus performing exceptionally well under lenient validation. LOGO, however, severs this dependency; once the test object is switched to "unseen device," performance drops sharply. This conclusion has clear methodological implications: in modeling research oriented towards online monitoring deployments, random partitioning or leave-one-out-of-sample precision is insufficient to assess the model's cross-device generalization ability.

[0054] S106. Time Drift Identification and Impact Analysis From the differences in target distribution and spectral shape, there is a significant time drift between C1 and C2: the mean CODMn of C1 is about 1.072 mg / L higher than that of C2, and the average RMSE among the mean spectra of each device reaches 0.144, indicating that time change not only alters the target value level but also changes the overall structure of the spectral response. Intertemporal migration validation further confirms that the empirical three-index model has R² values ​​of [missing values] in the C1→C2 and C2→C1 directions, respectively. 1.050 and The intertemporal results for Lasso and RF_RFE20 are even worse at 0.297, indicating that the unified model cannot achieve stable intertemporal predictions under the current data scale and period range.

[0055] like Figure 5 As shown, the average spectral curves of C1 and C2 have similar overall shapes, but there is a continuous shift in the main band range. This indicates that the difference between the two observations is not only reflected in the change of the target value level, but also in the systematic change of the spectral background, which provides direct evidence for subsequent drift identification.

[0056] S107. Robust Modeling and Drift Mitigation Table 3 shows the comparison results of preprocessing and drift correction. It is evident that conventional spectral preprocessing failed to provide a stable and consistent improvement in LOGO accuracy. The empirical three-exponential model and Lasso performed relatively better under the raw spectrum condition; SNV, first derivative, and detrending all led to performance degradation under most model combinations; PLSR showed a slight improvement under the first derivative condition, but the improvement was limited. Particularly noteworthy is that the empirical three-exponential model exhibited numerical instability under the detrending condition (R² = ...). The result (786.553, RMSE=41.494 mg / L) is due to the fact that detrending alters the relative relationship between wave bands, causing the denominator in the empirical index to approach zero and resulting in an abnormally amplified predicted value. This result indicates that detrending is structurally incompatible with the current empirical index structure, rather than simply being ineffective.

[0057] Table 3

[0058] Figure 6 The paper presents a comparison of LOGO RMSE under different preprocessing methods. The phased modeling experiment shows that phased modeling can only reduce the error locally in the C2 subset and fails to significantly outperform the overall performance of the unified model. This further proves that time drift originates from the joint change in the mapping relationship between the two phases, rather than a simple mean shift. Based on the evidence from these three aspects, time drift is a real factor restricting the reliability of the model across phases in the scenario presented in this paper, and cannot be simply regarded as technical noise that can be eliminated through input transformation.

[0059] The robust modeling strategy proposed in this embodiment is as follows: Table 4 shows the top 8 robust bands obtained from Lasso cross-fold frequency statistics. The Lasso cross-fold frequency statistics show that the high-frequency selected bands are mainly concentrated around 221.400, 235.700, 307.100, 871.400, and 950.000 nm, exhibiting a distribution characteristic involving both near-ultraviolet and near-infrared regions. The near-ultraviolet region may reflect colored dissolved organic matter and related absorption effects, while the near-infrared region is related to particle scattering, turbidity background, and changes in water structure. It is noteworthy that the selection frequencies of some bands differ significantly between C1 and C2: 221.400 nm and 871.400 nm have extremely high frequencies in the overall sample and C1, but are almost zero in C2; 878.600 nm is prominent in C2, indicating a local response difference between the two periods. This also suggests that selecting bands solely based on the best results from a single full sample carries the risk of misjudging local random relationships as stable patterns. It should be noted that the C1 / C2 frequencies in Table 2 show the differences in the selection distribution of each robust band in the two periods. The main ranking of robust bands in this paper is still based on the overall stability performance.

[0060] Table 4

[0061] S108. Results Output and Conclusions The LOGO model serves as a standard for evaluating real-world deployment capabilities: Rigorous validation results using the LOGO-based framework demonstrate that the triple-exponential quadratic empirical model achieves optimal results under current data conditions (R²=0.446, RMSE=1.101 mg / L, MAE=0.900 mg / L), outperforming the best machine learning model, Lasso (R²=0.253, RMSE=1.278 mg / L). In scenarios with small sample sizes and multiple devices, the structurally constrained empirical model exhibits more stable cross-device generalization capabilities than high-degree-of-freedom machine learning models.

[0062] Time drift is a real-world factor constraining intertemporal stability: significant time drift exists between samples C1 and C2, with a difference in mean CODMn of up to [percentage missing]. The concentration was 1.072 mg / L, and the RMSE of the mean spectral shape was 0.144. Conventional spectral preprocessing (SNV, first derivative, detrending, mean correction) could not fundamentally eliminate this drift, and the improvement effect of phased modeling was also relatively limited. This indicates that the drift originates from the combined effect of changes in water composition and structure and the relationship between band response, and is a real-world factor that restricts the stability of the model across phases.

[0063] In the case of small sample size and multiple devices, the structural constraint empirical model is more robust than the high degree of freedom complex model: the stability constraint three-exponential model based on Lasso cross-fold frequency (R²=0.422, RMSE=1.124 mg / L) trades a limited loss of accuracy for a more stable band interpretation structure, providing an operable path for robust modeling of weak spectral activity parameters in small sample online monitoring scenarios.

[0064] This embodiment uses Lasso cross-fold frequency statistics for robust band data screening. An alternative is to perform feature selection based on the entire sample in one go. One-time selection may select accidental features that are only effective for a specific subset of devices, resulting in poor stability. Cross-fold frequency statistics can screen out robust features that are not sensitive to data partitioning, which is better than the one-time selection method. This embodiment uses an empirical model as a robust baseline, but an alternative is to directly use the most efficient complex machine learning model. Advantages and disadvantages: Complex models may have higher accuracy under lenient validation, but under strict LOGO validation, their performance is often much lower than that of empirical models, resulting in poor generalization robustness. Therefore, in scenarios with small samples and multiple devices, empirical models are superior.

[0065] In summary, this embodiment used 20 online hyperspectral monitoring devices to collect 240 valid spectral-CODMn paired data points, divided into two batches: August-September (C1 period) and October (C2 period). Using this approach, LOGO validation was first performed, revealing that the three-exponential empirical model had an R² of 0.446, while the random forest model's R² plummeted from 0.846 (leave-one-out at the sample level) to 0.011. Analysis showed a 1.072 mg / L difference in the mean CODMn values ​​between C1 and C2 periods, with an average spectral RMSE of 0.144, and a negative R² value in the inter-period migration validation, confirming significant time drift. Conventional preprocessing failed to improve LOGO performance. Figure 7 As shown, robust wavelength bands such as 221.4nm and 871.4nm were selected through Lasso frequency filtering. The constructed stability-constrained three-exponential model has an R² of 0.422 under LOGO conditions, maintaining good robustness. Ultimately, the stability-constrained three-exponential model was selected as the deployment model for in-situ CODMn inversion in this region. Its evaluation results accurately reflect the expected performance on the new equipment. The implementation process of this embodiment is as follows: Figure 8 As shown.

[0066] Based on the same technological concept, such as Figure 9 As shown, this disclosure also proposes an in-situ hyperspectral inversion system for CODMn in river water, which includes: The data acquisition module is used to collect in-situ hyperspectral data of the river water in the study area and CODMn measured label data through several in-situ hyperspectral monitoring devices set up in the study area; The framework building module is used to build a validation framework that includes multiple model validation methods; The model training module is used to train the preset empirical model and machine learning model in parallel based on the in-situ hyperspectral data and CODMn measured label data, and to compare the performance of the trained models based on the verification framework, and select the model with the best performance as the target model. The data inversion module is used to perform in-situ hyperspectral inversion of CODMn in river water based on the target model.

[0067] like Figure 10 As shown, this disclosure also proposes a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement at least the above-described in-situ hyperspectral inversion method for CODMn in river water bodies.

[0068] This disclosure also proposes a computer program product stored in a computer-readable storage medium, which, when executed by a processor, is used to at least implement the above-described in-situ hyperspectral inversion method for CODMn in river water bodies.

[0069] Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for in-situ hyperspectral inversion of CODMn in river water, characterized in that, include: In-situ hyperspectral data and CODMn measured label data of the river water in the study area were collected by several in-situ hyperspectral monitoring devices set up in the study area. Construct a validation framework that includes multiple model validation methods; Based on the in-situ hyperspectral data and CODMn measured label data, the preset empirical model and machine learning model are trained in parallel, and the performance of the trained models is compared based on the verification framework, and the model with the best performance is selected as the target model. In-situ hyperspectral inversion of CODMn in river water was performed based on the target model.

2. The in-situ hyperspectral inversion method for CODMn in river water as described in claim 1, characterized in that, The model validation methods corresponding to the validation framework include: random partitioning validation, sample-level leave-one-out cross-validation, device-level leave-one-out cross-validation, and inter-period migration validation.

3. The in-situ hyperspectral inversion method for CODMn in river water as described in claim 1, characterized in that, The empirical model includes at least one of the following: a single-band linear model, an NDSI linear model, and an exponential quadratic empirical model; the machine learning model includes at least one of the following: a Lasso model, a Ridge model, a PLSR model, a random forest model, and an SVR model.

4. The in-situ hyperspectral inversion method for CODMn in river water as described in claim 1, characterized in that, Before performing in-situ hyperspectral inversion of CODMn in river water based on the target model, the process also includes: dividing the in-situ hyperspectral data and the measured label data of CODMn into time periods and identifying time drift, and correcting the target model based on the time drift identification results.

5. The in-situ hyperspectral inversion method for CODMn in river water as described in claim 4, characterized in that, The time drift identification includes: identifying the differences in data distribution between different periods, identifying the differences in the curve structure of the corresponding spectral curves of data between different periods, and identifying the impact of cross-period migration of data between different periods on the stability of the model across periods.

6. The in-situ hyperspectral inversion method for CODMn in flowing water as described in claim 4 or 5, characterized in that, Before performing parallel training of the preset empirical model and machine learning model based on the in-situ hyperspectral data and CODMn measured label data, the following steps are also included: The in-situ hyperspectral data is subjected to robust band data screening; the robust band data are the data corresponding to bands that are stable across devices and across periods.

7. The in-situ hyperspectral inversion method for CODMn in flowing water as described in claim 6, characterized in that, Robust band data screening of the in-situ hyperspectral data includes: robust band data screening of the in-situ hyperspectral data based on the Lasso cross-fold frequency statistical method.

8. A river water CODMn in-situ hyperspectral inversion system, characterized in that, include: The data acquisition module is used to collect in-situ hyperspectral data of the river water in the study area and CODMn measured label data through several in-situ hyperspectral monitoring devices set up in the study area; The framework building module is used to build a validation framework that includes multiple model validation methods; The model training module is used to train the preset empirical model and machine learning model in parallel based on the in-situ hyperspectral data and CODMn measured label data, and to compare the performance of the trained models based on the verification framework, and select the model with the best performance as the target model. The data inversion module is used to perform in-situ hyperspectral inversion of CODMn in river water based on the target model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, which, when executed by a processor, are used to implement at least the in-situ hyperspectral inversion method for CODMn in river water bodies as described in any one of claims 1-7.

10. A computer program product, said computer program product being stored in a computer-readable storage medium, characterized in that, When the computer program product is executed by a processor, it is used to implement at least the in-situ hyperspectral inversion method for CODMn in river water bodies as described in any one of claims 1-7.

Images