A method for tracing pollution of a water body based on DOM

Abstract

The application discloses a DOM-based pollution water body tracing method, and belongs to the technical field of water environment pollution tracing and fluorescence spectrum data analysis. The method collects three-dimensional fluorescence matrices of to-be-detected samples and end member samples, extracts low-dimensional features through pretreatment, full-spectrum two-dimensional development and principal component analysis, trains a classification model to realize source identification, constructs a mixed inversion model in a spectral series characteristic space, and completes qualitative identification and quantitative tracing. The method fully utilizes full-spectrum information, does not need artificial component setting and experience interpretation, can automatically process in batches, realizes quantitative contribution of mixed pollution sources, solves the defects of insufficient information utilization, dependence on manual operation, difficulty in quantitative analysis and weak standardization of traditional methods, and has the advantages of high precision, simple process, strong stability and suitability for watershed scale application.

Description

Technical Field

[0001] This invention relates to the field of water pollution source tracing and spectral data processing technology, and in particular to a method for tracing polluted water bodies based on DOM. Background Technology

[0002] Dissolved organic matter (DOM) is widely found in rivers, lakes, reservoirs, groundwater, and various wastewater systems, serving as a crucial medium for pollutant migration, microbial metabolism, and carbon cycling. Due to the wide range of DOM sources and their complex composition, different pollution sources often overlap and mix significantly after entering water bodies, posing challenges to watershed pollution identification and liability determination.

[0003] Current methods for tracing water pollution sources based on DOM mainly employ fluorescence index and PARAFAC decomposition methods, which present the following technical problems in practical applications: Insufficient information utilization: Relying only on a few fluorescence indices such as FI, BIX, and HIX or a limited number of components, it is impossible to characterize the overall spectral differences of DOM, and the ability to distinguish between multiple sources mixed with similar sources is limited; Reliance on human experience and low degree of automation: PARAFAC requires repeated manual setting of component numbers, verification of model consistency and manual interpretation of components, which is cumbersome, highly subjective and difficult to scale up. The ability to analyze mixed pollution is weak: traditional methods are mostly applicable to the identification of single sources and are difficult to directly quantify the contribution ratio of each pollution source in mixed water bodies. Poor engineering applicability: Parameters and models need to be readjusted for different river basins, different batches, and different water periods, making it impossible to form a standardized and automated traceability process. Summary of the Invention

[0004] This invention provides a method for tracing the source of polluted water bodies based on DOM (Discretionary Origin and Determination), which solves the problems of low information utilization rate, reliance on human experience, weak quantitative ability of mixed sources, and insufficient standardization in existing technologies, and realizes automated qualitative identification and accurate quantitative tracing of multi-source mixed polluted water bodies.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: This invention provides a method for tracing the source of polluted water bodies based on DOM (Domain-Oriented Analysis), comprising: S1: Collect water samples to be tested and end-member samples from known pollution sources, and obtain three-dimensional fluorescence excitation-emission matrix data for each sample; S2: The three-dimensional fluorescence matrix is ​​preprocessed by scattering removal, internal filtering effect correction, normalization and unified wavelength grid interpolation to obtain a standardized matrix; S3: Expand the normalized matrix into a two-dimensional feature matrix in the order of excitation wavelength - emission wavelength pixels; S4: Perform principal component analysis on the two-dimensional feature matrix to extract low-dimensional features that characterize the overall spectral differences of the DOM and construct the DOM spectral feature space; S5: Input the low-dimensional features corresponding to the known pollution source end-member samples into the classification model to train the pollution source identification model; S6: Input the low-dimensional features of the water sample to be tested into the pollution source identification model to complete the qualitative identification of the pollution source; S7: Based on the low-dimensional features corresponding to the DOM spectrum feature space and the known pollution source end-member samples, a mixed source contribution inversion model is constructed to calculate the contribution ratio of each pollution source in multi-source polluted water bodies. S8: Output the pollution source identification results and the proportion of pollution source contribution, and present them in a visual representation.

[0006] Furthermore, the known pollution source end-member samples mentioned in step S1 include at least one of domestic sewage, industrial wastewater, agricultural runoff, aquaculture wastewater, and natural background water.

[0007] Furthermore, the preprocessing described in step S2 includes blank subtraction, instrument baseline correction, Rayleigh and Raman scattering shielding or interpolation replacement, internal filter effect correction, Raman unit normalization, anomalous noise suppression, and uniform wavelength grid resampling.

[0008] Furthermore, in step S3, the normalized matrix is ​​expanded row by row or column by column to preserve the complete DOM spectral topology, peak position shift, and relative intensity distribution information.

[0009] Furthermore, in step S4, principal component analysis extracts low-dimensional features that explain 85% to 95% of the cumulative variance.

[0010] Furthermore, the classification model mentioned in step S5 is one or more of the following: support vector machine, random forest, extreme gradient boosting, K-nearest neighbors, Naive Bayes, and neural network.

[0011] Furthermore, in step S7, the hybrid source contribution inversion model is solved using least squares optimization with non-negativity constraints and total quantity conservation constraints.

[0012] Compared with the prior art, the technical solution disclosed in this invention has the following beneficial effects: Through standardized preprocessing in step S2, full-spectrum two-dimensional expansion in step S3, and principal component feature extraction in step S4, the three-dimensional fluorescence full-spectrum information of the DOM is fully preserved, which fully characterizes the overall spectral differences and significantly improves the differentiation accuracy of water bodies with similar sources and mixed pollution. Through the construction of a mixed source contribution inversion model in step S7, the contribution ratio of each pollution source can be directly output, realizing qualitative identification and quantitative source tracing at the same time, meeting the needs of pollution investigation and responsibility determination. Through steps S1-S8, an integrated fully automated process is formed, which does not require manual setting of component numbers, repeated model verification, or manual interpretation of components, greatly reducing manual intervention, improving batch processing efficiency, and is suitable for watershed-scale application. Through unified preprocessing, unified feature extraction, and unified classification inversion, a standardized framework for the entire process is formed, which does not require repeated parameter tuning across watersheds, water periods, and batches, has strong transferability and high stability, and is suitable for long-term operational use. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the polluted water source tracing method based on DOM provided in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] This invention provides a method for tracing the source of polluted water bodies based on DOM (Discretionary Origin and Determination), which can solve the problems of low information utilization, reliance on human experience, weak quantitative ability of mixed sources, and insufficient standardization, and realize the automated qualitative identification and accurate quantitative tracing of multi-source mixed polluted water bodies.

[0018] like Figure 1 As shown, this embodiment of the invention provides a method for tracing the source of polluted water bodies based on DOM, including: S1: Collect water samples to be tested and end-member samples from known pollution sources, and obtain three-dimensional fluorescence excitation-emission matrix data for each sample.

[0019] The known pollution source end-member samples mentioned in step S1 include at least one of domestic sewage, industrial wastewater, agricultural runoff, aquaculture wastewater, and natural background water.

[0020] That is, step S1: Sample collection and three-dimensional fluorescence matrix construction. Specifically, this includes: collecting water samples to be tested and end-member samples from known pollution sources, wherein the end-member samples include at least domestic sewage, industrial wastewater, agricultural runoff, aquaculture wastewater, natural background water, or combinations thereof; using a fluorescence spectrometer to obtain the fluorescence response values ​​of each sample under a preset excitation wavelength set Ex={ex1, ex2, …, exm} and an emission wavelength set Em={em1, em2, …, emn}, forming a three-dimensional fluorescence matrix X∈R^(m×n) for a single sample. Here, the matrix element x_ij represents the fluorescence intensity under the combination of the i-th excitation wavelength and the j-th emission wavelength.

[0021] S2: The three-dimensional fluorescence matrix is ​​preprocessed by scattering removal, internal filtering effect correction, normalization and unified wavelength grid interpolation to obtain a standardized matrix.

[0022] The preprocessing described in step S2 includes blank subtraction, instrument baseline correction, Rayleigh and Raman scattering shielding or interpolation replacement, internal filter effect correction, Raman unit normalization, anomalous noise suppression, and uniform wavelength grid resampling.

[0023] That is, step S2: spectral domain correction and normalization preprocessing. Specifically, this includes: sequentially performing blank subtraction, instrument baseline correction, Rayleigh and Raman scattering region shielding or interpolation replacement, internal filtering effect correction, Raman unit normalization, anomalous noise suppression, and unified wavelength grid resampling on the three-dimensional fluorescence matrix X to obtain the normalized matrix X′. Preferably, the internal filtering effect correction can be expressed as: X_ifc(ex,em)=X_raw(ex,em)×10^((A_ex+A_em) / 2), where A_ex and A_em represent the absorbance of the sample at the excitation and emission wavelengths, respectively; after normalization, the dimensionless matrix X_norm=X′ / ‖X′‖_F can be further obtained, where ‖·‖_F is the Frobenius norm. Through the above processing, the interference of different batches, different concentrations, and different instrument drifts on spectral shape comparison can be eliminated.

[0024] S3: Expand the normalized matrix into a two-dimensional feature matrix in the order of excitation wavelength - emission wavelength pixels.

[0025] In step S3, the normalized matrix is ​​expanded row by row or column by column to preserve the complete spectral topology, peak position shift and relative intensity distribution information of the DOM.

[0026] That is, step S3: DOM full-spectrum vectorization encoding. Specifically, this includes: expanding the preprocessed matrix X_norm row by row or column by column according to a preset excitation-emission order to generate a one-dimensional spectral vector v=[x_11, x_12, …, x_1n, x_21, …, x_mn]^T∈R^(mn); for N samples, constructing a full-spectrum feature matrix V∈R^(N×mn). Unlike methods that only extract fluorescence index or local peak features, this step preserves the overall topological information, peak position migration information, and relative intensity distribution information of the complete DOM spectrum, providing a high-dimensional information foundation for subsequent complex source discrimination.

[0027] S4: Perform principal component analysis on the two-dimensional feature matrix to extract low-dimensional features that characterize the overall spectral differences of the DOM and construct the DOM spectral feature space.

[0028] In step S4, principal component analysis extracts low-dimensional features that explain 85% to 95% of the cumulative variance.

[0029] That is, step S4: Feature compression and spectral space construction. Specifically, this includes: centering and standardizing matrix V, extracting low-dimensional latent variables using principal component analysis (PCA), partial least squares (PLS), kernel principal component analysis (KPCA), or manifold learning methods, and establishing the DOM spectral feature space Z. Preferably, PCA can be expressed as: Z = V_cP, where V_c is the centered feature matrix, and P is the loading matrix; the cumulative explained variance satisfies Σ_{k=1}^K λ_k / Σ_{k=1}^M λ_k ≥ η, where η is preferably 85%–95%. Through this step, the original high-dimensional spectrum is mapped into a compact feature subspace that can characterize source differences, and noise and redundant variables are compressed.

[0030] S5: Input the low-dimensional features corresponding to the known pollution source end-member samples into the classification model to train the pollution source identification model.

[0031] The classification model mentioned in step S5 is one or more of the following: support vector machine, random forest, extreme gradient boosting, K-nearest neighbors, Naive Bayes, and neural network.

[0032] That is, step S5: Source discrimination model construction. Specifically, this includes: using the coordinates of known endmember samples in the spectral feature space Z as the training dataset, and inputting it into a supervised learning classifier to establish a pollution source identification model f(z). The classifier can be one or more of support vector machines, random forests, extreme gradient boosting, K-nearest neighbors, Naive Bayes, or neural networks. Preferably, for multi-class support vector machines, endmember category separation can be achieved by solving the objective function min (1 / 2)‖w‖² + CΣξ_i, under the constraints y_i(w·φ(z_i)+b)≥1-ξ_i, ξ_i≥0; for probabilistic classifiers, the posterior probability P(c_j|z) of the unknown sample belonging to each endmember category is output.

[0033] S6: Input the low-dimensional features of the water sample to be tested into the pollution source identification model to complete the qualitative identification of the pollution source.

[0034] That is, step S6: Identification of unknown sample sources and similarity calculation. Specifically, this includes: mapping the unknown sample to the spectral feature space after processing in steps S1-S4, inputting it into the source discrimination model f(z_u), and obtaining its dominant pollution source category, candidate source ranking, and corresponding probability distribution. Further, to enhance the interpretability of the results, the Mahalanobis distance, Euclidean distance, or cosine similarity between the unknown sample and the centers of various endmembers can be calculated to form an auxiliary discrimination index D_j=(z_u-μ_j)^TΣ_j^(-1)(z_u-μ_j), where μ_j and Σ_j represent the center vector and covariance matrix of the j-th endmember in the feature space, respectively.

[0035] S7: Based on the low-dimensional features corresponding to the DOM spectral feature space and the known pollution source end-member samples, a mixed source contribution inversion model is constructed to calculate the contribution ratio of each pollution source in multi-source polluted water bodies.

[0036] In step S7, the hybrid source contribution inversion model is solved using least squares optimization with non-negativity constraints and total amount conservation constraints.

[0037] That is, step S7: Inversion of contributions from mixed pollution sources. Specifically, this includes: for cases where an unknown sample may be simultaneously affected by multiple pollution sources, establishing an endmember mixing model z_u≈Σ_{j=1}^p α_j z_j in the spectral feature space, and solving for the contribution coefficient α_j of each pollution source. Preferably, the contribution coefficients are obtained through constrained non-negative least squares optimization, i.e., solving the objective function: min ‖z_u-Eα‖² + λ‖α‖1, satisfying α_j≥0 and Σα_j=1; where E is the endmember feature matrix, α=[α_1, α_2, …, α_p]^T is the contribution vector to be estimated, and λ is the regularization coefficient. By introducing non-negativity constraints and total amount conservation constraints, the physical rationality and numerical stability of the contribution allocation results under multi-source mixing can be significantly improved.

[0038] S8: Output the pollution source identification results and the proportion of pollution source contribution, and present them in a visual representation.

[0039] That is, step S8: Result output and visualization. Specifically, this includes: outputting the main pollution source categories, candidate source probabilities, contribution ratios of each pollution source, and their confidence intervals for the sample to be tested, and projecting them into a two-dimensional or three-dimensional feature space; preferably, it can also output the sample-endmember distance matrix, classification confusion matrix, mixture inversion residual R=‖z_u-Eα̂‖2, and contribution change maps of different pollution sources on time series or spatial sections, to support watershed-scale pollution investigation, abnormal sample identification, and pollution process tracking.

[0040] Furthermore, embodiments of the present invention can also perform feature-level fusion or decision-level fusion of the DOM full-spectrum features with water quality parameters, absorption spectral parameters, fluorescence index parameters, isotope parameters or microbial community information to construct a multimodal joint source tracing model; however, even when using only DOM-EEM full-spectrum data, the present invention can still complete the source identification and quantitative mixing contribution of complex polluted water bodies.

[0041] In summary, the embodiments of the present invention do not rely on empirical interpretation of a single fluorescence peak, a single index, or a few components. Instead, they establish a systematic source tracing method based on the full spectrum information of the DOM, which combines high-dimensional spectral characterization, low-dimensional spectral mapping, supervised classification and discrimination, and constrained contribution inversion. This method can more fully release the information carrying capacity of EEM data and significantly improve the automation, precision, and engineering applicability of source tracing analysis of complex polluted water bodies.

[0042] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The embodiments of the present invention directly use EEM full-spectrum pixel information, rather than relying on only a few indices or a few components, thereby improving the retention of source differences, and are particularly suitable for polluted water bodies with similar sources and significant mixing; (2) The embodiments of the present invention avoid the process of repeated screening of components, interpretation of components and repeated verification of model consistency in the PARAFAC method, which significantly shortens the analysis process, reduces the degree of human intervention and improves the efficiency of batch analysis; (3) The embodiments of the present invention simultaneously realize two functions: "source identification + contribution quantification". It can output both the pollution source category and the mixed contribution ratio, which is more in line with the engineering requirements of watershed pollution investigation and responsibility identification. (4) The embodiments of the present invention are based on unified preprocessing, unified wavelength grid, unified feature extraction and unified classification process. The method has a high degree of standardization and is easy to migrate and apply in different water periods, different watersheds or different monitoring batches.

[0043] To further illustrate the technical advancements of this invention, the key obstacles in the prior art and the solutions provided by this invention are described below: Relying solely on a few indicators such as FI, BIX, and HIX Complex mixed samples lack discriminative power and are prone to misclassification when samples come from similar sources. The EEM is used for two-dimensional expansion to preserve full-spectrum information, and then PCA is used to extract main features. PARAFAC requires repeated setting of group numbers and conducting consistency verification. The process is lengthy, requires significant manual intervention, and results in low efficiency for batch analysis. Bypassing component decomposition, directly perform full-spectrum feature modeling. Traditional methods tend to focus on single-source identification. It is difficult to output the source proportion of mixed pollution. Constructing a hybrid source contribution inversion model in the principal component space Parameters often need to be reinterpreted for different batches of samples. Insufficient reproducibility and standardization of the method Establish a unified preprocessing-expansion-dimensionality reduction-classification-inversion process Specific Implementation

[0044] The following example, using the Liaohe River Basin as a case study, further illustrates the method provided by the embodiments of the present invention, but the scope of protection of the present invention is not limited to these embodiments.

[0045] Example: Source tracing of DOM in polluted water bodies in the Liaohe River Basin.

[0046] Representative pollution source end-member samples and river water samples were selected from the Liaohe River Basin. End-member samples may include domestic sewage, industrial wastewater, agricultural runoff, and natural background water samples. EEM data were measured for each sample, and preprocessing, two-dimensional expansion, principal component extraction, classification modeling, and mixed source inversion were completed according to steps S1-S8.

[0047] Traditional technical approaches typically require calculating fluorescence indices or establishing a PARAFAC model first, followed by manual interpretation of components before source identification can be performed. For mixed samples, further contribution analysis using empirical rules or additional models is also necessary. This process suffers from numerous modeling steps, significant human intervention, and high reliance on operator experience.

[0048] After adopting the method of this invention, all samples enter a unified "preprocessing-expansion-PCA-classification-inversion" process, eliminating the need for repeated screening of PARAFAC components and for first naming and interpreting each individual component. The source classification results and contribution ratios can be obtained directly. Therefore, in application scenarios such as the Liaohe River Basin, where sources are complex, mixing is significant, and sample batches are large, this invention has significant advantages in terms of process standardization, automation, and batch analysis efficiency. (a) The total analysis time required to complete source identification for a single batch of samples is reduced; (b) The number of steps requiring human intervention is reduced; (c) The number of repeated training sessions or parameter adjustments for the model is reduced; (d) The number of additional analytical steps required to output quantitative contribution results from mixed samples is reduced.

[0049] The embodiments of the present invention significantly reduce the time required for batch sample analysis by reducing the steps of component screening, component interpretation and repeated modeling, and improve the automation efficiency of DOM source identification and mixed contribution analysis in complex polluted water bodies.

[0050] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0051] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0052] It should also be noted that in the apparatus, device, and method of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of the present invention.

[0053] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0054] It should be understood that the qualifying terms "first", "second", "third", "fourth", "fifth" and "sixth" used in the description of the embodiments of the present invention are only used to more clearly illustrate the technical solutions and are not intended to limit the scope of protection of the present invention.

[0055] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for tracing the source of polluted water bodies based on DOM, characterized in that, include: S1: Collect water samples to be tested and end-member samples from known pollution sources, and obtain three-dimensional fluorescence excitation-emission matrix data for each sample; S2: The three-dimensional fluorescence matrix is ​​preprocessed by scattering removal, internal filtering effect correction, normalization and unified wavelength grid interpolation to obtain a standardized matrix; S3: Expand the normalized matrix into a two-dimensional feature matrix in the order of excitation wavelength - emission wavelength pixels; S4: Perform principal component analysis on the two-dimensional feature matrix to extract low-dimensional features that characterize the overall spectral differences of the DOM and construct the DOM spectral feature space; S5: Input the low-dimensional features corresponding to the known pollution source end-member samples into the classification model to train the pollution source identification model; S6: Input the low-dimensional features of the water sample to be tested into the pollution source identification model to complete the qualitative identification of the pollution source; S7: Based on the low-dimensional features corresponding to the DOM spectrum feature space and the known pollution source end-member samples, a mixed source contribution inversion model is constructed to calculate the contribution ratio of each pollution source in multi-source polluted water bodies. S8: Output the pollution source identification results and the proportion of pollution source contribution, and present them in a visual representation.

2. The method according to claim 1, characterized in that, The known pollution source end-member samples mentioned in step S1 include at least one of domestic sewage, industrial wastewater, agricultural runoff, aquaculture wastewater, and natural background water.

3. The method according to claim 1, characterized in that, The preprocessing described in step S2 includes blank subtraction, instrument baseline correction, Rayleigh and Raman scattering shielding or interpolation replacement, internal filter effect correction, Raman unit normalization, anomalous noise suppression, and uniform wavelength grid resampling.

4. The method according to claim 1, characterized in that, In step S3, the normalized matrix is ​​expanded row by row or column by column to preserve the complete spectral topology, peak position shift and relative intensity distribution information of the DOM.

5. The method according to claim 1, characterized in that, In step S4, principal component analysis extracts low-dimensional features that explain 85% to 95% of the cumulative variance.

6. The method according to claim 1, characterized in that, The classification model mentioned in step S5 is one or more of the following: support vector machine, random forest, extreme gradient boosting, K-nearest neighbors, Naive Bayes, and neural network.

7. The method according to claim 1, characterized in that, In step S7, the hybrid source contribution inversion model is solved using least squares optimization with non-negativity constraints and total amount conservation constraints.

Images