A Machine Learning-Based Fluorescence Spectroscopic Image Diagnosis Method for Water Pollution
By combining a three-dimensional fluorescence spectrometer with a CNN-LSTM model, the problems of long detection cycles and low accuracy in water environment monitoring have been solved, enabling rapid and accurate identification and on-site monitoring of water pollution, and supporting multi-level early warning and pollution source tracing.
Patent Information
- Application Number
- CN202510591606.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies for water environment monitoring suffer from problems such as long detection cycles, low accuracy, high cost, and poor portability, making it difficult to achieve rapid and accurate identification and real-time on-site monitoring of water pollution.
Spectral data were acquired using a three-dimensional fluorescence spectrometer. Preprocessing was performed using wavelet transform, adaptive baseline correction, and vector normalization. Data analysis was conducted using a CNN-LSTM hybrid model to construct pollution indicators and generate diagnostic reports.
It enables rapid and accurate identification and on-site monitoring of water pollution, reduces the detection cycle, improves data quality and model analysis capabilities, and supports multi-level early warning and pollution source tracing.
Smart Images

Figure CN123066038A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of environmental monitoring technology, and in particular to a machine learning-based method for diagnosing water pollution using fluorescence spectral images. Background Technology
[0002] In the field of water environment monitoring, dissolved organic matter (DOM) in water is a core parameter reflecting water quality health, and its composition and concentration changes are directly related to the degree of water pollution and ecological risk assessment. However, current mainstream detection methods, such as chemical analysis and chromatography-mass spectrometry, generally face multiple technical bottlenecks: On the one hand, these methods rely on cumbersome sample pretreatment processes (such as extraction, concentration, and derivatization), with a single detection cycle lasting several hours to several days, making it difficult to meet the emergency response needs of sudden pollution events; on the other hand, DOM in the aquatic environment usually presents a complex mixture of multiple components (such as proteins, humic acids, and microbial metabolites), and traditional technologies are limited by insufficient spectral resolution or low chemical separation efficiency, making it difficult to accurately analyze the pollution contribution of each component, and are easily affected by background interference factors such as water color, turbidity, and pH value, leading to deviations in characteristic signal extraction. In addition, existing technologies generally rely on large laboratory equipment and professional technicians, resulting in high hardware costs and poor portability, making it impossible to achieve real-time on-site monitoring and dynamic tracking of water pollution. With the increasing diversification of new pollution sources such as industrial wastewater and agricultural non-point source pollution, the limitations of existing technologies in terms of pollution type identification timeliness, multi-component analysis accuracy, and environmental adaptability are becoming increasingly prominent. There is an urgent need to innovate a DOM pollution diagnosis technology that integrates rapid detection, intelligent analysis, and anti-interference capabilities to overcome the multiple constraints of traditional methods in terms of efficiency, accuracy, and application scenarios, and to provide key technical support for the intelligent and precise development of water environment monitoring. Summary of the Invention
[0003] The embodiments of the present invention provide a machine learning-based fluorescent spectral image diagnostic method for water pollution, which achieves rapid and accurate identification of DOM pollution through spatiotemporal joint modeling of spectral features.
[0004] To achieve the above objectives, this invention provides a machine learning-based method for diagnosing water pollution using fluorescence spectral images, comprising the following steps:
[0005] The spectral data of water samples were collected using a three-dimensional fluorescence spectrometer to obtain a three-dimensional fluorescence spectral matrix with excitation wavelength a and emission wavelength b.
[0006] The three-dimensional fluorescence spectral matrix is preprocessed by wavelet transform denoising, adaptive baseline correction, and vector normalization.
[0007] Principal component analysis was used to extract the principal components of the three-dimensional fluorescence spectral matrix and construct the spectral feature vector.
[0008] The spectral feature vector is input into a predetermined CNN-LSTM hybrid model, which outputs pollution indicators, including pollution level, component type and concentration prediction value.
[0009] The pollution index is calculated using the pollution indicators. When the comprehensive pollution index exceeds a preset threshold, a pollution warning is triggered and a diagnostic report is generated.
[0010] Furthermore, the excitation wavelength a is 200-450nm, and the emission wavelength b is 250-550nm.
[0011] Furthermore, the wavelet transform is performed using the db4 wavelet basis function for a 3-level decomposition, and the baseline correction is performed using an adaptive iterative reweighting algorithm to fit a polynomial baseline.
[0012] Furthermore, the calculation formula for the vector normalization algorithm is as follows:
[0013]
[0014] In the formula, A ij C represents the fluorescence intensity value corresponding to the i-th excitation wavelength and the j-th emission wavelength in the three-dimensional fluorescence spectral matrix. ij B represents the fluorescence intensity value after normalization and background subtraction; ij Σ(A) represents the fluorescence intensity value of the blank background at the i-th excitation wavelength and the j-th emission wavelength; ij -B ij ) 2 This represents the magnitude of the vector calculated by summing the squares of all fluorescence intensity values after background subtraction.
[0015] Furthermore, the CNN-LSTM hybrid model includes:
[0016] Data preprocessing: The preprocessed three-dimensional fluorescence spectral matrix is converted into a model-compatible tensor format. For time series features, the spectral matrices of consecutive time steps are concatenated into a sequence that includes the time dimension.
[0017] A convolutional neural network module is constructed to extract the spatial features of the spectrum: the spectral matrix is scanned by multiple two-dimensional convolutional layers, and nonlinearity is introduced by using the ReLU activation function to capture the position, intensity distribution and spatial correlation patterns of fluorescence peaks; after each convolutional layer, a max pooling layer is connected to reduce the feature dimension, and a flattening layer is used to convert the multidimensional feature map into a one-dimensional vector;
[0018] The output one-dimensional vector is reshaped into a temporal sequence and input into a long short-term memory network module: the dynamic evolution of spectral spatial features over time is captured through a two-layer LSTM layer. The first LSTM layer is used to preserve temporal information, and the second layer is used to extract long-term dependencies. The temporal features are integrated through a fully connected layer to provide a comprehensive representation for multi-task output.
[0019] The design incorporates a dual-branch output layer to implement classification and regression tasks: the classification branch uses a fully connected layer combined with a Softmax activation function to output the probability distribution of each pollution level; the regression branch directly outputs the concentration values of each component through a fully connected layer.
[0020] The dataset is divided into training, validation, and test sets, and data augmentation techniques are used to increase sample diversity. The Adam optimizer is used in combination with a learning rate decay strategy to minimize the loss function, and an early stopping mechanism is used to prevent overfitting. After training, the model performance is evaluated by accuracy and / or root mean square error.
[0021] Furthermore, the comprehensive pollution index is calculated using the following formula:
[0022]
[0023] In the formula, CPI represents the comprehensive pollution index; w k D represents the preset influence weight of the k-th pollutant component; k S represents the predicted concentration of the k-th pollutant component; k This represents the national standard threshold for the k-th polluting component.
[0024] Furthermore, the process of triggering pollution warnings and generating diagnostic reports also generates a structured diagnostic report containing pollution levels, main components, and source tracing suggestions based on the proportion of each pollution component and the source association algorithm.
[0025] Furthermore, the source association algorithm determines the mapping relationship between pollutant components and pollutant sources by constructing a mapping table based on historical data.
[0026] The above technical solution has the following technical effects:
[0027] (1) A UV-Vis dual-source spectrometer is used to achieve rapid scanning of the full spectral range of 200-550nm. By repeatedly scanning, taking the mean, and subtracting the blank background, a three-dimensional fluorescence intensity matrix with a high signal-to-noise ratio is constructed. This solves the inefficient mode of existing technologies that rely on complex preprocessing procedures, compressing the single detection cycle from several hours to minutes, meeting the emergency response needs of sudden pollution events, and realizing rapid on-site screening of water pollution.
[0028] (2) High-frequency noise was removed by db4 wavelet 3-level decomposition and soft thresholding algorithm, and a polynomial baseline was fitted by adaptive iterative reweighting algorithm, followed by vector normalization to unify the feature scale. This effectively suppressed particle scattering and baseline drift interference in high-turbidity and high-chromatic water bodies, providing a clean and standardized data foundation for subsequent model analysis.
[0029] (3) The spatial distribution features of fluorescence peaks are extracted by using the multi-layer convolution and pooling operations of the CNN module, and the time-series spectral data are modeled by the LSTM module to capture the dynamic evolution law of DOM components and achieve accurate identification of multi-component mixed pollution.
[0030] (4) The pollution level is quantitatively assessed by using the comprehensive pollution index based on component weights and national standard limits, supporting multi-level early warning and pollution source inference. Attached Figure Description
[0031] Figure 1 This is a schematic flowchart of a machine learning-based fluorescence spectral image diagnosis method for water pollution according to an embodiment of the present invention.
[0032] Figure 2 This is a schematic diagram of the process of constructing a CNN-LSTM hybrid model in one embodiment of the present invention. Detailed Implementation
[0033] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0034] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.
[0035] Figure 1 This is a flowchart illustrating a machine learning-based fluorescence spectral image diagnostic method for water pollution according to an embodiment of the present invention. The method of this embodiment includes the following steps:
[0036] S1. Using a three-dimensional fluorescence spectrometer equipped with an ultraviolet-visible light source, water samples were scanned within a wide range of excitation wavelengths (200-450 nm, 5 nm intervals) and emission wavelengths (250-550 nm, 1 nm intervals). Spectral data for each sample were repeatedly acquired three times and averaged. Combined with ultrapure water background subtraction, a three-dimensional fluorescence spectral matrix containing 51 × 301 fluorescence intensity points was constructed. This process achieved full-spectrum coverage of characteristic fluorescence signals such as protein-like peaks and humic acid peaks of dissolved organic matter (DOM) in water, effectively reducing random noise interference and improving the original spectral signal-to-noise ratio to over 5:1, providing a high-fidelity raw data foundation for subsequent analysis.
[0037] S2. To address issues in the original spectra such as turbidity particle scattering, instrument electronic noise, and baseline drift caused by light source attenuation, a multi-stage preprocessing technique was employed: First, high-frequency noise was suppressed in the spectral matrix using db4 wavelet 3-level decomposition and a soft thresholding algorithm, improving the signal-to-noise ratio of high-turbidity water (SS = 200 mg / L) from 2.1:1 to 18:1; second, an adaptive iterative reweighting algorithm was used to fit a 5th-order polynomial baseline, controlling the baseline drift error to ≤2 nm and avoiding characteristic peak position shifts; finally, a vector normalization algorithm was used to eliminate the influence of sample concentration differences on fluorescence intensity and unify the feature space scale. After processing, the spectral characteristic peak position error was ≤1.5 nm, and the effective feature retention rate increased from 58% to 89%, significantly improving data quality.
[0038] In one specific implementation, the calculation formula for the vector normalization algorithm is as follows:
[0039]
[0040] In the formula, A ij C represents the fluorescence intensity value corresponding to the i-th excitation wavelength and the j-th emission wavelength in the three-dimensional fluorescence spectral matrix. ij B represents the fluorescence intensity value after normalization and background subtraction; ij The fluorescence intensity values of a blank background (such as ultrapure water) at the i-th excitation wavelength and the j-th emission wavelength; ∑(A ij -B ij ) 2 This represents the sum of squares of all fluorescence intensity values after background subtraction, used to calculate the vector magnitude.
[0041] S3. Principal Component Analysis (PCA) was used to reduce the dimensionality of the preprocessed three-dimensional spectral matrix. Through linear transformation, the multidimensional original data was compressed into the top 10 principal components with a cumulative contribution rate ≥95%, constructing a spectral feature vector containing the core fluorescence information of the DOM. This process not only removed redundant components from the data and reduced the computational complexity of the model, but also enhanced the feature vector's ability to represent pollution characteristics by highlighting the weights of key pollution indicators such as protein-like peaks and humic acid peaks. This allows subsequent machine learning models to more efficiently capture the intrinsic relationship between spectral data and the degree and type of pollution.
[0042] S4. The dimensionality-reduced spectral feature vectors are input into a hybrid Convolutional Long Short-Term Memory (CNN-LSTM) model: The CNN module extracts the spatial distribution features of fluorescence peaks (such as peak position and intensity gradient) through three 2D convolutional layers (32 / 64 / 128 channels, 3×3 convolutional kernels) and two max-pooling layers, effectively distinguishing overlapping peaks such as those of proteins and humic acids; the LSTM module models the spectral features of continuous time series through two 128-unit recurrent layers, capturing the dynamic evolution of DOM components over time (such as abrupt changes in humic acid concentration after heavy rain). The final model outputs the pollution level probability (mild / moderate / severe) and the concentrations of five main components simultaneously through the Softmax function and linear layers, achieving accurate analysis of multi-component mixed pollution.
[0043] Figure 2 This is a schematic diagram of the process for constructing a CNN-LSTM hybrid model according to one embodiment of the present invention, including:
[0044] S41. Convert the preprocessed spectral data into a model-compatible tensor format. If analyzing time series features, the spectral data from multiple consecutive time points need to be concatenated into a sequence that includes the time dimension. Simultaneously, normalize or standardize the data to eliminate dimensional differences, ensuring stable input feature distribution and providing structured and standardized input data for subsequent modeling.
[0045] S42. Construct a convolutional neural network module to extract spatial features of the spectrum: The spectral matrix is scanned by multiple two-dimensional convolutional layers (e.g., 3 layers, equipped with 32 / 64 / 128 3×3 convolutional kernels respectively). The ReLU activation function is used to introduce nonlinearity to capture the position, intensity distribution and spatial correlation patterns of fluorescence peaks. After each convolutional layer, a max pooling layer (e.g., 2×2 pooling kernel) is connected to reduce the feature dimension, reduce the amount of computation while retaining key spatial features. Finally, the multidimensional feature map is converted into a one-dimensional vector through a flattening layer to provide high-dimensional abstract features for the LSTM module.
[0046] S43. Reshape the one-dimensional feature vector output by the CNN into a temporal sequence and input it into the Long Short-Term Memory network module: capture the dynamic evolution of spectral features over time through multiple LSTM layers (e.g., 2 layers, each with 128 units). The first LSTM layer is used to preserve temporal information, and the second layer is used to extract long-term dependencies. Then, integrate the temporal features through a fully connected layer to provide a comprehensive representation for multi-task output.
[0047] S44. Design a dual-branch output layer to implement classification and regression tasks: The classification branch uses a fully connected layer (number of neurons = number of pollution levels, e.g., 3) combined with a Softmax activation function to output the probability distribution of each pollution level (e.g., mild / moderate / severe); the regression branch uses a fully connected layer (number of neurons = number of components, e.g., 5) to directly output the concentration values of DOM components (e.g., proteins, humic acids, etc.). A weighted loss function (e.g., classification loss × 0.7 + regression loss × 0.3) is used to simultaneously optimize the two tasks, improving the model's ability to jointly analyze pollution levels and component concentrations.
[0048] S45. Divide the dataset into a training set (80%), a validation set (10%), and a test set (10%). Use data augmentation techniques (such as random translation and scaling of the spectral matrix) to expand sample diversity. Use the Adam optimizer combined with a learning rate decay strategy (such as cosine annealing) to minimize the loss function. Use an early stopping mechanism (terminating training if the loss on the validation set does not decrease after 10 consecutive rounds) to prevent overfitting. After training, evaluate the model performance using metrics such as accuracy (classification task) and root mean square error (RMSE, regression task) to ensure its effectiveness and generalization ability in the spatiotemporal feature analysis of spectral data.
[0049] S5. Based on the pollution level, component type, and concentration prediction values output by the model, calculate the Comprehensive Pollution Index (CPI). A quantitative assessment of the pollution level is achieved through preset thresholds (e.g., CPI ≥ 70 triggers a Level 3 warning). When the pollution index exceeds the threshold, the system automatically triggers an audible and visual alarm. Combined with the proportion of pollution components (e.g., protein accounts for 68%) and source association algorithms (e.g., high humic acid values indicate agricultural non-point source pollution), a structured diagnostic report containing the pollution level, main components, and source tracing suggestions is generated.
[0050] In one specific implementation, the formula for calculating the comprehensive pollution index is as follows:
[0051]
[0052] In the formula, CPI represents the comprehensive pollution index; w k D represents the preset influence weight of the k-th pollutant component; k S represents the predicted concentration of the k-th pollutant component; kThis represents the national standard threshold (e.g., the limit for surface water quality standards) for the k-th polluting component.
[0053] In one specific implementation, the source association algorithm determines the relationship between pollutant components and pollution sources by constructing a mapping table based on historical data. The pollutant component-source association algorithm is a core technology for achieving accurate source tracing in the field of environmental monitoring. It integrates heterogeneous historical data from multiple sources to build a dynamic mapping system between the characteristics of water body DOM components and pollution sources. Specifically, the algorithm first systematically collects water samples from typical sources such as domestic sewage, industrial wastewater, and agricultural non-point source pollution (the sample size for each category is typically ≥200 cases). Using techniques such as three-dimensional fluorescence spectroscopy and gas chromatography-mass spectrometry, it accurately determines the concentration of more than 20 DOM components in the samples, including proteins, humic acids, phenols, and polycyclic aromatic hydrocarbons. It then combines this with information such as pollution discharge inventories and field surveys to complete the source category labeling (e.g., refined to industrial sub-categories or agricultural planting types). Based on this, statistical methods (such as the mean, standard deviation, and contribution rate distribution of each source component concentration) are used to mine feature differences. For example, it was found that the average proportion of protein components in domestic sewage is 55%-70%, while the concentration of phenols in chemical wastewater generally exceeds 10 mg / L and is negatively correlated with humic acid (p = -0.68). At the same time, machine learning models such as random forest and gradient boosting tree are used to train a mapping model of "component concentration vector - source probability distribution" to identify key source tracing features (such as the intensity of tyrosine-like peaks and the humic acid / fulvic acid ratio). The resulting association rule base can achieve rapid source tracing of unknown samples: when a water sample test shows that the proportion of protein is 62%, phenols are <0.8 mg / L, and humic acid is 18%, the algorithm infers that the pollution source is domestic sewage by matching the feature distribution pattern of domestic sewage samples in historical data (match degree >92%), and at the same time gives the probability of excluding agricultural non-point source pollution (<8%).
[0054] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.
Claims
1. A machine learning-based method for diagnosing water pollution using fluorescence spectral images, characterized in that, Includes the following steps: The spectral data of water samples were collected using a three-dimensional fluorescence spectrometer to obtain a three-dimensional fluorescence spectral matrix with excitation wavelength a and emission wavelength b. The three-dimensional fluorescence spectral matrix is preprocessed by wavelet transform denoising, adaptive baseline correction, and vector normalization. Principal component analysis was used to extract the principal components of the three-dimensional fluorescence spectral matrix and construct the spectral feature vector. The spectral feature vector is input into a predetermined CNN-LSTM hybrid model, which outputs pollution indicators, including pollution level, component type and concentration prediction value. The pollution index is calculated using the pollution indicators. When the comprehensive pollution index exceeds a preset threshold, a pollution warning is triggered and a diagnostic report is generated.
2. The water pollution fluorescence spectral image diagnosis method based on machine learning according to claim 1, characterized in that, The excitation wavelength a is 200-450nm, and the emission wavelength b is 250-550nm.
3. The water pollution fluorescence spectral image diagnosis method based on machine learning according to claim 1, characterized in that, The wavelet transform uses the db4 wavelet basis function for 3-level decomposition, and the baseline correction uses an adaptive iterative reweighting algorithm to fit a polynomial baseline.
4. The water pollution fluorescence spectral image diagnosis method based on machine learning according to claim 1, characterized in that, The formula for calculating vector normalization is as follows: In the formula, A ij C represents the fluorescence intensity value corresponding to the i-th excitation wavelength and the j-th emission wavelength in the three-dimensional fluorescence spectral matrix. ij B represents the fluorescence intensity value after normalization and background subtraction; ij The fluorescence intensity values of the blank background at the i-th excitation wavelength and the j-th emission wavelength; ∑(A ij -B ij ) 2 This represents the magnitude of the vector calculated by summing the squares of all fluorescence intensity values after background subtraction.
5. The water pollution fluorescence spectral image diagnosis method based on machine learning according to claim 1, characterized in that, The CNN-LSTM hybrid model includes: Data preprocessing: The preprocessed three-dimensional fluorescence spectral matrix is converted into a model-compatible tensor format. For time series features, the spectral matrices of consecutive time steps are concatenated into a sequence that includes the time dimension. A convolutional neural network module is constructed to extract the spatial features of the spectrum: the spectral matrix is scanned by multiple two-dimensional convolutional layers, and nonlinearity is introduced by using the ReLU activation function to capture the position, intensity distribution and spatial correlation patterns of fluorescence peaks; after each convolutional layer, a max pooling layer is connected to reduce the feature dimension, and a flattening layer is used to convert the multidimensional feature map into a one-dimensional vector; The output one-dimensional vector is reshaped into a temporal sequence and input into a long short-term memory network module: the dynamic evolution of spectral spatial features over time is captured through a two-layer LSTM layer. The first LSTM layer is used to preserve temporal information, and the second layer is used to extract long-term dependencies. The temporal features are integrated through a fully connected layer to provide a comprehensive representation for multi-task output. The design incorporates a dual-branch output layer to implement classification and regression tasks: the classification branch uses a fully connected layer combined with a Softmax activation function to output the probability distribution of each pollution level; the regression branch directly outputs the concentration values of each component through a fully connected layer. The dataset is divided into training, validation, and test sets, and data augmentation techniques are used to increase sample diversity. The Adam optimizer is used in combination with a learning rate decay strategy to minimize the loss function, and an early stopping mechanism is used to prevent overfitting. After training, the model performance is evaluated by accuracy and / or root mean square error.
6. The water pollution fluorescence spectral image diagnosis method based on machine learning according to claim 1, characterized in that, The comprehensive pollution index is calculated using the following formula: In the formula, CPI represents the comprehensive pollution index; w k D represents the preset influence weight of the k-th pollutant component; k S represents the predicted concentration of the k-th pollutant component; k This represents the national standard threshold for the k-th polluting component.
7. The water pollution fluorescence spectral image diagnosis method based on machine learning according to claim 1, characterized in that, The system triggers pollution warnings and generates diagnostic reports, and also generates structured diagnostic reports containing pollution levels, main components, and source tracing suggestions based on the proportion of each pollution component and the source association algorithm.
8. The water pollution fluorescence spectral image diagnosis method based on machine learning according to claim 1, characterized in that, The source association algorithm is determined by constructing a mapping table between pollutant components and pollutant sources using historical data.
