Water quality detection method, equipment, medium and product based on ultraviolet-visible light spectrum

Abstract

The invention discloses an ultraviolet-visible light spectrum water quality detection method and device, a medium and a product, and relates to the field of spectrum analysis and data processing.The method comprises the steps that an absorption spectrum corresponding to a transmittance spectrum of a sample is determined; carrying out de-noising treatment on the absorption spectrum of the sample; constructing corresponding spectral data generation models based on different neural networks; determining an optimal spectral data generation model; a concentration label generated in a preset concentration interval is searched by using a grid method, and a simulated absorption spectrum of the sample under the corresponding concentration label is generated by using the optimal spectrum data generation model; constructing a concentration regression model based on the simulated absorption spectrum and the corresponding concentration label; and performing pollutant concentration prediction on different pollutant solutions by using the concentration regression model. According to the invention, accurate estimation of the concentration of each pollutant in a complex water quality environment can be improved, a model can be quickly reconstructed when a detection task is changed, and an intelligent water quality sensing system is established.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of spectral analysis and data processing, in particular to a method and device for water quality detection based on ultraviolet-visible light spectrum, a medium and a product. BACKGROUND

[0002] Water resources are a key element for the sustainable development of human society and the ecological environment, and its quality directly affects human health, ecological system balance and social and economic development. However, with the acceleration of industrialization, the increase in the use of agricultural chemicals and the continuous advancement of urbanization, water pollution problems have become increasingly serious, becoming one of the important challenges in the field of global environmental protection and sustainable development. High-frequency, multi-index and real-time water quality monitoring methods need to be developed to realize real-time sensing and concentration analysis of the concentration changes of pollutants.

[0003] Traditional water quality monitoring methods such as chemical titration, photometry, electrochemistry and chromatography, although have high quantitative accuracy and detection stability, generally have defects such as complex detection process, long analysis period, cumbersome sample pretreatment and possible secondary pollution, which are difficult to meet the detection requirements of "reagent-free, rapid and online" in modern water environment monitoring. In contrast, ultraviolet-visible spectrum (UV-Vis) analysis technology, as a reagent-free and fast optical detection method, has received widespread attention in the field of online water quality monitoring in recent years.

[0004] However, UV-Vis technology still faces many challenges in practical application. On the one hand, the composition of pollutants in water is complex, and the background interference is significant, resulting in significant spectral overlap and absorption interference between different parameters, and the prediction of a single pollutant by the model is easily affected by other components. On the other hand, for the parameter regression problem in multi-component mixed water, most existing methods use differential spectrum processing or multi-source spectrum fusion strategy, the former suppresses background interference through band difference, but cannot completely eliminate the nonlinear influence brought by component interaction; the latter increases the information dimension, but needs to introduce multiple devices such as fluorescence, which is complex and high in cost, and is not suitable for actual engineering deployment. Some studies use band selection algorithms (such as SPA, PLS, etc.) for spectral dimension reduction to extract the characteristic absorption region for modeling, but this kind of method has insufficient adaptability to nonlinear coupling structure, and it is difficult to cover the high concentration absorption saturation region, so the generalization ability and robustness of the model are limited. It can be seen that the existing technology has poor adaptability when detecting changes in pollutants in water quality, and it takes a long time to re-model.

[0005] In summary, in the existing research on ultraviolet-visible spectrum water quality detection, there are problems such as single detection parameter, poor accuracy of concentration estimation of pollutant parameters and poor model transferability when adapting to new detection tasks. SUMMARY

[0006] The purpose of the present application is to provide a water quality detection method, device, medium and product based on ultraviolet-visible light spectrum, which can improve the accurate prediction of the concentration of various pollutants in a complex water quality environment and quickly rebuild the model when the detection task changes, and establish an intelligent water quality sensing system.

[0007] To achieve the above purpose, the present application provides the following solutions. In a first aspect, the present application provides a water quality detection method based on ultraviolet-visible light spectrum, which comprises: obtaining the transmittance spectrum and the calibration concentration of a sample, and determining the absorption spectrum corresponding to the transmittance spectrum of the sample; the sample includes solutions of various concentrations of chemical oxygen demand single component and total nitrogen single component, and mixed solutions of different concentration combinations of chemical oxygen demand and total nitrogen; performing denoising processing on the absorption spectrum of the sample to obtain a denoised absorption spectrum; constructing a corresponding spectrum data generation model based on different neural networks; each spectrum data generation model takes the concentration combination of chemical oxygen demand and / or total nitrogen as input, and takes the corresponding denoised absorption spectrum as output; the different neural networks include a support vector regression model, a random forest model and an adversarial autoencoder-conditional Wasserstein generative adversarial network; respectively evaluating the absorption spectrum generated by each spectrum data generation model and the corresponding denoised absorption spectrum to determine the optimal spectrum data generation model; searching for concentration labels within a preset concentration range using a grid method, and generating a simulated absorption spectrum of the sample under the corresponding concentration label using the optimal spectrum data generation model; constructing a concentration regression model based on the simulated absorption spectrum and the corresponding concentration label; using the concentration regression model to predict the concentration of pollutants in different pollutant solutions.

[0008] Optionally, the obtaining the transmittance spectrum and the calibration concentration of a sample, and determining the absorption spectrum corresponding to the transmittance spectrum of the sample specifically comprises: obtaining the transmittance spectrum of the sample using an ultraviolet-visible spectrometer; determining the absorption spectrum of the sample using the Lambert-Beer law according to the transmittance spectrum of the sample.

[0009] Optionally, the performing denoising processing on the absorption spectrum of the sample to obtain a denoised absorption spectrum specifically comprises: performing db8 wavelet denoising on the absorption spectrum of the sample; The absorption spectrum after wavelet denoising is smoothed using Savitzky-Golay polynomials to obtain the denoised absorption spectrum.

[0010] Optionally, algorithms for evaluating the absorption spectrum generated by each spectral data generation model and the corresponding denoised absorption spectrum include: principal component analysis, t-distribution random neighborhood embedding, Mahalanobis distance, and cosine similarity.

[0011] Optionally, the concentration regression models include: partial least squares support vector concentration regression model, partial least squares concentration regression model, and weighted neural concentration regression model.

[0012] Optionally, the concentration-sensitive weighted loss function of the weighted neural concentration regression model for: ; in, For the sample The loss weighting coefficient, N is the number of samples. , These are the concentration thresholds for chemical oxygen demand (COD) and total nitrogen. The loss weight decay coefficient for the sample. and These are the actual concentration labels and the predicted concentration labels, respectively. and These are the actual concentration labels for chemical oxygen demand and total nitrogen, respectively.

[0013] Secondly, this application provides a water quality detection device using ultraviolet-visible spectroscopy, the water quality detection device including: The data acquisition module is used to acquire the transmittance spectrum and calibrated concentration of the sample; and to determine the absorption spectrum corresponding to the transmittance spectrum of the sample; the sample includes: solutions of each concentration of chemical oxygen demand single component and total nitrogen single component, as well as mixed solutions of different concentration combinations of chemical oxygen demand and total nitrogen; A noise reduction module is used to denoise the absorption spectrum of the sample to obtain a denoised absorption spectrum. The generative model building module is used to construct corresponding spectral data generation models based on different neural networks. Each spectral data generation model takes the combination of chemical oxygen demand and / or total nitrogen concentrations as input and the corresponding denoised absorption spectrum as output. The different neural networks include: support vector regression model, random forest model, and adversarial autoencoder-conditional Wasserstein generative adversarial network. The optimal generation model determination module is used to evaluate the absorption spectrum generated by each spectral data generation model and the corresponding denoised absorption spectrum to determine the optimal spectral data generation model. The simulated absorption spectrum generation module is used to search for and generate concentration labels within a preset concentration range using a grid method, and to generate simulated absorption spectra of samples under the corresponding concentration labels using the optimal spectral data generation model. The concentration regression model construction module is used to construct concentration regression models based on the simulated absorption spectrum and the corresponding concentration labels, respectively. The pollutant concentration prediction module is used to predict the pollutant concentration of different pollutant solutions using the concentration regression model.

[0014] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the ultraviolet-visible light spectroscopy water quality detection method.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned ultraviolet-visible light spectroscopy water quality detection method.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned ultraviolet-visible light spectroscopy water quality detection method.

[0017] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, device, medium, and product for water quality detection using ultraviolet-visible spectroscopy. It involves acquiring transmittance spectra of solutions with individual concentrations of chemical oxygen demand (COD) and total nitrogen (TN), as well as mixed solutions with different combinations of COD and TN concentrations, and calculating their corresponding absorption spectra. The absorption spectra are then denoised to obtain denoised absorption spectra. Based on the absorption spectral data and concentration labels of solutions with different pollutant conditions, optimal spectral data generation models are trained. The changes in absorption spectra and component concentrations under different pollutant conditions are analyzed. Simulated absorption spectra corresponding to specified concentration labels under different pollutant conditions are generated using the optimal spectral data generation models, expanding the absorption spectral dataset. Using the simulated absorption spectra, corresponding concentration regression models are determined for different pollutant conditions, resulting in concentration regression models for each pollutant under different pollutant conditions. This application can improve the accuracy of concentration prediction for various pollutant parameters in complex polluted water and enhance model reconstruction capabilities, achieving intelligent water quality detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of a water quality detection method using ultraviolet-visible spectroscopy in one embodiment of this application; Figure 2 This is a schematic diagram of the regression process for TN concentration in a UV-Vis spectroscopy water quality detection method based on a fusion model. Figure 3 This is a schematic diagram of the COD concentration regression process of the UV-Vis spectroscopy water quality detection method based on the fusion model. Figure 4 This is a schematic diagram of the concentration regression process for a multi-component mixed solution using a UV-Vis spectroscopy water quality detection method based on a fusion model. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] In one exemplary embodiment, such as Figure 1 As shown, a water quality detection method using ultraviolet-visible spectroscopy is provided, comprising the following steps S101 to S107: Wherein: S101, Obtain the transmittance spectrum and calibrated concentration of the sample; and determine the absorption spectrum corresponding to the transmittance spectrum of the sample; The sample includes: solutions of each concentration of chemical oxygen demand single component and total nitrogen single component, as well as mixed solutions of different concentration combinations of chemical oxygen demand and total nitrogen; S101 specifically includes: S11, the transmittance spectrum of the sample was obtained using a UV-Vis spectrometer; S12. Based on the transmittance spectrum of the sample, the absorption spectrum of the sample is determined using the Lambert-Beer law.

[0023] S102, the absorption spectrum of the sample is denoised to obtain the denoised absorption spectrum, thereby removing noise caused by instrumentation and background interference in the absorption spectrum; S102 specifically includes: S21, Perform db8 wavelet denoising on the absorption spectrum of the sample; S22, the absorption spectrum after wavelet denoising is smoothed by Savitzky-Golay polynomial to obtain the denoised absorption spectrum.

[0024] S103, corresponding spectral data generation models are constructed based on different neural networks; each spectral data generation model takes the combination of chemical oxygen demand and / or total nitrogen concentrations as input and the corresponding denoised absorption spectrum as output; the different neural networks include: Support Vector Regression (SVR), Random Forest (RF), and Adversarial Autoencoder – Conditional Wasserstein Generative Adversarial Network (AAE-CWGAN). S104, evaluate the absorption spectrum generated by each spectral data generation model and the corresponding denoised absorption spectrum to determine the optimal spectral data generation model. The evaluation algorithms include: Principal Component Analysis (PCA), t-distributed random neighborhood embedding (t-SNE), Mahalanobis Distance, and Cosine Similarity.

[0025] Specifically, based on the aforementioned algorithm, the similarity between the simulated spectrum and the real sample is compared and analyzed from two levels: global distribution structure and local spectral structure. This is to measure whether the simulated data maintains consistency with the spatial distribution of the real dataset while ensuring accurate absorption peak positions and reasonable spectral shape changes with concentration. This yields multiple dimensional reduction and similarity indices for evaluating the spectral data generation model. Evaluation results are obtained based on these indices. These results are used to assess the similarity between the generated simulated absorption spectrum and the real spectrum in terms of spectral shape and spatial distribution, in order to select the optimal spectral data generation model.

[0026] S105, use the grid method to search and generate concentration labels within the preset concentration range, and use the optimal spectral data generation model to generate the simulated absorption spectrum of the sample under the corresponding concentration label. S106, Concentration regression models are constructed based on the simulated absorption spectrum and the corresponding concentration labels; the concentration regression models include: partial least squares support vector concentration regression model, partial least squares concentration regression model and weighted neural concentration regression model.

[0027] S106 specifically includes: Using the optimal spectral data generation model obtained above—the SVR spectral generation model—simulated absorption spectra under a specified TN concentration label are generated. Analyzing the relationship between the absorption peaks of the simulated absorption spectra and concentration changes, we established a mapping relationship between concentration and absorption spectra by using the simulated absorption spectra as input data and the corresponding concentrations as output data, and trained a partial least squares (PLR) concentration regression model. Using the original data as input data, we input the trained PLR concentration regression model to obtain the corresponding concentrations; and then used RMSE and other metrics to measure the results. Evaluate the accuracy of the concentration regression model.

[0028] Using the simulated absorption spectra as input data and the corresponding concentrations as output data, a mapping relationship between concentration and absorption spectra is established, and a partial least squares support vector (LS-SVM) concentration regression model is trained. The original data is then input into the trained LS-SVM concentration regression model to obtain the corresponding concentrations. The RMSE (Recovery Rate) is then used as the output data. The accuracy of the concentration regression model was evaluated and compared with the PLR ​​model. The analysis revealed the most accurate concentration regression model suitable for TN single-component solutions: the Partial Least Squares (PLR) concentration regression model.

[0029] Using the optimal spectral data generation model obtained above—the SVR spectral generation model—simulated absorption spectra under a specified COD concentration label are generated. Analyzing the relationship between the absorption peaks of the simulated absorption spectra and concentration changes, we established a mapping relationship between concentration and absorption spectra by using the simulated absorption spectra as input data and the corresponding concentrations as output data, and trained a partial least squares (PLR) concentration regression model. Using the original data as input data, we input the trained PLR concentration regression model to obtain the corresponding concentrations; and then used RMSE and other metrics to measure the results. Evaluate the accuracy of the concentration regression model.

[0030] Using the simulated absorption spectra as input data and the corresponding concentrations as output data, a mapping relationship between concentration and absorption spectra is established, and a partial least squares support vector (LS-SVM) concentration regression model is trained. The original data is then input into the trained LS-SVM concentration regression model to obtain the corresponding concentrations. The RMSE (Recovery Rate) is then used as the output data. The accuracy of the concentration regression model was evaluated and compared with the PLR ​​model. The analysis revealed that the most accurate concentration regression model suitable for COD single-component solutions is the Partial Least Squares Support Vector (LS-SVM) concentration regression model.

[0031] Using the optimal spectral data generation model obtained above—the SVR spectral generation model—simulated absorption spectra of the COD and TN mixed solution at specified concentration labels are generated. Analysis of the absorption spectra and absorption coefficient spectra of COD and TN, as well as COD and TN single-component solutions, revealed that the absorption peak of TN overlaps with the first absorption peak of COD. The effects of changes in COD and TN concentrations on the absorption peaks of the mixed solution's absorption spectrum were analyzed to determine the inflection point concentration that causes the nonlinear change in the absorption spectrum. Using the simulated absorption spectrum of the above mixed solution as input data and the corresponding concentration as output data, a mapping relationship between concentration and absorption spectrum is established, and a weighted neural network (Weighted-NN) concentration regression model is trained. The original data is then input into the trained Weighted-NN concentration regression model to obtain the concentrations of COD and TN in the mixed solution. The RMSE (Recovery Rate) and other parameters are then used to calculate the concentrations. Evaluate the accuracy of the concentration regression model.

[0032] The concentration-sensitive weighted loss function of the weighted neural concentration regression model for: ; in, For the sample The loss weighting coefficient, N is the number of samples. , These are the concentration thresholds for chemical oxygen demand (COD) and total nitrogen. The loss weight decay coefficient for the sample. and These are the actual concentration labels and the predicted concentration labels, respectively. and These are the actual concentration labels for chemical oxygen demand and total nitrogen, respectively.

[0033] S107, The concentration regression model is used to predict the concentration of pollutants in different pollutant solutions; the prediction result represents the concentration of each pollutant in the solution.

[0034] Spectra are the "fingerprints" of different substances. Different substances have different internal dynamic forms, generating different energy levels. When irradiated by light of a specific wavelength, electrons undergo transitions, usually accompanied by changes in the vibrational and rotational energy levels of the molecules. This results in absorption lines of electron transitions forming banded spectra. When electrons in a molecule undergo transitions, they need to absorb or emit photons of wavelengths corresponding to the energy level difference (200nm~800nm). This wavelength range is the ultraviolet-visible spectrum. The chemical bonds of different organic compounds exhibit selective absorption of electron radiation in the ultraviolet-visible band. Therefore, based on the different characteristics of absorption peaks, multiple parameters in water quality can be identified. Ultraviolet-visible spectroscopy (UV-Vis) technology quantitatively analyzes the concentration of organic and inorganic substances in water by measuring the absorption of specific wavelengths of light by dissolved substances. Furthermore, the intensity of a substance's absorption spectrum is usually directly proportional to its concentration. Therefore, based on the specific absorption spectra of different substances and their concentration mapping, spectral generation models of absorption spectrum versus concentration and concentration regression models of concentration versus absorption spectrum can be trained.

[0035] Furthermore, taking the case of a mixed COD and TN solution as an example, the preprocessing of wavelet denoising and Savitzky-Golay polynomial smoothing of the original absorption spectrum specifically includes: The original discrete data Represented as wavelet coefficients and wavelet basis functions Forms of convolution: ; Select soft-threshold wavelet coefficients to smooth the coefficients in the original signal that exceed the threshold, subtracting the threshold: ; Then, the spectral data after wavelet denoising is fitted using least-squares polynomial fitting within the moving window, representing it as a convolution of polynomial weight coefficients and the signal: ; The spectral data preprocessed by wavelet denoising and SG filtering can effectively remove noise from the original data while ensuring that the absorption peak positions and spectral shapes remain unchanged.

[0036] Furthermore, taking the case of a mixed COD and TN solution as an example, the training of the spectral data generation model specifically includes: During the experimental design phase, 52 groups of COD and TN mixed solution samples were configured, which did not undergo complete absorption saturation, had high modeling usability and representativeness, and had good coverage in the two-dimensional space of COD and TN concentrations. Their corresponding UV-Vis absorption spectra were also collected. Considering the fixed intensity of the light source used, under low concentration conditions, the absorption of each component has not yet reached saturation, and the absorption spectrum as a whole satisfies the Lambert-Beer law, with absorbance showing a linear relationship with the concentration of the substance. ; in, Absorbance The intensity of transmitted light. For the incident light intensity, Transmittance, The molar absorption coefficient, For optical path, Let be the concentration of the substance. Furthermore, the components satisfy the linear superposition property: ; in, This represents the total absorbance of the mixed solution. This represents the absorbance of each substance. Because the characteristic absorption peak of TN completely overlaps with the first characteristic absorption peak of COD, when the concentration of any component in the solution increases, the overlapping absorption region saturates first, while the non-overlapping region remains unsaturated, causing the linear relationship between absorbance and concentration across the entire wavelength band to break down. At this point, the overall spectrum exhibits nonlinear changes and absorption saturation effects, breaking the Lambert-Beer linear superposition assumption. Therefore, under these conditions, a linear model based on Lambert's law is insufficient to accurately generate mixed absorption spectra; a generation model capable of modeling nonlinear coupling relationships between wavelength bands is needed. To achieve the automatic generation of absorption spectra corresponding to concentration labels in multi-component mixed solutions, three representative generative models were constructed and compared: Support Vector Regression (SVR), Random Forest Regression (RF), and an AAE-CWGAN model combining an Autoencoder Adversarial Network (AAE) and a Conditional Wasserstein Generative Adversarial Network (CWGAN). These models all use two-dimensional label vectors composed of COD and TN concentrations. As input, the output wavelength range covers the absorption spectrum of 190–800 nm (1014 bands in total). ,in The absorption spectrum-concentration mapping function established by SVR is: ; Among them, radial basis function kernels are used. The final output simulated absorption spectrum is expressed as follows: .

[0037] The RF-established mapping function between absorption spectrum and concentration is: ; in, The number of decision trees; Indicates the i-th tree for wavelength Predicted absorbance results.

[0038] The AAE-CWGAN-established absorption spectrum-concentration mapping function is as follows: ; in, For generator, The light source spectrum is used as the basis for the above theoretical basis. The measured absorption spectra of 52 mixed solutions and their corresponding COD and TN concentration labels are used as real training data inputs. At the same time, concentration label samples generated by the grid method are used to train three models, namely SVR, RF and AAE-CWGAN, respectively, so as to learn the nonlinear mapping relationship between COD and TN component concentrations and spectral shapes.

[0039] Furthermore, when using a fusion model-based UV-Vis spectroscopy water quality detection system to detect TN concentration in solution, the training of the PLR ​​concentration regression model specifically includes: The absorption spectra of TN samples with different concentrations measured in the original samples were processed by wavelet denoising and SG filtering to remove high-frequency noise from the spectra; The processed absorption spectra and corresponding concentrations are input into the SVR spectral data generation model for training. Then, a more refined TN concentration label is generated using a grid method as input to the SVR generation model, and the corresponding TN simulated absorption spectrum data is output. Then, the TN simulated absorption spectrum data were divided into training and validation sets at a ratio of 90% and 10%, respectively. Principal component decomposition was then performed simultaneously on the simulated absorption spectrum matrix X and the corresponding concentration matrix Y of the training set using the least squares method. ; ; Where T and U are the score matrices of X and Y, respectively; P and Q are the loading matrices of X and Y, respectively; and E and F are the errors introduced during the fitting of X and Y, respectively. Then, a linear regression is performed on T and U: ; ; Then, the validation set and the original preprocessed absorption spectra are used as the test set and input into the trained PLR model to obtain the corresponding predicted concentration values: ; Then calculate the training set, validation set, and test set respectively. and To evaluate the prediction accuracy and generalization ability of the model. Root mean square error (RMSE) The difference between predicted and actual values ​​is described by the coefficient of determination (COP). Evaluate the extent to which the inversion model can explain the data: ; ; in For the true value, For predicted values, For the sample size, This represents the true mean. The training set for the final model. , ; Validation set , Test set , .

[0040] Furthermore, when using a UV-Vis spectroscopy water quality detection system with a fusion model to detect COD concentration in solution, the training of the LS-SVM concentration regression model specifically includes: The absorption spectra of COD samples with different concentrations were measured in the original samples and then processed by wavelet denoising and SG filtering to remove high-frequency noise from the spectra. The preprocessed absorption spectra and corresponding concentrations are input into the SVR spectral data generation model for training. Then, a more refined COD concentration label is generated using the grid method as input to the SVR generation model, and the corresponding COD simulated absorption spectrum data is output. Then, the COD simulated absorption spectrum data was divided into training and validation sets at a ratio of 80% and 20%, respectively. Using the training set as input data, LS-SVM represented the concentration and absorption spectrum as follows: ; in, For Lagrange multipliers, These are bias terms. These two parameters are highly dependent on the regularization parameters. and the width of the mapping kernel function Continuously optimize by combining grid search and cross-validation. , Hyperparameters are combined, and the optimal model structure is determined by minimizing the root mean square error (RMSE) of the validation set. The training set of the final model... , ; Validation set , Test set , .

[0041] The root mean square error used ( The coefficient of determination (COP) describes the difference between the predicted and actual values; Evaluate the extent to which the inversion model can explain the data: ; ; in, For the true value, For predicted values, For the sample size, This represents the true mean.

[0042] Furthermore, when using a UV-Vis spectroscopy water quality detection system with a fusion model to detect the concentrations of each component in a mixed COD and TN solution, step S6, the training of the Weighted-NN concentration regression model, specifically includes: The absorption spectra of COD and TN mixed samples with different concentration combinations measured in the original samples were processed by wavelet denoising and SG filtering to remove high-frequency noise from the spectra; The preprocessed absorption spectra and corresponding concentrations are input into the SVR spectral data generation model for training. Then, the grid method is used to generate concentration labels for all concentration combinations of COD and TN mixed solutions as input to the SVR generation model, and the corresponding simulated absorption spectral data are output. Then, the simulated absorption spectrum data is split into training and validation sets using five-fold cross-validation. The training set is used as input data to train a Weighted-NN concentration regression model. The Weighted-NN model represents the concentration and absorption spectrum as follows: ; in, Indicated by network parameters Controlled neural network mapping. Considering that the influence of high-concentration samples on the spectrum may mask low-concentration features, a concentration-sensitive weighted loss function is introduced during training to improve the prediction accuracy and stability of the model in the low-concentration range: ; in, For the sample The loss weighting coefficient is defined as follows: ; In the formula, , The concentration threshold, This is the loss weight decay coefficient for high-concentration samples. and These are the actual and predicted concentration labels, respectively. This design allows the model to focus more on the prediction accuracy in low-concentration regions, avoiding high concentrations dominating the regression direction, thereby improving the generalization performance across the entire concentration range. By analyzing the influence of each component in the mixed solution on the absorption spectrum, the concentration thresholds and weight ratios for COD and TN were determined. Finally, on the training set, the WNN model achieved extremely high fitting accuracy for both COD and TN concentrations, with very small deviations between predicted and actual values. Specifically, the COD values ​​in the training set... , Prediction accuracy within the instrument resolution range (2 mg / L) ;TN , Prediction accuracy within the instrument resolution range (2 mg / L) This indicates that the model has fully learned the nonlinear mapping relationship between the mixed spectrum and the target concentration on the training data. In the five-fold cross-validation validation set, the model still maintains good predictive ability for COD. , Prediction accuracy within the instrument resolution range (2 mg / L) ;TN , Prediction accuracy within the instrument resolution range (2 mg / L) Using the original denoised absorption spectrum as an additional test set also yielded good results for COD. , Prediction accuracy within the instrument resolution range (2 mg / L) ;TN , Prediction accuracy within the instrument resolution range (2 mg / L) .

[0043] The root mean square error used ( The coefficient of determination (COP) describes the difference between the predicted and actual values; Evaluate the extent to which the inversion model can explain the data: ; ; in For the true value, For predicted values, For the sample size, This represents the true mean.

[0044] The training results of the aforementioned model demonstrate that the UV-Vis spectral water quality detection method, based on the fusion of the spectral generation model and the concentration regression model, can not only achieve high-precision concentration prediction for parameters with different absorption spectral structures, but also alleviate the masking effect of absorption peaks in overlapping absorption regions while maintaining high-precision regression of the concentrations of each component in water quality conditions where multiple pollutants exist and their absorption peaks mask each other. It exhibits good accuracy and generalization ability in multi-parameter concentration prediction tasks for single-component and mixed-polluted water samples. Furthermore, the model training speed is fast, and the spectral generation model significantly shortens the data measurement time in the early stages of modeling. It can adapt to remodeling when detection parameters change, making it an effective regression modeling method suitable for practical water quality detection tasks.

[0045] The present application will be further described in detail below through specific embodiments.

[0046] like Figure 2 The UV-Vis spectroscopy water quality detection method based on a fusion model shown here, which regresses TN concentration, specifically includes: S1. Prepare a partial TN single-component solution covering the detection concentration range of 2 mg / L - 40 mg / L, and measure its transmittance using a UV-Vis spectrometer. Calculate the absorption spectrum of the sample according to the Lambert-Beer law. The specific measurement steps are as follows: Take approximately 1 ml of the prepared sample, transfer it into a clean cuvette, place the cuvette behind the light source of the transmission UV-Vis spectrometer, and cover the cuvette with a black light shield to prevent background light interference. The light absorbed by the substance in the cuvette is received by the fiber optic port of the spectrometer detector, converted into a digital signal of transmittance spectral lines, and then the absorption spectrum is calculated.

[0047] S2, the absorption spectrum obtained above is preprocessed by wavelet denoising and Savitzky-Golay polynomial smoothing to obtain the denoised absorption spectrum signal.

[0048] S3. Build a support vector (SVR) spectral data generation model. Take the concentration labels corresponding to the above absorption spectral signals as input and the above denoised absorption spectra as output to obtain the mapping relationship between absorption spectra and concentration, and obtain the trained spectral data generation model.

[0049] S4. Use the grid method to generate refined TN single-component concentration labels covering the above concentration range, and use the trained SVR spectral data generation model to generate simulated absorption spectral data corresponding to the concentration labels.

[0050] S5. Construct a partial least squares (PLR) concentration regression model. Use the simulated absorption spectrum data and the corresponding concentration label data as input data to train the mapping relationship between concentration and absorption spectrum, and obtain the trained PLR concentration regression model.

[0051] S6, the absorption spectrum of the pre-processed TN sample obtained from the original measurement is used as input data for the trained PLR concentration regression model to obtain the corresponding estimated concentration; and RMSE is used as the reference. As an evaluation indicator, it is used to evaluate the accuracy of the concentration regression model.

[0052] like Figure 3 The UV-Vis spectroscopy water quality detection method based on a fusion model shown here, which regresses COD concentration, specifically includes: S10 involves preparing a partial COD single-component solution covering the detection concentration range of 5-40 mg / L, and measuring its transmittance using a UV-Vis spectrometer. The absorption spectrum of the sample is then calculated based on the Lambert-Beer law. The specific measurement steps are as follows: approximately 1 ml of the prepared sample is transferred to a clean cuvette. The cuvette is placed behind the light source of the transmission UV-Vis spectrometer and covered with a black light shield to prevent background light interference. The light absorbed by the substance in the cuvette is received by the fiber optic port of the spectrometer detector, converted into a digital signal of transmittance spectral lines, and then the absorption spectrum is calculated.

[0053] S20. The absorption spectrum obtained above is preprocessed by wavelet denoising and Savitzky-Golay polynomial smoothing to obtain the denoised absorption spectrum signal.

[0054] S30. Build a support vector (SVR) spectral data generation model. Take the concentration labels corresponding to the above absorption spectral signals as input and the above denoised absorption spectra as output to obtain the mapping relationship between absorption spectra and concentration, and obtain the trained absorption spectral data generation model.

[0055] S40, use the grid method to generate refined COD single-component concentration labels covering the above concentration range, and use the trained SVR spectral data generation model to generate simulated absorption spectral data corresponding to the concentration labels.

[0056] S50. Construct a partial least squares-support vector (LS-SVM) concentration regression model. Use the simulated absorption spectrum data and the corresponding concentration label data as input data to train the mapping relationship between concentration and absorption spectrum, and obtain the trained LS-SVM concentration regression model.

[0057] S60, the absorption spectrum of the pre-processed COD sample obtained from the original measurement is used as input data for the trained LS-SVM concentration regression model to obtain the corresponding estimated concentration; and RMSE is used as the input data. As an evaluation indicator, it is used to evaluate the accuracy of the concentration regression model.

[0058] like Figure 4 The UV-Vis spectroscopy water quality detection method based on a fusion model, shown, performs concentration regression on multi-component mixed solutions, specifically including: The S100 is used to prepare a mixed solution with varying concentrations of TN (2 mg / L - 40 mg / L) and COD (5 mg / L - 40 mg / L), covering the detection concentration range. The transmittance is measured using a UV-Vis spectrometer, and the absorption spectrum is calculated based on the Lambert-Beer law. The specific measurement procedure is as follows: approximately 1 ml of the prepared sample is transferred to a clean cuvette. The cuvette is placed behind the light source of the transmission UV-Vis spectrometer and covered with a black light shield to prevent background light interference. The light absorbed by the substances in the cuvette is received by the fiber optic port of the spectrometer detector, converted into a digital signal of transmittance spectral lines, and then the absorption spectrum is calculated.

[0059] S200, the absorption spectrum obtained above is preprocessed by wavelet denoising and Savitzky-Golay polynomial smoothing to obtain the denoised absorption spectrum signal.

[0060] S300, build a support vector (SVR) spectral data generation model, take the concentration label corresponding to the above absorption spectral signal as input, take the above denoised absorption spectrum as output, obtain the mapping relationship between absorption spectrum and concentration, and obtain the trained spectral data generation model.

[0061] S400: Use the grid method to generate concentration labels covering all concentration combinations in the entire concentration range mentioned above. Use the trained SVR spectral data generation model to generate simulated absorption spectral data for the corresponding concentration combination labels.

[0062] S500 analyzes the changes in absorption peaks of the mixed solution absorption spectrum with COD or TN concentrations, and determines the transition concentration based on the COD or TN concentrations corresponding to the saturation of the absorption peaks.

[0063] S600, using the aforementioned transition concentration as the threshold concentration, a weighted neural network (Weighted-NN) concentration regression model is constructed. The aforementioned simulated absorption spectrum data and the aforementioned corresponding concentration label data are used as input data. The weights of the COD and TN loss functions are continuously adjusted to train the mapping relationship between concentration and absorption spectrum, thus obtaining the trained Weighted-NN concentration regression model.

[0064] S700, the preprocessed absorption spectra of the original measured COD and TN mixture are used as input data for the trained Weighted-NN concentration regression model to obtain the corresponding estimated concentrations of COD and TN; and RMSE is used to calculate the concentrations. As an evaluation indicator, it is used to evaluate the accuracy of the concentration regression model.

[0065] Based on the same inventive concept, this application also provides an ultraviolet-visible spectroscopy water quality detection device for implementing the ultraviolet-visible spectroscopy water quality detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more ultraviolet-visible spectroscopy water quality detection device embodiments provided below can be found in the limitations of the ultraviolet-visible spectroscopy water quality detection method described above, and will not be repeated here.

[0066] In one exemplary embodiment, a water quality detection device using ultraviolet-visible spectroscopy is provided, comprising: The data acquisition module is used to acquire the transmittance spectrum and calibrated concentration of the sample; and to determine the absorption spectrum corresponding to the transmittance spectrum of the sample; the sample includes: solutions of each concentration of chemical oxygen demand single component and total nitrogen single component, as well as mixed solutions of different concentration combinations of chemical oxygen demand and total nitrogen; A noise reduction module is used to denoise the absorption spectrum of the sample to obtain a denoised absorption spectrum. The generative model building module is used to construct corresponding spectral data generation models based on different neural networks. Each spectral data generation model takes the combination of chemical oxygen demand and / or total nitrogen concentrations as input and the corresponding denoised absorption spectrum as output. The different neural networks include: support vector regression model, random forest model, and adversarial autoencoder-conditional Wasserstein generative adversarial network. The optimal generation model determination module is used to evaluate the absorption spectrum generated by each spectral data generation model and the corresponding denoised absorption spectrum to determine the optimal spectral data generation model. The simulated absorption spectrum generation module is used to search for and generate concentration labels within a preset concentration range using a grid method, and to generate simulated absorption spectra of samples under the corresponding concentration labels using the optimal spectral data generation model. The concentration regression model construction module is used to construct concentration regression models based on the simulated absorption spectrum and the corresponding concentration labels, respectively. The pollutant concentration prediction module is used to predict the pollutant concentration of different pollutant solutions using the concentration regression model.

[0067] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a water quality detection method using ultraviolet-visible spectroscopy.

[0068] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0069] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0070] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0071] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0072] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0073] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the embodiments provided in this application may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0074] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0075] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0076] Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. In summary, the content of this specification should not be construed as a limitation of this application.

Claims

1. A water quality detection method using ultraviolet-visible spectroscopy, characterized in that, The ultraviolet-visible spectroscopy method for water quality detection includes: The transmittance spectrum and calibrated concentration of the sample are obtained; and the absorption spectrum corresponding to the transmittance spectrum of the sample is determined; the sample includes: solutions of each concentration of chemical oxygen demand single component and total nitrogen single component, as well as mixed solutions of different concentration combinations of chemical oxygen demand and total nitrogen; The absorption spectrum of the sample is denoised to obtain the denoised absorption spectrum; Different spectral data generation models are constructed based on different neural networks. Each spectral data generation model takes the combination of chemical oxygen demand and / or total nitrogen concentrations as input and the corresponding denoised absorption spectrum as output. The different neural networks include: support vector regression model, random forest model and adversarial autoencoder-conditional Wasserstein generative adversarial network. The absorption spectra generated by each spectral data generation model and the corresponding denoised absorption spectra are evaluated to determine the optimal spectral data generation model. Concentration labels within a preset concentration range are generated using a grid search method, and the simulated absorption spectra of the samples under the corresponding concentration labels are generated using the optimal spectral data generation model. Concentration regression models were constructed based on the simulated absorption spectra and the corresponding concentration labels, respectively. The concentration regression model described above is used to predict the concentration of pollutants in different pollutant solutions.

2. The water quality detection method using ultraviolet-visible spectroscopy according to claim 1, characterized in that, The acquisition of the transmittance spectrum and calibration concentration of the sample, and the determination of the absorption spectrum corresponding to the transmittance spectrum of the sample, specifically include: The transmittance spectrum of the sample was obtained using a UV-Vis spectrometer. The absorption spectrum of the sample was determined using Lambert-Beer's law based on the sample's transmittance spectrum.

3. The water quality detection method using ultraviolet-visible spectroscopy according to claim 1, characterized in that, The denoising process for the absorption spectrum of the sample to obtain the denoised absorption spectrum specifically includes: The absorption spectrum of the sample was denoised using db8 wavelet denoising. The absorption spectrum after wavelet denoising is smoothed using Savitzky-Golay polynomials to obtain the denoised absorption spectrum.

4. The water quality detection method using ultraviolet-visible spectroscopy according to claim 1, characterized in that, Algorithms for evaluating the absorption spectra generated by each spectral data generation model and their corresponding denoised absorption spectra include: principal component analysis, t-distribution random neighborhood embedding, Mahalanobis distance, and cosine similarity.

5. The water quality detection method using ultraviolet-visible spectroscopy according to claim 1, characterized in that, Concentration regression models include: partial least squares support vector concentration regression model, partial least squares concentration regression model, and weighted neural concentration regression model.

6. The water quality detection method using ultraviolet-visible spectroscopy according to claim 5, characterized in that, The concentration-sensitive weighted loss function of the weighted neural concentration regression model for: ； in, For the sample The loss weighting coefficient, N is the number of samples. , These are the concentration thresholds for chemical oxygen demand (COD) and total nitrogen. The loss weight decay coefficient for the sample. and These are the actual concentration labels and the predicted concentration labels, respectively. and These are the actual concentration labels for chemical oxygen demand and total nitrogen, respectively.

7. A water quality detection device using ultraviolet-visible spectroscopy, characterized in that, The ultraviolet-visible spectroscopy water quality detection equipment includes: The data acquisition module is used to acquire the transmittance spectrum and calibrated concentration of the sample; and to determine the absorption spectrum corresponding to the transmittance spectrum of the sample; the sample includes: solutions of each concentration of chemical oxygen demand single component and total nitrogen single component, as well as mixed solutions of different concentration combinations of chemical oxygen demand and total nitrogen; A noise reduction module is used to denoise the absorption spectrum of the sample to obtain a denoised absorption spectrum. The generative model building module is used to construct corresponding spectral data generation models based on different neural networks. Each spectral data generation model takes the combination of chemical oxygen demand and / or total nitrogen concentrations as input and the corresponding denoised absorption spectrum as output. The different neural networks include: support vector regression model, random forest model, and adversarial autoencoder-conditional Wasserstein generative adversarial network. The optimal generation model determination module is used to evaluate the absorption spectrum generated by each spectral data generation model and the corresponding denoised absorption spectrum to determine the optimal spectral data generation model. The simulated absorption spectrum generation module is used to search for and generate concentration labels within a preset concentration range using a grid method, and to generate simulated absorption spectra of samples under the corresponding concentration labels using the optimal spectral data generation model. The concentration regression model construction module is used to construct concentration regression models based on the simulated absorption spectrum and the corresponding concentration labels, respectively. The pollutant concentration prediction module is used to predict the pollutant concentration of different pollutant solutions using the concentration regression model.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the water quality detection method using ultraviolet-visible spectroscopy as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the water quality detection method using ultraviolet-visible spectroscopy as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the water quality detection method using ultraviolet-visible spectroscopy as described in any one of claims 1-6.

Images