Pollutant detection method and system in sludge treatment process

Abstract

The invention provides a pollutant detection method and system in a sludge treatment process. The method comprises the following steps: determining a full-wave band equivalent spectrum in a digestion process of a sludge sample to be detected according to a sparse base of spectral data of the sludge sample to be detected and a compressed light intensity signal; determining a global contribution characteristic variable and a local contribution characteristic variable of the full-wave band equivalent spectrum to sludge total nitrogen detection, and screening out a key characteristic wavelength from the full-wave band equivalent spectrum based on the global contribution characteristic variable and the local contribution characteristic variable; constructing a sample similarity matrix of the to-be-detected sludge sample according to the difference of the spectrum data at different moments and the light intensity change of the wavelength interval in the compressed light intensity signal; and determining the total nitrogen inversion concentration of the sludge sample to be detected through the key characteristic wavelength and the sample similarity matrix. By adopting the scheme provided by the invention, the influence of the dynamic conversion characteristic of the nitrogen form on the detection of the pollutants in the sludge in the digestion process of the sludge can be reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pollutant detection, and more particularly to a pollutant detection method and system in a sludge treatment process. BACKGROUND

[0002] Pollutant detection is a process of identifying, quantifying and analyzing the types, concentrations, existence forms and distribution characteristics of harmful pollutants in environmental media (air, water, soil, solid waste, etc.), organisms or industrial products through professional technical means. The core goal is to provide scientific basis for environmental governance, health protection, compliance supervision and process optimization, and it is a key technical support for ensuring ecological environment safety and human health.

[0003] Sludge is a byproduct of wastewater treatment, and its total nitrogen content is a key indicator for assessing the pollution level of sludge and guiding subsequent treatment (such as land use and incineration). Currently, the detection of total nitrogen in sludge mainly relies on traditional chemical methods (such as Kjeldahl method and alkaline potassium persulfate digestion-ultraviolet spectrophotometry). These methods require sample pretreatment (such as digestion and filtration), chemical reagent reaction, laboratory instrument analysis and other steps, which have limitations such as long detection period (usually several hours), complicated operation, dependence on manual and chemical reagents, and inability to monitor the oxidation digestion process in real time. At the same time, although existing spectral detection techniques can achieve rapid analysis, the sludge system is complex, containing high concentrations of organic matter, suspended particles, water, etc. Its spectrum is easily affected by scattering and interference component absorption, resulting in a blurred correlation between spectral signal and total nitrogen concentration. Moreover, during the oxidation digestion process, the nitrogen form dynamically changes, and traditional spectral methods are difficult to capture the key characteristics of this process, often resulting in problems such as static characteristic modeling not being able to adapt to dynamic reaction, leading to low detection accuracy and poor generalization ability. Therefore, how to reduce the influence of nitrogen form dynamic transformation characteristics of sludge during digestion on the detection of pollutants in sludge has become a problem faced by the industry. SUMMARY

[0004] The present application provides a pollutant detection method and system in a sludge treatment process, which can reduce the influence of nitrogen form dynamic transformation characteristics of sludge during digestion on the detection of pollutants in sludge.

[0005] In a first aspect, the present application provides a pollutant detection method in a sludge treatment process, wherein a to-be-detected sludge sample is mixed with an alkaline potassium persulfate oxidant and injected into a digestion cup for oxidation digestion reaction to detect the total nitrogen of pollutants in the sludge. The method includes the following steps: Collecting spectral data of the to-be-detected sludge sample changing with time during the oxidation digestion reaction and compressed light intensity signals after absorption and scattering of the to-be-detected sludge sample in the digestion cup; reconstruct the compressed light intensity signal based on a sparse basis of the spectral data to obtain a full-waveband equivalent spectrum of the sludge sample under digestion; determine characteristic variables of global contribution and characteristic variables of local contribution of the full-waveband equivalent spectrum to total nitrogen detection of sludge, and screen key characteristic wavelengths representing the oxidation and digestion process from the full-waveband equivalent spectrum based on the characteristic variables of global contribution and the characteristic variables of local contribution; construct a sample similarity matrix of the sludge sample under oxidation and digestion at different time points based on the difference of spectral data at different time points and the light intensity change of wavelength interval in the compressed light intensity signal; quantitatively invert the total nitrogen concentration in the sludge sample through the key characteristic wavelengths and the sample similarity matrix to obtain the total nitrogen inversion concentration of the sludge sample.

[0006] In some embodiments, the equivalent reconstruction of the compressed light intensity signal based on the sparse basis of the spectral data to obtain a full-waveband equivalent spectrum of the sludge sample under digestion specifically includes: perform denoising processing on the spectral data, and determine a sparse basis of the denoised spectral data; determine the linear relationship between the spectral data and the compressed light intensity signal; equivalently reconstruct the compressed light intensity signal according to the sparse basis and the linear relationship to obtain a full-waveband equivalent spectrum of the sludge sample under digestion.

[0007] In some embodiments, the determination of the characteristic variables of global contribution and the characteristic variables of local contribution of the full-waveband equivalent spectrum to total nitrogen detection of sludge specifically includes: perform standardization processing on the full-waveband equivalent spectrum to obtain a standardized full-waveband equivalent spectrum; calculate the characteristic variables of global contribution of the full-waveband equivalent spectrum to total nitrogen concentration based on the standardized full-waveband equivalent spectrum; divide the standardized full-waveband equivalent spectrum into multiple spectral subintervals, and calculate the characteristic variables of local contribution of each spectral subinterval to total nitrogen concentration.

[0008] In some embodiments, the screening of key characteristic wavelengths representing the oxidation and digestion process from the full-waveband equivalent spectrum based on the characteristic variables of global contribution and the characteristic variables of local contribution specifically includes: determine an initial characteristic variable pool after removing duplicate wavelengths according to the characteristic variables of global contribution and the characteristic variables of local contribution; screen a core candidate set of wavelengths from the full-waveband equivalent spectrum through the initial characteristic variable pool; obtaining a chemical mechanism of total nitrogen oxidation digestion; screening the core candidate set based on the chemical mechanism to obtain key characteristic wavelengths representing the oxidation digestion process.

[0009] In some embodiments, constructing a sample similarity matrix representing the similarity relationship between the test sludge samples at different time points in the oxidation digestion process according to the difference of the spectral data at different time points and the light intensity change of the wavelength interval in the compressed light intensity signal specifically includes: determining the difference of the spectral data at different time points; determining the light intensity change of the wavelength interval in the compressed light intensity signal; calculating the similarity between any two test sludge samples at different time points according to the difference and the light intensity change; constructing the sample similarity matrix representing the similarity relationship between the test sludge samples at different time points in the oxidation digestion process through all the similarities.

[0010] In some embodiments, quantitatively inverting the total nitrogen concentration in the test sludge sample through the key characteristic wavelengths and the sample similarity matrix to obtain the total nitrogen inversion concentration of the test sludge sample specifically includes: converting the key characteristic wavelengths into a feature matrix of time-key wavelength number; converting the sample similarity matrix into a diagonal weight matrix of the test sludge sample at different time points; fusing the feature matrix and the diagonal weight matrix to obtain a weighted feature matrix; constructing a quantitative inversion model of the weighted feature matrix and the total nitrogen concentration in the test sludge sample; determining the initial inversion concentration of the total nitrogen in the test sludge sample through the quantitative inversion model; determining the similarity score of the test sludge sample and the sample with known concentration; correcting the initial inversion concentration according to the similarity score to obtain the total nitrogen inversion concentration of the test sludge sample.

[0011] In some embodiments, the digestion cup is made of quartz material.

[0012] In some embodiments, an experimental device including a constant temperature digestion module, a spectral detection module, a compressed light intensity detection module, and a synchronous control module is built to collect the spectral data of the test sludge sample changing with time in the oxidation digestion reaction process and the compressed light intensity signal after being absorbed and scattered by the test sludge sample in the digestion cup.

[0013] In some embodiments, the total nitrogen includes organic nitrogen, ammonia nitrogen, and nitrite nitrogen.

[0014] In a second aspect, the present application provides a sludge treatment process pollutant detection system, comprising: The acquisition module is configured to acquire spectral data of the to-be-tested sludge sample changing with time in the oxidation digestion reaction process and compressed light intensity signals after absorption and scattering of the to-be-tested sludge sample in the digestion cup. The processing module is configured to perform equivalent reconstruction on the compressed light intensity signals based on the sparse basis of the spectral data, to obtain full-waveband equivalent spectra of the to-be-tested sludge sample in the digestion process. The processing module is further configured to determine characteristic variables with global contribution and characteristic variables with local contribution of the full-waveband equivalent spectra to the total nitrogen detection of the sludge, and screen out key characteristic wavelengths representing the oxidation digestion process from the full-waveband equivalent spectra based on the characteristic variables with global contribution and the characteristic variables with local contribution. The processing module is further configured to construct a sample similarity matrix of the to-be-tested sludge sample in the oxidation digestion process, representing similarity relationships between to-be-tested sludge samples at different time points, according to differences between spectral data at different time points and changes in light intensity in wavelength intervals in the compressed light intensity signals. The execution module is configured to quantitatively invert the total nitrogen concentration in the to-be-tested sludge sample through the key characteristic wavelengths and the sample similarity matrix, to obtain the total nitrogen inversion concentration of the to-be-tested sludge sample.

[0015] The technical scheme provided by the embodiments of the present application has the following beneficial effects: In the sludge treatment process pollutant detection method and system provided by the present application, first, spectral data of the to-be-tested sludge sample changing with time in the oxidation digestion reaction process and compressed light intensity signals after absorption and scattering of the to-be-tested sludge sample in the digestion cup are acquired; the compressed light intensity signals are reconstructed based on the sparse basis of the spectral data, to obtain full-waveband equivalent spectra of the to-be-tested sludge sample in the digestion process; characteristic variables with global contribution and characteristic variables with local contribution of the full-waveband equivalent spectra to the total nitrogen detection of the sludge are determined, and key characteristic wavelengths representing the oxidation digestion process are screened out from the full-waveband equivalent spectra based on the characteristic variables with global contribution and the characteristic variables with local contribution; a sample similarity matrix of the to-be-tested sludge sample in the oxidation digestion process, representing similarity relationships between to-be-tested sludge samples at different time points, is constructed according to differences between spectral data at different time points and changes in light intensity in wavelength intervals in the compressed light intensity signals; and the total nitrogen concentration in the to-be-tested sludge sample is quantitatively inverted through the key characteristic wavelengths and the sample similarity matrix, to obtain the total nitrogen inversion concentration of the to-be-tested sludge sample.

[0016] It can be seen that, in the pollutant detection process in the sludge treatment process, by collecting the time-varying spectral data and the compressed light intensity signal, the dynamic spectral characteristics of the nitrogen form (ammonium-nitrite-nitrate) conversion can be captured in real time, and the information loss caused by missing the key conversion node due to static data can be avoided; based on the spectral sparse basis reconstruction full-band equivalent spectrum, the lost spectral details related to the nitrogen form conversion in the compressed light intensity signal can be recovered, the characteristic absorption signals of different forms of nitrogen can be completely retained, the problem of missing key information caused by signal compression can be solved; the global contribution (universal characteristics shared by each nitrogen form) and the local contribution (specific characteristics of a specific form) variables are distinguished and the key wavelengths are screened, the characteristic absorption interval of each form of nitrogen in the conversion process can be accurately covered, the key signals in the dynamic conversion can be avoided to be missed by a single characteristic variable, and the spectral interference caused by the form switching can be reduced; the sample similarity matrix is constructed, the sample characteristic differences caused by the nitrogen form conversion at different times are quantified, the stable period (high similarity) and the intense period (low similarity) are identified, the basis for subsequent weighting is provided, and the influence of abnormal signals at the intense conversion moment is weakened; the concentration is inversed by combining the key wavelengths and the similarity matrix, the stable period characteristic contribution is highlighted by weighted fusion, and then the deviation correction is performed to adapt to the dynamic law of the form conversion, finally the interference of the dynamic change of the nitrogen form on the detection precision is reduced, and the accurate inversion of the total nitrogen concentration is realized. By using the above scheme, the influence of the dynamic conversion characteristics of the nitrogen form in the sludge in the sludge digestion process on the detection of pollutants in the sludge can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is an example flowchart of a pollutant detection method in a sludge treatment process according to some embodiments of the present application; Figure 2 is an example flowchart of determining a full-band equivalent spectrum according to some embodiments of the present application; Figure 3 is an example flowchart of determining a sample similarity matrix according to some embodiments of the present application; Figure 4 is a structural schematic diagram of a pollutant detection system in a sludge treatment process according to some embodiments of the present application; Figure 5 is a structural schematic diagram of a computer device for implementing a pollutant detection method in a sludge treatment process according to some embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in combination with the drawings in the specification and specific embodiments.

[0019] Reference Figure 1FIG. 1 is an exemplary flowchart of a method for detecting pollutants in a sludge treatment process according to some embodiments of the present application, which mainly includes the following steps: It should be noted that the to-be-detected sludge sample is mixed with an alkaline potassium persulfate oxidant and injected into a digestion cup for an oxidation digestion reaction to detect total nitrogen in the sludge, which is one of the key pollutants in the sludge. If it enters the environment with the sludge disposal (such as land use, landfill), it is easy to cause water eutrophication and destroy the balance of soil nitrogen cycle, and it is a pollution index that needs to be monitored. Therefore, it is included in the scope of sludge pollutants. From the design of the digestion reaction, according to the core principle of the total nitrogen detection standard method, alkaline potassium persulfate as an oxidant can efficiently oxidize and digest various forms of nitrogen in the sludge sample, such as organic nitrogen, ammonia nitrogen, and nitrite nitrogen, into nitrate nitrogen under heating conditions. The alkaline environment can avoid the loss of nitrogen in the form of ammonia nitrogen during the digestion process, ensuring that the subsequent detection can accurately reflect the true content of total nitrogen in the sludge. At the same time, the mixed sample is injected into the digestion cup, which can be made of quartz material. The light transmittance does not affect the spectral signal collection, which is to provide a stable reaction container to ensure uniform and sufficient oxidation digestion reaction and meet the consistency requirements of the pretreatment for quantitative detection.

[0020] In step 101, the spectral data of the to-be-detected sludge sample during the oxidation digestion reaction and the compressed light intensity signal after the to-be-detected sludge sample absorbs and scatters in the digestion cup are collected.

[0021] In a specific implementation, first, an experimental device including a constant temperature digestion module, a spectral detection module, a compressed light intensity detection module, and a synchronous control module is built, for example, by open source hardware integration technology or spectral-light intensity collaborative detection technology. The spectral data of the sludge sample to be measured changing with time during the oxidation digestion reaction and the compressed light intensity signal of the sludge sample to be measured after absorption and scattering in the digestion cup are collected by the experimental device, for example, in the following manner: the constant temperature digestion module can use an intelligent constant temperature digestion instrument, and the digestion cup is placed in a constant temperature environment to ensure that the oxidation digestion reaction proceeds stably; the spectral detection module can use a fiber-optic ultraviolet-visible spectrometer, and the fiber-optic probe is aimed at the reaction area of the digestion cup, and spectral data is collected every 5-10 seconds, which can cover a wavelength band of 200-800 nm. A time sequence collection algorithm, such as a real-time data recording program developed based on a laboratory virtual instrument engineering platform, is used to record the spectral data and the corresponding time stamp synchronously, thereby forming the spectral data of the sludge sample to be measured changing with time during the oxidation digestion reaction. The compressed light intensity signal is collected based on the compressed sensing technology, and a preset random binary coding mask, such as a 512x1024 dimension random matrix generated by a matrix laboratory, is added to the light path as a coding template. When the probe light passes through the sludge sample to be measured in the digestion cup, the light signal after absorption and scattering by the sample is coded by the mask, and then a high-sensitivity charge-coupled detector receives the coded signal to form the compressed light intensity signal after absorption and scattering by the sludge sample to be measured in the digestion cup. The signal is converted into a digital quantity through an interface matched with the detector, and is accurately aligned with the spectral data in the time dimension by the synchronous control module. In other embodiments, other ways of collection can also be used, which are not limited here.

[0022] It should be noted that the spectral data in the present application represents a set of light signal values of the sludge sample to be measured changing with time during the oxidation digestion reaction. The change of the light intensity at a specific wavelength in the spectral data, such as the decrease of the absorption peak intensity and the peak shape shift, can quantitatively reflect the concentration change of the target pollutant, the rate and completion degree of the digestion reaction. The compressed light intensity signal represents the light intensity distribution signal after compression of the light intensity after absorption and scattering by the sludge sample to be measured in the digestion cup. The compressed light intensity signal retains the overall characteristics of the light intensity, such as the total absorption degree of the sample to light and the light intensity attenuation trend caused by particle scattering, and can reflect the macroscopic optical state of the sludge system, such as the change of the scattered light intensity caused by the breakage of sludge flocs during the digestion process.

[0023] In step 102, the compressed light intensity signal is equivalently reconstructed based on the sparse basis of the spectral data, and the full-band equivalent spectrum of the sludge sample to be measured during the digestion process is obtained.

[0024] In some embodiments, with reference to Figure 2As shown, the figure is an exemplary flow chart for determining the full-band equivalent spectrum in some embodiments of the present application, and the full-band equivalent spectrum of the sludge sample to be tested during the digestion process is obtained by equivalent reconstruction of the compressed light intensity signal based on the sparse basis of the spectrum data in the embodiments. First, in step 1021, the spectrum data is denoised, and the sparse basis of the denoised spectrum data is determined. Second, in step 1022, the linear relationship between the spectrum data and the compressed light intensity signal is determined. Finally, in step 1023, the compressed light intensity signal is equivalent reconstructed according to the sparse basis and the linear relationship, and the full-band equivalent spectrum of the sludge sample to be tested during the digestion process is obtained.

[0025] In specific implementation, first, the spectrum data is filtered and standardized to eliminate baseline drift to obtain denoised spectrum data, for example, the filtering can use the Savitzky-Golay filtering algorithm (window size is set to 11 points, and polynomial order is 2), and the spectrum data is locally polynomial fitted through a sliding window to smooth the noise while retaining the characteristic absorption peaks at 220 nm, 275 nm and total nitrogen; then, the sparse basis is constructed based on the denoised spectrum data, for example, K-singular value decomposition dictionary learning algorithm or discrete wavelet transform can be used for construction, taking K-singular value decomposition dictionary learning algorithm as an example, the spectrum data containing different total nitrogen concentrations and different digestion stages is used as a training sample, a random dictionary is initialized, the number of atoms is set to 1.5 times the number of samples, the atom dimension is consistent with the number of spectrum wavelengths, sparse coding is performed through iteration, the orthogonal matching pursuit algorithm is used to solve the sparse coefficients of each sample under the current dictionary and update the dictionary, and the dictionary atoms are updated by singular value decomposition of the residual matrix until the dictionary converges, the number of iterations can be set to 50 times, and the residual is less than 1e-4. The final obtained dictionary is the sparse basis of the spectrum data, wherein the sparse basis is a set of representative basic elements of the spectrum data; in other embodiments, other ways can also be used to determine the sparse basis, which is not limited here.

[0026] Second, the linear relationship between the spectrum data and the compressed light intensity signal is determined, which is specifically implemented as follows: the compressed light intensity signal y is linearly mapped from the spectrum data x by the coding matrix Φ, and satisfies y=Φx. The coding matrix Φ is a random binary mask matrix generated by the matrix laboratory used during acquisition, and the dimension is the number of detector pixels x the number of full-band wavelengths. The spectrum data x and the compressed light intensity signal y are fitted to determine the linear relationship between the spectrum data and the compressed light intensity signal, for example, the least squares method can be used for fitting; in other embodiments, other ways can also be used to determine the linear relationship, which is not limited here.

[0027] Finally, the equivalent reconstruction of the compressed light intensity signal based on the sparse basis and linear relationship is specifically implemented as follows: the spectral data x is expressed as the product of a sparse basis dictionary D and a sparse coefficient a (x=Da), which is converted into solving an optimization problem, for example, the orthogonal matching pursuit algorithm can be used to iteratively solve, the compressed light intensity signal is taken as an initialization residual, each time the column vector with the highest correlation with the residual (i.e. the most relevant atom) is selected from ΦD, the sparse coefficient a is updated and the residual is corrected, and the iteration is repeated until the residual is less than a preset threshold, such as 1% of the original signal energy, to obtain the sparse coefficient a; the sparse coefficient a is multiplied by the sparse basis dictionary D (x=Da), and the full-band equivalent spectrum of the sludge sample to be tested in the digestion process is reconstructed; in other embodiments, other ways can also be used for construction, which are not limited here.

[0028] It should be noted that the full-band equivalent spectrum in the present application represents the spectral data of the sludge sample to be tested in the oxidation digestion process, which is consistent with the key features of the original full-band spectrum, solves the problems of cost and data volume of traditional full-band measurement, and can support subsequent quantitative analysis, which is a key bridge from "theoretically feasible" to "engineering practical" for spectral detection of complex samples such as sludge.

[0029] In step 103, the characteristic variables with global contribution and the characteristic variables with local contribution of the full-band equivalent spectrum to the detection of total nitrogen in sludge are determined, and the key characteristic wavelengths representing the oxidation digestion process are selected from the full-band equivalent spectrum based on the characteristic variables with global contribution and the characteristic variables with local contribution.

[0030] In some embodiments, the determination of the characteristic variables with global contribution and the characteristic variables with local contribution of the full-band equivalent spectrum to the detection of total nitrogen in sludge can be achieved by the following steps: The full-band equivalent spectrum is standardized to obtain a standardized full-band equivalent spectrum; The characteristic variables with global contribution of the full-band equivalent spectrum to the total nitrogen concentration are calculated based on the standardized full-band equivalent spectrum; The standardized full-band equivalent spectrum is divided into multiple spectral subintervals, and the characteristic variables with local contribution of each spectral subinterval to the total nitrogen concentration are calculated.

[0031] In specific implementation, first, the full-band equivalent spectrum is standardized to obtain a standardized full-band equivalent spectrum, for example, the standard normal variable transformation is used for processing, that is, for each wavelength point of each sample spectrum, the conversion is performed according to (light intensity value-mean value of all wavelengths of the sample spectrum) / standard deviation of all wavelengths of the sample spectrum, so as to eliminate the overall shift of the spectrum caused by sludge particle scattering and instrument baseline drift. In other embodiments, other ways can also be used for standardization processing, which are not described here.

[0032] In addition, in specific implementation, the global contribution variable of the total nitrogen concentration with respect to the full-band equivalent spectrum can be calculated by using, for example, partial least squares regression, that is, the normalized full-band equivalent spectrum is used as an independent variable (each wavelength is a feature), the measured value of the total nitrogen concentration at the corresponding time is used as a dependent variable, a partial least squares model is constructed, the optimal principal component number is determined by 5-fold cross-validation, and generally 8-12 principal components are used to minimize the cross-validation error. The variable importance projection value output by the model is used to measure the global contribution of each wavelength, and the wavelengths with a variable importance projection value greater than 1 are selected as the feature variables of the global contribution. In other embodiments, the random forest regression algorithm or the recursive feature elimination algorithm can also be used for calculation, which is not limited here.

[0033] In addition, in specific implementation, the full-band equivalent spectrum is divided into a plurality of spectral subintervals, and the local contribution variable of the total nitrogen concentration with respect to each spectral subinterval is calculated. For example, the moving window partial least squares can be used to divide the full-band into continuous spectral subintervals with a window size of 10 nm and a step size of 2 nm, such as 200-210 nm, 202-212 nm, and the like. A partial least squares submodel is constructed for the spectral data and the total nitrogen concentration in each spectral subinterval. The principal component number is the same as that of the global model. The prediction root mean square error and the determination coefficient of each submodel are calculated. The determination coefficient reflects the fitting degree between the predicted value and the actual observation value of the model. The subintervals with a prediction root mean square error less than 0.5 mg / L and a determination coefficient greater than 0.9 are selected as the key local intervals, such as 215-225 nm, which correspond to the absorption characteristics of the key stage of total nitrogen digestion. All wavelengths in the interval are the feature variables of the local contribution. The window size can be adjusted according to the width of the characteristic peak, for example, the window size can be reduced to 5 nm for a narrow peak. In other embodiments, the interval division based on wavelet transform or the competitive adaptive reweighted sampling combined with interval analysis can also be used for calculation, which is not limited here.

[0034] It should be noted that the feature variables of the global contribution in the present application represent spectral characteristics that have a universal, stable and overall correlation effect on the detection of total nitrogen concentration in the full-band spectrum. The feature variables of the local contribution represent spectral characteristics that have a significant and targeted influence on the detection of total nitrogen concentration in a specific subinterval after the full-band spectrum is divided into a plurality of subintervals. The feature variables of the local contribution can provide a key basis for the quantitative prediction of total nitrogen concentration, form a global stable and local precise sludge total nitrogen detection system, and ultimately achieve efficient and accurate detection of sludge total nitrogen under different conditions.

[0035] In some embodiments, the key feature wavelengths representing the oxidation and digestion process can be selected from the full-band equivalent spectrum based on the feature variables of the global contribution and the feature variables of the local contribution by using the following steps: An initial feature variable pool after removing duplicate wavelengths is determined according to the feature variables of the global contribution and the feature variables of the local contribution; filtering a core candidate set of wavelengths from the full-band equivalent spectrum through the initial feature variable pool; obtaining a chemical mechanism of total nitrogen oxidation and digestion; filtering the core candidate set based on the chemical mechanism to obtain key feature wavelengths representing the oxidation and digestion process.

[0036] In a specific implementation, first, the initial feature variable pool after removing duplicate wavelengths is determined according to the globally contributed feature variables and the locally contributed feature variables, which can be implemented in the following manner, that is, removing duplicate wavelengths in the feature variables, for example, using set operations in Python to remove, and then sorting the wavelengths after removing duplicates in ascending order of numerical values to form an ordered initial feature variable pool after removing duplicate wavelengths; then, filtering a core candidate set of wavelengths from the full-band equivalent spectrum through the initial feature variable pool can be implemented in the following manner, that is, setting the number of neighbors and the number of features to be selected to calculate the discrimination weights of all wavelengths in the initial feature variable pool, and selecting wavelengths with a weight value > 0.1 as the core candidate set, for example, calculating through the relief F feature selection algorithm, and at the same time, a model containing 100 decision trees can be constructed by combining the feature importance class of the random forest regression, and the wavelengths with the top 30% feature importance are extracted for complementary verification to further ensure the reliability of the core candidate set; in other embodiments, other ways can also be used for implementation, which are not limited here.

[0037] Among them, the chemical mechanism of total nitrogen oxidation and digestion is obtained, for example, based on the known nitrogen form transformation rule, such as the chemical mechanism in the oxidation and digestion process, ammonium nitrogen (NH4⁺-N) is first converted into nitrite nitrogen (NO2⁻-N), and then further converted into nitrate nitrogen (NO3⁻-N), among them, ammonium nitrogen has characteristic absorption in the wavelength interval of 200-210 nm, nitrate nitrogen has strong absorption at 220 nm, and 275 nm is the interference absorption peak of organic matter oxidation products), the feature wavelength range related to the digestion process is sorted out, the feature wavelength range includes the target absorption interval: 200-230 nm, the interference interval: 270-280 nm, in other embodiments, other ways can also be used for implementation, which are not limited here.

[0038] In addition, in specific implementation, the core candidate set is screened based on the aforementioned chemical mechanism to obtain the key characteristic wavelengths characterizing the oxidation digestion process. This can be achieved by: screening the core candidate wavelengths using Boolean indexing, retaining wavelengths that fall within the target absorption range (200-230nm) and are not within the interference range (270-280nm). For example, the pandas library in Python can be used for screening. The screening results are then validated using a model to obtain the key characteristic wavelengths characterizing the oxidation digestion process. For example, this can be validated using partial least squares regression. The model is constructed with the equivalent spectral intensity of the full-band wavelengths corresponding to the screened wavelengths as the independent variable and the total nitrogen concentration as the dependent variable. The principal component count is set to 10. If the model's determination coefficient is >0.9 and the root mean square error of prediction is <0.4mg / L, then the wavelength set is determined to be the key characteristic wavelengths characterizing the oxidation digestion process. For example, the wavelengths corresponding to the absorption of ammonium nitrogen at 208nm and the wavelengths corresponding to the absorption of nitrate nitrogen at 220nm are finally screened. These wavelengths can accurately reflect the spectral changes corresponding to the nitrogen form transformation during the digestion process. Other screening methods can also be used in other embodiments, which are not limited here.

[0039] It should be noted that the key characteristic wavelengths in this application represent the key specific signal carriers at the spectral level during the total nitrogen oxidation and digestion process. They avoid the absorption wavelengths of interfering substances and can be used to build a bridge between spectral signals and the chemical digestion process, so as to achieve the goal of rapidly and accurately back-calculating the total nitrogen concentration of sludge based on spectral data.

[0040] In step 104, a sample similarity matrix is ​​constructed based on the differences in the spectrum of the spectral data at different times and the changes in light intensity in the wavelength range of the compressed light intensity signal to determine the similarity between the sludge samples to be tested at different times during the oxidation and digestion process.

[0041] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flowchart for determining the sample similarity matrix in some embodiments of this application. In this embodiment, the sample similarity matrix for constructing the similarity relationship between the sludge samples to be tested at different times during the oxidation digestion process can be achieved by the following steps based on the differences in the spectrum of the spectral data at different times and the changes in light intensity in the wavelength range of the compressed light intensity signal: First, in step 1041, the differences in the spectra of the spectral data at different times are determined; Secondly, in step 1042, the intensity variation in the wavelength range of the compressed light intensity signal is determined; Then, in step 1043, the similarity between the sludge samples to be tested at any two times is calculated based on the differences and the changes in light intensity. Finally, in step 1044, a sample similarity matrix of the similarity relationship between the to-be-tested sludge samples at different time points in the oxidation digestion process is constructed through all the similarities.

[0042] In a specific implementation, first, the difference of the spectral data at different time points is determined, and the spectral curve at each time point is extracted from the equivalent spectrum of the full waveband in time sequence, such as 0 min, 5 min, 10 min, and the like in the digestion process. The L2 norm of the spectral vectors at any two time points is calculated, for example, the Euclidean distance is used to calculate the difference function, that is, √Σ(x_i-x_j)², where x_i and x_j are the light intensity values at the same wavelength at two time points. At the same time, the cosine similarity is used for auxiliary verification, and all L2 norms are standardized to the interval [0, 1] as the difference, for example, the minimum-maximum normalization method is used for standardization. The difference indicates the difference degree of the spectral data at different time points. Second, the light intensity change in the wavelength interval in the compressed light intensity signal is determined, and the wavelength interval corresponding to the compressed light intensity signal is divided into a target absorption interval and an interference interval according to the key characteristic wavelength. The absolute change amount of the light intensity at two time points is calculated for each interval. The absolute change amount indicates the difference between the mean values of the light intensity in the interval, and all the absolute change amounts are standardized to the interval [0, 1] as the light intensity change. The light intensity change indicates the light intensity change in the wavelength interval in the compressed light intensity signal. In other embodiments, other ways can also be used to determine the light intensity change, which is not limited here.

[0043] In addition, in a specific implementation, the similarity of samples at any two time points is calculated, for example, the weighted fusion strategy is used to calculate the similarity, and the weights ω1 and ω2 are determined through grid search, such as ω1=0.6 and ω2=0.4. The similarity=1-(ω1×difference index+ω2×light intensity change) ensures that the smaller the difference and the slower the light intensity change, the closer the similarity to 1. Finally, a sample similarity matrix is constructed, an n*n matrix (n is the number of time points) is initialized, the similarity of all two-time points is filled into the matrix according to the corresponding time index, and the diagonal elements are set to 1, that is, the similarity of the same time sample to itself is 1. A symmetric matrix reflecting the similarity relationship between samples at different time points is obtained. The symmetric matrix is used as the sample similarity matrix of the similarity relationship between the to-be-tested sludge samples at different time points in the oxidation digestion process of the to-be-tested sludge sample. In other embodiments, other ways can also be used to determine the sample similarity matrix, which is not limited here.

[0044] It should be noted that the sample similarity matrix in the present application represents the similarity degree of the similarity relationship between the to-be-tested sludge samples at different time points in the oxidation digestion process of the to-be-tested sludge sample. The sample similarity matrix can be used for sample clustering of the samples at different time points in the oxidation digestion process, classifying the samples at similar time points into one category, dividing different stages of digestion, and detecting abnormal samples, so as to facilitate in-depth understanding of the sample change process and optimization of experimental parameters.

[0045] In step 105, the total nitrogen concentration in the sludge sample to be measured is quantitatively inversed by the key feature wavelength and the sample similarity matrix to obtain the total nitrogen inversion concentration of the sludge sample to be measured.

[0046] In some embodiments, the total nitrogen concentration in the sludge sample to be measured is quantitatively inversed by the key feature wavelength and the sample similarity matrix to obtain the total nitrogen inversion concentration of the sludge sample to be measured can be achieved by the following steps: The key feature wavelength is converted into a feature matrix of time-key wavelength number; The sample similarity matrix is converted into a diagonal weight matrix of the sludge sample to be measured at different times; The feature matrix and the diagonal weight matrix are fused to obtain a weighted feature matrix; A quantitative inversion model of the weighted feature matrix and the total nitrogen concentration in the sludge sample to be measured is constructed; The initial inversion concentration of total nitrogen in the sludge sample to be measured is determined by the quantitative inversion model; The similarity score of the sludge sample to be measured and the sample with known concentration is determined; The initial inversion concentration is corrected according to the similarity score to obtain the total nitrogen inversion concentration of the sludge sample to be measured.

[0047] In specific implementation, first, the key feature wavelength is converted into a feature matrix of time-key wavelength number, for example, if k key feature wavelengths are screened out and there are n monitoring times in the digestion process, the light intensity values corresponding to the k key wavelengths at each time are extracted from the full-band equivalent spectrum, which is organized into a two-dimensional array of n rows and k columns by the array creation library of Python, the rows correspond to the time and the columns correspond to the key wavelength, that is, the feature matrix X (dimension n x k), wherein the feature matrix represents a two-dimensional data structure carrying the time-key wavelength core spectral information; then the sample similarity matrix is converted into a diagonal weight matrix, the average similarity of each time corresponding row in the sample similarity matrix (n x n) is calculated, the average similarity reflects the association strength of the sample at this time and the overall sample, the average similarity array is converted into an n x n diagonal weight matrix W, only the average similarity value on the diagonal line, the rest of the elements are 0, realizing the weight distribution of samples at different times, the higher the average similarity, the greater the weight corresponding to the time, the conversion can be realized by using the diagonal line function of Python array, wherein the diagonal weight matrix represents a diagonal matrix of the importance of samples at different times; in other embodiments, other ways can also be used for conversion, which is not limited here.

[0048] In addition, in specific implementation, the weighted feature matrix is obtained by fusing the feature matrix and the diagonal weight matrix, that is, element-level multiplication is performed on the feature matrix and the diagonal weight matrix to obtain the weighted feature matrix, which can be implemented by using the broadcast mechanism of the array creation function of Python, so that the key wavelength light intensity value at each moment is multiplied by the weight at the corresponding moment, and the weighted feature matrix represents a matrix highlighting the feature contribution of the moment with high correlation degree.

[0049] In the quantitative inversion model, a partial least squares regression algorithm is used to adapt the linear / weak nonlinear correlation between the spectral features and the concentration, the partial least squares regression algorithm is imported from a cross-decomposition module of a machine learning library in Python, the weighted feature matrix is used as an independent variable, and the measured concentration of the total nitrogen in the sludge at the corresponding moment is used as a dependent variable, 70% of the data is divided into a training set, and 30% is divided into a test set, the number of principal components is set to 8-12, the optimal number of principal components is determined by 5-fold cross-validation, the cross-validation determination coefficient is maximized, and the quantitative inversion model is trained to obtain a quantitative inversion model, which is a machine learning model representing the correlation between the spectral features and the total nitrogen concentration. The quantitative inversion model can be used to convert input spectral feature data into a corresponding total nitrogen concentration prediction value. In other embodiments, other methods can be used to construct the quantitative inversion model, which is not limited here.

[0050] In addition, in specific implementation, the initial inversion concentration of the total nitrogen in the to-be-measured sludge sample is determined, the complete weighted feature matrix is input into the trained quantitative inversion model, the model is called to predict the weighted feature matrix to output the initial inversion concentration of the total nitrogen at each moment, and the initial inversion concentration represents the total nitrogen concentration prediction value without bias correction. The similarity score of the to-be-measured sludge sample and the known concentration sample is determined, the weighted feature matrices of a plurality of groups of known total nitrogen concentration sludge samples are collected, the cosine similarity of each sample vector in the weighted feature matrix of the to-be-measured sludge sample and the weighted feature matrix of the known total nitrogen concentration sludge sample is calculated, the mean value of all cosine similarities is taken as the similarity score, and the similarity score represents a numerical value of the similarity degree of the sample at the monitoring moment and the sample at other moments. The higher the score is, the closer the characteristics of the to-be-measured sample and the known sample are. In other embodiments, other methods can be used to determine the similarity score, which is not limited here.

[0051] In addition, in specific implementation, the initial inversion concentration is corrected in deviation according to the similarity score, to obtain the total nitrogen inversion concentration of the to-be-tested sludge sample, that is, taking the similarity score and the initial inversion concentration of the known sample as independent variables, and taking the total nitrogen inversion concentration as a dependent variable, a correction formula is fitted, for example, the least square function in the optimization module of the SciPy library can be used for fitting, to obtain the correction formula total nitrogen inversion concentration = initial inversion concentration * (0.8 * similarity score + 0.2), and finally the total nitrogen inversion concentration of the to-be-tested sludge sample is obtained; in other embodiments, other ways can be used for correction, which is not limited here.

[0052] It should be noted that the total nitrogen inversion concentration in the present application represents the accurate concentration value representing the total nitrogen content of the sludge after deviation correction, which is directly used as the quantitative result of the total nitrogen concentration of the sludge and can be used in the actual application scenarios of sludge treatment process optimization, such as adjusting the digestion time, reagent dosage, and pollutant emission evaluation.

[0053] In addition, another aspect of the present application, in some embodiments, the present application provides a pollutant detection system in a sludge treatment process, referring to Figure 4 The figure is a structural schematic diagram of a pollutant detection system in a sludge treatment process according to some embodiments of the present application, the pollutant detection system 400 in the sludge treatment process includes a collection module 401, a processing module 402 and an execution module 403, which are described as follows: The collection module 401 is mainly used for collecting the spectral data of the to-be-tested sludge sample changing with time in the oxidation and digestion reaction process and the compressed light intensity signal after absorption and scattering of the to-be-tested sludge sample in the digestion cup in the present application; The processing module 402 is used for equivalent reconstruction of the compressed light intensity signal based on the sparse basis of the spectral data, to obtain the full-waveband equivalent spectrum of the to-be-tested sludge sample in the digestion process in the present application; It should be noted that the processing module 402 is also used for determining the characteristic variable with global contribution and the characteristic variable with local contribution of the full-waveband equivalent spectrum to the total nitrogen detection of the sludge, and screening the key characteristic wavelength representing the oxidation and digestion process from the full-waveband equivalent spectrum based on the characteristic variable with global contribution and the characteristic variable with local contribution; In addition, it should be noted that the processing module 402 is also used for constructing the sample similarity matrix of the similarity relationship between the to-be-tested sludge samples at different time in the oxidation and digestion process of the to-be-tested sludge sample according to the difference of the spectral data at different time and the light intensity change of the wavelength interval in the compressed light intensity signal in the present application; The execution module 403 in this application is mainly used to quantitatively invert the total nitrogen concentration in the sludge sample to be tested by using the key feature wavelength and the sample similarity matrix, so as to obtain the total nitrogen inversion concentration of the sludge sample to be tested.

[0054] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the pollutant detection method described above in the sludge treatment process.

[0055] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device for implementing a pollutant detection method in the sludge treatment process, according to some embodiments of this application. The pollutant detection method in the sludge treatment process described in the above embodiments can be achieved through... Figure 5 The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0056] Processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0057] The communication bus 502 can be used to transmit information between the aforementioned components.

[0058] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CDROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0059] The memory 503 is configured to store program codes of the embodiments of the present application, and the processor 501 is configured to execute the program codes stored in the memory 503. The program codes can include one or more software modules. The methods used in the above-described embodiments can be implemented by the processor 501 and one or more software modules in the program codes in the memory 503.

[0060] The communication interface 504 is configured to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc., using any transceiver-like mechanism.

[0061] In a specific implementation, as an example, the computer device can include a plurality of processors, each of which can be a single CPU processor or a multi-CPU processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0062] The computer device described above can be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0063] In addition, the present application also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned sludge treatment process and pollutant detection method.

[0064] Although the preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to cover all changes and modifications falling within the scope of the present application.

[0065] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting pollutants during sludge treatment, wherein, The method involves pre-mixing the sludge sample to be tested with alkaline potassium persulfate oxidant and injecting it into a digestion cup for oxidative digestion to detect the total nitrogen of pollutants in the sludge. The method is characterized by the following steps: The spectral data of the sludge sample under test during the oxidation and digestion reaction and the compressed light intensity signal after absorption and scattering in the digestion cup were collected. The compressed light intensity signal is equivalently reconstructed based on the sparse basis of the spectral data to obtain the full-band equivalent spectrum of the sludge sample during the digestion process. The characteristic variables that contribute globally and locally to the detection of total nitrogen in sludge by the full-band equivalent spectrum are determined, and key characteristic wavelengths characterizing the oxidation and digestion process are screened from the full-band equivalent spectrum based on the global and local contribution characteristic variables. Based on the differences in the spectrum at different times and the changes in light intensity in the wavelength range of the compressed light intensity signal, a sample similarity matrix is ​​constructed to show the similarity between the sludge samples under test at different times during the oxidation and digestion process. The total nitrogen concentration in the sludge sample to be tested is quantitatively retrieved by using the key characteristic wavelength and the sample similarity matrix, thus obtaining the total nitrogen inversion concentration of the sludge sample to be tested.

2. The method as described in claim 1, characterized in that, Based on the sparse basis of the spectral data, the compressed light intensity signal is equivalently reconstructed to obtain the full-band equivalent spectrum of the sludge sample during digestion. Specifically, this includes: The spectral data is denoised, and the sparse basis of the denoised spectral data is determined. Determine the linear relationship between the spectral data and the compressed light intensity signal; The compressed light intensity signal is reconstructed based on the sparse basis and the linear relationship to obtain the full-band equivalent spectrum of the sludge sample during the digestion process.

3. The method as described in claim 1, characterized in that, The specific characteristic variables that contribute globally and locally to the detection of total nitrogen in sludge using the full-band equivalent spectrum include: The full-band equivalent spectrum is standardized to obtain a standardized full-band equivalent spectrum. The characteristic variables of the global contribution of the full-band equivalent spectrum to the total nitrogen concentration are calculated based on the standardized full-band equivalent spectrum. The standardized full-band equivalent spectrum is divided into multiple spectral sub-intervals, and the characteristic variables of the local contribution of each spectral sub-interval to the total nitrogen concentration are calculated.

4. The method as described in claim 1, characterized in that, Based on the feature variables of the global contribution and the feature variables of the local contribution, key characteristic wavelengths characterizing the oxidation digestion process are screened from the full-band equivalent spectrum, specifically including: The initial feature variable pool after removing duplicate wavelengths is determined based on the feature variables of the global contribution and the feature variables of the local contribution. The core candidate set of wavelengths is selected from the full-band equivalent spectrum through the initial feature variable pool; To obtain the chemical mechanism of total nitrogen oxidation and decomposition; Based on the aforementioned chemical mechanism, the core candidate set was screened to obtain key characteristic wavelengths that characterize the oxidative digestion process.

5. The method as described in claim 1, characterized in that, Based on the differences in the spectrum of the spectral data at different times and the changes in light intensity within the wavelength range of the compressed light intensity signal, a sample similarity matrix is ​​constructed to represent the similarity between the sludge samples under test at different times during the oxidation digestion process. Specifically, this matrix includes: Determine the differences in the spectra of the spectral data at different times; Determine the intensity variation within the wavelength range of the compressed light intensity signal; Calculate the similarity between any two sludge samples at any given time based on the differences and the changes in light intensity. A sample similarity matrix was constructed based on all similarities to determine the similarity relationships between the sludge samples at different times during the oxidation and digestion process.

6. The method as described in claim 1, characterized in that, The total nitrogen concentration in the sludge sample to be tested is quantitatively retrieved using the key characteristic wavelengths and the sample similarity matrix. Specifically, the retrieved total nitrogen concentration of the sludge sample to be tested includes: Convert the key feature wavelengths into a feature matrix of time-key wavelength number; The sample similarity matrix is ​​converted into a diagonal weight matrix of the sludge sample to be tested at different times; The feature matrix is ​​fused with the diagonal weight matrix to obtain a weighted feature matrix; Construct a quantitative inversion model of the weighted feature matrix and the total nitrogen concentration in the sludge sample to be tested; The initial inversion concentration of total nitrogen in the sludge sample to be tested was determined using the quantitative inversion model. Determine the similarity score between the sludge sample to be tested and the sample with known concentration; The initial inversion concentration is corrected for deviation based on the similarity score to obtain the total nitrogen inversion concentration of the sludge sample to be tested.

7. The method as described in claim 1, characterized in that, The digestion cup is made of quartz.

8. The method as described in claim 1, characterized in that, An experimental setup was constructed that included a constant-temperature digestion module, a spectral detection module, a compressed light intensity detection module, and a synchronous control module. The spectral data of the sludge sample under test during the oxidative digestion reaction and the compressed light intensity signal after absorption and scattering of the sludge sample under test in the digestion cup were collected.

9. The method as described in claim 1, characterized in that, Total nitrogen includes organic nitrogen, ammonia nitrogen, and nitrite nitrogen.

10. A pollutant detection system for sludge treatment, characterized in that, include: The acquisition module is used to acquire the spectral data of the sludge sample under test during the oxidation and digestion reaction process and the compressed light intensity signal after absorption and scattering of the sludge sample under test in the digestion cup. The processing module is used to perform equivalent reconstruction of the compressed light intensity signal based on the sparse basis of the spectral data to obtain the full-band equivalent spectrum of the sludge sample during the digestion process. The processing module is further configured to determine the feature variables that contribute globally and locally to the detection of total nitrogen in sludge from the full-band equivalent spectrum, and to screen out key feature wavelengths characterizing the oxidation and digestion process from the full-band equivalent spectrum based on the feature variables that contribute globally and locally. The processing module is also used to construct a sample similarity matrix based on the differences in the spectrum of the spectral data at different times and the changes in light intensity in the wavelength range of the compressed light intensity signal, to determine the similarity between the sludge samples to be tested at different times during the oxidation digestion process. The execution module is used to quantitatively invert the total nitrogen concentration in the sludge sample to be tested using the key feature wavelength and the sample similarity matrix, so as to obtain the total nitrogen inversion concentration of the sludge sample to be tested.

Images