A full-spectrum water quality detection method with self-consistent compensation for transient interference

Abstract

This invention discloses a full-spectrum water quality detection method with self-consistent compensation for transient interference, comprising: acquiring absorbance; performing transient interference separation processing on the absorbance to generate absorbance after stripping scattering; performing thin-film interference de-curling processing on the absorbance after stripping scattering to generate de-curled absorbance; and quantitatively calculating the target analyte concentration vector based on the de-curled absorbance. This invention can cascade compensation for two key types of interference: transient scattering and thin-film interference, and achieves self-consistent closed-loop processing through uncertainty propagation and self-cleaning triggering, thereby improving the accuracy and robustness of online water quality detection.

Description

Technical Field

[0001] This invention belongs to the field of online water quality monitoring, and in particular, it is a full-spectrum water quality detection method with self-consistent compensation for transient interference. Background Technology

[0002] Full-spectrum analysis technology is a core tool for online water quality monitoring, and it is of great significance for ensuring drinking water safety, monitoring surface water environment, and optimizing industrial process water treatment. This technology can non-invasively and rapidly acquire optical fingerprint information of multiple components in water, such as nitrates, chemical oxygen demand (COD), and dissolved organic carbon (DOC), enabling real-time, multi-parameter assessment of water quality.

[0003] Currently, full-spectrum water quality monitoring systems are already in practical application. These systems typically use immersion spectral probes to acquire the raw transmission or absorption spectra of water bodies. To cope with complex field environments, existing technologies employ a series of data processing and maintenance methods. For example, a reference optical channel is set up to compensate for slow drift in light source intensity, or dark current measurements are used to subtract the sensor's background noise. For random noise in spectral data, spectral smoothing algorithms, such as moving averages or Savitzky-Golay filtering, are employed. For signal drift caused by biofilm contamination of the probe's optical window, current methods primarily rely on periodic, pre-set automatic cleaning devices (such as mechanical scrapers or high-pressure water jets) for physical removal, or offline manual maintenance by maintenance personnel.

[0004] However, existing full-spectrum water quality detection technologies suffer from insufficient fine-grained compensation when faced with complex and dynamic field disturbances, especially in simultaneously handling transient and slowly varying disturbances with vastly different characteristics. Specifically, this manifests in a lack of robust identification capability for sub-second transient disturbances and the inability to quantify and compensate for biofilm contamination within the probe window online in real time. Summary of the Invention

[0005] The purpose of this invention is to provide a full-spectrum water quality detection method with self-consistent compensation for transient interference, so as to solve the above-mentioned problems existing in the prior art.

[0006] The technical solution is a full-spectrum water quality detection method with self-consistent compensation for transient interference, including:

[0007] Obtain absorbance;

[0008] Transient interference separation processing is performed on the absorbance to generate absorbance after stripping scattering;

[0009] Thin-film interference de-curling processing is performed on the absorbance after stripping and scattering to generate de-curling absorbance;

[0010] Based on the de-curling absorbance, the concentration vector of the target analyte is quantitatively calculated.

[0011] Beneficial effects: This invention can compensate for two key types of interference, transient scattering and thin-film interference, in a cascade manner, and achieve a self-consistent closed loop through uncertainty propagation and self-cleaning triggering, thereby improving the accuracy and robustness of online water quality detection. Attached Figure Description

[0012] Figure 1 A flowchart illustrating the steps of a full-spectrum water quality detection method with transient interference self-consistent compensation provided in this application embodiment.

[0013] Figure 2 This is a flowchart illustrating the steps for generating de-curling absorbance in an embodiment of this application.

[0014] Figure 3 A flowchart illustrating the steps for generating absorbance after stripping and scattering, as provided in an embodiment of this application.

[0015] Figure 4 A flowchart illustrating the steps for obtaining event tags provided in this application embodiment. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0018] The study found that existing methods struggle to accurately identify and separate transient scattering interference caused by bubbles and suspended particles. Simply relying on time-domain threshold detection, such as monitoring the rapid rate of change of a single channel signal, easily misinterprets rapid fluctuations in water quality or circuit noise as interference, leading to incorrect data rejection. Conversely, detection methods relying solely on spectral morphology are too slow to capture sub-second transient events. This fragmented time-spectral information approach results in both false negatives and false positives for transient interference. Furthermore, existing technologies treat biofilm contamination at optical windows as a slow-varying baseline drift that can only be reset through physical cleaning, neglecting the thin-film interference effect introduced by the biofilm as an optical thin film. This interference effect superimposes periodic ripples on the spectrum. These ripples differ significantly from the absorption peaks of the contaminants but are multiplied or added to their absorption signals, causing significant quantitative calculation errors. Current technologies lack the ability to model this interference effect online, making it impossible to invert the equivalent optical thickness of the biofilm in real time, and therefore unable to mathematically separate it from the spectrum at the signal level. Moreover, the two types of interference are coupled: transient scattering interference that is not effectively separated will further distort the spectral morphology, and the noise it introduces will make the already weak thin-film interference fringes even more difficult to identify, causing the compensation algorithms for both interference sources to fail, ultimately damaging the accuracy and continuity of water quality monitoring data.

[0019] Specifically, in online water quality monitoring, full-spectrum probes, such as immersion UV-Vis spectral probes, face two main types of interference sources with distinctly different characteristics. The first type is transient interference, such as sub-second rapid scattering events caused by bubbles, suspended particles, or impurities flowing through the water. This type of interference causes a sharp, non-specific baseline rise or distortion in the spectral signal within a short period. The second type is slowly varying interference, typically such as biofilms that gradually grow on the probe's optical window. These biofilms form optical thin films, producing thin-film interference effects that result in periodic curls superimposed on the spectral data. Existing technologies typically address these two types of interference in isolation. For example, transient interference might be removed using simple time-domain thresholding, but this leads to data loss; while biofilm interference usually relies on offline, periodic physical cleaning and maintenance, making it impossible to compensate for its effects in real time during measurement.

[0020] like Figure 1 As shown, a full-spectrum water quality detection method with self-consistent compensation for transient interference is proposed, including:

[0021] Obtain absorbance.

[0022] Alternatively, one can say that by obtaining the absorbance and performing preprocessing, an optimized absorbance can be obtained.

[0023] Specifically, obtaining absorbance can include: acquiring the original spectral time series signal S using a spectrometer. raw (λ, t), and simultaneously acquire the reference spectrum S used for calibration. ref The absorbance A(λ,t) and dark current D(λ,t) are given by the time interval. λ represents wavelength, and t represents time. The absorbance A(λ,t) can be obtained through a logarithmic transformation, for example, A(λ,t) = -logt. 10 (T(λ,t)), and the transmittance T(λ,t) is calculated from the original signal, the reference signal, and the dark current signal, for example, T = (S raw - D) / (S ref -D). In some preferred embodiments, obtaining absorbance may also include a series of online self-calibration processes, such as dark current subtraction, reference normalization, and peak drift compensation, to ensure that the obtained absorbance A(λ,t) has high stability and accuracy.

[0024] Transient interference separation processing is performed on the absorbance to generate the absorbance after stripping scattering.

[0025] In this embodiment, rapid scattering events caused by bubbles or particles are processed. Exemplarily, transient interference separation processing can be an algorithm based on signal processing and state estimation. The absorbance A after stripping scattering is... deSca (λ, t) refers to the estimated scattering component S subtracted from the original absorbance A(λ, t). sca The net spectrum obtained after (λ, t). That is, A deSca = A -S sca In some preferred embodiments, an index (e.g., transient scattering index) that can sensitively reflect transient events is constructed. Event labels are generated based on this index. For the time period labeled as an event, a state-space model is established, and a recursive filter is applied to dynamically estimate and separate the scattering component S. sca .

[0026] Thin-film interference de-curling processing is performed on the absorbance after stripping and scattering to generate de-curled absorbance.

[0027] In this embodiment, the thin-film interference effect caused by the biological membrane is addressed. Since the absorbance after stripping scattering already provides absorbance A free from transient scattering... deSca This provides a clean input for interference feature extraction. For example, thin-film interference de-creep processing can be a solution process based on an inverse optical problem. De-creep absorbance A clean (λ, t) refers to the absorbance A after stripping and scattering. deSca The spectrum representing the true absorption of water quality was obtained after removing interference fringes modulation. In some preferred embodiments, the absorbance A after stripping scattering is... deScaExtract fringe features, such as fringe spacing; obtain the equivalent optical thickness of the biofilm by inverse solving based on the thin-film optical model; construct a forward interference model based on this thickness and solve the inverse optical problem to obtain the de-fringe absorbance A. clean .

[0028] Based on the de-curling absorbance, the concentration vector of the target analyte is quantitatively calculated.

[0029] Specifically, the target concentration vector Q targets (t) can include various pollutants or water quality parameters in the water body, such as nitrate concentration, dissolved organic carbon (DOC), color, etc. For example, quantitative calculations can employ methods from chemometrics, such as least-squares constraint solving based on the Lambert-Beer law, i.e., solving for A. clean = E · Q targets Where E is a known extinction coefficient matrix. Due to the decremented absorbance A... clean The two main interferences, transient scattering and thin-film interference, have been self-consistently compensated for, therefore, based on the de-crissional absorbance A... clean The calculated target concentration vector Q targets (t) has higher accuracy and robustness.

[0030] Accordingly, a full-spectrum water quality detection system with self-consistent compensation for transient interference is also proposed. This system may include, but is not limited to, a spectrometer, a light source, an optical probe, and one or more data processors. The data processor is configured to execute a full-spectrum water quality detection method with self-consistent compensation for transient interference. Exemplarily, the data processor may internally include multiple functional modules. Specifically, the system includes:

[0031] The transient interference separation module is used to receive absorbance and perform transient interference separation processing on absorbance to generate absorbance after stripping scattering;

[0032] The thin-film interference de-curling module is used to receive the absorbance after stripping and scattering, and to perform thin-film interference de-curling processing on the absorbance after stripping and scattering to generate de-curling absorbance.

[0033] The quantitative calculation module is used to receive the de-curling absorbance and quantitatively calculate the target analyte concentration vector based on the de-curling absorbance.

[0034] In one exemplary embodiment, after obtaining the absorbance, online self-calibration and quality gating are performed, specifically including:

[0035] Perform data collection and synchronization.

[0036] In this embodiment, not only is the main sensor data acquired, but also auxiliary data for self-calibration and quality assessment. It is understood that this system may include a main spectral channel and one or more auxiliary channels; wherein the main spectral channel is used to acquire the full-spectrum signal. Exemplarily, the raw full-spectrum signal S is acquired. raw (λ, t), reference channel signal S ref (λ, t) and dark current signal D(λ, t); acquire independent high-speed channel signal I fast (t), for example, this channel uses a high-sampling-rate photodiode to capture the temporal characteristics of transient events; it acquires a set of sensor health vectors H sens (t) can be internal state parameters read from the device driver layer, such as light source current, light source temperature, spectrometer detector temperature, circuit gain settings, and power supply voltage. After acquisition, the timestamps of different channels are interpolated, aligned, and clock drift corrected to ensure that all data (S) are accurate. raw S ref D, I fast H sens The data stream is synchronized over time t. In some preferred embodiments, a packet loss and resampling strategy is also included. For example, missing data points are checked in the data stream; for short-term missing data points, such as 1-2 sampling points, local spline interpolation or linear interpolation can be performed; for missing data points exceeding a preset time limit, such as 5 sampling points, no interpolation is performed, and a resampling / invalidity flag M is recorded. resamp (t). All non-spectral auxiliary data, such as I fast (t), H sens (t), M resamp (t) and other calibration parameters, such as temperature T(t), are encapsulated as a synchronization data packet P. sync (t).

[0037] Perform calibration and marking.

[0038] In this embodiment, the raw signal is converted into preliminary absorbance, and unreliable regions in the data are marked. Specifically, this process includes performing dark current temperature regression correction. It is understood that the sensor's dark current D(λ, t) is significantly affected by temperature. Preferably, instead of using a fixed dark current template, a synchronization data packet P is utilized. sync The detector temperature T(t) collected in (t) is used to calculate the dynamic dark current D at the current temperature using a pre-calibrated regression model f(λ, T). corr (λ, t) = f(λ, T(t)); where the regression model can be a multinomial model or a lookup table. For the reference channel signal S... ref (λ, t) itself also performs dark current subtraction (using D) corrIt can selectively perform sliding baseline correction to compensate for slow drift of the light source, resulting in a corrected reference spectrum S. ref_corr (λ, t). The transmittance T(λ, t) is calculated using the corrected signal, and the formula is: T(λ, t) = (S raw (λ,t) - D corr (λ,t)) / (S ref_corr (λ,t) - D corr (λ, t)). Calculate the initial absorbance A. pre (λ, t) = -log 10 (T(λ, t)). During this process, extremely low transmittance values ​​are thresholded to avoid calculating the initial absorbance A. pre Infinity values ​​appear. Because the Lambert-Beer law exhibits nonlinearity in the high-concentration (high-absorbance) region due to internal filtration effects, these regions need to be labeled. Specifically, an upper limit threshold for the instrument's linear response is set, for example, A. th = 2.5 AU. For the initial absorbance A pre For all bands λ and time t in (λ, t) that exceed the threshold, generate range label B. range (λ, t), which is labeled as nonlinear or unreliable. This B range The tag will be passed, for example, packed into P. sync (t), and used in subsequent quantitative calculations to reduce or correct the high concentration region.

[0039] Perform peak position drift compensation.

[0040] In this embodiment, the drift of the spectrometer peak position (wavelength axis) caused by temperature changes or minor mechanical vibrations is corrected. Specifically, one or more stable and distinctive anchor points λ are selected in the spectrum. anchors These can be known characteristic absorption peaks of the water body itself, such as in the near-infrared band, or markers built into the instrument. Using anchor point λ... anchors Based on the current preliminary absorbance A pre Cross-correlation calculations are performed between (λ, t) and a standard template, and the sub-pixel-level peak offset Δλ(t) is calculated using methods such as spline interpolation or parabolic fitting. In some preferred embodiments, the spectral drift may not be linearly consistent across different wavelength bands. In this case, the spectrum can be divided into multiple segments, and each segment can be registered independently, or a nonlinear polynomial function can be used to fit the distortion of the entire spectrum, followed by resampling to obtain the registered absorbance A. reg (λ, t). After registration, calculate the registration residual R. align (t), for example, the root mean square error between the registered spectrum and the template. R align(t) serves as a confidence index; a high residual indicates registration failure or severe noise in the spectrum itself.

[0041] Implement quality gating during the data acquisition phase.

[0042] In this embodiment, a comprehensive score is performed on the quality of the data acquired at time t. Specifically, a quality score Q is calculated. s1 (t) is used as a comprehensive indicator, preferably by integrating multiple quality indicators through a weighted function. For example: Q s1 (t)= f(w H · H sens_score (t), w M · M resamp_score (t), w R · R align_score (t)); where H sens_score (t) is based on the sensor health status H sens The score obtained by (t), for example, if the light source temperature exceeds H sens The best range in the range is low; M resamp_score (t) is based on the resampling flag M resamp (t) The score obtained, for example, if M resamp If (t) is true, the score is 0; R align_score (t) is based on the registration confidence level R. align The score obtained by (t), for example, if R align (t) is very high, so the score should be low; w H w M w R These are their respective preset weights. Based on the calculated quality score Q... s1 (t), setting the quality gate threshold, for example, Q. th = 0.7. If Q s1 (t) < Q th Then the data at time t is marked as a low-quality sample, labeled B. lowq (t). Finally, the absorbance A(λ,t) after online self-calibration (i.e., A) is obtained. reg (λ, t) and the updated synchronization data packet P sync (t), this data packet is now rich in quality information, including but not limited to B. range (λ, t) (nonlinear notation), Q s1 (t) (overall quality score) and B lowq (t) (low-quality sample markers). These markers will be used to guide the algorithm in subsequent interference separation and quantitative calculations, thus reflecting self-consistency.

[0043] like Figure 3As shown, according to one aspect of this application, generating absorbance after stripping scattering includes:

[0044] Obtain event labels to identify time periods in absorbance where transient interference exists.

[0045] like Figure 4 As shown, in one optional implementation, obtaining the event label includes:

[0046] Obtain the transient scattering index.

[0047] In a preferred embodiment, obtaining the transient scattering index includes: acquiring a high-speed channel signal; determining a time-domain component based on the rapid time-domain changes of the high-speed channel signal; determining a spectral component based on the spectral slope anomaly of absorbance; and fusing the time-domain component and the spectral component to generate the transient scattering index. The transient scattering index I is... ts (t) is preferably formed by a weighted fusion of time-domain components and spectral-domain components. Specifically, the time-domain component I is determined. t (t), used to capture the rapid temporal dynamics of events. It can be obtained from the synchronization data packet P. sync Obtain the high-speed channel signal I from (t) fast (t), this signal has a high sampling rate, such as 100Hz, and is sensitive to rapid changes in transmittance. For the high-speed channel signal I... fast (t) Apply a bandpass filter, for example, preserving the frequency components from 2Hz to 20Hz, to filter out slow baseline drift and high-frequency electronic noise. Calculate the time derivative of the filtered signal, for example, the first-order difference dI. fast / dt, and then standardize it, for example, calculate the z-score, i.e., z-score = (x - μ) / σ, where z-score is the standardized score, x is the time derivative value, μ is the mean of the time derivative sequence, and σ is the standard deviation of the time derivative sequence, thus obtaining the time domain component I. t (t). Time domain component I t The value of (t) is close to 0 when the signal is stable, but it produces strong positive and negative pulses when fast events occur. Determine the spectral domain component I. s(t) is used to capture the spectral characteristics of an event. Specifically, absorbance A(λ, t) is used. It is understood that transient scattering is typically wavelength-dependent and can cause nonlinear uplift or distortion of the spectral baseline across the entire or partial bands, i.e., spectral morphology anomalies. These anomalies can be quantified by calculating the local slope or derivative of the spectrum. For example, the difference slope |ΨA / Ψλ| of A(λ, t) between adjacent bands can be calculated, where Ψ is the partial derivative. To obtain a scalar value that summarizes the extent of spectral anomalies and suppresses the influence of individual noisy bands, robust aggregate statistics are preferably employed. For example, the median of the difference slopes across all bands λ can be used as the spectral component I. s (t), i.e., I s (t) = median λ (|Ψ A(λ, t) / Ψ λ|). Spectral domain component I s The value of (t) remains low when the spectral morphology is stable (even with slow overall baseline drift), but increases rapidly when the spectral morphology is distorted. The time-domain and spectral-domain components are fused to generate the transient scattering index I. ts (t). The fusion formula can be I ts (t) = w t · I t (t) + w s · I s (t); where w t and w s These are the weights of the time domain and spectral domain components, respectively.

[0048] In a preferred embodiment, the weight w t and w s It is not fixed, but adaptively tuned, which ensures that the two components are consistent with I. ts The contributions of (t) are roughly balanced, avoiding any one component dominating the index. Alternatively, an adaptive tuning strategy is variance balancing, where the system calculates the time-domain component I within a rolling time window, for example, the most recent N=300 samples judged to be stable. t (t) and spectral domain component I s The variance Var(t) of (t) t ) and Var(I s The weights are set to be inversely proportional to the variance, for example, w. t Proportional to 1 / Var(I) t ), w s Proportional to 1 / Var(I) s ), and normalize (w t + w s = 1). Thus, if the time-domain component It The noise (variance) of channel (t) (high-speed channel) increased at a certain time, and its weight w t It will automatically decrease. In some implementations, the weight tuning can also utilize the quality score Q. s1 (t). For example, if the quality score Q s1 The sensor health vector H in (t) sens (t) indicates a hardware failure in the high-speed channel, which can force w to... t Set it to 0, then the transient scattering index I ts (t) is downgraded to depend only on the spectral domain component I s (t), thereby improving the robustness of the system.

[0049] Within a rolling time window, a robust statistical estimate is applied to the transient scattering index to determine its distribution parameters.

[0050] Specifically, a rolling time window W is set, for example, the most recent N=300 samples. Within this window W, the transient scattering index I is calculated. ts The statistical distribution parameters of (t), including the baseline (location) μ ts Scaling (discretion) σ ts A preferred implementation is to use robust statistical estimation, since the window W may already contain the events to be detected (outliers). If conventional means and standard deviations are used, the calculation results will be severely contaminated (inflated) by outliers, leading to a decrease in detection sensitivity. Therefore, preferably, the baseline μ... ts Calculated as the transient scattering index I within window W ts The median of (t); the scale σ ts It is calculated as the Median Absolute Deviation (MAD), for example, σ. ts = 1.4826 · MAD (I ts | W), where 1.4826 is the correction factor that makes MAD equal to the standard deviation under a normal distribution.

[0051] An adaptive threshold is generated based on the distribution parameters.

[0052] In this embodiment, due to the baseline μ ts and scale σ ts The threshold is calculated dynamically within a scrolling window, thus the generated threshold is adaptive and can automatically adapt to slow changes in the background turbidity of the water body. For example, at least one threshold is generated, such as a high threshold θ. ts_high = μ ts + k high · σ ts ;where k highIt is a high multiple factor, such as k high = 3 or 4, corresponding to the statistically significant 3-σ or 4-σ events. In some implementations, a low threshold θ can also be generated. ts_low = μ ts + k low ·σ ts ;k low For low-multiple factors, such as k low = 2 or 2.5, used for early warning or marking weak events. Optionally, the size N of the scrolling window W can also be adaptive. For example, from the synchronization data packet P sync The flow velocity information of the water body is read in (t). If the flow velocity is fast, the window N is automatically shortened, for example, N=100, to improve the detection response speed.

[0053] The transient scattering index is compared with an adaptive threshold to generate an event label, namely E. evt (t).

[0054] Specifically, for example, if I ts (t)≥θ ts_high Then E evt (t) is marked as a strong event, for example, E evt (t) = 2; if θ ts_low ≤I ts (t) < θ ts_high If it is, then it is marked as a weak event, E evt (t) = 1; otherwise, it is a non-event, E evt (t) = 0. Furthermore, we can consider E... evt (t) The tag sequence undergoes morphological processing, such as morphological closing and opening operations, to remove extremely short-duration glitches and fill in any brief interruptions that may occur in the event. In a preferred embodiment, event type segmentation may also be included. Specifically, in E evt Further analysis of the time domain component I within the time period where (t)>0. t Pulse morphology and spectral domain components I of (t) s The spectral characteristics of (t) can be used, for example, whether the anomaly occurs across the entire spectrum or only in the shortwave band. Based on these characteristics, a simple classifier can be used to subdivide the event into different types E. type (t), for example E type ∈{bubbles, particulate matter, other impurities}. This E type The (t) label will be used to select a more accurate physics model.

[0055] For the time period identified by the event label, a state-space model is established, in which the slowly varying absorbance is taken as the system state and the absorbance is taken as the observation.

[0056] Preferably, event boundary refinement is performed before modeling. That is, using the transient scattering index I. ts The first derivative (rate of ascent) and rate of descent of (t) are used to precisely pinpoint the start and end timestamps of the event. start and t end Forming event fragments Seg evt ={t start , ..., t end}. In this event fragment Seg evt Within this model, establish a linear Gaussian state-space model or other applicable model: the state equation is A slow (λ, t) = A slow (λ,t-1) + v t The state-space model uses slowly varying absorbance A as an example. slow (λ, t) represents the system state. A slow This represents the unscattered, true absorbance of the water quality that we want to track. This equation assumes A slow Transient events change slowly over a short period of time (typically less than 1 second), for example, approximating a random walk. t It is process noise, and its covariance is Q. λ The observation equation is A(λ, t) = A slow (λ,t) + S sca (λ,t) + e t The model uses absorbance A(λ, t) (i.e., the measured value) as the observation. The observed value is modeled as a slowly varying state A. slow Scattering component S sca and measurement noise e t (covariance is R) λ A linear superposition of states. The initial value A of the state. slow (λ,t start Preferably, the initial time t is used. start The robust mean of A(λ, t) over the most recent non-event period is used for initialization; the robust mean can be the median or a moving average. Noise covariance Q λ and R λ Online adaptive estimation can also be achieved by analyzing the residuals of non-event segments.

[0057] It is understandable that the above observation equation A(λ, t) = A slow (λ,t) + S sca (λ,t) + e t There are two unknowns (A) slow and S sca Therefore, the model is underdetermined. To make it solvable, we need to introduce information about S. sca(λ, t) prior structural information. Further, the state-space model includes a prior model of the scattering components. In a preferred embodiment, the prior model of the scattering components characterizes the scattering components as time-frequency prototypes composed of a spectral basis. In other words, S sca (λ, t) is assumed to have a specific spectral shape. sca (λ, t) is modeled as S sca (λ, t) = B sca (λ, k) · c(t, k); where B sca (λ, k) is the k-dimensional spectral basis matrix, i.e., k characteristic spectral shapes, and c(t, k) is the k-dimensional time coefficient. The amplitude of the time-frequency prototype is driven by the transient scattering exponent. This means that the amplitude (norm) of the coefficient c(t, k) is related to I. ts (t) are related. For example, ||c(t, k)||2 = g(I ts (t)), where g is the gain function. Spectral basis B sca (λ, k) is preferably obtained through offline learning. For example, a large number of detected event spectra (after baseline subtraction) are collected from historical data to form a sample matrix X. Principal component analysis (PCA) or singular value decomposition (SVD) is performed on the sample matrix X, and the top k principal components (eigenvectors) with the highest energy are taken as the spectral basis B. sca (λ, k). In another alternative embodiment, if event type segmentation E has already been performed... type (t), then here we can call E type (t) (e.g., bubbles) corresponds to a specific spectral substrate B sca_bubble (λ, k), thereby improving the physical accuracy of the model. The step of estimating the scattering components is performed based on the prior model of the scattering components. By using S sca Substituting the prior model into the observation equations, the state-space model becomes solvable.

[0058] A recursive filter is applied to the state-space model to dynamically estimate the slowly varying absorbance.

[0059] In a preferred embodiment, if the noise v t and e t Assuming a Gaussian distribution, a Kalman filter is used as the recursive filter. In another alternative implementation, if analysis of the residuals in non-event segments reveals that the noise exhibits non-Gaussian characteristics, such as impulsive or heavy-tailed distributions, a particle filter is preferably used as the recursive filter to obtain more robust estimation results. The system in event segment {t} start , ..., t end Within a given time period, the selected filter is run at each time step t to obtain the slowly varying absorbance A. slow The filtered estimate A* slow(λ, t|t). Furthermore, to improve estimation accuracy, especially at the end of the event, it is preferable to [evaluate at t]. end After a certain time, a reverse smoothing is performed. For example, the RTS smoother (Rauch-Tung-StriebelSmoother) is applied. The RTS smoother utilizes t end All previous data, for {t start , ..., t end The state estimate within the interval is backtracked and corrected to obtain a smoother and more accurate slowly varying absorbance estimate A*. slow_smooth (λ, t).

[0060] The absorbance is compared with the slowly varying absorbance to estimate the scattering component; the estimated scattering component is separated from the absorbance to generate the absorbance after stripping the scattering.

[0061] Specifically, the scattering component is estimated as: S sca (λ, t) = A(λ, t) - A* slow_smooth (λ, t). Absorbance A after stripping scattering deSca (λ, t) is the final estimate of the slowly varying absorbance, i.e., A. deSca (λ, t) = A* slow_smooth (λ, t). In the non-event segment (E) evt (t) = 0), the filter is not activated, and A is in this state. deSca (λ, t) = A(λ, t). In this embodiment, the original absorbance A(λ, t) is separated into A... deSca (λ, t) (slowly varying true absorption) and S sca (λ, t) (transient scattering component). Furthermore, this embodiment can also calculate quality indicators, such as the scattering energy percentage E. sca_ratio (t) = ||S sca ||2 2 / ||A||2 2 This metric quantifies the intensity of transient disturbances and will be combined with S... sca (λ, t) itself and E evt The (t) tag is passed along with the subsequent quality and self-diagnosis report.

[0062] This embodiment constructs a detection index that is extremely sensitive to transient events by fusing high-frequency time-domain information and full-spectrum spatial-domain (spectral domain) information; using this index as a sentinel, it drives the state-space model to dynamically separate the spectrum into slowly varying true absorption and fast scattering components.

[0063] like Figure 2 As shown, in one possible embodiment, generating de-wrinkled absorbance includes:

[0064] Stripe features were extracted from the absorbance after stripping and scattering.

[0065] Specifically, extracting fringe features includes: performing spectral domain analysis on the absorbance after stripping and scattering to obtain fringe features that include the fringe envelope amplitude and fringe spacing.

[0066] In a preferred implementation, event label E can be used. evt (t). If event label E evt (t) Marking the current time slice t as a strong event, the spectral data of this time slice is only recorded and not used for feature extraction and subsequent inversion. This is to prevent potentially residual strong scattering signals from contaminating the identification of the thin-film interference model. For time slices of non-strong events, the absorbance A after stripping and scattering of the input is... deSca A spectral-domain bandpass filter is applied at (λ, t). The passband of this filter is designed to cover the expected biomembrane interference frequencies, for example, corresponding to 5 cm⁻¹. -1 Up to 60 cm -1 The equivalent frequency range suppresses slow spectral baseline drift (below 5 cm⁻¹). -1 ) and high-frequency electronic noise (above 60 cm) -1 The filtered spectral signal is subjected to a Hilbert transform to obtain its analytic signal. From this analytic signal, two key instantaneous features can be demodulated: the fringe envelope amplitude R(λ,t) and the instantaneous phase Φ(λ,t). The fringe spacing Δ(1 / λ,t) is obtained by phase unfolding the instantaneous phase Φ(λ,t) and calculating its derivative with respect to the wavenumber (1 / λ), or by finding the wavenumber distance between adjacent extreme points of the phase Φ(λ,t). In a preferred embodiment, a fringe consistency index K(t) can also be calculated to quantify the quality and coherence of the extracted fringes. For example, K(t) can be calculated as the coefficient of determination of the linear fit between the unfolded phase Φ(λ,t) and the wavenumber 1 / λ. A K(t) value close to 1 indicates clear fringes and high consistency; a low K(t) value, such as below 0.8, indicates a weak fringe signal or severe noise interference. The fringe consistency index K(t) will be output as an important quality parameter in the subsequent self-diagnostic report. The extracted features, such as the fringe envelope amplitude R(λ,t), fringe spacing Δ(1 / λ,t), and fringe consistency index K(t), are robustly processed to form a fringe feature set F. ripple .

[0067] Based on the stripe features and a pre-configured thin-film optical model, the equivalent optical thickness is obtained by inverse solving.

[0068] In a further embodiment, the equivalent optical thickness is obtained by inverse calculation, including:

[0069] Based on the relationship between fringe spacing and optical approximation, an objective function is established to solve for the equivalent optical thickness. In the objective function, the fitting residual of the fringe spacing is weighted by the fringe envelope amplitude to obtain the weighted objective function. Solving the weighted objective function yields the equivalent optical thickness.

[0070] In this embodiment, the optical approximation relationship refers to the fundamental physical model of single-layer thin-film interference, namely, the relationship between the fringe spacing Δ(1 / λ) and the equivalent optical thickness n. film d film They are approximately inversely proportional: Δ(1 / λ) ≈ 1 / (2 · n film d film (t)); where n film d film (t) is the target variable to be solved, representing the refractive index n of the biomembrane. film With physical thickness d film The product of the observations and the model values. The objective function J(t) is established to minimize the residuals between the observations and the model values. A preferred, robust objective function J(t) is defined as follows: J(t) = ∑ λ w λ ·L δ (Δ obs (1 / λ, t) - 1 / (2·n) film d film (t))) + β·TV(n film d film J(t)), where J(t) is the total cost minimized at time t; Δ obs (1 / λ, t) is the observed fringe spacing; n film d film (t) is the equivalent optical thickness scalar to be solved; w λ It is a weighted sum based on the amplitude of the fringe envelope. Specifically, w λ The amplitude is proportional to the fringe envelope amplitude R(λ, t). Bands with larger fringe envelope amplitude R exhibit clearer fringe signals and higher signal-to-noise ratios, and therefore should be given higher weight when fitting the objective function. δ (·) represents a robust loss function, such as the Huber loss function. Its form is: if the absolute value of the residual a, |a| ≤ δ, then L δ (a) = 0.5 · a 2 (Square loss); if |a|> δ, then L δ (a) = δ · (|a| - 0.5 · δ) (linear loss); where δ is an adjustable parameter. The Huber loss is used instead of the traditional squared loss (a 2 ), can reduce Δ obs Individual outliers in the solution result n filmd film The adverse effects of (t). β · TV(·) is the time regularization term. β is the regularization coefficient, the value of which can be determined by the L-curve method or cross-validation; TV(n film d film (t) is the total time variation operator, which penalizes n film d film (t) Dramatic jumps at adjacent time points t. This term is added based on the physical prior that biofilm growth is a slow process, and its thickness should remain smooth over time, thus suppressing noise in the solution. In a highly preferred embodiment, temperature refractive index compensation is also included to further improve accuracy. It is understood that the refractive index n of the biofilm (primarily composed of water) is... film Sensitive to temperature. From synchronization data packet P sync The current water temperature T(t) is read from (t), and the refractive index fine-tuning term δ is calculated based on the water temperature T(t) and the known refractive index temperature coefficient dn / dT (e.g., the dn / dT value of water). n (t), where n is the refractive index of the biofilm and T is the water temperature. The model used to solve for the objective function was modified to 1 / (2 · (n) base + δ n (t)) · d film (t)), where n base The reference refractive index is used. This eliminates spurious drift caused by water temperature changes in thickness inversion, which is crucial for achieving high-precision measurements. Solving the objective function J(t) (e.g., through a convex optimization algorithm) yields the equivalent optical thickness n at time t. film d film (t).

[0071] A forward optical model of the interference effect is constructed using equivalent optical thickness.

[0072] Specifically, using the equivalent optical thickness n film d film A mathematical model M(λ,t) is constructed to forward simulate (predict) the multiplicative modulation effect of thin-film interference on the spectrum. This forward model M(λ,t) can be an exact model based on the optical transfer matrix, or a simplified cosine modulation model M(λ,t) = 1 + α(t) · cos(2π · (1 / λ) / Δ model + Φ0(t)), where Δ model It is determined by the equivalent optical thickness n film d film α(t) is uniquely determined and represents the reciprocal of the fringe spacing; α(t) is the modulation amplitude factor; Φ0(t) is the phase offset term.

[0073] Based on the forward optical model, and taking the absorbance after stripping scattering as the observation, the inverse optical problem is solved to generate the de-scratched absorbance.

[0074] In this embodiment, the absorbance A after wrinkle removal is calculated. clean (λ, t) represents the true absorbance in the absence of interference effects. This problem is constructed as an inverse optical problem, i.e., solving A... clean , such that M(λ,t) · A clean (The model-predicted, interferometric spectrum) and observed value A deSca The closest (spectrum after stripping scattering) is found. This inverse problem is usually ill-posed and is preferably solved by solving a regularized optimization problem: min Aclean ||A deSca - M(λ, t) · A clean ||2 2 + γ · ||Ψ A clean / Ψ λ ||2 2 ;where A clean These are the variables to be optimized; ||· ||2 2 Denotes the L2 norm (sum of squares); the first term ||A deSca - M · A clean ||2 2 This is a data fidelity term, and we need to solve for A. clean After M modulation, it should approximate the observed value A. deSca The second term is the Tikhonov regularization term, where γ is the regularization coefficient (determined via L-curve or generalized cross-validation GCV), and ΨA clean / Ψ λ is A clean The derivative with respect to wavelength. This regularization term represents the true absorbance spectrum A. clean It should be smooth; by penalizing its derivative, i.e., roughness, noise and ringing artifacts that may occur during the solution process can be suppressed. In some preferred embodiments, to further achieve anti-ringing constraints, in addition to Tikhonov regularization, soft constraints on the spectral derivative or higher-order smoothing constraints can be introduced to ensure A clean The physical reality of (λ, t). Solving this optimization problem yields the final de-curling absorbance A. clean (λ, t).

[0075] In one preferred implementation, it further includes:

[0076] Calculate the time derivative of the equivalent optical thickness over time to obtain the thickness growth rate.

[0077] For example, the growth rate G(t) = d(n) film dfilm ) / dt can be obtained by applying n film d film The (t) sequence is obtained by time difference or sliding fit.

[0078] An adaptive amplitude threshold is determined based on the historical statistical distribution of the fringe envelope amplitude.

[0079] Specifically, calculate the median R of the current fringe envelope amplitude. curr = median λ R(λ, t). Adaptive amplitude threshold A thr (t) is preferably determined by historical statistical distribution, for example, A thr (t) = 1.2 × p95(R|last 7 days); where p95 (95th percentile) is the value of R from the last 7 days. curr The values ​​are statistically derived from the value sequence, allowing the threshold to dynamically adapt to recent pollution conditions.

[0080] When the thickness growth rate exceeds the preset growth rate threshold, and the current value of the stripe envelope amplitude exceeds the adaptive amplitude threshold, a self-cleaning trigger value is generated.

[0081] In this embodiment, the preset growth rate threshold G thr (or denoted as η) is an empirical value set according to the on-site working conditions. For example, it can be set to 20 nm / hour in eutrophic water bodies and 5 nm / hour in oligotrophic water bodies.

[0082] In an optional implementation, generating the self-cleaning trigger amount can also be achieved by: calculating the time trend of the equivalent optical thickness; and generating the self-cleaning trigger amount when the time trend exceeds a preset growth rate threshold and the fringe envelope amplitude exceeds a preset amplitude threshold.

[0083] In a further embodiment, it also includes:

[0084] The difference between the reconstructed spectrum corresponding to the de-curling absorbance under the forward optical model and the absorbance after stripping and scattering is calculated to obtain the de-curling residual.

[0085] For example, the de-curling residual e film (λ, t) = A deSca - M(λ, t) · A clean This residual e film It represents the signal portion that the interference model M cannot explain, and is a key indicator for evaluating the goodness of fit of the model and triggering self-cleaning.

[0086] The residual threshold is determined based on the historical statistical distribution of the de-curling residual.

[0087] Specifically, calculate the de-curling residual e.film norm E curr = norm(e film Residual threshold θ film Preferably, it is determined by historical statistical distribution, for example, based on the residual statistical characteristics of the most recent stable period (such as a period immediately after cleaning), denoted as θ. film = μ res + 3 · σ res , where μ res and σ res These are the mean and standard deviation of the residual norm of the stable segment.

[0088] When generating the self-cleaning trigger value, it is further required that the de-curling residual exceeds the residual threshold.

[0089] In this embodiment, a self-cleaning trigger value U is generated. clean The triggering logic for (t) is that when (the median R of the current fringe envelope amplitude) curr ≥Adaptive amplitude threshold A thr With thickness growth rate G(t) ≥ preset growth rate threshold G thr ) or (norm E of the decurving residual) curr ≥ Residual threshold θ film If any combination of ) satisfies the condition, then U clean (t) = 1 (trigger). That is to say, cleaning should be performed when the biofilm is thick and grows fast, or when the de-curling model itself has failed (excessive residual).

[0090] After the self-cleaning trigger is triggered, the thickness growth rate, stripe envelope amplitude, and de-curling residual are continuously monitored. When all monitored indicators are lower than their respective preset reset thresholds and the preset hysteresis time is maintained, the self-cleaning trigger is reset.

[0091] Specifically, once the self-cleaning trigger amount U clean (t) is set to 1, and it will remain at 1 until all three metrics (R) are set to 1. curr G(t), E curr All of them fall back below their respective preset reset thresholds, for example, 70% of their respective trigger thresholds, and this state must continue for a preset hysteresis time T. hys For example, T hys =10 minutes, then U clean (t) is then reset to 0. This prevents the trigger from oscillating near the threshold. In a preferred embodiment, scheduling and mutual exclusion are also included. That is, U clean (t) = 1 does not immediately perform physical cleaning, but instead generates a cleaning schedule recommendation, Sched. clean (t). The system will query P. sync(t) Package, check the current process cycle time, for example, whether it is in the sampling gap, to ensure that the cleaning action does not interrupt the critical measurement process, reflecting the self-consistency of the system operation.

[0092] According to one aspect of this application, quantitatively calculating the target analyte concentration vector includes: performing a quantitative inversion of the target analyte.

[0093] Specifically, the absorbance A of the de-curling method clean (λ, t) are input to the multi-objective inversion module with physical prior constraints. The objective of this module is to solve a constrained least squares problem, such as: min Qtargets(t) ||A clean_w (λ, t) - E(λ, C) · Q targets (t)||2 2 + ρ · ||L · Q targets (t)||2 2 ; where Q targets (t) is the target concentration vector to be solved; E(λ, C) is a pre-stored, known extinction coefficient matrix, representing the extinction coefficient of compound C at wavelength λ; ρ·||L·Q targets (t)||2 2 The term is a regularization term, ρ is a coefficient, and L is a constraint matrix that can be used to impose, for example, concentration nonnegativity (Q). targets Constraints include (≥0) or known stoichiometric ratios. Solve the optimization problem to obtain the target analyte concentration vector Q. targets (t). In a highly preferred embodiment, internal filtration effect consistency correction can also be performed. It is understood that in the high concentration region, the linear relationship between absorbance and concentration (Lambert-Beer law) will fail. This is achieved using the nonlinear region and the range marker B. range (λ, t), when constructing the least squares problem, does not use the de-curling absorbance A. clean Instead, it is the weighted absorbance A after removing the curl lines. clean_w (λ, t). Weighted de-curling absorbance A clean_w Absorbance A for removing curl lines clean B range The band λ, marked as unreliable, is given zero or very low weight. This avoids the influence of signals from the high-concentration nonlinear region on the inversion result Q. targets (t) This causes contamination, improving the robustness and accuracy of the quantitative model. In another alternative implementation, standard sample bias correction may also be included. For example, the system can be configured to automatically introduce standard samples of known concentrations at preset intervals and perform a complete measurement procedure. The calculated target analyte concentration vector Q is then used to... targets By comparing the system bias vector (Bias) with the true value of the standard sample, the system bias vector is estimated. corr(C). This deviation vector is stored and used to correct subsequent real-time measurement results, resulting in the corrected target concentration vector Q. targets_final = Q targets - Bias corr (C).

[0094] In a further embodiment, it also includes:

[0095] Summarize the target concentration vector, scattering component, equivalent optical thickness, event tag, and self-cleaning trigger amount.

[0096] Specifically, creating quality and self-diagnostic reports R qc (t). This report is a data structure that compiles all key intermediate variables and final results generated at time t, including at least: the target concentration vector Q. targets (t); Scattering component S sca (λ, t) (or its energy percentage E) sca_ratio (t) and event label E evt (t); equivalent optical thickness n film d film (t) (and its time trend) and self-cleaning trigger U clean (t).

[0097] Based on the summarized results, a quality and self-diagnosis report is generated.

[0098] In this embodiment, report R is generated. qc (t) is not just a simple summary, but also includes a comprehensive interpretation of this information. For example, the report may include: the current biofilm thickness is X nm, the growth rate is Y nm / hour, a strong transient event (type: bubble) was detected at t-10 seconds, and a self-cleaning recommendation: yes (trigger amount U). clean =1).

[0099] In a preferred embodiment, the quality and self-diagnostic report R qc (t) also includes detailed uncertainty (confidence interval) information. Specifically, it also includes:

[0100] The transient separation uncertainty is determined based on the estimated residuals of the scattering components.

[0101] Specifically, when performing state-space filtering and smoothing, an estimated residual R is generated. sep (t), i.e., A* slow_smooth The discrepancy in consistency with the observed values. This residual R0 sep The covariance of (t) is used as the transient separation uncertainty Σ sep Σ sep Quantization was performed on the stripped scattering component S scaThe uncertainty introduced at that time.

[0102] The uncertainty of decurving is determined based on the decurving residual.

[0103] Specifically, when solving the optical inverse problem, the de-curling residual e was calculated. film (λ, t), i.e., A deSca With the reconstruction model M·A clean The difference between them. The residual e film The covariance is used as the decurving uncertainty Σ film Σ film The uncertainty introduced when stripping interference fringes was quantified.

[0104] The transient separation uncertainty and the decurving uncertainty are combined into the spectral error covariance.

[0105] In this embodiment, the final de-curling absorbance A clean The total uncertainty is modeled as the sum of multiple independent error sources. Its spectral error covariance Σ A It is synthesized into: Σ A = Σ base + Σ sep + Σ film ;wherein Σ base This represents the fundamental measurement uncertainty, for example, from the dark current D. corr Covariance Σ of estimation error dark and the reference spectrum after reference correction S ref_corr Covariance Σ of drift ref ;Σ sep and Σ film These are the introduced uncertainties.

[0106] Determine the solution Jacobi for the quantitative calculation steps.

[0107] Specifically, the Jacobian J is the sensitivity matrix of the quantitative inversion model, representing the output concentration Q. targets Relative to the input spectrum A clean The partial derivative, i.e., J = ΨQ targets / Ψ A clean The inversion model is the least squares solver. The Jacobian matrix J can be determined by performing local numerical differentiation or the adjoint method on the solver.

[0108] The spectral error covariance is propagated by solving the Jacobian to obtain the uncertainty of the target substance concentration.

[0109] For example, the standard law of uncertainty propagation can be applied. The uncertainty of the target analyte concentration (expressed as a covariance matrix Σ)Q (represented) is calculated as: Σ Q ≈ J · Σ A · J T ;in T This is a transpose.

[0110] The quality and self-diagnostic report further includes the uncertainty of the target substance concentration.

[0111] Specifically, R qc (t) The concentration output in the report is no longer a single point estimate, but an interval estimate including a confidence interval. For example, for the i-th target analyte, its concentration is reported as Q. targets_i (t) ±1.96 · sqrt(Σ Q_ii (t)), where Σ Q_ii The uncertainty of the target concentration Σ Q The diagonal element, i.e., the variance; 1.96 corresponds to a 95% confidence level.

[0112] In some preferred embodiments, in order to verify the target concentration uncertainty Σ based on Jacobi (linear) propagation Q In addition to verifying its effectiveness, the system can also perform Monte Carlo calibration. Specifically, the system measures the spectral error covariance Σ. A M random samples are performed from the defined multidimensional Gaussian distribution, for example, M=1000 times, to generate M simulated de-curling absorbance values ​​A. clean_MC，i Spectra. These M spectra are input one by one into the quantitative solver to obtain M corresponding target analyte concentration vectors Q. targets_MC，i Results. Directly analyze the distribution of these M concentration results, for example, calculate their 95th percentile intervals (CI). MC R qc (t) The report will be distributed in CI MC The confidence interval CI obtained from Jacobi propagation Jacob That is, ±1.96 · sqrt(Σ) Q_ii A comparison is made to generate a consistency index C. cons If C cons A lower value indicates that the linear propagation assumption may be invalid; for example, the inversion model may be too nonlinear. In this case, CI should be used preferentially. MC As a measure of uncertainty.

[0113] In a preferred embodiment, R qc (t) The report also includes an alarm level mapping. The system maps the final concentration uncertainty to alarm level A. lvl For example: A lvl∈ {Excellent (<5%), Good (5-15%), Poor (>15%)}; where the concentration uncertainty can be the relative uncertainty sqrt(Σ) Q_ii ) / Q targets_i To achieve a closed loop in the self-consistent compensation framework, R... qc (t) The report is sent back to the host computer or system controller. If the self-cleaning trigger value U in the report... clean (t) = 1, and the controller then follows R. qc The clean scheduling recommendation in (t) is Schedule clean (t), issuing a physical self-cleaning command at a suitable process cycle. Meanwhile, E in the report... evt (t) record, n film d film (t) Trend and A lvl Alarm levels provide operators with a complete and transparent view of system health and data quality.

[0114] This application establishes a self-consistent compensation closed-loop framework, no longer treating transient scattering and thin-film interference as noise that needs to be independently eliminated, but rather as identifiable, modelable, and separable signal components within the system. Specifically, this framework employs a cascaded processing flow: identifying and stripping transient scattering interference to provide a cleaner spectral baseline for subsequent thin-film interference identification; performing interference identification and de-curling on the stripped spectrum; and using the de-curled net spectrum for quantitative calculations. Furthermore, the framework includes closed-loop feedback; for example, summarizing the identification results of thin-film interference (e.g., biofilm thickness) and transient scattering into a quality and self-diagnostic report to guide when the system needs self-cleaning or maintenance, thus achieving a self-consistent closed loop between interference compensation and system operation and maintenance.

[0115] In a detailed embodiment, assume a full-spectrum detector deployed in the water body, whose goal is to measure nitrate concentration. The actual water quality (target) is the nitrate concentration Q during the measurement period from t=1 to t=100. target The concentration was kept constant at 5.0 mg / L. The slowly varying interference (biofilm) was observed when the probe window was clean at t=1 (n...). film d film (t=1) = 0nm). Due to biofilm growth, the equivalent optical thickness n film d filmThe absorbance (t) increases linearly with time t, reaching 200 nm at t=100. Transient interference (bubbles) occurs at t=50, 51, and 52, where large bubble clusters pass through the probe, causing strong transient scattering. During the period from t=1 to t=100, the water temperature T(t) slowly rises from 10°C to 20°C. The system collects data at t=1, 2, ..., 100. The acquired absorbance A(λ, t) is as follows: at t=1, ..., 49; 53, ..., 100, spectrum A is the true absorption spectrum of 5.0 mg / L nitrate superimposed with interference fringes that increase with t; at t=50, 51, and 52, A is the true absorption spectrum + interference fringes + strong scattering spectrum (severe baseline rise). Simultaneously, the synchronization data packet P... sync (t) is encapsulated, where I fast (t) (high-speed channel) records strong pulse signals at t=50, 51, 52; T(t) records the temperature change from 10°C to 20°C. The system calculates the transient scattering index I. ts (t). Time domain component I t (t) (from I) fast ) and spectral domain component I s The spectral slope (t) from A exhibits strong peaks at t=50, 51, and 52. The system is based on the transient scattering exponent I. ts Robust statistics (median+MAD) of (t) are used to calculate the adaptive high threshold θ. ts_high Transient scattering index I ts (t=50, 51, 52) exceeds θ ts_high The system generates E evt (t=50, 51, 52) = 2 (strong event). In the interval t=50, 51, 52, the system activates the Kalman filter (assuming the noise is Gaussian) and the scattering prior model (e.g., using a pre-stored bubble spectral basis B). sca_bubble The system output absorbance A after stripping and scattering. deSca (λ, t). In the interval t = 50, 51, 52, A deSca A* is estimated by Kalman filtering. slow_smooth (Slowly varying spectrum), scattering component S sca Successfully separated; at t=1,...,49;53,...,100; A deSca = A. At this time, A deSca The sequence has eliminated transient impacts of the bubble at all 100 points, but still contains interference fringes that increase with t. The system is applied to A... deSca The (λ, t) sequence is processed. At t=50, 51, 52, due to E evt=2 (strong events), these points are masked by the system and are not used for thickness inversion. The system uses data t=1,...,49;53,...,100 to perform a robust objective function J(t) minimization. In the inversion n film d film When (t), the system calls P. sync The T(t) (10°C to 20°C) data in (t) are used to perform temperature refractive index compensation. The system accurately inverts n film d film The (t) curve shows a linear increase in value from 0 nm to 200 nm, consistent with the scene setting. Using n film d film The system constructs a forward model M(λ, t) for each t and solves the inverse optical problem. The system outputs the de-curved absorbance A. clean (λ, t). In this sequence, the interference fringes have been successfully removed. A clean (λ, t) stably approximates the true absorption spectrum of 5.0 mg / L nitrate at all time points t = 1, 2, ..., 100 (including t = 50, 51, 52). At t = 100, the system detects R... curr Both the (stripe amplitude) and G(t) (growth rate) are already high. Assuming that both simultaneously meet their respective trigger thresholds, the system generates U. clean (t=100) = 1. Let A clean The (λ, t) sequence is input into the quantitative solver to obtain Q. targets The (t=1, ..., 100) sequence has values ​​that stabilize around 5.0 mg / L (e.g., 4.95 to 5.05 mg / L). Calculate Σ. sep (larger at t=50, 51, 52) and Σ film (Increases with increasing t). Through Jacobi propagation J, we obtain Σ. Q At t=100, the concentration uncertainty may be ±0.1 mg / L; at t=51 (the center of the transient event), the uncertainty may instantaneously increase to ±0.3 mg / L. At t=100, the system generates R qc (t=100) Report, including: Nitrate concentration: 5.01 ± 0.1 mg / L (95% confidence interval), quality grade: Excellent; Event log: Strong events (bubbles) detected at t=50, 51, and 52; Biofilm status: Current thickness 200.4 nm, growth rate 2.1 nm / hour; Maintenance recommendation: Trigger self-cleaning U. clean =1. A comparison reveals that without transient separation, at t=50, 51, and 52, the A signal is severely contaminated by bubbles. The feature extraction step will fail, and n cannot be calculated. film d film This led to Aclean The image is severely distorted during this period. Final Q targets (t=50, 51, 52) will report (for example) a false high value of 20.0 mg / L. If no de-curling is performed, A deSca Interference fringes are still present (even after removing only the air bubbles). A deSca Direct input to the quantitative solver, Q targets (t) will change with n film d film The increase in refractive index causes violent oscillations, for example, fluctuating between 4.0 mg / L and 6.0 mg / L, making it impossible to provide stable readings. Without temperature refractive index compensation, at t=100 (20°C), due to the inversion of n film d film The refractive index n at 10°C is still used. base The inversion results will be biased, for example, misclassified as 190nm. The M(λ,t) model built based on this incorrect thickness will be inaccurate, leading to incomplete de-curling. clean There are still residual stripes in Q. targets There may still be deviations, such as ± 0.3 mg / L.

[0116] This embodiment can simultaneously compensate for transient interference and slowly varying interference (and its temperature effect), and perform quality monitoring and uncertainty propagation throughout the entire process, ultimately outputting high-precision, high-confidence water quality concentration results, and realizing a closed loop for intelligent maintenance.

[0117] In one embodiment of this application, generating the absorbance after stripping scattering can also be achieved by: constructing a transient scattering index using the absorbance and the high-speed channel, and combining the rapid changes in the time domain with the slope anomalies in the spectral domain; performing bandpass differentiation and robust normalization on the high-speed channel to obtain the time domain component I. t (t) = z-score(dI) fast / dt); Calculate the difference slope norm of adjacent bands for absorbance to obtain a robust measure of the spectral domain components. For example, the median can be used to avoid the dominance of anomalous bands. Transient scattering index I ts (t) =w t ·I t (t)+w s ·I s (t), where w t with w s Automatic tuning from historical stable periods aims to bring the variance contributions of the two components closer together. Within a rolling window W, a robust distribution threshold is estimated for the transient scattering exponent: baseline and scale are determined using the median and MAD (median-nab-solute deviation): μ ts =median(I ts |W), σ ts=1.4826·MAD(I ts |W); Adaptive threshold θ ts_high =μ ts +k high ·σ ts θ ts_low =μ ts +k low ·σ ts If the transient scattering index is greater than or equal to the adaptive high threshold, it is labeled as a strong event; if the adaptive low threshold is less than or equal to the transient scattering index and less than the adaptive high threshold, it is labeled as a weak event. Morphological closing operations are performed on the labels using 3–5 samples to suppress glitches, resulting in event labels. The time axis is divided into event segments and non-event segments according to the event labels. A linear Gaussian state space with slowly varying absorption terms as states is established for the event segments: the state equation is: A slow (λ, t) = A slow (λ,t-1) + v t v t ~N(0, Q) λ The observation equation is: A(λ, t) = A slow (λ,t) + S sca (λ,t) + e t e t ~N(0,R) λ ), where N is the number of samples in the rolling window; the scattering components adopt a low-rank time-frequency prototype S within the event segment. sca =λ space_basis ·time AR (1), whose amplitude is driven by the transient scattering exponent; where λ space_basis Let time be the scattering basis matrix in the wavelength dimension. AR (1) This is a first-order autoregressive model over time. The slowly varying absorption term is initialized with the mean absorbance of the most recent non-event period, and the process noise covariance Q is... λ With R λ Adaptive estimation from the residuals of the non-event segment. Within the event segment, run a switched Kalman filter (particle filter for nonlinear cases): for each t, ​​update the filtered estimate and covariance of the slowly varying absorbance; scattering component S... sca (λ, t) = A(λ, t) - A slow (λ, t|t); where A slow (λ, t|t) is the filtered estimate of the slowly varying absorbance, and A(λ, t) is the absorbance; RTS smoothing is performed after the event segment ends to reduce boundary bias; the output is the absorbance A after stripping scattering. deSca (λ, t) = A(λ, t) - S sca (λ, t).

[0118] Specifically, the interpolated high-speed channel sequence and quality score are read from the synchronization data packet. Detrending, bandpass (typically 2–20Hz), and robust normalization are performed on the high-speed channels to obtain normalized high-speed channels. The normalized high-speed channels are read, and the first-order difference and z-score are calculated to obtain the time-domain components; the weight of low-quality samples is reduced using the quality score. The absorbance is read, and the median aggregation of the difference slope |ΨA / Ψλ| is calculated on adjacent bands to obtain the spectral domain components. The time-domain components, spectral domain components, and health vector are read, and the results are analyzed for w. t w s Online adaptive optimization (making the variance contributions of the two components similar), output weight w t w s Read the time domain components, spectral domain components, and weight w. t and w s The transient scattering index is obtained by fusion, and stabilization is suppressed using long and short window extrema. The transient scattering index is read along with the flow rate and valve position information from the synchronization data packet, and the rolling window size N is set. win Read the transient scattering index and the rolling window size, and calculate the baseline μ. ts , scale σ ts (median+MAD) yields the low threshold θ ts_low High threshold θ ts_high Read the transient scattering index and low threshold θ. ts_low High threshold θ ts_high The process generates event labels (strong / weak events) and adds hysteresis to avoid jitter. It reads the event labels, performs opening and closing operations, and outputs smoothed event labels. It then reads the time-domain components, spectral-domain components, and event labels, and subdivides the events into different types E based on impulse morphology and spectral consistency. type (t)∈{bubbles, particles}; Write to the quality and self-diagnosis report R qc (t). Read the event labels and transient scattering index, refine the event start and end points using the rise / fall rates, and output the event segment table. Read the absorbance and the most recent non-event segment, calculate the robust mean, and obtain the initial state value and initial covariance of the slowly varying absorption term. Read the residual sequence of absorbance in the non-event segment and estimate the process noise covariance Q online. λ With R λ Read the time-frequency blocks of absorbance during the event segment, and perform SVD / dictionary learning according to different types to obtain the scattering basis. Read the transient scattering index, fit the coefficients of the first-order autoregressive model AR(1), and form the amplitude-driven model g(I). ts )=gain·I ts Where gain is the scaling factor. The noise covariance Q during the reading process. λ With R λIf the observation is nonlinear or the noise is non-Gaussian, select particle filtering; otherwise, select Kalman filtering. Configure the output filter. Read the absorbance, filter configuration, initial state value of the slowly varying absorption term, initial covariance, and process noise covariance. Recursively obtain the filtered estimate of the slowly varying absorbance and the covariance matrix of the slowly varying absorption term at each time step. Read the filtered estimate of the slowly varying absorbance and the covariance matrix of the slowly varying absorption term. Perform RTS smoothing after each event segment to obtain a smoother slowly varying absorbance estimate. Read the absorbance, the smoother slowly varying absorbance estimate, the scattering substrate, and the amplitude-driven model. Calculate the scattering components and obtain the absorbance after stripping scattering. Perform consistency verification with the nearest neighbor non-event segments, output the consistency residual, and write it into the quality and self-diagnostic report. Read the scattering components and absorbance, calculate the energy percentage, and write it into the quality and self-diagnostic report for subsequent quality assessment.

[0119] In another embodiment of this application, generating de-crimped absorbance can also involve: performing spectral bandpass and analytical signal analysis on the absorbance after stripping and scattering; applying an tunable bandpass in the wavelength domain, for example, 5–60 cm⁻¹. -1 The equivalent frequency is used to suppress the baseline and high-frequency noise, resulting in a bandpass signal. A Hilbert transform is performed, and the analytic signal is obtained to acquire the instantaneous phase Φ(λ,t) and fringe envelope amplitude R(λ,t). The dΦ / d(1 / λ) or the spacing between adjacent extrema Δ(1 / λ) is calculated to form the fringe feature F. ripple (λ, t) = {R(λ, t), Δ(1 / λ, t), phase coherence K(t)}. If the event label is a strong event, the fringe characteristics at time t are only recorded and not used for inversion. Based on the interference approximation of a single-layer thin film, the fringe spacing is related to the equivalent optical thickness n. film d film (t) Correlation: The physical relationship (approximate formula) is: Δ(1 / λ)≈1 / (2·n) film d film The objective function is: to minimize J(t) = Σ on samples of non-strong events. λ w λ ·(Δ obs (1 / λ, t) - 1 / (2·n) film d film (t))) 2 + β·TV(n film d film (t)), where w λThe amplitude of the fringe envelope is ∝, with a higher weight for stronger envelopes; TV is a time total variation regularization to suppress spurious jumps. Huber loss is applied to anomalous bands to obtain the equivalent optical thickness. The thin-film interference is treated as a multiplicative modulation of the ideal absorbance using a transfer matrix model: M(λ,t)=1+α(t)·cos(2π·(1 / λ) / Δ(1 / λ)+Φ0(t)), where Δ(1 / λ) is determined by the equivalent optical thickness; its inverse operator is constructed and Tikhonov regularization is applied for defringe optimization: min Aclean ||A deSca - M(λ, t) ·A clean ||2 2 + γ · || Ψ A clean / Ψ λ ||2 2 The absorbance A after wrinkle removal is obtained by solving time-by-time. clean (λ, t), and record the residual e film (λ, t) = A deSca - M(λ, t) · A clean As a quality metric, calculate the self-cleaning trigger value U. clean (t) and set a multi-condition threshold with hysteresis: Condition A (amplitude threshold): median λ R(λ, t) ≥ 1.2 times p95(R|last 7 days); Condition B (growth rate threshold): d(n) film d film ) / dt ≥ η, such as 5–20 nm equivalent thickness / hour, corresponding to the operating condition setting; Condition C (residual threshold): norm(e film ) ≥ θ film θ film =μ res +3·σ res μ res σ res The residual distribution of the recently stable segment is shown. If conditions A and B are satisfied simultaneously and one of conditions C is satisfied, then the self-cleaning trigger value = 1. After cleaning, the self-cleaning trigger value is only set for a preset hysteresis time T when all three conditions are below the low threshold (70% of conditions A, B, and C). hys It was then reset to 0.

[0120] Specifically, the absorbance after stripping and scattering is read in relation to the instrument resolution, and the bandpass is set, for example, to 5–60 cm. -1Equivalent frequency, output bandpass parameters. Read the absorbance and bandpass parameters after stripping scattering, perform Hilbert transform on the bandpass signal in the spectral domain to obtain the fringe envelope amplitude and instantaneous phase. Read the instantaneous phase, perform phase expansion, calculate the fringe period Δ(1 / λ, t) and consistency K(t) (measures fringe coherence), forming fringe features. If the event label is a strong event, it is only recorded and not used for inversion. Read the fringe features, perform Huber suppression on outliers to obtain robust fringe features. Read the robust fringe features, and obtain a robust estimate of the fringe period based on the adjacent extremum spacing and frequency tracking. Read the robust estimate of the fringe period and the fringe envelope amplitude to establish an objective function. Read the temperature in the calibration parameter set, construct a refractive index fine-tuning term, and embed it into the solution process. Read the robust estimate of the fringe period, the parameter set of the objective function, and the refractive index fine-tuning term to solve for the equivalent optical thickness. Read the equivalent optical thickness, construct forward modulation M(λ, t), prioritizing single-layer modulation, and approximating multiple layers if necessary. Read the absorbance and forward modulation after stripping scattering, determine the regularization coefficient γ using L-curve or generalized cross-validation, and construct a de-curling optimization. Read the absorbance, forward modulation, and regularization coefficient after stripping scattering, add spectral derivative soft constraints to suppress ringing, and obtain the de-curling absorbance. Read the absorbance, forward modulation, and de-curling absorbance after stripping scattering, calculate the de-curling residual and spectral / time statistics, and write them into the quality and self-diagnostic report. Read the historical distribution of fringe envelope amplitude, equivalent optical thickness, and residual, self-learn the baseline, and output amplitude threshold, growth rate threshold, and residual threshold. Read the fringe envelope amplitude, equivalent optical thickness, residual, amplitude threshold, growth rate threshold, and residual threshold, and calculate the self-cleaning trigger amount according to the rule that amplitude and growth rate must be satisfied, and residual must also be satisfied; add a hysteresis window to avoid oscillation. Read the self-cleaning trigger amount and process cycle time in the synchronization data packet, generate cleaning scheduling suggestions, and write them into the quality and self-diagnostic report. Read the self-cleaning trigger value, stripe envelope amplitude, equivalent optical thickness, and residual after cleaning. After verifying that all three indicators are below the low threshold and continue for a preset hysteresis time, reset the self-cleaning trigger value.

[0121] This invention no longer relies on the time domain or spectral domain in isolation, but rather adaptively weights and fuses the time domain components from independent high-speed channels with the spectral domain components from the full spectrum. This allows for both sub-second response (like time domain detection) and accurate identification (like spectral domain detection), solving the problem of robust identification of transient interference. Simultaneously, it no longer treats the biomembrane as a simple baseline drift, but precisely models it as an optical thin film. By analyzing the fringe features in the spectrum and based on physical optical relationships, the equivalent optical thickness of the current biomembrane is solved in real-time. After constructing a forward optical model based on this thickness, interference fringes are mathematically extracted from the signal by solving a regularized inverse optical problem, thus achieving real-time, online compensation for biomembrane interference. Through a sophisticated cascaded processing flow, the vicious cycle of two types of interference coupling, leading to the failure of both compensation methods, is resolved. Specifically, transient interference separation is performed before thin-film interference processing, ensuring that the spectral signal used to identify interference fringes is clean and free from scattering contamination. Furthermore, the event tags generated by transient detection are used to actively shield subsequent fringe feature extraction, preventing data contamination. By decoupling the two types of interference, both compensation algorithms can run stably and accurately, resulting in high-precision final quantitative results.

[0122] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A full spectrum water quality detection method with transient interference self-consistent compensation, characterized in that, The method comprises: obtaining absorbance; performing transient interference separation processing on the absorbance to generate scattering-stripped absorbance; performing thin film interference de-wrinkling processing on the scattering-stripped absorbance to generate de-wrinkled absorbance; quantitatively calculating a target object concentration vector based on the de-wrinkled absorbance; generating the de-wrinkled absorbance, comprising: extracting fringe features from the scattering-stripped absorbance; based on the fringe features and a preconfigured thin film optical model, inversely solving to obtain equivalent optical thickness; using the equivalent optical thickness, constructing a forward optical model of interference effects; based on the forward optical model and taking the scattering-stripped absorbance as observation, solving an optical inverse problem to generate the de-wrinkled absorbance.

2. The method of claim 1, wherein, extracting the fringe features, comprising: performing spectral domain analytical signal analysis on the scattering-stripped absorbance to obtain fringe features including fringe envelope amplitude and fringe spacing; inversely solving to obtain the equivalent optical thickness, comprising: based on the fringe spacing and an optical approximation relationship, establishing a target function for solving the equivalent optical thickness; in the target function, using the fringe envelope amplitude to weight the fitting residual of the fringe spacing to obtain a weighted target function; solving the weighted target function to obtain the equivalent optical thickness.

3. The method of claim 2, wherein, Further comprising: calculating a time derivative of the equivalent optical thickness with respect to time to obtain a thickness growth rate; based on a historical statistical distribution of the fringe envelope amplitude, determining an adaptive amplitude threshold; when the thickness growth rate exceeds a preset growth rate threshold and the current value of the fringe envelope amplitude exceeds the adaptive amplitude threshold, generating a self-cleaning trigger.

4. The method of claim 3, wherein, Further comprising: calculating a difference between a reconstructed spectrum corresponding to the de-wrinkled absorbance under the forward optical model and the scattering-stripped absorbance to obtain a de-wrinkling residual; based on a historical statistical distribution of the de-wrinkling residual, determining a residual threshold; further requiring the de-wrinkling residual to exceed the residual threshold when the self-cleaning trigger is generated; after the self-cleaning trigger is triggered, continuously monitoring the thickness growth rate, the fringe envelope amplitude and the de-wrinkling residual; when all monitoring indicators are lower than the respective preset reset thresholds and continue for a preset hysteresis time, resetting the self-cleaning trigger.

5. The method of claim 3, wherein, generating the scattering-stripped absorbance, comprising: obtaining an event label to identify a time period in which transient interference exists in the absorbance; establishing a state space model for the time period identified by the event label, wherein the state space model takes slowly varying absorbance as system state and takes absorbance as observation; applying a recursive filter to the state space model to dynamically estimate the slowly varying absorbance; comparing the absorbance with the slowly varying absorbance to estimate the scattering component; separating the estimated scattering component from the absorbance to generate the scattering-stripped absorbance.

6. The method of claim 5, wherein, obtaining the event label, comprising: obtaining a transient scattering index; within a rolling time window, applying a robust statistical estimation to the transient scattering index to determine distribution parameters of the transient scattering index; based on the distribution parameters, generating an adaptive threshold; comparing the transient scattering index with the adaptive threshold to generate the event label.

7. The method of claim 6, wherein, obtaining the transient scattering index, comprising: obtaining a high-speed channel signal; based on the time domain rapid change of the high-speed channel signal, determining a time domain component; based on the spectral domain slope abnormality of the absorbance, determining a spectral domain component; The time-domain component is fused with the spectral-domain component to generate a transient scattering index.

8. The method of claim 7, wherein, The state-space model further comprises a scattering component prior model; The scattering component prior model characterizes the scattering component as a time-frequency prototype composed of spectral bases; The amplitudes of the time-frequency prototype are driven by the transient scattering index; The step of estimating the scattering component is performed based on the scattering component prior model.

9. The method of claim 5, wherein, Further comprising: The target concentration vector, the scattering component, the equivalent optical thickness, the event label, and the self-cleaning trigger amount are summarized; Based on the summary result, a quality and self-diagnosis report is generated.

Images