Multi-parameter water quality intelligent spectrum analysis system and self-adaptive correction method

Abstract

The invention relates to the technical field of water quality intelligent detection, and discloses a multi-parameter water quality intelligent spectrum analysis system and a self-adaptive correction method. An environment inhibition module is used for constructing an intelligent spectrum analysis scene, and then a multi-spectrum acquisition module is used for acquiring original spectrum data in the whole intelligent spectrum analysis scene; the multi-parameter inversion module analyzes data characteristics of the corrected spectral data and the compensation parameters after receiving the corrected spectral data and the compensation parameters, and then inputs the data characteristics into a 1D-CNN + LSTM multi-task learning network model to obtain a parameter gradient and a residual error; on the basis of the parameter gradient, parameters in the 1D-CNN + LSTM multi-task learning network model are updated in a back propagation calculation mode, meanwhile, a residual error is transmitted to an adaptive correction engine to update parameters of a mathematical model for dynamic compensation of the ambient light, and finally, a cloud edge collaboration module is utilized to carry out dynamic compensation on the ambient light. The overall parameter updating of the water quality intelligent spectrum analysis system is promoted; and the effectiveness and the robustness of the system and the method are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent water quality detection, and in particular to a multi-parameter water quality intelligent spectrum analysis system and an adaptive correction method. Background Art

[0002] With the implementation of the Water Pollution Prevention and Control Law and the promotion of the River Chief System, my country's demand for real-time water quality monitoring has surged. Traditional laboratory testing takes several hours, making it difficult to meet the rapid response requirements for sudden pollution incidents. While spectral analysis technology can rapidly detect water pollution, it faces two major bottlenecks:

[0003] 1. Ambient light interference problem: During outdoor testing, the intensity of sunlight (up to 10^5 lux) far exceeds that of artificial light (usually 10 3 lux), resulting in: the water body reflectance spectrum signal-to-noise ratio drops by >15dB, cloud movement causes spectral baseline drift, and COD detection errors as high as 20%;

[0004] 2. Deficiencies in multi-parameter simultaneous analysis: Existing equipment is limited by the following factors: a single detector cannot cover the entire UV-visible-near-infrared band (200-850nm), the COD characteristic peak (254nm) partially overlaps with the ammonia nitrogen absorption peak (210nm), the cross-validation error of the traditional PLS algorithm reaches 30%, and the water body components in different river basins vary greatly. The accuracy of the fixed calibration model drops sharply to below 70% when applied across regions.

[0005] In order to solve the above technical problems, technical personnel in this field usually adopt frequency domain filtering and time-sharing detection methods. Among them, although the frequency domain filtering method can reduce the interference of external factors by modulating the light source, this method has insufficient high-frequency noise suppression, and the signal-to-noise ratio improvement under strong light is <10dB; and the time-sharing detection method controls interference factors by rotating the filter wheel to switch the band, but its detection cycle will be extended by 3 times, and it cannot capture the instantaneous correlation between parameters in time; both cannot meet the increasingly high demand for water quality detection.

[0006] Therefore, those skilled in the art are in urgent need of an intelligent analysis system and method for water quality detection that maintains environmental robustness and multi-parameter synchronization accuracy. Summary of the Invention

[0007] The purpose of the present invention is to solve the above problems and to design a multi-parameter water quality intelligent spectral analysis system and an adaptive correction method.

[0008] The technical solution of the present invention to achieve the above-mentioned purpose is a multi-parameter water quality intelligent spectral analysis system, which includes the following parts:

[0009] The environmental suppression module, whose main function is to eliminate external light interference, forms an intelligent spectral analysis scene with a closed dark box and an electric shutter installed on the closed dark box. The closed dark box is used to accommodate the collected water samples, and after the electric shutter is closed, the intelligent spectral analysis scene of the water samples is formed. The closed dark box is provided with a pulse-modulated wide-spectrum LED light (the wavelength of the light emitted by the LED light is 200-850nm) and an ambient light sensor. The light emitted by the pulse-modulated wide-spectrum LED light serves as the background light inside the closed dark box, and the ambient light sensor is used to collect background spectral data inside the closed dark box in real time; wherein, the sampling rate of the pulse-modulated wide-spectrum LED light is ≥1kHz;

[0010] The multi-spectral acquisition module is mainly used to collect high-precision water sample spectral data. It uses a multi-channel InGaAs / CMOS sensor array (band-wise acquisition) to collect the original spectral data within the entire intelligent spectral analysis scene, and calculates the corrected spectral data based on the mathematical model of ambient light dynamic compensation. The corrected spectral data is then input into the multi-parameter inversion module.

[0011] The calculation process of the mathematical model of ambient light dynamic compensation is as follows: the original spectral data is optimized by using a lock-in amplifier to synchronize the light source pulse, and then the corrected spectral data is calculated based on the optimized original spectral data and the background spectral data;

[0012] The multi-parameter inversion module receives the corrected spectral data from the multi-spectral acquisition module and the compensation parameters from the environmental suppression module, analyzes their data characteristics, then inputs the data characteristics into the 1D-CNN+LSTM multi-task learning network model to obtain parameter predictions. The parameter predictions and real-time parameter measurements are then input into the loss function to obtain parameter gradients and residuals. Based on the parameter gradients, the multi-parameter inversion module updates the parameters in the 1D-CNN+LSTM multi-task learning network model through backpropagation calculations, and transmits the residuals to the adaptive correction engine.

[0013] The adaptive correction engine updates the parameters of the mathematical model for dynamic compensation of ambient light and detects the residual output of the multi-parameter inversion module. If the residual exceeds the set threshold, the incremental learning mechanism and virtual sample generator are triggered. After the virtual sample generator is triggered, it synthesizes training samples. The incremental learning mechanism updates the parameters of the mathematical model for dynamic compensation of ambient light through incremental learning of the synthesized training samples, thereby realizing the feedback control loop of the adaptive correction engine on the multi-spectral acquisition module and the multi-parameter inversion module.

[0014] The cloud-edge collaboration module uses the MQTT over 4G / 5G communication protocol to implement cloud data interaction in a 24-hour cycle, receives data uploaded by the multi-spectral acquisition module, multi-parameter inversion module and adaptive correction engine, and uses the cloud-based federated learning aggregation edge node data algorithm to optimize and provide feedback on the uploaded data, promoting the parameter update of the multi-spectral acquisition module, multi-parameter inversion module and adaptive correction engine in the water quality intelligent spectral analysis system, and storing the optimized data in the time series spectral database for backup and management.

[0015] The mathematical expression of the mathematical model of the ambient light dynamic compensation is:

[0016]

[0017] Where S corrected (λ) is the corrected spectral vector at wavelength λ; S raw (λ) is the original spectrum vector collected by the sensor at wavelength λ, which contains water sample signal and ambient light interference),

[0018] S ambient (λ) is the ambient light background spectrum vector collected by the sensor when the active light source is turned off at wavelength λ. K(λ) is the light source-ambient light cross-interference kernel function, whose physical meaning is the frequency domain response generated by ambient light in the optical path. Its discrete form is an n-dimensional vector that satisfies the normalization condition ∑i=1nK(λi)=1. represents the convolution operation, λ is the wavelength, and α(λ) is the wavelength-dependent adaptive weight coefficient, which is determined by the ambient light intensity and spectral similarity.

[0019] The mathematical expression of the adaptive weight coefficient α(λ) is:

[0020]

[0021] Where β is the adjustment factor, which controls the weight of light intensity and spectral characteristics, I ref is the reference light intensity threshold, S ref It is a typical ambient light spectrum template;

[0022] sim(·) is the spectral similarity function, and its mathematical expression is:

[0023]

[0024] Where A is S ambient , B is S ref .

[0025] The structure of the 1D-CNN+LSTM multi-task learning network model includes:

[0026] Input layer, used to receive data features;

[0027] 1D-CNN neural network layer, mainly used to extract the spatial features of the spectrum, which includes three layers of convolution, each followed by ReLU+MaxPooling;

[0028] The LSTM neural network layer is mainly used to construct the temporal characteristics of the spectrum and adopts a bidirectional LSTM neural network design;

[0029] A multi-task learning network layer that uses a prediction model to predict water quality parameters based on the temporal characteristics of the spectrum;

[0030] The loss calculation layer is mainly used to balance the multi-task learning objectives, use the loss function to adjust the task weights and output the predicted value and residual.

[0031] The mathematical model of the loss function is:

[0032]

[0033] Where K is the number of parameters to be measured, is the loss function of the kth parameter, y k and are the true value and predicted value of the kth parameter, ω k is the task weight, γ is the L2 regularization coefficient, is the Frobenius norm of the model parameter matrix; where the task weight ω k Dynamic normalization is required based on the parameter measurement range:

[0034] σ k =std(y k )

[0035] Where, σ k is the standard deviation of the kth parameter in the training set.

[0036] The data feature is a feature vector formed by fusing the band ratio feature and the differential spectrum feature;

[0037] in,

[0038] The band ratio characteristics are:

[0039]

[0040] The differential spectral characteristics are:

[0041]

[0042] In the above formula, R i / j is the band ratio index; is the first-order differential spectrum; Δλ is the spectrum sampling interval; λ i1 is the starting wavelength of the characteristic band of parameter i, λ i2 is the end wavelength of the characteristic band, which is preset according to the absorption characteristics of the material; j1 is the starting wavelength of the reference band, λ j2 is the end wavelength of the reference band; i is the target wavelength point; S(λ i +1) is the spectrum value of the right neighbor point; S(λ i -1) is the spectrum value of the left neighboring point;

[0043] The prediction model of the data feature input is:

[0044]

[0045] Where, is the predicted value of COD, is the predicted value of ammonia nitrogen, W1,b1 are the hidden layer weight matrix and bias vector, W2,b2 are the output layer weight matrix and bias vector, and ReLU(·) is the activation function.

[0046] The triggering condition for the residual error during detection is:

[0047] ε=||S actual -S pred ||2>3σ base

[0048] Where S actual is the current measured spectrum vector, S pred is the spectrum vector reconstructed by the current model, σ base is the standard deviation of the historical residuals.

[0049] The mathematical model of the virtual sample generator synthesizing the virtual sample is:

[0050]

[0051] Where I0(λ) is the initial intensity of the light source, ε m (λ) is the molar absorption coefficient of the mth pollutant (physicochemical parameters are known), c m is the randomly generated pollutant concentration (uniformly distributed within a reasonable range), l is the optical path length, N(0,Σ) is Gaussian noise, and the covariance matrix Σ is estimated by historical noise;

[0052] The mathematical expression of the incremental learning mechanism using online gradient descent update is:

[0053]

[0054] Among them, the dynamic learning rate is:

[0055]

[0056] Where, Θ t is the model parameter at time t, η0 is the initial learning rate (0.01), k is the decay coefficient (0.1), N update is the cumulative number of updates, and T is the decay period.

[0057] An adaptive correction method, which uses the multi-parameter water quality intelligent spectral analysis system described in any one of claims 1 to 9, and comprises the following steps:

[0058] Step 1: Implement ambient light motion compensation:

[0059] First, the pulse light source is turned off, and the ambient light background spectrum is collected, and the light intensity and solar incident angle are recorded simultaneously. Then, the pulse modulated light source is turned on, and the original spectral data of the water sample is collected. The original spectral data of the water sample is dynamically compensated in real time. At an illumination of 100,000 lux, the signal-to-noise ratio is improved from 20dB to 45dB, and the correlation coefficient between the compensated spectral data and the darkroom measurement results is improved.

[0060] Step 2: Multi-parameter joint inversion:

[0061] Construct band ratio features and differential spectral features, use the 1D-CNN+LSTM multi-task learning network model to predict parameters, use the loss function to optimize and update the parameters of the 1D-CNN+LSTM multi-task learning network model, and output the residual at the same time;

[0062] Step 3: Residual trigger judgment:

[0063] Calculate the output residual to determine whether the residual meets the trigger condition, effectively increasing the sensitivity of anomaly detection to 90%;

[0064] Step 4: Incremental learning correction:

[0065] The COD / ammonia nitrogen / turbidity / total phosphorus concentration combinations randomly generated by virtual sample generation are used as training samples, and the learning rate decay strategy of the incremental learning mechanism is used to update the parameters of the dynamic compensation of ambient light.

[0066] Step 5: Cloud-edge collaborative optimization:

[0067] Edge nodes are used to upload encrypted spectral data to the cloud every 24 hours. The cloud aggregates edge node data through federated learning algorithms, regenerates the global model and optimizes model parameters.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The present invention has a strong ability to resist strong light interference. It uses the ambient light suppression module to realize the separation of pulse modulation signals and dynamic subtraction of background spectrum, effectively isolating the interference of strong light;

[0070] 2. The present invention achieves synchronous high-precision detection effects based on multiple parameters, and realizes multi-task learning joint optimization process through a multi-parameter inversion module, making the intelligent detection process more timely and accurate;

[0071] 3. The present invention can effectively improve the adaptability of the system, automatically expand the virtual sample training set using the adaptive correction engine, and realize the adaptive adjustment of model parameters through the incremental learning mechanism, thereby realizing intelligent and precise water quality detection, which can better meet the needs of actual use;

[0072] 4. The present invention has strong robustness and uses cloud-edge collaborative federated learning to periodically optimize the global model to ensure long-term steady-state operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 This is a structural block diagram of a multi-parameter water quality intelligent spectral analysis system according to the present invention;

[0074] Figure 2 is a flow chart of the multi-parameter inversion and adaptive correction according to the present invention;

[0075] Figure 3 This is a schematic diagram of the federated learning algorithm for aggregating edge node data according to the present invention;

[0076] Figure 4 This is a data comparison and analysis table of Example 1 of the present invention;

[0077] Figure 5 This is a data comparison and analysis table of Example 2 of the present invention;

[0078] Figure 6 This is a data comparison and analysis table of Example 3 of the present invention;

[0079] Figure 7 is a comparative analysis table of Examples 1-3 of the present invention;

[0080] Figure 8 It is a flow chart of the adaptive correction method of the present invention. DETAILED DESCRIPTION

[0081] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1-3 As shown;

[0082] A multi-parameter water quality intelligent spectral analysis system, the system includes the following parts:

[0083] The environmental suppression module, whose main function is to eliminate external light interference, forms an intelligent spectral analysis scene with a closed dark box and an electric shutter installed on the closed dark box. The closed dark box is used to accommodate the collected water samples, and after the electric shutter is closed, the intelligent spectral analysis scene of the water samples is formed. The closed dark box is provided with a pulse-modulated wide-spectrum LED light (the wavelength of the light emitted by the LED light is 200-850nm) and an ambient light sensor. The light emitted by the pulse-modulated wide-spectrum LED light serves as the background light inside the closed dark box, and the ambient light sensor is used to collect background spectral data inside the closed dark box in real time; wherein, the sampling rate of the pulse-modulated wide-spectrum LED light is ≥1kHz;

[0084] It should be noted that the environmental suppression module uses the ambient light sensor to collect background spectrum data after the electric shutter is closed, and uses the temperature sensor and pH sensor to obtain compensation parameters, namely temperature T and pH;

[0085] The multi-spectral acquisition module is mainly used to collect high-precision water sample spectral data. It uses a multi-channel InGaAs / CMOS sensor array (band-wise acquisition) to collect the original spectral data within the entire intelligent spectral analysis scene, and calculates the corrected spectral data based on the mathematical model of ambient light dynamic compensation. The corrected spectral data is then input into the multi-parameter inversion module.

[0086] The calculation process of the mathematical model of ambient light dynamic compensation is as follows: using a phase-locked amplifier to detect and optimize the original spectral data, and then calculating the corrected spectral data based on the optimized original spectral data and background spectral data;

[0087] It should be noted that the multi-spectral acquisition module controls the pulse-modulated broadband LED lamp to excite at a frequency of 1kHz, and then uses a multi-channel InGaAs / CMOS sensor array to collect water sample spectral data S raw , and finally get real-time calculation;

[0088] The main function of the multi-parameter inversion module is to synchronously output the predicted values ​​of parameters such as COD / ammonia nitrogen as the analysis results. The specific process is: after receiving the corrected spectral data output by the multi-spectral acquisition module and the compensation parameters output by the environmental suppression module, the multi-parameter inversion module analyzes the data characteristics of the two, and then inputs the data characteristics into the 1D-CNN+LSTM multi-task learning network model (i.e.: 1D-CNN (one-dimensional convolutional neural network) + LSTM (long short-term memory neural network) and multi-task learning network) and obtains the parameter prediction value, and then compares the parameter prediction value with the parameter real-time measurement value (i.e. the real-time measurement value of the corrected spectral data output by the multi-spectral acquisition module and the real-time measurement value of the compensation parameters output by the environmental suppression module). The measured value) is input into the loss function to obtain the parameter gradient (i.e., the slope of the loss function in the parameter space, indicating the direction in which the loss function value decreases fastest) and the residual (i.e., the scalar value output by the loss function after receiving the predicted value and the real-time measured value as input, which is mainly used to measure the prediction error of the 1D-CNN+LSTM multi-task learning network model - that is, the residual of the spectral vector). The multi-parameter inversion module updates the parameters in the 1D-CNN+LSTM multi-task learning network model based on the parameter gradient through back propagation calculation (i.e., the loss value of the loss function → calculating the parameter gradient using the back propagation algorithm → updating the CNN / LSTM weights), and transmits the residual to the adaptive correction engine.

[0089] The adaptive correction engine is mainly used to dynamically correct the disturbance of the environment on the water sample, especially to update the parameters of the mathematical model of dynamic compensation of ambient light. The specific correction process is as follows: the adaptive correction engine detects the residual output of the multi-parameter inversion module. If the residual exceeds the set threshold, the incremental learning mechanism and the virtual sample generator are triggered. After the virtual sample generator is triggered, it synthesizes training samples. The incremental learning mechanism updates the parameters of the mathematical model of dynamic compensation of ambient light through incremental learning of the synthesized training samples, thereby realizing the feedback control loop of the adaptive correction engine on the multi-spectral acquisition module and the multi-parameter inversion module.

[0090] The cloud-edge collaboration module mainly performs long-term optimization and data management of the above-mentioned models, generally in a 24-hour cycle. It uses the MQTT over 4G / 5G communication protocol to implement cloud data interaction, receives data uploaded by the multi-spectral acquisition module, multi-parameter inversion module and adaptive correction engine, and uses the cloud-based federated learning aggregation edge node data algorithm to optimize and feedback the uploaded data, promote the parameter update of the multi-spectral acquisition module, multi-parameter inversion module and adaptive correction engine in the water quality intelligent spectral analysis system, and store the optimized data in the time series spectral database for backup and management.

[0091] The mathematical expression of the mathematical model of the ambient light dynamic compensation is:

[0092]

[0093] Where S corrected (λ) is the corrected spectral vector at wavelength λ (i.e., corrected spectral data); S raw (λ) is the original spectral vector collected by the sensor at wavelength λ (i.e., the original spectral data (usually voltage or digital count value), which contains water sample signal and ambient light interference), S ambient (λ) is the ambient light background spectrum vector collected by the sensor when the active light source is turned off at wavelength λ (i.e., background spectrum data, measured under the same conditions by turning off the active light source), K(λ) is the light source-ambient light cross-interference kernel function (describing the interaction between the active light source and the ambient light), and its physical meaning is the frequency domain response generated by the ambient light in the optical path; its discrete form is an n-dimensional vector (n is the number of wavelength channels), satisfying the normalization condition ∑ i =1nK(λi)=1, represents a convolution operation (used to simulate the frequency-domain coupling effect of ambient light in an optical system), where λ is the wavelength in nanometers (nm), covering the system operating band (200-850 nm), and α(λ) is the wavelength-dependent adaptive weight coefficient, which is determined by the ambient light intensity and spectral similarity.

[0094] The mathematical expression of the adaptive weight coefficient α(λ) is:

[0095]

[0096] Where β is the adjustment factor (0.6-0.8), which controls the weight of light intensity and spectral characteristics, I ref is the reference light intensity threshold (100,000 lux), S ref Typical ambient light spectrum template (such as sunlight, cloudy day, incandescent lamp, etc.);

[0097] sim(·) is the spectral similarity function (using cosine similarity), and its mathematical expression is:

[0098]

[0099] Its main function is to accurately simulate the coupling mechanism of ambient light in the optical system through convolution operation, and use adaptive weight α to achieve frequency-domain selective subtraction of ambient light interference, solving the problem of spectral baseline drift under strong light.

[0100] Where A is S ambient , B is S ref .

[0101] The structure of the 1D-CNN+LSTM multi-task learning network model includes:

[0102] Input layer, used to receive data features;

[0103] 1D-CNN neural network layer, mainly used to extract the spatial features of the spectrum, which includes 3 layers of convolution (kernel = 3, stride = 1), each layer is followed by ReLU + MaxPooling (pool_size = 2);

[0104] LSTM neural network layer is mainly used to construct the time series characteristics of the spectrum. It adopts bidirectional LSTM neural network design. size =64;

[0105] The multi-task learning network layer uses the temporal characteristics of the spectrum as the basis and uses the prediction model to predict water quality parameters, including: COD branch prediction, full connection (64→1) + Sigmoid (activation function); ammonia nitrogen branch prediction, full connection (64→1) + LeakyReLU (0.1);

[0106] The loss calculation layer is mainly used to balance the multi-task learning objectives, use the loss function to adjust the task weights and output the predicted value and residual.

[0107] The mathematical model of the loss function is:

[0108]

[0109] Where K is the number of parameters to be measured (such as COD, ammonia nitrogen, turbidity, etc.), is the loss function (mean square error MSE) of the kth parameter, y k and are the true value and predicted value of the kth parameter, ω k is the task weight (dynamically adjusted according to the importance of the parameter), γ is the L2 regularization coefficient, is the Frobenius norm of the model parameter matrix; where the task weight ω k Dynamic normalization is required based on the parameter measurement range:

[0110] σ k =std(y k )

[0111] Where, σ k is the standard deviation of the kth parameter in the training set.

[0112] The data feature is a feature vector formed by fusing the band ratio feature (whose main function is to enhance feature differentiation and suppress background interference) and the differential spectrum feature (whose main function is to highlight the absorption peak / valley position and suppress baseline drift);

[0113] in,

[0114] The band ratio characteristics are:

[0115]

[0116] The differential spectral characteristics are:

[0117]

[0118] In the above formula, R i / j is the band ratio index; is the first-order differential spectrum; Δλ is the spectrum sampling interval (unit: nm), which is determined by the resolution of the spectrometer; λ i1 is the starting wavelength of the characteristic band of parameter i, λ i2 The end wavelength of the characteristic band is preset according to the absorption characteristics of the substance (such as the main absorption band of COD is 254nm); j1 is the starting wavelength of the reference band, λ j2 is the end wavelength of the reference band; i is the target wavelength point; S(λ i +1) is the spectrum value of the right neighbor point; S(λ i -1) is the spectrum value of the left neighboring point;

[0119] The prediction model of the data feature input is:

[0120]

[0121] Where, is the predicted value of COD, is the predicted value of ammonia nitrogen, W1,b1 are the hidden layer weight matrix and bias vector, W2,b2 are the output layer weight matrix and bias vector, and ReLU(·) is the activation function (i.e., rectified linear unit).

[0122] The triggering condition for the residual error during detection is:

[0123] ε=||S actual -S pred ||2>3σ base

[0124] Where S actual is the current measured spectrum vector, S pred is the spectral vector reconstructed by the current model (via Calculation), σ base is the standard deviation of the historical residuals (obtained by rolling calculation of the last 100 measurements).

[0125] The mathematical model for synthesizing virtual samples by the virtual sample generator (mainly a synthesis model based on the Beer-Lambert law) is:

[0126]

[0127] Where I0(λ) is the initial intensity of the light source, ε m (λ) is the molar absorption coefficient of the mth pollutant (physicochemical parameters are known), c m is the randomly generated pollutant concentration (uniformly distributed within a reasonable range), l is the optical path length (fixed value), N(0,Σ) is Gaussian noise, and the covariance matrix Σ is estimated by historical noise;

[0128] The mathematical expression of the incremental learning mechanism using online gradient descent update is:

[0129]

[0130] Among them, the dynamic learning rate is:

[0131]

[0132] Where, Θ t is the model parameter at time t, η0 is the initial learning rate (0.01), k is the decay coefficient (0.1), N update is the cumulative number of updates, and T is the decay period (usually set to 100).

[0133] An adaptive correction method, the method adopts the multi-parameter water quality intelligent spectral analysis system described in any one of claims 1 to 9 above, such as Figure 8 As shown, it includes the following steps:

[0134] Step 1: Implement ambient light motion compensation:

[0135] Its main purpose is to eliminate the interference of sunlight (10-5lux) on active light sources (10^3lux) during outdoor detection, and to solve the problem of traditional methods with a sudden drop in signal-to-noise ratio of more than 15dB under strong light.

[0136] First, the pulse light source was turned off, and the ambient light background spectrum was collected, and the light intensity (unit: lux) and solar incidence angle were simultaneously recorded. Then, the pulse modulated light source (1kHz square wave) was turned on to collect the original spectral data of the water sample. The original spectral data of the water sample was dynamically compensated in real time. At an illumination of 100,000 lux, the signal-to-noise ratio was improved from 20dB to 45dB, and the correlation coefficient between the compensated spectral data and the darkroom measurement results was improved.

[0137] Step 2: Multi-parameter joint inversion:

[0138] It mainly solves the spectral overlap problem of COD (254nm), ammonia nitrogen (210nm), turbidity (full band) and other parameters, and avoids the error accumulation of traditional single parameter models;

[0139] Construct band ratio features and differential spectral features, use a 1D-CNN+LSTM multi-task learning network model to predict parameters, and use a loss function to optimize and update the parameters of the 1D-CNN+LSTM multi-task learning network model (only updating the weights of the last two layers of the network to avoid catastrophic forgetting), while outputting the residual.

[0140] Step 3: Residual trigger judgment:

[0141] Its main purpose is to identify sudden changes in water quality or model drift (such as the emergence of new pollutants) to avoid continuous output of erroneous results;

[0142] The output residual is calculated to determine whether the residual meets the trigger condition (the trigger condition is that it exceeds the standard three times in a row to be considered a valid trigger), effectively increasing the sensitivity of anomaly detection to 90% (the traditional fixed threshold method is only 60%).

[0143] Step 4: Incremental learning correction:

[0144] Its main purpose is to achieve online model update and solve the problem that traditional methods must return to the laboratory for recalibration;

[0145] The COD / ammonia nitrogen / turbidity / total phosphorus concentration combinations randomly generated by virtual sample generation are used as training samples, and the learning rate decay strategy of the incremental learning mechanism is used to update the parameters of the ambient light dynamic compensation;

[0146] Step 5: Cloud-edge collaborative optimization: Edge nodes upload encrypted spectral data to the cloud every 24 hours. The cloud aggregates edge node data through federated learning algorithms, regenerates the global model, and optimizes model parameters.

[0147] The content of the federated learning algorithm for aggregating edge node data is as follows:

[0148] Initialization: The cloud initializes global model parameters, and each edge node downloads the corresponding parameters as the local model initial values;

[0149] Local training allows edge nodes to train using local data and calculate parameter updates;

[0150] Secure upload: Edge nodes use homomorphic encryption to encrypt gradients and add differential privacy noise.

[0151] Cloud aggregation, decryption and weighted aggregation are performed to update the global model;

[0152] Model distribution: Send the updated global model to all edge nodes. The nodes verify the model signature and then load it for use.

[0153] Example 1:

[0154] A portable field rapid detection instrument, the test data source: Yellow River water samples, such as Figure 4 As shown, it includes:

[0155] Environmental suppression module: uses a foldable dark box (unfolded size 20×15×10cm), a miniature electric shutter (response time <50ms) and a pulsed LED array (peak wavelength 254 / 365 / 690nm);

[0156] Multispectral acquisition module: uses low-cost CMOS sensor (resolution 5nm) and dynamic compensation model;

[0157] Multi-parameter inversion module: uses lightweight 1D-CNN (only 3 layers, 50KB parameters), supporting COD / ammonia nitrogen dual parameter prediction;

[0158] Adaptive correction engine: The threshold is set to be greater than 15%, and the virtual sample generation cycle is 24 hours;

[0159] Cloud-edge collaboration module: uses 4G transmission and synchronizes once a day.

[0160] Example 2:

[0161] A fixed water quality monitoring station, the test data source: Yangtze River estuary; Figure 5 As shown, it includes:

[0162] Environmental suppression module: uses a stainless steel dark box (anti-corrosion design), a double shutter redundancy system and a high-stability LED (lifespan > 10,000 hours);

[0163] Multispectral acquisition module: using InGaAs sensor (resolution 1nm), real-time full spectrum compensation;

[0164] Multi-parameter inversion module: uses a 1D-CNN-LSTM hybrid network (parameter size 2MB) to simultaneously output COD / ammonia nitrogen / turbidity / total phosphorus;

[0165] Adaptive correction engine: intelligent threshold setting: greater than 3, and real-time virtual sample generation mode;

[0166] Cloud-edge collaboration module: uses 5G private network, synchronizes data interaction every hour, and aggregates 10 nodes for federated learning.

[0167] Example 3

[0168] A drone-mounted mobile monitoring network, with test data sourced from the Taihu Lake cyanobacteria outbreak area; Figure 6 As shown, it includes:

[0169] Environmental Suppression Module: Utilizes a carbon fiber dark box (weight <300g), an anti-vibration shutter mechanism, and a wide-angle ambient light sensor;

[0170] Multispectral acquisition module: uses a micro-spectrometer (wavelength range 400-800nm) superimposed motion compensation algorithm;

[0171] Multi-parameter inversion module: uses the knowledge distillation model (the teacher model parameters are compressed by 80%) and only outputs COD / ammonia nitrogen;

[0172] Adaptive correction engine: uses cluster collaborative correction mode, that is, 10 drones share residual data and implement aerial virtual sample generation mode;

[0173] Cloud-edge collaboration module: uses Starlink transmission and real-time federated learning mechanism to automatically generate heat maps.

[0174] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include," "comprise," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations. The phrase "includes an element defined by..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0175] The above technical solutions only reflect the preferred technical solutions of the technical solutions of the present invention. Any changes that may be made to certain parts thereof by those skilled in the art all reflect the principles of the present invention and fall within the scope of protection of the present invention.

Claims

1. A multi-parameter water quality intelligent spectral analysis system, characterized in that: The system consists of the following parts: The environmental suppression module uses a closed darkroom and electric shutter to form an intelligent spectral analysis scene. The closed darkroom is equipped with a pulse-modulated wide-spectrum LED light and an ambient light sensor. The multi-spectral acquisition module collects the original spectral data in the entire intelligent spectral analysis scene, calculates the corrected spectral data based on the mathematical model of ambient light dynamic compensation, and then inputs the corrected spectral data into the multi-parameter inversion module; The multi-parameter inversion module receives the corrected spectral data and compensation parameters, analyzes their data characteristics, and then inputs the data characteristics into the 1D-CNN+LSTM multi-task learning network model to obtain parameter prediction values. The parameter prediction values ​​and real-time parameter measurements are then input into the loss function to obtain parameter gradients and residuals. The multi-parameter inversion module updates the parameters in the 1D-CNN+LSTM multi-task learning network model based on the parameter gradients through backpropagation calculations, and transmits the residuals to the adaptive correction engine. The adaptive correction engine detects residuals. If the residual exceeds a set threshold, it triggers the incremental learning mechanism and virtual sample generator. Once triggered, the virtual sample generator synthesizes training samples. The incremental learning mechanism updates the parameters of the mathematical model for ambient light dynamic compensation through incremental learning of the synthesized training samples. The cloud-edge collaboration module implements cloud-side data interaction, uses the cloud-based federated learning aggregation edge node data algorithm to optimize and feedback the uploaded data, promotes the parameter update of the water quality intelligent spectral analysis system, and stores the optimized data in the time series spectral database for backup and management.

2. A multi-parameter water quality intelligent spectral analysis system according to claim 1, characterized in that: The mathematical expression of the mathematical model of the ambient light dynamic compensation is: Where S corrected (λ) is the corrected spectral vector at wavelength λ; S raw (λ) is the original spectrum vector collected by the sensor at wavelength λ, S ambient (λ) is the ambient light background spectrum vector collected by the sensor when the active light source is turned off at wavelength λ, K(λ) is the light source-ambient light cross-interference kernel function, represents the convolution operation, λ is the wavelength, and α(λ) is the wavelength-dependent adaptive weight coefficient, which is determined by the ambient light intensity and spectral similarity.

3. A multi-parameter water quality intelligent spectral analysis system according to claim 2, characterized in that: The mathematical expression of the adaptive weight coefficient α(λ) is: Where β is the adjustment factor, which controls the weight of light intensity and spectral characteristics, I ref is the reference light intensity threshold, S ref It is a typical ambient light spectrum template; sim(·) is the spectral similarity function, and its mathematical expression is: Where A is S ambient , B is S re f.

4. A multi-parameter water quality intelligent spectral analysis system according to claim 1, characterized in that: The structure of the 1D-CNN+LSTM multi-task learning network model includes: Input layer, used to receive data features; 1D-CNN neural network layer, mainly used to extract the spatial features of the spectrum, which includes three layers of convolution, each followed by ReLU+MaxPooling; The LSTM neural network layer is mainly used to construct the temporal characteristics of the spectrum and adopts a bidirectional LSTM neural network design; A multi-task learning network layer that uses a prediction model to predict water quality parameters based on the temporal characteristics of the spectrum; The loss calculation layer is mainly used to balance the multi-task learning objectives, use the loss function to adjust the task weights and output the predicted value and residual.

5. A multi-parameter water quality intelligent spectral analysis system according to claim 4, characterized in that: The mathematical model of the loss function is: Where K is the number of parameters to be measured, is the loss function of the kth parameter, y k and are the true value and predicted value of the kth parameter, ω k is the task weight, γ is the L2 regularization coefficient, is the Frobenius norm of the model parameter matrix; where the task weight ω k Dynamic normalization is required based on the parameter measurement range: s k =std(y k ) Where, σ k is the standard deviation of the kth parameter in the training set.

6. A multi-parameter water quality intelligent spectral analysis system according to claim 4, characterized in that: The data feature is a feature vector formed by fusing the band ratio feature and the differential spectrum feature; in, The band ratio characteristics are: The differential spectral characteristics are: In the above formula, R i / j is the band ratio index; is the first-order differential spectrum; Δλ is the spectrum sampling interval; λ i1 is the starting wavelength of the characteristic band of parameter i, λ i2 is the end wavelength of the characteristic band, which is preset according to the absorption characteristics of the material; j1 is the starting wavelength of the reference band, λ j2 is the end wavelength of the reference band; i is the target wavelength point; S(λ i +1) is the spectrum value of the right neighbor point; S(λ i -1) is the spectrum value of the left neighboring point.

7. A multi-parameter water quality intelligent spectral analysis system according to claim 4, characterized in that: The prediction model of the data feature input is: Where, is the predicted value of COD, is the predicted value of ammonia nitrogen, W1, b1 are the hidden layer weight matrix and bias vector, W2, b2 are the output layer weight matrix and bias vector, and ReLU (ε) is the activation function.

8. The multi-parameter water quality intelligent spectral analysis system according to claim 4, characterized in that: The triggering condition for the residual error during detection is: ε=||S actual -S pred ||2>3σbase Where S actual is the current measured spectrum vector, S pred is the spectrum vector reconstructed by the current model, σ base is the standard deviation of the historical residuals.

9. The multi-parameter water quality intelligent spectral analysis system according to claim 4, characterized in that: The mathematical model of the virtual sample generator synthesizing the virtual sample is: Where I0(λ) is the initial intensity of the light source, ε m (λ) is the molar absorption coefficient of the mth pollutant, c m is the randomly generated pollutant concentration, l is the optical path length, N(0,Σ) is Gaussian noise, and the covariance matrix Σ is estimated by historical noise; The incremental learning mechanism uses the online gradient descent update mathematical expression as follows: Among them, the dynamic learning rate is: Where, Θ t is the model parameter at time t, η0 is the initial learning rate (0.01), k is the decay coefficient (0.1), N update is the cumulative number of updates, and T is the decay period.

10. An adaptive correction method, characterized in that: The method adopts the multi-parameter water quality intelligent spectral analysis system described in any one of claims 1 to 9, and comprises the following steps: Step 1: Implement ambient light motion compensation: First, turn off the pulse light source, collect the ambient light background spectrum and synchronously record the light intensity and solar incident angle, then turn on the pulse modulated light source, collect the original spectrum data of the water sample, and perform real-time dynamic compensation on the original spectrum data of the water sample; Step 2: Multi-parameter joint inversion: Construct band ratio features and differential spectral features, use the 1D-CNN+LSTM multi-task learning network model to predict parameters, use the loss function to optimize and update the parameters of the 1D-CNN+LSTM multi-task learning network model, and output the residual at the same time; Step 3: Residual trigger judgment: Calculate the output residual to determine whether the residual meets the trigger condition; Step 4: Incremental learning correction: The training samples are randomly generated by virtual sample generation, and the learning rate decay strategy of the incremental learning mechanism is used to update the parameters of the ambient light dynamic compensation. Step 5: Cloud-edge collaborative optimization: Edge nodes are used to upload encrypted spectral data to the cloud every 24 hours. The cloud aggregates edge node data through federated learning algorithms, regenerates the global model and optimizes model parameters.

Images