Visible spectrum noise reduction and heavy metal ion detection method

Abstract

The invention relates to the technical field of spectral analysis, and discloses a visible spectrum noise reduction and heavy metal ion detection method, which comprises the following steps: reacting a water sample to be detected with a color developing agent to generate a color developing system, obtaining a gray spectrum signal through optical imaging equipment, carrying out variational mode decomposition, optimizing the signal by adopting singular value decomposition, and calculating the noise of the heavy metal ions. And determining an optimal reconstruction order by using a particle swarm optimization algorithm and reconstructing a signal, finally extracting a feature value from the denoised spectral signal, and predicting the type and concentration of metal ions through a neural network model. Spectral global / local features are modeled through cooperation of multi-head self-attention and learnable position coding, signal frequency-time domain cross-modal fusion is realized in combination with variational mode decomposition and cross attention, training parameters are dynamically optimized by innovatively adopting a reinforcement learning strategy, classification and regression tasks are self-adaptively balanced based on a gradient diversity reward mechanism, and frequency-time domain cross-modal fusion is realized. And finally, dual precision improvement of ion type identification and concentration prediction and model robustness enhancement are realized in a complex noise scene.

Description

Technical Field

[0001] The present invention relates to the technical field of spectrum analysis, and in particular to a visible spectrum noise reduction and heavy metal ion detection method. Background Art

[0002] With the continuous advancement of industrialization, heavy metal ion pollution has become a major challenge in environmental monitoring. Traditional detection methods such as atomic absorption spectrometry (AAS) and inductively coupled plasma mass spectrometry (ICP-MS) have high accuracy, but rely on complex operations and expensive equipment, making it difficult to meet the needs of real-time monitoring. In recent years, visible spectroscopy has become a research hotspot due to its advantages such as non-contact and rapidity, but its practical application is limited by two major bottlenecks: First, spectral signals are easily affected by noise, and traditional noise reduction methods (such as wavelet transform) over-smoothe the signal, resulting in the loss of key features; second, existing deep learning models (such as CNN and MLP) are limited by their local feature extraction capabilities and fixed training strategies, making it difficult to simultaneously achieve high-precision classification and concentration regression.

[0003] Specifically, existing spectral analysis models based on convolutional neural networks can extract local wavelength features through convolution kernels, but are unable to model long-range wavelength dependencies, resulting in insufficient analysis capabilities for overlapping peaks and wide-band signals. At the same time, fixed network structures and loss function designs are prone to gradient conflicts in multi-task learning, restricting the stability of the model in complex noisy environments. Although recent studies have attempted to introduce transfer learning or lightweight design to improve efficiency, there is still a lack of a mechanism for integrating spectral physical priors (such as frequency domain modal features), and the training process relies on manual parameter adjustment, making it difficult to adaptively optimize the balance between classification and regression objectives.

[0004] In response to the above problems, the present invention proposes a visible spectrum noise reduction and heavy metal ion detection method. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a visible spectrum noise reduction and heavy metal ion detection method, which solves the problems of high cost, low efficiency and spectral signal noise interference in existing heavy metal ion detection methods.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: A visible spectrum noise reduction and heavy metal ion detection method comprises the following steps:

[0007] S1, reacting the water sample to be tested with a preset color developer to generate a visual color system;

[0008] S2. Capturing an image of the color development system and obtaining a corresponding grayscale spectrum signal by an optical imaging device;

[0009] S3. performing variational modal decomposition on the grayscale spectrum signal to obtain multiple modal components;

[0010] S4. Constructing Hankel matrices for each of the plurality of modal components and performing singular value decomposition to obtain singular value components of each mode;

[0011] S5. Optimizing the singular value components using a particle swarm optimization algorithm to determine an optimal reconstruction order;

[0012] S6. Reconstructing the modal components based on the optimal reconstruction order to obtain a one-dimensional spectral signal after noise reduction;

[0013] S7, extracting characteristic values of the target wavelength range from the one-dimensional spectral signal after noise reduction as model input, and using a neural network model to predict the type and concentration of metal ions.

[0014] Preferably, the color developer in step S1 is a complex compound configured according to different metal ions, and the color developer in step S1 is used to undergo a complex reaction with the target metal ion. The reaction is carried out at a temperature of 25°C-30°C, a pH value of 6.5-7.5, and a reaction time of 5-15 minutes. The color developer used to detect cadmium ions is a complex formed by disulfide hydrazone and benzothiazole connected by a sulfide bond, and the complex reacts with cadmium ions to produce a purple color.

[0015] Preferably, the image acquisition process in the S2 step includes: irradiating the reaction liquid with a narrow-band LED light source with a wavelength of 510-630 nanometers under dark box conditions, and using an industrial camera with a resolution of not less than 2000×600 pixels to capture the color image in the colorimetric tank, converting the obtained image into a grayscale spectral signal and normalizing it to between 0 and 1.

[0016] Preferably, the modal number K of the variational modal decomposition in step S3 is adaptively calculated according to the frequency domain bandwidth of the signal, and the calculation formula of the modal number K is:

[0017]

[0018] Among them, f max With f min are the maximum and minimum values in the frequency domain, respectively, in Hertz; is the frequency resolution, which is a real value between 0.05 and 0.5 Hz; Δf is the frequency domain resolution.

[0019] Preferably, the Hankel matrix in step S4 is a two-dimensional structure in which the sum of the number of rows and the number of columns is less than the length of the original signal. The singular value decomposition operation in step S4 takes the matrix as input and extracts the first m-order singular values and the corresponding left and right singular vectors, where m does not exceed half of the rank of the original matrix.

[0020] Preferably, the objective function of the particle swarm optimization algorithm in step S5 is:

[0021] F = α·SNR + β·MSE -1 ;

[0022] Wherein, SNR is the signal-to-noise ratio, MSE is the mean square error, and weight parameters α and β satisfy α+β=1. α is a floating point number between 0.4 and 0.8, and β is a floating point number between 0.2 and 0.6, which are automatically set according to the noise type.

[0023] Preferably, the one-dimensional spectral signal after noise reduction in step S6 is a signal reconstructed based on the optimal reconstruction order k determined in step S5, and the reconstructed signal is expressed as:

[0024]

[0025] Wherein, k is a positive integer not less than 3 and not more than 20; σ i 、u i 、v i are the singular values and the corresponding left and right singular vectors respectively; is the reconstructed signal; It represents the accumulation of the first k singular value components, that is, only the first k-order principal components are used for signal reconstruction; is the transpose of the i-th right singular vector, derived from V T Matrix, representing an orthogonal basis in column space.

[0026] Preferably, the characteristic value of the target wavelength range extracted in step S7 is the average grayscale value in the range of ±2 nm to ±10 nm of the color peak wavelength in the noise-reduced signal, and the peak wavelength is determined according to the maximum absorption wavelength of the color developer and the band is scanned in steps of 1 nm.

[0027] Preferably, the neural network model in step S7 is a hybrid model based on the Transformer architecture, comprising:

[0028] A cascaded multi-head self-attention module, where each attention head calculates the correlation between wavelength positions in the spectral sequence using a learnable weight matrix;

[0029] A learnable position encoding layer that uses a one-dimensional convolution kernel to generate wavelength position feature embeddings;

[0030] The cross-modal fusion module inputs the singular value spectrum of each modal component obtained by variational modal decomposition as prior knowledge into the cross-attention layer;

[0031] The parallel output head generates ion species classification probability and concentration regression value through the fully connected layer respectively.

[0032] Preferably, in step S7, the neural network adopts an adaptive training strategy driven by reinforcement learning, specifically including:

[0033] Define the actions of the reinforcement learning agent as learning rate adjustment, batch normalization parameter update, and loss function weight assignment;

[0034] Constructing the state space includes the number of training iterations, the classification accuracy of the validation set, the regression mean square error, and the gradient sparsity index;

[0035] The reward function is designed as:

[0036] R t =0.5ΔAccuracy+0.3(1-MSE t / MSE t-1 )+0.2Grad_Diversity;

[0037] Among them, Grad_Diversity is calculated by the singular value entropy of the parameter gradient matrix, ΔAccuracy represents the increment of classification accuracy between adjacent training time steps, and MSE t is the mean square error of the current time step t, MSE t-1 is the mean square error of the previous time step, R t is the total reward;

[0038] The double-delayed deep deterministic policy gradient algorithm is used to update the network parameters, and the policy network and target network parameters are synchronized every 50 iterations.

[0039] The present invention provides a method for visible spectrum noise reduction and heavy metal ion detection. It has the following beneficial effects:

[0040] 1. Compared with the existing methods using multi-layer perceptrons or fixed-depth networks, the present invention dynamically captures the global correlation between the wavelength positions of the spectral sequence through a cascaded multi-head self-attention mechanism, and combines the learnable position encoding layer to fine-tune the embedding of local wavelength features, thereby significantly improving the integrity of spectral feature extraction. The adaptive training strategy driven by reinforcement learning effectively suppresses the risk of model overfitting by real-time optimization of the learning rate, loss weight, and gradient update direction, while reducing the demand for hardware computing resources.

[0041] 2. The present invention solves the problem of improper handling of signal complexity in traditional methods by extracting features within the target wavelength range and combining it with the technology of cross-modal fusion module. The existing technology often adopts a relatively simple feature extraction method, which easily misses key signal details. The present invention enhances the model's ability to capture signal details through the cross-attention interaction of the variational mode decomposition singular value spectrum and the original spectrum, making the prediction of metal ion types and concentrations more accurate.

[0042] 3. The reinforcement learning-driven dynamic loss function optimization strategy adopted in the present invention effectively combines regression and classification tasks, improving the overall accuracy of the prediction results. Compared with the existing solutions that optimize regression or classification tasks separately, the present invention uses a dynamic loss weight allocation mechanism based on gradient diversity rewards to ensure that the prediction results can achieve high accuracy in the dual dimensions of metal ion types and concentrations, thereby maintaining stability and reliability in complex experimental environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the present specification. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] Please see the attached Figure 1 The embodiment of the present invention provides a visible spectrum noise reduction and heavy metal ion detection method, comprising the following steps:

[0046] S1, reacting the water sample to be tested with a preset color developer to generate a visual color system;

[0047] In the present method for visible spectrum noise reduction and heavy metal ion detection, the target metal ions in the water sample are first converted into a colorimetric system with optical response characteristics through a chemical reaction, facilitating subsequent image acquisition and spectral modeling analysis. This step, as the starting point of the entire detection process, has a significant impact on the ultimate detection sensitivity and accuracy. Therefore, it is necessary to systematically optimize the control of reaction conditions, colorimetric reagent design, and complexation mechanism.

[0048] Generally speaking, in order to obtain a color development system with strong selectivity and identifiable spectral characteristics, specific organic composite color developers can be configured for different metal ions to undergo complexation and color development reactions with metal ions to generate stable complexes with obvious absorption peaks in a specific wavelength range.

[0049] In this embodiment, step S1 includes the following contents:

[0050] The water sample to be tested may contain a variety of heavy metal ions, such as lead (Pb 2+ ), cadmium (Cd 2+ ), copper (Cu 2+) etc. In order to improve the recognition, in one possible implementation, a composite color developer that specifically reacts with the target metal ion is selected as the detection precursor. 2+ ) detection, the color developing agent is a complex formed by connecting a disulfide hydrazone (Dithizone) and a benzothiazole (Benzothiazole) group through a thioether bond.

[0051] Specifically, the composite color developer contains two thiol groups (-SH) and two imine groups (=NH) in its molecular structure, and can exist stably and maintain complexation activity in the pH range of 6.5 to 7.5. 2+ After the complexation reaction occurs, a purple complex is formed, the maximum absorption wavelength of which is between 520 nanometers and 530 nanometers, and it has good color specificity.

[0052] As an option, the reaction process is carried out in a constant temperature environment between 25°C and 30°C, and the reaction time is controlled between 5 and 15 minutes to ensure sufficient complexation reaction and stable color development, and to avoid the generation of by-products that affect spectral data collection.

[0053] In some embodiments, to ensure the selectivity of the color development reaction, the interference reaction of non-target metal ions can be suppressed by adjusting the buffer composition or introducing specific masking agents. 2+ In the system, citrate can be introduced as a chelating agent to increase the Cd 2+ response priority.

[0054] In order to facilitate the subsequent image and spectrum acquisition, the color reaction system is carried out in a preset colorimetric tank, and the color depth of the solution after color development is consistent with that of Cd 2+ The concentration shows a quantifiable relationship, which provides a basis for grayscale signal extraction and band scanning in subsequent steps.

[0055] In a preferred embodiment, in order to achieve simultaneous detection of multiple ions, different color developers can be respectively placed in a multi-channel colorimetric cell to form a parallel reaction array, thereby completing the pretreatment reactions of multiple metal ions within one acquisition cycle.

[0056] Through the above design, step S1 not only completes the color development reaction of the target metal ions, but also enhances the selectivity and detection repeatability of the system by controlling the reaction conditions and reagent structure, providing a reliable chemical basis for the subsequent spectral signal noise reduction and model recognition of the present invention.

[0057] S2. Capturing an image of the color development system and obtaining a corresponding grayscale spectrum signal by an optical imaging device;

[0058] In the implementation of the present invention, after the complexation reaction between the color developer and the metal ion to be detected in step S1, an image of the resulting color system is captured to obtain its visible light response characteristics. Step S2, which focuses on image capture and grayscale signal acquisition of the color system, is the core step in converting the chemical color development effect into a digital spectral signal.

[0059] In general, to minimize the impact of external interference on image quality and improve the reproducibility and comparison stability of collected data, it is recommended that the color development system be placed in an optical darkroom for imaging. The interior of the darkroom should be made of low-reflective black material to avoid background interference, and a colorimetric card slot can be provided at the bottom to ensure consistent sample positioning.

[0060] In this embodiment, the light source system uses a narrowband LED array with a central wavelength between 510nm and 630nm, with LED spacing controlled to 10nm. The full width at half maximum (FWHM) of each LED is controlled within 15nm, and the overall light source emission intensity is controlled between 200mA and 350mA to ensure a good distribution of image brightness across the grayscale value range.

[0061] In one possible implementation, the LED array emits light sequentially in a time-multiplexed manner. Specifically, the control chip sends PWM signals to sequentially drive LED modules of different wavelengths, synchronizing them with the camera shutter signal so that each exposure corresponds to a single wavelength. This enables a structured spectroscopic mode of "single-band excitation + single-frame acquisition."

[0062] The camera uses an industrial-grade CCD image sensor with a pixel size of 3.45μm × 3.45μm, an effective resolution of 2048 × 600 pixels, and an image frame rate of 15 frames per second. The lens has a focal length of 16mm and an aperture of f / 4.0, which, when combined with LED lighting, enables clear imaging of the colorimetric tank area.

[0063] Specifically, the captured color image is first gamma corrected (γ=1.0-1.2) and white balanced, and then converted to a grayscale image. In some embodiments, the grayscale conversion method adopts the ITU-R BT.601 standard conversion, that is:

[0064] I gray (x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y);

[0065] Among them, I gray (x,y) is the grayscale value of the pixel with coordinates (x,y) in the image; R(x,y), G(x,y), and B(x,y) correspond to the red, green, and blue channel values of the pixel, respectively; the coefficients are derived from the weighted sensitivity of the human eye to different bands.

[0066] To extract one-dimensional spectral signals, this paper introduces a vertical projection method. This method calculates the vertical average of the grayscale values of each column of pixels to form a vector g(t) corresponding to the spatial position and grayscale value. This vector is used as the grayscale spectral signal input. The expression is as follows:

[0067] Where g(t) is the grayscale value at the horizontal position t; H is the image height; W is the image width; G(t,y) is the grayscale value at the horizontal pixel t and the vertical pixel y.

[0068] This vector grayscale spectral signal g(t) will be used as the original input signal f(t) of the variational mode decomposition (VMD) algorithm in subsequent steps, so the definitions of the two must be consistent. That is, f(t) = g(t), t∈[1,W];

[0069] The grayscale signal is further normalized to the interval [0,1] so that the subsequent signal processing algorithm is not affected by the absolute brightness difference of the image. The normalization process uses the following linear mapping model:

[0070] Among them, f(t) is the original grayscale value; f min ,f max is the minimum and maximum grayscale values of all pixels in the image; f norm (t) is the normalized signal value.

[0071] In some embodiments, to prevent image edge artifacts from interfering with feature extraction, an edge mask can be introduced. This involves setting the effective region t∈[t0,t1] to ignore the edge noise. The specific edge parameter can be set based on the actual colorimetric cell width, typically within 10% of the image width.

[0072] Through the above-mentioned image acquisition system construction and grayscale signal generation method, step S2 realizes the complete conversion from the color reaction image to the standardized one-dimensional spectral signal, laying a stable and standardized data foundation for the modal decomposition processing and subsequent noise reduction analysis in step S3.

[0073] S3. performing variational modal decomposition on the grayscale spectrum signal to obtain multiple modal components;

[0074] The core of variational modal decomposition lies in the frequency domain decomposition of the signal, and its process is based on the frequency domain bandwidth of a given signal. In order to adaptively calculate the modal number K based on the frequency domain characteristics, we first calculate the appropriate modal number based on the frequency domain bandwidth (i.e., the difference between the maximum and minimum frequencies of the signal) and the frequency resolution. Specifically, the formula for calculating the modal number K is:

[0075] Among them, f max is the maximum frequency of the signal in the frequency domain, in Hertz (Hz), indicating the highest frequency in the signal frequency domain; f min is the minimum frequency of the signal in the frequency domain, in Hertz (Hz), indicating the lowest frequency in the signal's frequency domain; Δf is the frequency resolution, indicating the size of the frequency interval in the frequency domain, usually a real value ranging from 0.05Hz to 0.5Hz. The smaller the frequency resolution, the finer the resolution and the more modes there are; the larger the frequency resolution, the fewer modes there are; Indicates a rounding operation, that is, rounding the division result to the largest integer not greater than the result.

[0076] Implementation details of the variational mode decomposition process and adaptive mode number calculation:

[0077] In the actual variational mode decomposition process, the calculated mode number K is adaptive and dynamically adjusted based on the signal's frequency domain bandwidth and frequency resolution. The purpose of this adaptive calculation is to more accurately adapt to the frequency characteristics of different signals and ensure that each mode can capture the changes in the signal across different frequency ranges.

[0078] Maximum frequency f in the frequency domain max :The spectrum of the signal is usually obtained by Fourier transform or other frequency domain analysis methods, f max It is the maximum effective frequency in the signal spectrum. If the signal contains high-frequency noise or other unimportant high-frequency components, the frequencies of these components may also affect f max Therefore, in some application scenarios, it is possible to consider preprocessing or bandpass filtering the signal to eliminate unnecessary frequency components and obtain a more accurate frequency domain boundary.

[0079] Frequency domain minimum frequency f min :Similarly, f min Represents the lowest significant frequency in the signal spectrum. This value can usually be obtained by analyzing the signal's behavior at the low frequency end. For most periodic signals, f min may be close to zero, while for non-periodic or noisy signals, f min It may need to be determined by signal stationarity or denoising process.

[0080] Frequency Resolution Δf: Frequency resolution Δf controls the spacing between frequency points in the frequency domain and determines the accuracy of frequency domain analysis. In practical applications, the frequency resolution Δf may need to be selected based on the characteristics of the signal. If the signal contains subtle frequency variations, a smaller Δf may be more appropriate; if the signal spectrum is relatively smooth, a larger Δf may be more effective. Generally speaking, Δf should be selected between 0.05Hz and 0.5Hz to ensure the effectiveness of variational mode decomposition.

[0081] The influence of adaptive mode number K:

[0082] The larger the modal number K is, the wider the frequency domain bandwidth (f max -f min ) is large, or the frequency resolution Δf is small, resulting in more modes being decomposed. In this case, variational mode decomposition can capture the frequency components of the signal more carefully and is suitable for signals with rich spectra and drastic frequency changes.

[0083] A smaller modal number, K, reduces the frequency domain bandwidth or frequency resolution. This helps avoid over-decomposition in signals with smooth frequency variations or noise. For signals with smooth frequency variations, a smaller modal number provides a more concise frequency domain representation.

[0084] Generally speaking, in practical signal processing applications, the adaptive modal number calculation of variational mode decomposition can be dynamically adjusted according to the different frequency bandwidths of the signal, ensuring that the decomposed modes effectively reflect the signal's frequency domain structure. As an option, the level of decomposition can be controlled by setting different frequency resolutions. For example, when the signal level is high, a larger Δf value can be selected to reduce the number of modes and avoid decomposing too many noise components. On the other hand, when the signal bandwidth is wide and the frequency characteristics are rich, a smaller Δf value can be selected to obtain more modes and more carefully capture the frequency components in the signal.

[0085] Through the above calculation formula and implementation details, the frequency domain bandwidth (f max -f min ) and frequency resolution Δf, adaptively determining the number of modes in variational modal decomposition (VMD), thereby achieving efficient signal decomposition and precise analysis. This method is highly adaptable and flexible, capable of adjusting the decomposition accuracy based on the signal's characteristics, providing a reliable foundation for subsequent signal processing, feature extraction, and denoising tasks.

[0086] S4. Constructing Hankel matrices for each of the plurality of modal components and performing singular value decomposition to obtain singular value components of each mode;

[0087] After the grayscale spectral signal is processed by adaptively calculating the number of modes based on the frequency domain bandwidth and completing the variational mode decomposition (VMD) in step S3, the signal is decomposed into multiple modes with different frequency components. To further quantify the time domain structural characteristics of each mode and provide a low-rank discrimination basis for subsequent mode selection, reconstruction, and interference removal operations, the Hankel matrix construction and singular value decomposition (SVD) method are introduced in step S4. This step takes the time series data of each mode as input, constructs a two-dimensional matrix with structural constraints, extracts its principal component energy distribution, and quantitatively analyzes the effectiveness and sparsity of each mode.

[0088] In this embodiment, for each modal component u k (t) is handled as follows:

[0089] In a typical implementation, the modal component u k (t) is the frequency-localized time series obtained in step S3, whose length is T, i.e., t∈{1,2,…,T}. In order to extract its internal time-correlated structure from this series, it is necessary to construct it as a Hankel matrix H with Toeplitz diagonal symmetry. k .

[0090] The construction formula is as follows:

[0091]

[0092] in, is the Hankel matrix corresponding to the kth mode; u k (t) is the amplitude of the kth modal component at time t, which comes from VMD decomposition; represents the total duration (or length) of the modal component; is the number of rows of the Hankel matrix; is the number of columns in the Hankel matrix; m+n-1≤T must be satisfied to ensure that matrix element indices do not cross the bounds and that all anti-diagonal elements are identical.

[0093] In some embodiments, to improve the sensitivity of time-dependent features, the Hankel matrix can be constructed using a sliding window. The window length is set to m, the step size is 1, and each column is generated by sequentially intercepting subsequences of length m from the original modality until the end.

[0094] Singular value decomposition operation details:

[0095] The constructed Hankel matrix H k It is input into the singular value decomposition module for SVD decomposition:

[0096] in, is a left singular vector matrix, and the column vectors are orthogonal; is a right singular vector matrix, and the column vectors are orthogonal; is a singular value matrix, satisfying ∑ k (i,j)=σ k,i ·δ i,j , where σ k,1 ≥σ k,2 ≥…≥0 are the singular values in descending order; T is the transpose operation; δ i,j is the Kroneckerdelta symbol, which is used to define the diagonal elements.

[0097] Singular value truncation and principal component extraction:

[0098] In order to extract the principal components that best represent the modal structure from the singular spectrum, only the first m s The singular values and corresponding singular vectors of order . The truncation rule is:

[0099] in, is the truncation order, that is, the number of singular values to be retained; rank(H k ) is the Hankel matrix H k The numerical rank of ; Indicates rounding down.

[0100] As an option, singular value truncation can also be implemented based on the energy accumulation threshold. Let the total energy be Then choose the smallest m s So that:

[0101] Among them, η∈(0,1) is the energy retention threshold, which is generally between 0.90 and 0.99. For example, setting η=0.95 can ensure that 95% of the signal energy is concentrated in the first m s order principal component.

[0102] In an implementable optimization strategy, the original modality can be further restored using the projection signal in the principal component direction to compress the dimension and suppress high-frequency noise:

[0103] in represents the time series reconstructed corresponding to the i-th order main singular vector, represents the reconstructed signal value of the kth modal component at time point t, σ k,i represents the Hankel matrix H k In this way, the main dynamic information is retained and invalid interference is filtered out.

[0104] In spectral signature analysis, the decay rate of singular values can also serve as a reference for evaluating modal component quality. Generally, a higher proportion of principal components indicates a high-quality mode; conversely, a lower proportion may contain redundant or noisy information.

[0105] S5. Optimizing the singular value components using a particle swarm optimization algorithm to determine an optimal reconstruction order;

[0106] In step S4, the Hankel matrix is constructed for each modal component and the first m are extracted by singular value decomposition (SVD). s After the order main singular components are obtained, in order to further improve the accuracy and robustness of modal reconstruction, the particle swarm optimization algorithm (PSO) is introduced in step S5. This step is based on the aforementioned singular value components and aims to determine the optimal singular value truncation order m. s , thus obtaining a reconstruction result that achieves a global optimal balance between signal-to-noise ratio and reconstruction error.

[0107] In this embodiment, the particle swarm optimization algorithm constructs a composite objective function to evaluate the modal reconstruction effect under different truncation orders. The optimization objective function is as follows:

[0108] F = α·SNR + β·MSE -1 ;

[0109] Among them, F is the fitness function value in particle swarm optimization, which is used to measure the signal reconstruction effect corresponding to the current particle (i.e., truncation order selection). The larger the value, the better the result.

[0110] SNR is the signal-to-noise ratio (SNR), which is used to evaluate the ratio of useful components to noise components in the reconstructed signal. The unit is decibel (dB) and is defined as:

[0111]

[0112] MSE is the mean square error (MSE), which represents the average error between the reconstructed signal and the original signal and is defined as:

[0113] α∈(0.4,0.8) is the signal-to-noise ratio weight coefficient, which is set according to the noise characteristics and is used to control the importance of the signal-to-noise ratio in the objective function;

[0114] β∈(0.2,0.6) is the weight parameter of the inverse of the mean square error, reflecting the contribution of reconstruction accuracy to the optimization objective;

[0115] The parameter constraints are: α + β = 1;

[0116] This constraint ensures that the overall weight of the objective function is balanced and normalized.

[0117] In one possible implementation, each particle in the PSO represents a candidate singular value cutoff order m s , its search space is:

[0118] The particle adjusts its position and velocity iteratively, gradually approaching the optimal truncation order that maximizes the objective function F. In each iteration, the system adjusts the m represented by the current particle s The singular value truncation and reconstruction are performed on the value, and the corresponding SNR and MSE are calculated and the fitness is updated accordingly.

[0119] Generally speaking, if the background noise component is strong or the spectrum pollution is serious, the β value can be increased to enhance the constraint on low error; when the main component of the signal is obvious and the structure is clear, the α value can be appropriately increased to prioritize the preservation of the signal structure integrity.

[0120] As an option, in order to improve search efficiency and stability, the particle swarm optimization algorithm can introduce inertia factor ω, individual cognitive factor c1, and group social factor c2 for update operations. The speed update formula is:

[0121] v i (t+1)=ωv i (t)+c1r1(p i -x i (t))+c2r2(gx i (t));

[0122] The location is updated to:

[0123] x i (t+1)=x i (t)+v i (t+1);

[0124] Among them, x i (t) is the current position of the i-th particle at the t-th iteration (truncation order); v i (t) is the velocity of the i-th particle; p i is the individual historical optimal solution of the particle; g is the global optimal solution of the group; r1, r2~U(0,1) are random factors; ω, c1, c2 are empirical parameters, usually with values of ω∈[0.5,0.9], c1=c2≈1.5.

[0125] In some embodiments, different modalities u k (t) Perform independent optimization processes to obtain the optimal truncation parameters for each modal component This method is more suitable for situations where the spectrum differences between modalities are significant, and can improve the overall reconstruction accuracy.

[0126] In the extended implementation, particle swarm optimization can also be combined with other meta-heuristic methods such as genetic algorithms and simulated annealing to improve the global search capability and prevent falling into local optimality.

[0127] In summary, step S5 intelligently controls the singular value truncation strategy by introducing the particle swarm optimization algorithm, and effectively achieves optimal control of the modal reconstruction quality with SNR and MSE as dual criteria, providing a solid optimization foundation for signal post-processing, denoising recognition and data compression.

[0128] S6. Reconstructing the modal components based on the optimal reconstruction order to obtain a one-dimensional spectral signal after noise reduction;

[0129] After completing the particle swarm optimization in step S5, the optimal reconstruction order k is obtained, a key parameter in the reconstruction process. Using this reconstruction order, we perform a weighted reconstruction of the modal components to obtain the denoised signal. The denoised signal is reconstructed based on the first k principal components, minimizing noise while preserving the signal's essential information. This process constructs the denoised spectral signal by accumulating the products of the singular values and the corresponding singular vectors.

[0130] In this embodiment, the signal after noise reduction This is achieved by weighted combination of the first k singular value components, as follows:

[0131] in, The goal is to remove the noise components from the original signal while preserving the main structure of the signal. The reconstructed signal is a one-dimensional time series, representing the signal amplitude at each time point t.

[0132] k is the optimal reconstruction order determined by the particle swarm optimization algorithm in step S5. Its value range is generally a positive integer not less than 3 and not more than 20. This value determines the number of principal components selected for signal reconstruction. The particle swarm optimization algorithm determines the most suitable value of k based on the optimal trade-off between signal-to-noise ratio and mean squared error, achieving the optimal balance between denoising and signal fidelity in signal reconstruction.

[0133] σ i is the i-th singular value, which comes from the singular value matrix ∑ in the SVD decomposition k , that is: ∑ k =diag(σ1,σ2,…,σ r ), where σ iThe size of reflects the energy proportion of the singular value in the modal component. Larger singular values usually correspond to the main components in the signal, while smaller singular values usually represent noise or redundant information.

[0134] u i is the i-th left singular vector, representing the main characteristics of the modal component in the time domain, which is obtained by SVD decomposition and describes the projection of the signal in the time dimension. Specifically, u i is the matrix u k The i-th column of , which captures the main dynamic information in the time series.

[0135] v i is the i-th right singular vector, which represents the projection of the modal component in the frequency domain or feature space. It is obtained by SVD decomposition and describes the distribution of the signal on the spectrum. Specifically, v i is the matrix v k The i-th column represents the main component of the signal in the frequency domain.

[0136] is the transpose of the i-th right singular vector, is a row vector. By transposing, and the left singular vector u i and singular values σ i Combined to obtain the reconstructed components of the signal.

[0137] The cumulative part of the formula Represents weighted reconstruction of the first k principal components, that is, using the first k largest singular values and their corresponding left and right singular vectors to reconstruct the signal. In this process, smaller singular values and their corresponding vectors are discarded because they correspond to noise or minor components.

[0138] During implementation, the reconstructed signal It is achieved by weighted summing of the first k principal components. The contribution of each principal component is determined by its corresponding singular value σ i Weighting ensures that the most important features of the reconstructed signal are preserved. By using smaller singular value components for weighting, the noise component in the signal can be effectively removed, thereby improving the clarity of the signal.

[0139] For example, if the original signal contains multiple noise sources or redundant information, reconstructing it using the first k principal components can maximize the preservation of the original signal's useful components while suppressing the noise and redundant components. Using different reconstruction orders k for different noise types allows for flexible signal optimization and denoising.

[0140] In some embodiments, to ensure the quality of signal reconstruction, the emphasis on the signal-to-noise ratio (SNR) and mean square error (MSE) during signal reconstruction can be adjusted using different weight coefficients (such as α and β). In the case of strong noise, appropriately increasing the k value can improve the quality of signal reconstruction by increasing the number of principal components.

[0141] Furthermore, reconstructed signals can be combined with other signal processing techniques. For example, in certain high-precision applications, combining filters, time series analysis, or spectrum analysis techniques can further optimize the quality of the reconstructed signal. Furthermore, if the signal characteristics during reconstruction are complex, adaptive signal processing algorithms can be considered to further enhance noise reduction.

[0142] In some embodiments, the reconstruction order k can be adjusted to meet different application requirements. For example, in scenarios with high ambient noise, a higher k value can be selected for better denoising; while in scenarios with good signal quality or less noise, k can be appropriately reduced to improve computational efficiency.

[0143] Through the reconstruction process in step S6, we can perform a weighted reconstruction of the modal components based on the optimal reconstruction order k, thereby obtaining a de-noised spectral signal. This process effectively removes noise and redundant information while preserving the signal's key characteristics. In applications, selecting the appropriate reconstruction order k based on the signal characteristics and noise conditions is key to improving reconstruction results. This reconstruction technique can obtain high-quality signals in various noise environments, facilitating subsequent signal analysis and processing tasks.

[0144] S7, extracting characteristic values of the target wavelength range from the one-dimensional spectral signal after noise reduction as model input, and using a neural network model to predict the type and concentration of metal ions.

[0145] In the metal ion detection process, the joint classification and regression prediction of spectral features through a neural network model is the core step in achieving intelligent analysis. In this embodiment, step S7 builds a multi-task prediction model based on a deep learning algorithm. By fusing the global contextual features of the spectral sequence with the prior knowledge of variational mode decomposition, it simultaneously outputs the ion type determination results and concentration quantification values. The architecture design and training strategy of this model fully consider the noise interference characteristics of spectral data and the requirements of multi-objective optimization. The specific implementation method includes the following technical features:

[0146] Typically, the neural network model employs a hybrid network based on the Transformer architecture, with the input being the feature vector of a denoised one-dimensional spectral signal within the target wavelength range. Alternatively, a learnable position encoding layer is implemented at the model input, embedding the wavelength sequence using a one-dimensional convolutional kernel. The kernel size is set to 7 wavelengths, with a step size of 3 and 64 output channels, thereby capturing the local wavelength correlation of the spectral signal.

[0147] Specifically, the main structure of the model consists of a cascade of multi-head self-attention modules, each of which calculates the correlation between the wavelength positions of the spectral sequence through a learnable weight matrix. In one possible implementation, given an input spectral sequence (L is the number of wavelength points, d is the feature dimension), the self-attention operation is defined as:

[0148]

[0149] Where Q = XW Q , K=XW K , V=XW V are query, key, and value matrices respectively, W Q , W K , is the trainable parameter matrix, d k is the scaling factor, usually d k = 64. The multi-head attention output is layer-normalized and then cross-modally fused with the singular value spectrum generated by variational mode decomposition.

[0150] In particular, the cross-modal fusion module decomposes the singular value spectrum of each modal component obtained by variational modal decomposition (m is the number of modalities, n is the singular value dimension) serves as prior knowledge and interacts with the original spectral features through a cross-attention mechanism. The key-value pairs of the cross-attention layer are derived from the linear projection of the singular value spectrum, while the query still comes from the spectral sequence, thereby achieving a dynamic weighted fusion of decomposed features and original features.

[0151] The model's output layer is designed as a parallel dual-branch structure, generating ion species classification probabilities and concentration regression values through fully connected layers. The classification branch uses a softmax activation function to output the probability distribution of each category, while the regression branch uses linear activation to output the concentration prediction value. To avoid gradient conflicts in multi-task learning, the loss function is defined as:

[0152]

[0153] in, is the cross entropy classification loss, is the mean square error regression loss, and λ1, λ2 are dynamically adjusted weight coefficients.

[0154] In terms of model training strategy, the adaptive optimization method driven by reinforcement learning is adopted. The state space s of the reinforcement learning agent t Contains the current number of training iterations t and the classification accuracy of the validation set Acc t , regression mean square error MSE t and gradient sparsity index Grad Sparsity . Action space a t It is defined as the joint adjustment of the learning rate η, batch normalization parameters γ, β and loss weights λ1, λ2. The reward function is designed as:

[0155]

[0156] Where ΔAcc=Acc t -Acc t-1 Indicates the accuracy increment, ΔAccuracy indicates the increment of classification accuracy between adjacent training time steps, MSE t is the mean square error of the current time step t, MSE t-1 is the mean square error of the previous time step, R t For the total reward, Grad_Diversity calculates the parameter gradient matrix The singular value entropy of is obtained:

[0157]

[0158] Where, σ i is the i-th singular value after the singular value decomposition of the gradient matrix, and k = min(p,q). The policy network is updated using the double-delayed deep deterministic policy gradient algorithm (TD3), synchronizing the target network parameters every 50 iterations.

[0159] To improve the model's ability to resist interference, synthetic noise patterns were injected during training. These included pulse noise with an amplitude less than 5% of the signal intensity, baseline drift with a slope less than 0.01 / wavelength, and spectral shifts to simulate interference from coexisting ions. The gradient reversal layer within the domain-adaptive training framework forced the model to learn noise-invariant feature representations.

[0160] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for visible spectrum noise reduction and heavy metal ion detection, characterized in that: The following steps are involved: S1, reacting the water sample to be tested with a preset color developer to generate a visual color system; S2. Capturing an image of the color development system and obtaining a corresponding grayscale spectrum signal by an optical imaging device; S3. performing variational modal decomposition on the grayscale spectrum signal to obtain multiple modal components; S4. Constructing Hankel matrices for each of the plurality of modal components and performing singular value decomposition to obtain singular value components of each mode; S5. Optimizing the singular value components using a particle swarm optimization algorithm to determine an optimal reconstruction order; S6. Reconstructing the modal components based on the optimal reconstruction order to obtain a one-dimensional spectral signal after noise reduction; S7, extracting characteristic values of the target wavelength range from the one-dimensional spectral signal after noise reduction as model input, and using a neural network model to predict the type and concentration of metal ions.

2. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The color developer in step S1 is a complex compound configured according to different metal ions. The color developer in step S1 is used to undergo a complex reaction with the target metal ion. The reaction is carried out at a temperature of 25°C-30°C, a pH value of 6.5-7.5, and a reaction time of 5-15 minutes. The color developer used to detect cadmium ions is a complex formed by connecting disulfide hydrazone and benzothiazole through a sulfide bond, and the complex reacts with cadmium ions to produce a purple color.

3. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The image acquisition process in step S2 includes: irradiating the reaction liquid with a narrow-band LED light source with a wavelength of 510-630 nanometers under dark box conditions, and using an industrial camera with a resolution of not less than 2000×600 pixels to capture the color image in the colorimetric tank, converting the obtained image into a grayscale spectral signal and normalizing it to between 0 and 1.

4. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The modal number K of the variational modal decomposition in step S3 is adaptively calculated according to the frequency domain bandwidth of the signal. The calculation formula of the modal number K is: Among them, f max With f min are the maximum and minimum values in the frequency domain, respectively, in Hertz; is the frequency resolution, which is a real value between 0.05 and 0.5 Hz; Δf is the frequency domain resolution.

5. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The Hankel matrix in step S4 is a two-dimensional structure in which the sum of the number of rows and the number of columns is less than the length of the original signal. The singular value decomposition operation in step S4 takes the matrix as input and extracts the first m-order singular values and the corresponding left and right singular vectors, where m does not exceed half of the rank of the original matrix.

6. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The objective function of the particle swarm optimization algorithm in step S5 is: F=α·SNR+β·MSE -1 ; Wherein, SNR is the signal-to-noise ratio, MSE is the mean square error, and weight parameters α and β satisfy α+β=1. α is a floating point number between 0.4 and 0.8, and β is a floating point number between 0.2 and 0.6, which are automatically set according to the noise type.

7. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The one-dimensional spectral signal after noise reduction in step S6 is a signal reconstructed based on the optimal reconstruction order k determined in step S5. The reconstructed signal is expressed as: Wherein, k is a positive integer not less than 3 and not more than 20; σ i 、u i 、v i are the singular values and the corresponding left and right singular vectors respectively; is the reconstructed signal; It represents the accumulation of the first k singular value components, that is, only the first k-order principal components are used for signal reconstruction; is the transpose of the i-th right singular vector, derived from V T Matrix, representing an orthogonal basis in column space.

8. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The characteristic value of the target wavelength range extracted in step S7 is the average grayscale value within the range of ±2 nm to ±10 nm of the color peak wavelength in the noise-reduced signal, and the peak wavelength is determined according to the maximum absorption wavelength of the color developer and the band is scanned in steps of 1 nm.

9. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: The neural network model in step S7 is a hybrid model based on the Transformer architecture, including: A cascaded multi-head self-attention module, where each attention head calculates the correlation between wavelength positions in the spectral sequence using a learnable weight matrix; A learnable position encoding layer that uses a one-dimensional convolution kernel to generate wavelength position feature embeddings; The cross-modal fusion module uses the singular value spectrum of each modal component obtained by variational modal decomposition as prior knowledge and inputs it into the cross-attention layer; The parallel output head generates ion species classification probability and concentration regression value through the fully connected layer respectively.

10. The method for visible spectrum noise reduction and heavy metal ion detection according to claim 1, characterized in that: In step S7, the neural network adopts an adaptive training strategy driven by reinforcement learning, which specifically includes: Define the actions of the reinforcement learning agent as learning rate adjustment, batch normalization parameter update, and loss function weight assignment; Constructing the state space includes the number of training iterations, the classification accuracy of the validation set, the regression mean square error, and the gradient sparsity index; The reward function is designed as: R t =0.5ΔAccuracy+0.3(1-MSE t / MSE t-1 )+0.2Grad_Diversity; Among them, Grad_Diversity is calculated by the singular value entropy of the parameter gradient matrix, ΔAccuracy represents the increment of classification accuracy between adjacent training time steps, and MSE t is the mean square error of the current time step t, MSE t-1 is the mean square error of the previous time step, R t is the total reward; The double-delayed deep deterministic policy gradient algorithm is used to update the network parameters, and the policy network and target network parameters are synchronized every 50 iterations.

Images