Absorption spectrum measurement method for simultaneously predicting concentrations of nitrate and nitrite in water body

Abstract

An absorption spectrum measurement method for simultaneously predicting concentrations of nitrate and nitrite in a water body comprises the following steps: firstly, collecting spectral data of single-component samples of nitrate and nitrite and a mixed sample of the single-component samples, and sequentially carrying out moving average filtering and normalization treatment; a BP neural network structure with two parallel channels is constructed, spectral features of nitrate and nitrite are extracted respectively, and respective concentration prediction is carried out; in the single-substance training stage, similar and heterogeneous sample pairs are constructed by using single-component samples, a differential learning mechanism is introduced, the distinguishing capability of the model on nitrate and nitrite characteristics is enhanced, and in the mixed substance fine tuning stage, part of a characteristic extraction layer is unfrozen, a regression layer is finely tuned, and interference information of mixed substances is learned; and inputting spectral data of a mixed solution of the nitrate and the nitrite into the trained model, and outputting concentration values of the nitrate and the nitrite. According to the method, under the condition that the spectrums of the two are seriously overlapped, simultaneous prediction can still be realized, and an accurate prediction result is obtained.

Description

Technical Field

[0001] This invention relates to an absorption spectroscopy-based measurement method that utilizes differential learning to simultaneously predict the concentrations of nitrates and nitrites in water. Background Technology

[0002] Nitrates and nitrites are common pollutants in water bodies, posing a serious threat to aquatic life and human health. Accurately predicting nitrate and nitrite concentrations has become a key research focus in water quality monitoring. Currently, the main methods for detecting nitrates and nitrites include colorimetry, spectrometry, chemiluminescence, chromatography, and electrochemical methods. Among these, spectrometry offers advantages such as high sensitivity, no secondary pollution, reagent-free operation, accuracy, and stability, making it a significant strength in pollutant detection.

[0003] Nitrates and nitrites exhibit strong absorption peaks in the 210-230 nm range, leading many researchers to explore concentration prediction methods based on absorption spectra. However, the significant overlap in their spectra makes simultaneous and accurate prediction of nitrate and nitrite concentrations a challenge. In practical applications, an effective method capable of simultaneously and accurately predicting the concentrations of these two substances is urgently needed. Therefore, developing a novel and efficient prediction method is of great importance.

[0004] In recent years, differential learning has been widely applied in fields such as image processing and computer vision. Its core idea is to learn differential features that can distinguish similar samples by constructing pairs of similar and dissimilar samples. Essentially, it requires samples of the same type to have higher similarity and dissimilar samples to have lower similarity, thereby improving the model's discriminative ability. Because the absorption spectra of nitrate and nitrite severely overlap, traditional methods struggle to extract highly discriminative features, resulting in poor discriminative ability and low prediction accuracy. Differential learning, however, can learn the essential differences between the spectra to uncover more discriminative features, thus enabling simultaneous prediction of nitrate and nitrite concentrations, improving prediction accuracy and generalizability. Therefore, introducing differential learning methods into absorption spectral analysis provides a new approach to solving the problem of simultaneously and accurately predicting nitrate and nitrite concentrations. Summary of the Invention

[0005] To overcome the shortcomings of traditional methods and to extract more distinguishable features of nitrate and nitrite, this invention provides an absorption spectral measurement method for simultaneously predicting the concentrations of nitrate and nitrite in water using differential learning. Even when the spectra of the two substances severely overlap, simultaneous prediction can still be achieved, and accurate prediction results can be obtained.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] An absorption spectroscopy method for simultaneously predicting the concentrations of nitrate and nitrite in water, the method comprising the following steps:

[0008] 1) Data acquisition and preprocessing, the process is as follows:

[0009] 1-1) First, use a spectrometer to measure the spectral data of nitrate, nitrite and their mixtures in water samples at different concentration levels, and divide them into training set and test set;

[0010] 1-2) Filter and normalize the raw spectral data;

[0011] 2) Construction of the dual-channel BP neural network structure: The BP neural network structure has two parallel channels to extract the spectral features of nitrate and nitrite respectively and predict their respective concentrations. Each channel includes a feature extraction model and a regression layer. The feature extraction model consists of an input layer, a fully connected layer, an activation function, and an output layer.

[0012] 3) Differentiated learning and training, the process is as follows:

[0013] 3-1) In the single-substance training stage, single-component samples are used to construct similar and dissimilar sample pairs, and a differential learning mechanism is introduced to enhance the model's ability to distinguish between nitrate and nitrite features.

[0014] 3-2) In the fine-tuning stage of mixed substances, some feature extraction layers are unfrozen and the regression layer is fine-tuned to learn the interference information of mixed substances;

[0015] 4) Concentration prediction: After the above training phase is completed, input the spectral data of the mixed solution of the two into the trained model, and output the concentration values ​​of nitrate and nitrite.

[0016] Furthermore, the process described in 3-1) is as follows:

[0017] 3-1-1) Similar sample pairs consist of absorbance data of the same substance at different concentrations, reflecting the consistency of the spectral characteristics of the substance under concentration changes; dissimilar sample pairs consist of absorbance data of nitrate and nitrite, reflecting the differences in spectral characteristics between the two substances.

[0018] 3-1-2) In order to measure the similarity and difference between features, cosine similarity is used as the measurement method, as shown in formula (1). For samples of the same class, it is desirable that the features are highly similar; for samples of different classes, it is desirable that their feature similarity is as low as possible.

[0019]

[0020] Where a and b represent the feature vectors of the sample pair, and S represents the cosine similarity between the two vectors;

[0021] 3-1-3) To enhance the network's learning of the differences between spectra, a similarity loss function was constructed to measure the similarity between sample pairs: samples of the same class should have higher similarity, while samples of different classes should have lower similarity, as shown in formula (2); the goal of this similarity loss function is to continuously enhance the network's ability to learn the differences between nitrate and nitrite through the training process, and to extract features with stronger discrimination ability. On this basis, in order to take into account both feature discrimination and concentration prediction performance, the similarity loss and regression loss are combined to construct the total loss function, as shown in formula (3);

[0022]

[0023] Among them, L sim (t) is the similarity loss, used to learn the ability to distinguish between nitrate and nitrite. t represents the number of iterations, n represents the number of sample pairs used in each iteration, α = 1 for similar sample pairs and α = 0 for dissimilar sample pairs, S represents the cosine similarity between the two vectors in the sample pair, and L... total (t) represents the total loss, L reg (t) is the regression loss, and λ and β are weight parameters that balance the contributions of each component during training. The regression loss function L reg (t), achieved through mean squared error (MSE) to minimize the difference between predicted and actual concentrations;

[0024] 3-1-4) The constructed total loss function is used as the objective function. The network parameters are updated by gradient through backpropagation algorithm. The network weights are updated iteratively by stochastic gradient descent (SGD) to gradually optimize the feature extraction and discrimination capabilities, as shown in formula (4). After training, the feature extraction model is frozen. After multiple rounds of training, the model can improve the discrimination capability and output discriminative features that can be used to predict the concentrations of nitrate and nitrite at the same time. The model can also accurately predict the concentrations of each substance through the regression layer.

[0025]

[0026] Where t represents the number of iterations, w t This represents the network parameters at the current moment, where η represents the learning rate. w represents the gradient of the total loss function with respect to the network parameters. t+1 This indicates the updated network parameters.

[0027] Furthermore, the process described in 3-2) is as follows:

[0028] 3-2-1) In the fine-tuning stage of mixed substances, the last few layers of the above feature extraction network are unfrozen and the learning rate is reduced. The model can learn the interference information in the mixed features. At the same time, the two regression layers are fine-tuned to adapt to the concentration prediction task in the mixed samples. In order to measure the difference between the predicted value and the true value, the mean squared error (MSE) is designed as the loss function, as shown in formula (5):

[0029]

[0030] Among them, y j Let x represent the true concentration value of the j-th sample. j L represents the concentration value predicted by the model, N represents the total number of samples, and L represents the concentration value predicted by the model. MSE (k) represents the loss function;

[0031] 3-2-2) Using the loss function as the objective function, the network parameters are updated using the gradient through the backpropagation algorithm, and the network weights are iteratively updated using stochastic gradient descent (SGD), as shown in formula (6):

[0032]

[0033] Where k represents the number of iterations, θ k This represents the network parameters at the current moment, and ε represents the learning rate. θ represents the gradient of the total loss function with respect to the network parameters. k+1 This indicates the updated network parameters.

[0034] Furthermore, in step 4), the concentration prediction is as shown in formulas (7) and (8):

[0035] C = v T h+r (7)

[0036] D = u T f+s (8)

[0037] Where C represents the predicted concentration of nitrate, h represents the nitrate feature vector extracted by the model based on the input spectral data, v and r represent the regression layer parameters of the nitrate channel, D represents the predicted concentration of nitrite, f represents the nitrite feature vector extracted by the model based on the input spectral data, and u and s represent the regression layer parameters of the nitrite channel.

[0038] Preferably, in step 2), the constructed dual-channel BP neural network consists of two parallel channels, each of which comprises a feature extraction model and a regression layer, with identical parameter settings. The specific parameter design of the feature extraction model is shown in Table 1:

[0039]

[0040] Table 1.

[0041] The technical concept of this invention is as follows: In the data construction stage, this invention proposes a strategy for constructing similar and dissimilar sample pairs based on concentration levels and substance categories. Similar sample pairs are constructed using samples of the same substance but different concentrations, while dissimilar sample pairs are constructed using samples of different substances, guiding the network to learn the consistency of similar substances and the differences between dissimilar substances. Regarding the network structure, a dual-channel BP neural network is employed to extract the spectral features of nitrate and nitrite separately, avoiding inter-channel interference. Cosine similarity is used to measure feature differences, thereby enhancing the discriminative ability in the feature space. Furthermore, this invention designs a total loss function that includes similarity loss and regression loss to optimize feature extraction and concentration prediction. In terms of the training strategy, a two-stage training approach is adopted: first, single-substance training is performed; then, mixed substances are used to fine-tune the feature extraction model and regression layer to adapt to the concentration prediction task in mixed samples.

[0042] The beneficial effects of this invention are as follows:

[0043] 1. This invention introduces a differential learning mechanism, which can effectively guide the model to learn the distinguishing features between nitrates and nitrites, and can significantly improve prediction accuracy even when the spectra of the two substances highly overlap.

[0044] 2. This invention employs a dual-channel network structure to extract features from nitrates and nitrites separately, effectively avoiding feature interference between different substances and thus enhancing the model's ability to represent a single substance.

[0045] 3. The two-stage training strategy enables the model to better learn the interference information in the mixed spectrum, further improving its adaptability in complex real-world scenarios. Attached Figure Description

[0046] Figure 1 This is a flowchart of the method of the present invention.

[0047] Figure 2 These are absorbance plots of nitrate and nitrite during the single-substance training phase. (a) is the nitrate dataset, and (b) is the nitrite dataset.

[0048] Figure 3 This is an absorbance graph of the mixture of nitrates and nitrites during the fine-tuning stage of the mixture.

[0049] Figure 4 These are the predicted results of the measured samples. (a) is the predicted result of nitrate, and (b) is the predicted result of nitrite. Detailed Implementation

[0050] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0051] Reference Figures 1-4 An absorption spectroscopy measurement method for simultaneously predicting nitrate and nitrite concentrations in water is proposed. This method utilizes differential learning to model the simultaneous prediction of nitrate and nitrite concentrations in water. First, it constructs similar and dissimilar sample pairs using spectral data of nitrate and nitrite, inputting them into their respective feature extraction models. The differential learning mechanism continuously learns discriminative features. Subsequently, a fine-tuning stage is performed to enable the model to learn interference information in the mixed features. Simultaneously, the two regression layers are fine-tuned to adapt to the concentration prediction task in the mixed samples. In the testing stage, measured mixed data are input, and the respective concentration values ​​are output.

[0052] The absorption spectroscopy measurement method for simultaneously predicting the concentrations of nitrate and nitrite in water includes the following steps:

[0053] 1) Data acquisition and preprocessing, the process is as follows:

[0054] 1-1) In implementation, firstly, a spectrometer was used to measure the spectral data of single nitrate and nitrite components and their mixtures in water samples at different concentration levels, and these samples were then divided into a training set and a test set. The training set contained 60 single-substance samples and 20 mixed samples, while the test set contained 20 mixed samples.

[0055] 1-2) To reduce the interference of instrument noise and differences between samples, the original spectral data were subjected to moving average filtering and normalization.

[0056] 2) Construction of the dual-channel BP neural network structure: The constructed dual-channel BP neural network structure has two parallel channels, each including a feature extraction model and a regression layer, with identical parameter settings. The feature extraction model consists of an input layer, a fully connected layer, an activation function, and an output layer, with parameter settings as shown in Table 1.

[0057]

[0058] Table 1

[0059] 3) Differentiated learning and training, the process is as follows:

[0060] 3-1) Single-substance training phase, the process is as follows:

[0061] 3-1-1) Construction of similar and dissimilar sample pairs: Similar sample pairs consist of absorbance data of the same substance at different concentrations, reflecting the consistency of the spectral characteristics of the substance under concentration changes; dissimilar sample pairs consist of absorbance data of nitrate and nitrite, reflecting the differences in spectral characteristics between the two substances.

[0062] 3-1-2) In order to measure the similarity and difference between features, cosine similarity is used as the measurement method, as shown in formula (1). For samples of the same class, it is desirable that the features are highly similar; for samples of different classes, it is desirable that their feature similarity is as low as possible.

[0063]

[0064] Where a and b represent the feature vectors of the sample pair, and S represents the cosine similarity between the two vectors;

[0065] 3-1-3) To enhance the network's learning of differences between spectra, a similarity loss function was constructed to measure the similarity between sample pairs: samples of the same class should have higher similarity, while samples of different classes should have lower similarity, as shown in formula (2). The goal of this similarity loss function is to continuously enhance the network's ability to learn the differences between nitrate and nitrite through the training process, and to extract features with stronger discriminative power. Based on this, in order to simultaneously consider feature discrimination and concentration prediction performance, the similarity loss and regression loss are combined to construct a total loss function, as shown in formula (3).

[0066]

[0067] Among them, L sim (t) is the similarity loss, used to learn the ability to distinguish between nitrate and nitrite. t represents the number of iterations, n represents the number of sample pairs used in each iteration, α = 1 for similar sample pairs and α = 0 for dissimilar sample pairs, S represents the cosine similarity between the two vectors in the sample pair, and L... total (t) represents the total loss, L reg (t) is the regression loss, and λ and β are weight parameters that balance the contributions of each component during training. The regression loss function L... reg (t), achieved through mean squared error (MSE), to minimize the difference between the predicted concentration and the actual concentration;

[0068] 3-1-4) The constructed total loss function is used as the objective function, the number of iterations is 1000, the learning rate is set to 0.002, the network parameters are updated by gradient through backpropagation algorithm, and the network weights are updated iteratively using stochastic gradient descent (SGD) to gradually optimize the feature extraction and discrimination capabilities, as shown in formula (4). After training, the feature extraction model is frozen. After multiple rounds of training, the model can improve the discrimination capability and output discriminative features that can be used to simultaneously predict the concentrations of nitrate and nitrite, and accurately predict their respective concentrations through the regression layer;

[0069]

[0070] Where t represents the number of iterations, w t This represents the network parameters at the current moment, where η represents the learning rate. w represents the gradient of the total loss function with respect to the network parameters. t+1 This indicates the updated network parameters;

[0071] 3-2) Fine-tuning stage of mixed substances, the process is as follows:

[0072] 3-2-1) In the fine-tuning stage of mixed substances, the last few layers of the above feature extraction model are unfrozen and the learning rate is reduced. The model can learn the interference information in the mixed features and at the same time, the two regression layers are fine-tuned to adapt to the concentration prediction task in mixed samples. In order to measure the difference between the predicted value and the true value, the mean squared error (MSE) is designed as the loss function, as shown in formula (5).

[0073]

[0074] Among them, y j Let x represent the true concentration value of the j-th sample. j L represents the concentration value predicted by the model, N represents the total number of samples, and L represents the concentration value predicted by the model. MSE (k) represents the loss function.

[0075] 3-2-2) The loss function is used as the objective function, the number of iterations is 1000, the learning rate is set to 0.001, the network parameters are updated by gradient through backpropagation algorithm, and the network weights are updated iteratively using stochastic gradient descent (SGD), as shown in formula (6).

[0076]

[0077] Where k represents the number of iterations, θ k This represents the network parameters at the current moment, and ε represents the learning rate. θ represents the gradient of the total loss function with respect to the network parameters. k+1 This indicates the updated network parameters.

[0078] 4) Concentration prediction, the process is as follows:

[0079] After the above training phase is completed, input the spectral data of the mixed solution of the two into the trained model, and output the concentration values ​​of nitrate and nitrite, as shown in formulas (7) and (8). The prediction results are referenced. Figure 4 (a) and Figure 4 (b) The results show that the prediction accuracy is very high, indicating that the method is effective.

[0080] C = v T h+r (9)

[0081] D = uT f+s (10)

[0082] Where C represents the predicted concentration of nitrate, h represents the nitrate feature vector extracted by the model based on the input spectral data, v and r represent the regression layer parameters of the nitrate channel, D represents the predicted concentration of nitrite, f represents the nitrite feature vector extracted by the model based on the input spectral data, and u and s represent the regression layer parameters of the nitrite channel.

[0083] In the data construction phase, this embodiment proposes a strategy for constructing similar and dissimilar sample pairs based on concentration levels and substance categories. Similar sample pairs are constructed using samples of the same substance but different concentrations, while dissimilar sample pairs are constructed using samples of different substances. This guides the network to learn the consistency of similar substances and the differences between dissimilar substances. Regarding the network structure, a dual-channel BP neural network is employed to extract the spectral features of nitrate and nitrite separately, avoiding inter-channel interference. Cosine similarity is used to measure feature differences, thereby enhancing the discriminative ability in the feature space. Furthermore, this embodiment designs a total loss function that includes similarity loss and regression loss to optimize feature extraction and concentration prediction. In terms of training strategy, a two-stage training approach is adopted: first, single-substance training is performed, and then the feature extraction model and regression layer are fine-tuned using mixed substances to adapt to the concentration prediction task in mixed samples.

[0084] The embodiments described in this specification are merely examples of implementations of the inventive concept and are not intended to limit the invention. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for simultaneously predicting the concentrations of nitrate and nitrite in water using absorption spectroscopy, characterized in that, The method includes the following steps: 1) Data acquisition and preprocessing, the process is as follows: 1-1) First, use a spectrometer to measure the spectral data of nitrate, nitrite and their mixtures in water samples at different concentration levels, and divide them into training set and test set; 1-2) Filter and normalize the raw spectral data; 2) Construction of the dual-channel BP neural network structure: The BP neural network structure has two parallel channels to extract the spectral features of nitrate and nitrite respectively and predict their respective concentrations. Each channel includes a feature extraction model and a regression layer. The feature extraction model consists of an input layer, a fully connected layer, an activation function, and an output layer. 3) Differentiated learning and training, the process is as follows: 3-1) In the single-substance training stage, single-component samples are used to construct similar and dissimilar sample pairs, and a differential learning mechanism is introduced to enhance the model's ability to distinguish between nitrate and nitrite features. 3-2) In the fine-tuning stage of mixed substances, some feature extraction layers are unfrozen and the regression layer is fine-tuned to learn the interference information of mixed substances; 4) Concentration prediction: After the above training phase is completed, input the spectral data of the mixed solution of the two into the trained model, and output the concentration values ​​of nitrate and nitrite.

2. The absorption spectral measurement method for simultaneously predicting the concentrations of nitrate and nitrite in water as described in claim 1, characterized in that, The process described in 3-1) is as follows: 3-1-1) Similar sample pairs consist of absorbance data of the same substance at different concentrations, reflecting the consistency of the spectral characteristics of the substance under concentration changes; dissimilar sample pairs consist of absorbance data of nitrate and nitrite, reflecting the differences in spectral characteristics between the two substances. 3-1-2) In order to measure the similarity and difference between features, cosine similarity is used as the measurement method, as shown in formula (1). For samples of the same type, it is desirable that the features are highly similar. For outlier samples, we want their feature similarity to be as low as possible; Where a and b represent the feature vectors of the sample pair, and S represents the cosine similarity between the two vectors; 3-1-3) To enhance the network's learning of the differences between spectra, a similarity loss function was constructed to measure the similarity between sample pairs: samples of the same class should have higher similarity, while samples of different classes should have lower similarity, as shown in formula (2); the goal of this similarity loss function is to continuously enhance the network's ability to learn the differences between nitrate and nitrite through the training process, and to extract features with stronger discrimination ability. On this basis, in order to take into account both feature discrimination and concentration prediction performance, the similarity loss and regression loss are combined to construct the total loss function, as shown in formula (3); L total (t)=λ·L sim (t)+β·L reg (t) (3) Among them, L sim (t) is the similarity loss, used to learn the ability to distinguish between nitrate and nitrite. t represents the number of iterations, n represents the number of sample pairs used in each iteration, α = 1 for similar sample pairs and α = 0 for dissimilar sample pairs, S represents the cosine similarity between the two vectors in the sample pair, and L... total (t) represents the total loss, L reg (t) is the regression loss, and λ and β are weight parameters that balance the contributions of each component during training. The regression loss function L reg (t), achieved through mean squared error (MSE) to minimize the difference between predicted and actual concentrations; 3-1-4) The constructed total loss function is used as the objective function. The network parameters are updated by gradient through backpropagation algorithm. The network weights are updated iteratively by stochastic gradient descent (SGD) to gradually optimize the feature extraction and discrimination capabilities, as shown in formula (4). After training, the feature extraction model is frozen. After multiple rounds of training, the model can improve the discrimination capability and output discriminative features that can be used to predict the concentrations of nitrate and nitrite at the same time. The model can also accurately predict the concentrations of each substance through the regression layer. Where t represents the number of iterations, w t This represents the network parameters at the current moment, where η represents the learning rate. w represents the gradient of the total loss function with respect to the network parameters. t+1 This indicates the updated network parameters.

3. The absorption spectral measurement method for simultaneously predicting the concentrations of nitrate and nitrite in water as described in claim 1 or 2, characterized in that, The process described in 3-2) is as follows: 3-2-1) In the fine-tuning stage of mixed substances, the last few layers of the above feature extraction network are unfrozen and the learning rate is reduced. The model can learn the interference information in the mixed features. At the same time, the two regression layers are fine-tuned to adapt to the concentration prediction task in the mixed samples. In order to measure the difference between the predicted value and the true value, the mean squared error (MSE) is designed as the loss function, as shown in formula (5): Among them, y j Let x represent the true concentration value of the j-th sample. j L represents the concentration value predicted by the model, N represents the total number of samples, and L represents the concentration value predicted by the model. MSE (k) represents the loss function; 3-2-2) Using the loss function as the objective function, the network parameters are updated using the gradient through the backpropagation algorithm, and the network weights are updated iteratively using stochastic gradient descent (SGD), as shown in formula (6): Where k represents the number of iterations, θ k This represents the network parameters at the current moment, and ε represents the learning rate. θ represents the gradient of the total loss function with respect to the network parameters. k+1 This indicates the updated network parameters.

4. The absorption spectral measurement method for simultaneously predicting the concentrations of nitrate and nitrite in water as described in claim 3, characterized in that, In step 4), the concentration prediction is as shown in formulas (7) and (8): C=v T h+r (7) D=u T f+s (8) Where C represents the predicted concentration of nitrate, h represents the nitrate feature vector extracted by the model based on the input spectral data, v and r represent the regression layer parameters of the nitrate channel, D represents the predicted concentration of nitrite, f represents the nitrite feature vector extracted by the model based on the input spectral data, and u and s represent the regression layer parameters of the nitrite channel.

5. The absorption spectral measurement method for simultaneously predicting the concentrations of nitrate and nitrite in water as described in claim 1 or 2, characterized in that, In step 2), the constructed dual-channel BP neural network consists of two parallel channels. Each channel is composed of a feature extraction model and a regression layer, and the parameter settings are exactly the same. The specific parameter design of the feature extraction model is shown in Table 1. Table 1.

Images