Water quality parameter prediction model modeling method and water quality parameter prediction method and device

Abstract

The present invention provides a water quality parameter prediction model modeling method and a water quality parameter prediction method and device. The water quality parameter prediction model modeling method includes: step 1: collecting water quality data; step 2: performing a standardization transformation; step 3: obtaining B training sample matrices and B label matrices; step 4: position encoding the b-th training sample matrix and label matrix respectively to obtain a training sample matrix and label matrix after the b-th position encoding; step 5: using the training sample matrices after each position encoding and the label matrices after each position encoding to train a multi-head attention model, and using the trained multi-head attention model as a water quality parameter prediction model.

Description

Technical Field

[0001] The present invention relates to the field of water quality parameter monitoring, and in particular to a water quality parameter prediction model building method and a water quality parameter prediction method and device. Background Art

[0002] Water is the foundation of human survival and development. Water resources play an irreplaceable role in human material life and various social production activities. Water quality parameter prediction provides a basis for water quality control and water resource management. Environmental management personnel can take preemptive action based on water quality parameter predictions, prepare for water pollution control measures, prevent further water pollution, plan water use strategies, and improve water resource utilization efficiency. Water quality prediction enables the transition from post-treatment to preventive treatment.

[0003] Water quality prediction has always been a hot topic of research in countries around the world. Existing water quality prediction algorithms mainly include time series analysis, regression analysis, recurrent neural networks, decision trees, support vector machines, etc.

[0004] With the advancement of science and technology, water quality parameter data has shown explosive growth. Traditional algorithms find it difficult to effectively mine the patterns in massive data and cannot fully extract the effective information collected by sensors, resulting in poor prediction accuracy. Moreover, as water quality parameter indicators become increasingly perfect, the previous prediction algorithms based on data collected by a single sensor are difficult to meet the accuracy requirements of water quality parameter prediction. Summary of the Invention

[0005] The problem to be solved by the present invention is that the existing water quality prediction methods have poor prediction accuracy, and a water quality parameter prediction model modeling method and a water quality parameter prediction method and device are provided.

[0006] To solve the above technical problems, the present invention adopts a technical solution: a water quality parameter prediction model building method, the water quality parameter prediction model building method comprising the following steps:

[0007] Step 1: Collect N sets of water quality data according to a fixed sampling period. Each set of water quality data consists of M values ​​corresponding to M types of water quality parameters.

[0008] Step 2: Perform standardization transformation on all collected data to obtain N groups of standardized water quality data:

[0009] Step 3: Based on the N groups of standardized water quality data obtained in step 2, obtain B training sample matrices in the order of collection time, and obtain B label matrices in the order of collection time, where each training sample matrix is ​​a matrix with C rows and M columns composed of water quality data from C consecutive sampling periods, and each label matrix is ​​a matrix with λC rows and M columns composed of water quality data from λC consecutive sampling periods. The bth training sample matrix T b The corresponding water quality data of C sampling periods and the bth label matrix L b The water quality data of the corresponding λC sampling periods constitute the water quality data of continuous C+λC sampling periods, and the bth label matrix L b The corresponding λC sampling periods are located in the b-th training sample matrix T b After the corresponding C sampling periods, λC is an integer, the proportional coefficient λ≤0.5, C is a preset constant value, and b=1, 2, ..., B;

[0010] Step 4: For the b-th training sample matrix T b Perform position encoding to obtain the training sample matrix after the b-th position encoding And for the b-th label matrix L b Perform position encoding to obtain the label matrix after the b-th position encoding

[0011] Step 5: Use the training sample matrix after encoding each position and the label matrix after encoding each position to train the multi-head attention model, and use the trained multi-head attention model as the water quality parameter prediction model.

[0012] According to the above technical solution, the present invention constructs a multi-head attention model as a water quality parameter prediction model. That is, when predicting a certain type of water quality parameter, the water quality parameter prediction model can not only utilize the time series information of the water quality parameters of that type, but also utilize the time series information of gas-type water quality parameters, thereby achieving high accuracy of the water quality parameter prediction model. Through this model, each training session utilizes the sampled data of C sampling cycles and the water quality data of λC sampling cycles immediately following the water quality data of C sampling cycles. As a result, the resulting water quality parameter prediction model can predict water quality data for λC consecutive sampling cycles, thereby improving prediction efficiency.

[0013] In the above technical solution, in step 1, if a certain data collected is missing, the spline interpolation algorithm is used to calculate the estimated value of the missing data, thereby completing the data.

[0014] In the above technical solution, in step 2, the normalization transformation is performed using the following formula:

[0015]

[0016] Among them, x i,j represents the value of the j-th water quality parameter collected in the i-th sampling period, the value range of i is [1, N], and the value range of j is [1, M]; μ j and σ j represent the mean and standard deviation of the values ​​of N j-th water quality parameters respectively; Indicates that x i,j The corresponding standardized value is This constitutes the i-th group of standardized water quality data.

[0017] In the above technical solution, in step 3, the b-th training sample matrix T b and the b+1th training sample matrix T b+1 There is a gap of △C sampling periods between them, and the value range of △C is △C ≥ 1. That is, there can be a gap between the starting sampling periods of two adjacent training sample matrices.

[0018] In the above technical solution, in step 4, the training sample matrix after the b-th position encoding is The label matrix after encoding the b-th position The expression is as follows:

[0019]

[0020] Among them: matrix PE T The element PE at row ma and column na in T The expression of (ma,na) ​​is as follows:

[0021]

[0022] Matrix PE L The element PE at row mb and column nb in L The expression for (mb,nb) is as follows:

[0023]

[0024] Among them, the value range of ma is [1, C], the value range of na is [1, M], the value range of mb is [1, λC], and the value range of nb is [1, M].

[0025] In the above technical solution, the multi-head attention model includes Number encoders, Number decoders, and a linear mapping layer connected to the output of the Number decoder, the output of the linear mapping layer is the output of the multi-head attention model, and Number is a preset constant value;

[0026] The output of the ua-th encoder is the input of the ua+1-th encoder, where ua = 1, 2, ..., Number - 1;

[0027] The output of the ubth decoder is the input of the ub+1th decoder, where ub=1,2,……,Number-1;

[0028] The output of the number-th encoder is used as two inputs of the multi-head attention mechanism module of each decoder;

[0029] In step 5, the steps of training the multi-head attention model specifically include:

[0030] Step 5-1: Encode the training sample matrix after the bth position As the input matrix of the first encoder of the multi-head attention model, and the b-th first input matrix is ​​used as the input matrix of the first decoder of the multi-head attention model to obtain the b-th output matrix of the multi-head attention model;

[0031] Where b = 1, 2, ..., B, the number of rows of each first input matrix is ​​λC+1, the number of columns is M, the values ​​of the M elements in the first row of each first input matrix are all Set, the La+1th row of the bth first input matrix and the label matrix after the bth position encoding The Lath row of is the same, La=1,2,3,……,λC, and Set is a preset constant value.

[0032] Step 5-2: Calculate the value of the loss function based on the B output matrices and each label matrix L of the multi-head attention model obtained in step 5-1;

[0033] If the loss function converges, the training of the multi-head attention model is completed;

[0034] If the loss function does not converge, the error back propagation algorithm is used to back propagate the gradient information of the loss function, and the Adam algorithm is used to update the values ​​of all trained parameters in the multi-head attention model, and then jump to step 5-1.

[0035] According to the above technical solution, the multi-head attention model can be trained using the training sample matrix and label matrix after encoding each position, thereby updating the values ​​of all training parameters in the multi-head attention model. When the loss function converges, it means that the trained multi-head attention model meets the requirements.

[0036] In the above technical solution, in step 5, when training the multi-head attention model, the expression of the loss function used is:

[0037]

[0038] Among them, f(X b,pos,j ) represents the b-th label matrix L b The value output by the water quality parameter prediction model corresponding to the element in the posth row and jth column, L b,pos,j Represents the b-th label matrix L b The value of the element at row pos and column j in j Represents the weight of the j-th water quality parameter, and the value range of j is [1,M].

[0039] In the above technical solution, the M-type water quality parameters include at least chemical oxygen demand and total phosphorus, and also include at least one of the comprehensive pollution index, turbidity, dissolved oxygen, ammonia nitrogen, pH value, conductivity, and temperature.

[0040] The present invention also provides a water quality parameter prediction method, which comprises:

[0041] According to the above-mentioned water quality parameter prediction model modeling method, a water quality parameter prediction model is obtained, and the water quality parameter prediction model is a multi-head attention model;

[0042] The water quality parameter prediction method further comprises:

[0043] Step 6-1: Using the first matrix and the second matrix as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model, respectively, and obtaining an output matrix of the water quality parameter prediction model according to the water quality parameter prediction model, wherein the first matrix is ​​a matrix with C rows and M columns composed of water quality data collected after C consecutive sampling periods and performing a standardized transformation, the values ​​of the M elements in the first row of the second matrix are all Set, and the elements in the other λC rows of the second matrix are all 0, and Set is a preset constant value;

[0044] Step 6-2: Using the elements of the first row of the output matrix of the water quality parameter prediction model obtained in step 6-1, the elements of the second row of the second matrix are updated accordingly, and the first matrix and the updated second matrix are used as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model respectively, and the output matrix of the water quality parameter prediction model is obtained according to the water quality parameter prediction model;

[0045] Step 6-3: Using the elements of the second row of the output matrix of the water quality parameter prediction model obtained in step 6-2, the elements of the third row of the second matrix are updated accordingly. The first matrix and the updated second matrix are used as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model, respectively. The output matrix result of the linear mapping layer is obtained according to the water quality parameter prediction model;

[0046] And so on, until all elements in the 2nd, 3rd, ..., λC+1th rows of the second matrix are updated;

[0047] Perform the inverse transformation of the normalization transformation on each element in the obtained updated second matrix, and use the values ​​of each element in the 2nd row, the values ​​of each element in the 3rd row, ..., and the values ​​of each element in the λC+1th row of the obtained matrix as the prediction results of each water quality parameter in the 1st sampling period, the 2nd sampling period, ..., and the λCth sampling period after the said C consecutive sampling periods in step 6-1, respectively.

[0048] According to the above technical solution, when predicting the water quality parameters of the first sampling period after C sampling periods, the data of the corresponding rows in the second matrix are all 0. When predicting the water quality parameters of the second sampling period after C sampling periods, that is, the water quality parameters of the first sampling period after C sampling periods have been predicted, then the corresponding first row of the second matrix is ​​updated by the prediction result, and so on, thereby predicting the prediction results of each water quality parameter of the first sampling period, the second sampling period, ..., the λCth sampling period after C consecutive sampling periods.

[0049] The present invention also provides a water quality parameter prediction device, comprising a processor and M types of sensors respectively used to collect data of M types of water quality parameters, wherein the processor is configured to execute the steps of the above-mentioned water quality parameter prediction method.

[0050] The advantages and positive effects of the present invention are: compared with a single-category water quality parameter prediction model, the present invention supports the prediction of multiple categories of water quality parameters. When predicting a certain category of water quality parameters, the model not only utilizes the time series information of the water quality parameters of this category, but more importantly, it also utilizes the time series information of other categories of water quality parameters, and theoretically can achieve higher prediction accuracy. The present invention can predict water quality parameters at multiple time steps, which makes up for the deficiency of traditional water quality parameter prediction models that can only predict water quality parameters at one time step. The present invention adopts a transformer architecture cascaded encoder and decoder structure, and integrates a multi-head attention mechanism to fully explore the temporal correlation of data. Compared with traditional prediction models based on machine learning, it has stronger data processing capabilities and can achieve efficient prediction of multiple categories of water quality parameters at multiple time steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1Schematic diagram of the overall process of the water quality parameter prediction method according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of some data of multiple types of water quality parameters collected at the Shima River measuring point according to an embodiment of the present invention;

[0054] Figure 3 Schematic diagram of a single sample-label matrix pair consisting of a single sample matrix and a single label matrix after completion and standardization according to an embodiment of the present invention;

[0055] Figure 4-1 This is a simplified structural diagram of a water quality parameter prediction model based on a transformer architecture according to an embodiment of the present invention, in which only one decoder and one encoder are shown, and "6×" represents six decoders and six encoders.

[0056] Figure 4-2 For display Figure 4-1 Schematic diagram of data flow between 6 encoders and 6 decoders;

[0057] Figures 5-1 to 5-9 Schematic diagrams showing comparisons between predicted values ​​and true values ​​of nine types of water quality parameters of the Shima River using the water quality parameter prediction method of the present application. DETAILED DESCRIPTION

[0058] The present application will be further described below with reference to the accompanying drawings and examples. It should be noted that the following detailed description is illustrative and intended to further illustrate the present application. Unless otherwise indicated, all technical and scientific terms used herein have the same meanings as commonly understood by those of ordinary skill in the art to which this application belongs.

[0059] like Figure 4-1 、 Figure 4-2 As shown, the present invention provides a water quality parameter prediction model modeling method, the water quality parameter prediction model modeling method comprises the following steps:

[0060] Step 1: Collect N sets of water quality data according to a fixed sampling period. Each set of water quality data consists of M values ​​corresponding to M types of water quality parameters.

[0061] Step 2: Perform standardization transformation on all collected data to obtain N groups of standardized water quality data.

[0062] Step 3: Based on the N groups of standardized water quality data obtained in step 2, obtain B training sample matrices in the order of collection time, and obtain B label matrices in the order of collection time, where each training sample matrix is ​​a matrix with C rows and M columns composed of water quality data from C consecutive sampling periods, and each label matrix is ​​a matrix with λC rows and M columns composed of water quality data from λC consecutive sampling periods. The bth training sample matrix T b The corresponding water quality data of C sampling periods and the bth label matrix L b The water quality data of the corresponding λC sampling periods constitute the water quality data of continuous C+λC sampling periods, and the bth label matrix L b The corresponding λC sampling periods are located in the b-th training sample matrix T b After the corresponding C sampling periods, λC is an integer, the proportional coefficient λ≤0.5, C is a preset constant value, and b=1, 2, ..., B.

[0063] Step 4: For the b-th training sample matrix T b Perform position encoding to obtain the training sample matrix after the b-th position encoding And for the b-th label matrix L b Perform position encoding to obtain the label matrix after the b-th position encoding

[0064] Step 5: Use the training sample matrix after encoding each position and the label matrix after encoding each position to train the multi-head attention model, and use the trained multi-head attention model as the water quality parameter prediction model.

[0065] In step 1, if some of the collected data is missing, the spline interpolation algorithm is used to calculate the estimated value of the missing data, thereby completing the data.

[0066] In step 2, the normalization transformation is performed using the following formula:

[0067]

[0068] Among them, x i,j represents the value of the j-th water quality parameter collected in the i-th sampling period, the value range of i is [1, N], and the value range of j is [1, M]; μ j and σ j represent the mean and standard deviation of the values ​​of N j-th water quality parameters respectively; Indicates that x i,j The corresponding normalized value is This constitutes the i-th group of standardized water quality data.

[0069] In step 3, the bth label matrix Lb , the b+1th training sample matrix T b There is 0 or △C sampling periods between them, and the value range of △C is △C≥2λC.

[0070] In step 4, the training sample matrix after the b-th position encoding is The label matrix after encoding the b-th position The expression is as follows:

[0071]

[0072] Among them: matrix PE T The element PE at row ma and column na in T The expression of (ma,na) ​​is as follows:

[0073]

[0074] Matrix PE L The element PE at row mb and column nb in L The expression for (mb,nb) is as follows:

[0075]

[0076] Among them, the value range of ma is [1, C], the value range of na is [1, M], the value range of mb is [1, λC], and the value range of nb is [1, M].

[0077] The multi-head attention model includes Number encoders, Number decoders, and a linear mapping layer connected to the output of the Number decoder. The output of the linear mapping layer is the output of the multi-head attention model. Number is a preset constant value. Wherein Number is preferably 6.

[0078] The output of the ua-th encoder (also called the ua-th layer) is the input of the ua+1-th encoder, where ua = 1, 2, ..., Number - 1; the output of the ub-th decoder (also called the ub-th layer) is the input of the ub+1-th decoder, where ub = 1, 2, ..., Number - 1;

[0079] The output of the number-th encoder serves as two inputs to the multi-head attention mechanism module of each decoder. The input Q and input K of each decoding layer come from the top layer of the encoder (that is, the output from the number-th encoder).

[0080] In step 5, the steps of training the multi-head attention model specifically include:

[0081] Step 5-1: Encode the training sample matrix after the bth position As the input matrix of the first encoder of the multi-head attention model, and the bth first input matrix is ​​used as the input matrix of the first decoder of the multi-head attention model to obtain the bth output matrix of the multi-head attention model.

[0082] Where b = 1, 2, ..., B, the number of rows of each first input matrix is ​​λC+1, the number of columns is M, the values ​​of the M elements in the first row of each first input matrix are all Set, the La+1th row of the bth first input matrix and the label matrix after the bth position encoding The Lath row of the same data is the same, La = 1, 2, 3, ..., λC, and Set is a preset constant value. When setting the value of Set, the values ​​of each water quality parameter are clearly distinguished. For example, Set can be set to -1000.

[0083] Step 5-2: Calculate the value of the loss function based on the B output matrices and each label matrix L of the multi-head attention model obtained in step 5-1;

[0084] If the loss function converges, the training of the multi-head attention model is completed;

[0085] If the loss function does not converge, the error back propagation algorithm is used to back propagate the gradient information of the loss function, and the Adam algorithm is used to update the values ​​of all trained parameters in the multi-head attention model, and then jump to step 5-1.

[0086] The Adam (Adaptive Momentum) algorithm is a stochastic optimization method for adaptive momentum. It uses the error backpropagation algorithm to backpropagate the gradient information of the loss function and uses the Adam algorithm to update the values ​​of all trained parameters in the multi-head attention model. This is an existing technology.

[0087] In step 5, when training the multi-head attention model, the loss function used is expressed as:

[0088]

[0089] Among them, f(X b,pos,j ) represents the b-th label matrix L b The value output by the water quality parameter prediction model corresponding to the element in the posth row and jth column of b,pos,j ) is the value of the element located at the posth row and jth column in the bth output matrix of the multi-head attention model obtained in step 5-1. L b,pos,j Represents the b-th label matrix L b The value of the element at row pos and column j in jRepresents the weight of the j-th water quality parameter, and the value range of j is [1,M].

[0090] Class M water quality parameters include at least chemical oxygen demand and total phosphorus, and at least one of the following: comprehensive pollution index, turbidity, dissolved oxygen, ammonia nitrogen, pH value, conductivity, and temperature;

[0091] like Figure 1 As shown, the present invention also provides a water quality parameter prediction method, which includes:

[0092] According to the above-mentioned water quality parameter prediction model modeling method, a water quality parameter prediction model is obtained, and the water quality parameter prediction model is a multi-head attention model;

[0093] The water quality parameter prediction method further comprises:

[0094] Step 6-1: Use the first matrix and the second matrix as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model, respectively, and obtain the output matrix of the water quality parameter prediction model according to the water quality parameter prediction model, wherein the first matrix is ​​a matrix with C rows and M columns composed of the collected water quality data after C consecutive sampling periods and performing normalization transformation, the second matrix has λC+1 rows and M columns, the values ​​of the M elements in the first row of the second matrix are all Set, the elements in the other λC rows of the second matrix are all 0, and Set is a preset constant value;

[0095] Step 6-2: Use the elements in the first row of the output matrix of the water quality parameter prediction model obtained in step 6-1 to update (i.e., replace) the elements in the second row of the second matrix. The elements in rows 3 to λC+1 of the second matrix remain 0. The first matrix and the updated second matrix are used as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model, respectively. The output matrix of the water quality parameter prediction model is obtained according to the water quality parameter prediction model.

[0096] Step 6-3: Use the elements in the second row of the output matrix of the water quality parameter prediction model obtained in step 6-2 to update (i.e., replace) the elements in the third row of the second matrix. The elements in rows 4 to λC+1 of the second matrix remain 0. The first matrix and the updated second matrix are used as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model, respectively. The output matrix result of the linear mapping layer is obtained according to the water quality parameter prediction model.

[0097] And so on, until all elements in the 2nd, 3rd, ..., λC+1th rows of the second matrix are updated;

[0098] Perform the inverse transformation of the normalization transformation on each element in the obtained updated second matrix, and use the values ​​of each element in the 2nd row, the values ​​of each element in the 3rd row, ..., and the values ​​of each element in the λC+1th row of the obtained matrix as the prediction results of each water quality parameter in the 1st sampling period, the 2nd sampling period, ..., and the λCth sampling period after the said C consecutive sampling periods in step 6-1, respectively.

[0099] The present invention also provides a water quality parameter prediction device, comprising a processor and M types of sensors respectively used to collect data of M types of water quality parameters, wherein the processor is configured to execute the steps of the above-mentioned water quality parameter prediction method.

[0100] The following is a further detailed description of the present application.

[0101] The water quality parameter prediction method disclosed in the present invention is a water quality parameter prediction method based on multi-sensor data fusion and transformer, which solves the problem that the existing water quality parameter prediction algorithm is difficult to process the massive data collected by multiple sensors and the water quality parameter prediction accuracy is not high. In this application, a multi-class water quality parameter data with a time span of more than 1 year is collected by a water quality parameter sensor, and then the missing data is supplemented and standardized by an interpolation algorithm, from which the sample-label matrix pairs required for model training are sequentially intercepted, and then all samples and label matrices are positionally encoded; the regression loss function of the water quality parameter prediction model and the weight of each type of water quality parameter are defined; the model hyperparameters (including the number of heads h of the multi-head attention mechanism, the dimension d of the model, etc.) are specified, and the prepared sample-label matrix pairs are used to train the multi-class water quality parameter prediction model based on the transformer architecture; finally, multiple types of water quality parameters are predicted, and the output results of the model are denormalized to obtain the prediction results. The invention can simultaneously predict multiple types of water quality parameters. When predicting a certain type of water quality parameter, the model not only uses the time series information of the water quality parameter of this type, but also uses the time series information of other types of water quality parameters, so as to obtain a higher prediction accuracy.

[0102] Specifically, the water quality parameter prediction method of the present application includes:

[0103] 1. Data collection of multiple water quality parameters. Install multiple water quality parameter sensors at the test point, set the sampling period T, and collect M types of water quality parameters such as pH value, dissolved oxygen, chemical oxygen demand, turbidity, conductivity, and ammonia nitrogen every T period; the sampling time span is Y, which is no less than 1 year. Theoretically, the dimension of the water quality parameter data that can be collected is N×M, where N is the number of data groups collected within the time span Y. T is the sampling period.

[0104] 2. Water quality parameter data completion and standardization. For groups with missing data due to sensor failure or other reasons within the sampling time span Y, the spline interpolation algorithm is used to calculate the estimated value of the missing data and complete the missing data. The mean and standard deviation of each type of water quality parameter are calculated, and all water quality parameter values ​​are standardized using the following formula:

[0105]

[0106] x i,j represents the value of the j-th water quality parameter measured at the i-th time, where i ranges from [1, N] and j ranges from [1, M]; μ j and σ j represent the mean and standard deviation of the jth type of water quality parameter; Represents the standardized water quality parameter value.

[0107] 3. Prepare the model training data set. The data set for model training and validation is intercepted from the standardized water quality parameter data. The data of C consecutive time steps are intercepted in sequence as a training sample matrix. Each training sample matrix is ​​a matrix of dimension C×M, represented by T. The data of the subsequent λC time steps of the training sample matrix T are intercepted as the label matrix of the current training sample matrix. λ represents the ratio of the length of the label matrix to the length of the training sample matrix. Usually λ≤0.5. Each label matrix is ​​a matrix of dimension λC×M, represented by L. Move ΔC time steps to intercept the next training sample matrix and label matrix. The total number of sample-label matrix pairs that can be obtained is:

[0108]

[0109] in represents the rounding down operation, and B represents the number of sample-label matrix pairs.

[0110] 4. Perform position encoding on all training sample matrices T and label matrices L. The prediction of water quality parameters requires the use of sample time step information. Position encoding adds time step information to the sample and label matrices. The position encoding formula is as follows:

[0111]

[0112] Where pos represents the time step of the sample, the value range of pos of sample T is [1, C], the value range of pos of label matrix L is [1, λC], j represents the j-th water quality parameter, and the value range of j is [1, M]. PE has the same dimension as sample T or label matrix L. The encoding process is as follows:

[0113]

[0114] Among them, PET and PE L Represent the position encoding of the training sample matrix T and the label matrix L, and Represent the sample and label matrices after encoding respectively.

[0115] 5. Define the model regression loss function. The model predicts the values ​​of multiple water quality parameters for the next λC time steps. The dimension of the predicted data is consistent with the dimension of the label matrix L, which is λC×M. Define the weighted root mean square error as the loss function:

[0116]

[0117] Where f(X pos,j ) represents the standardized value of the j-th water quality parameter at the pos-th time step output by the water quality parameter prediction model, L pos,j represents the standardized value of the j-th water quality parameter at the pos-th time step, B represents the input model of B training sample matrix-label matrix pairs, w j Represents the weight of the j-th water quality parameter. Due to the different measurement accuracy and randomness of different water quality parameters, the loss function weights the prediction results of different water quality parameters. Water quality parameters with high measurement accuracy and low randomness are easier to predict and are assigned small weights, while water quality parameters with low measurement accuracy or high randomness are assigned large weights.

[0118] In the loss function formula, L pos,j That is, the data of λC time steps as labels in training, f(X pos,j ) is the data obtained from the water quality parameter prediction model (i.e. Figure 4-1 The “prediction results” obtained in

[0119] 6. Train a multi-class water quality parameter prediction model based on the Transformer architecture. The model consists of six encoders and six decoders. Training the model primarily involves two processes: forward propagation of data and backpropagation of errors. In each decoding layer, the self-attention Q and K are derived from the top encoder layer (i.e., the last encoder layer).

[0120] 6-1. Multi-head attention mechanism operation. During forward propagation, after the training sample matrix is ​​position-encoded, a multi-head attention operation is performed, and the sample T is copied twice to obtain Q, K, and V, which are then added to the matrix W. Q 、W K 、W V Multiplication:

[0121]

[0122] Where W Q 、WK 、W V The dimensions are all M×d, where d represents the model dimension and its value range is [32,128].

[0123] The attention calculation process is as follows:

[0124]

[0125] in Representative Matrix The transpose of , let a i,j is a matrix The element at row i and column j. After performing softmax operation, Multiply.

[0126] The softmax operation process is:

[0127]

[0128] Softmax implements row normalization of matrix A.

[0129] Multi-head attention means multiple attention calculations on sample T, which can be expressed as:

[0130]

[0131] Where h represents the number of heads of the attention mechanism, which is usually less than 16. Concat means splicing along the row direction of the matrix. After splicing, it is combined with the matrix W. O Multiply them and map the output to C × M. The number of columns of the matrix obtained after splicing is equal to the sum of the number of columns of each spliced ​​matrix.

[0132] In the encoder, Q, K, and V are equal. In the multi-head attention mechanism module of the decoder, K and V come from the output of the encoder, and Q is calculated by the decoder.

[0133] In the masked attention mechanism module, the upper triangle values ​​of the A matrix are 10 -9 Fill, the value of the lower triangle part remains unchanged, that is:

[0134]

[0135] It will participate in subsequent operations as a new element of A.

[0136] 6-2. Add and normalize the layers. Add the output of the attention mechanism to the data that has not been through the attention mechanism, and then perform layer normalization:

[0137]

[0138] Where x represents the matrix obtained by addition, with dimension C×M, E[·] and Var[·] are mean and variance operations, respectively, and γ and β are two learnable parameters, initialized to 1 and 0, respectively.

[0139] 6-3. Forward Propagation and Linear Mapping Layer. The forward propagation and linear mapping layers work in the same way: they multiply the input by a matrix to produce the output. The dimensions of the forward propagation matrix and the decoder linear mapping matrix are the same in both the encoder and decoder.

[0140] 6-4. Calculate the loss and update the model’s trainable parameters. The trainable parameters include W of the attention mechanism module. Q 、W K 、W V 、W O The loss function is solved by formula (5), and then the gradient of the trainable parameters is calculated by the error back propagation algorithm, and the value of the trainable parameters is updated by the Adam algorithm.

[0141] 6-5. Iterate the two processes of data forward propagation and error back propagation multiple times until the model loss function converges.

[0142] 7. Prediction and denormalization of multiple water quality parameters. The M types of water quality parameter measurements for C consecutive time steps are completed and normalized and then input into the model. The water quality parameter values ​​for λC time steps are gradually predicted. The starting input of the decoder is the start flag. The model outputs the predicted value of the first time step, which is added to the decoder input to predict the water quality parameter value for the next time step. This process is repeated until the water quality parameters for the λC time step are predicted. Finally, the output result is denormalized using the mean and standard deviation of the water quality parameters in formula (1) to obtain the predicted result.

[0143] When using the model for prediction, for the first time step after C sampling periods, the values ​​of the first nine elements in the first row of the first decoder are all -1000, and the values ​​of the water quality parameters in the other 2-7 rows are filled with 0. After obtaining the model output, the row corresponding to the model output is filled into the second row of the first decoder input matrix (i.e., the second row is updated), and the water quality parameters for the second time step are predicted. When predicting the water quality parameters for the second time step, the 3-7 rows of the first decoder input matrix are still filled with 0. And so on.

[0144] It is of great significance to make full use of the water quality parameter information collected by multiple sensors, explore the patterns of water quality parameters and water quality levels changing over time from massive historical data, and study high-precision water quality prediction models that meet the input of multiple water quality parameters.

[0145] The following is an example of the water quality parameter prediction method of the present invention:

[0146] S1: Collect water quality parameter data of Shima River from January 2021 to July 2022 as training data for the model. The format of the collected raw data is as follows: Figure 2 As shown, the collected raw data includes ten water quality parameters: surface water grade, comprehensive pollution index, chemical oxygen demand, conductivity, dissolved oxygen, ammonia nitrogen, pH value, total phosphorus, turbidity, and temperature. Among them, the surface water grade can be calculated based on the remaining water quality parameters (i.e., the water quality parameters that are not predicted by the model in this application). The model predicts nine water quality parameters other than surface water grade. The above-mentioned water quality parameters are all commonly used in this field. Those skilled in the art understand how to collect and calculate the above-mentioned water quality parameters.

[0147] S2: The sensor collects data every 4 hours, but some data are missing at certain times. The missing data are supplemented by the spline interpolation algorithm, and a total of more than 83,160 sets of water quality parameter data are obtained. The mean and standard deviation of each type of water quality parameter are calculated, and all water quality parameter values ​​are standardized using the following formula:

[0148]

[0149] x i,j represents the value of the j-th water quality parameter measured at the i-th time, i ranges from [1,83160], j ranges from [1,9]; μ j and σ j represent the mean and standard deviation of the jth type of water quality parameter; Represents the standardized water quality parameter value.

[0150] S3: The model predicts the next 6 sets (one day) of water quality parameter values ​​using 18 consecutive sets (three days) of data. Sequentially intercept 18 consecutive time steps of data as a training sample matrix with a dimension of 18×9. The data of the next 6 time steps of the training sample matrix are intercepted as the label matrix of the current training sample matrix. Move 12 time steps to intercept the next training sample matrix and label matrix. The total number of sample-label matrix pairs that can be obtained is:

[0151]

[0152] in The number of sampling periods between two sample-label pairs can be set as needed, for example, it can be set to any value between 1 and 12.

[0153] The values ​​of the first sample-label matrix pair are as follows Figure 3As shown, the column data of the sample and label matrices represent time steps, and the row data represents the type of water quality parameter. All data are normalized. The label matrix input to the decoder needs to add a start flag and set the start flag value to -1000 to clearly distinguish the label matrix values.

[0154] S4: Position encoding is performed on the sample and label matrices respectively. The prediction of water quality parameters requires the use of sample time step information. The position encoding module adds the time step information to the sample and label matrices. The position encoding formula is as follows:

[0155]

[0156] Where pos represents the time step of the sample. For the sample, the value range of pos is [1,18]. For the label matrix, the value range of pos is [1,6]. j represents the j-th water quality parameter, and the value range of j is [1,9]. PE has the same dimension as the sample or label matrix, and the data of their corresponding positions are added to achieve position encoding of the sample and label matrix.

[0157] S5: Define the model regression loss function. The model predicts multiple water quality parameters for the next 6 time steps. The dimension of the predicted data is the same as the label matrix dimension, which is 6×9. Define the weighted root mean square error as the loss function:

[0158]

[0159] Where f(X pos,j ) represents the standardized value of the j-th water quality parameter at the pos-th time step output by the water quality parameter prediction model, L pos,j represents the standardized value of the j-th water quality parameter at the pos-th time step, B represents the input model of B training sample matrix-label matrix pairs, w j Represents the weight of the jth water quality parameter. Due to the varying measurement accuracy and randomness of different water quality parameters, the loss function weights the predictions for each type of water quality parameter. Water quality parameters with high measurement accuracy and low randomness are more easily predicted and are assigned smaller weights, while water quality parameters with low measurement accuracy or high randomness are assigned larger weights. Turbidity and conductivity have a weight of 10, while the remaining water quality parameters have a weight of 1.

[0160] S6: Train a multi-class water quality parameter prediction model based on the transformer architecture. The Transformer model architecture is as follows Figure 4-1 、 Figure 4-2The model consists primarily of six encoder models stacked with six decoder modules. Each encoder and decoder module incorporates multiple attention mechanisms to correlate information across different time steps of the sample. The number of attention heads is set to two. The decoder inputs are the encoder output and a label matrix with a start marker. The masked multi-head attention mechanism masks all time steps after the current prediction time step, preventing the model from referencing future data when predicting the current value. After processing the data, the decoder uses a linear mapping layer to reshape the data into a 6×9 dimension, corresponding to the values ​​of the nine water quality parameters across the six time steps. The loss function is calculated based on the model output and the corresponding label matrix information. The gradients of the trainable parameters are then calculated using the error backpropagation algorithm, and the values ​​of the trainable parameters are updated using the Adam algorithm. The model converges after 10 rounds of iterative training.

[0161] During model training, all sample-label pairs are required. All training samples are divided into N batches. After all N batches are fed into the network, one iteration is completed. The loss function for this batch of samples is calculated, and the error is backpropagated. The next iteration feeds N batches of data back into the network.

[0162] For example, suppose there are 6,000 sample-label pairs. Input the first sample-label pair, i.e., the data from the first 18 time steps as training samples, and the data from the last 6 time steps as labels (the label matrix is ​​filled with the data from the collected 6 time steps, with no zero values ​​assigned). Calculate the first loss function value, backpropagate the error loss function gradient using the error backpropagation algorithm, and use the Adam algorithm to update the values ​​of all trainable model parameters. Substitute the second input sample-label pair and repeat the above steps. After substituting all 6,000 sample-label pairs, complete one iteration, and then start again from the first sample label for the second iteration.

[0163] During training, the decoder input label matrix is ​​a 7 × 9 matrix, and the model output is a 6 × 9 matrix.

[0164] The decoder output and the model output are always 6×9 matrices. The decoder input is always a 7×9 matrix. When using the model for prediction, once the model obtains a result, the row corresponding to the model output is added to the row corresponding to the decoder input (that is, one row of the label matrix is ​​updated each time), gradually obtaining the prediction results for six time steps.

[0165] S7: Prediction and Denormalization of Multiple Water Quality Parameters. A total of 18 data sets from the Shima River between August 5 and August 7, 2022, were used as test samples for the model. Greedy encoding was used to gradually predict the water quality parameter values ​​for six time steps. The decoder's initial input was a start flag. The model outputted the prediction for the first time step, which was then fed into the decoder's input to predict the water quality parameter value for the next time step. This process was repeated until the water quality parameters for the sixth time step were predicted.

[0166] The model predicts the water quality parameter values ​​for the whole day on August 8. In order to compare with the actual values, the prediction results are denormalized. The actual values ​​and the model prediction values ​​are as follows: Figures 5-1 to 5-9 As shown in the figure, the model has a good effect in predicting the trends of the nine types of water quality parameters.

[0167] In addition, the trained water quality parameter data can also be used to predict water quality parameters at different measuring points, which will not be elaborated here.

[0168] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0169] The embodiments of the present invention are described in detail above, but the contents are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of this patent. After reading this invention, modifications of various equivalent forms of the present invention by those skilled in the art fall within the scope defined by the claims attached to this application. In the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

Claims

1. A water quality parameter prediction modeling method, characterized by: The water quality parameter prediction model modeling method comprises the following steps: Step 1: Collect data according to a fixed sampling period N Group of water quality data, each group of water quality data is M The water quality parameters correspond to M The value composition; Step 2: Perform standardization transformation on all collected data to obtain N Water quality data after group standardization: Step 3: Based on the result from step 2 N The water quality data after group standardization are obtained in the order of collection time. B training sample matrices, and obtained in the order of collection time B label matrix, where each training sample matrix is ​​composed of continuous C The number of rows of water quality data for a sampling period is C , the number of columns is M Each label matrix is ​​composed of continuous λC The number of rows of water quality data for a sampling period is λC , the number of columns is M The matrix of b training sample matrix T b Corresponding C Water quality data of the sampling period, b Label matrix L b Corresponding λC The water quality data of each sampling period constitutes a continuous C + λC The water quality data of the sampling period, and the b Label matrix L b Corresponding λC The sampling period is located at b training sample matrix T b Corresponding C After a sampling period, λC is an integer, the proportional coefficient λ ≤0.5 ，C is a preset constant value, b =1,2,……, B ; Step 4: b training sample matrix T b Perform position encoding to obtain b The training sample matrix after position encoding , and for the b Label matrix L b Perform position encoding to obtain b The label matrix after position encoding ; Step 5: Use the training sample matrix encoded at each position 、 The label matrix after encoding each position is used to train the multi-head attention model, and the trained multi-head attention model is used as the water quality parameter prediction model; the multi-head attention model is a model based on the transformer architecture; Wherein: In step 5, when training the multi-head attention model, the expression of the loss function used is: ； in, f ( X b,pos,j ) indicates that b Label matrix L b No. POS Rank j The values ​​output by the water quality parameter prediction model corresponding to the column elements, L b,pos,j Indicates the b Label matrix L b Located in the middle POS Row, No. j The value of the column element, w j Representative j The weights of water quality parameters, j The value range is [1, M ]; M Water quality parameters include chemical oxygen demand, total phosphorus, comprehensive pollution index, turbidity, dissolved oxygen, ammonia nitrogen, pH value, conductivity, and temperature; Water quality parameters with high measurement accuracy and low randomness are assigned small weights, while water quality parameters with low measurement accuracy or high randomness are assigned large weights.

2. The water quality parameter prediction modeling method according to claim 1, wherein: In step 1, if some of the collected data is missing, the spline interpolation algorithm is used to calculate the estimated value of the missing data, thereby completing the data.

3. The water quality parameter prediction modeling method according to claim 1, wherein: In step 2, the standardization transformation is performed using the following formula: ； in, x i,j Indicates the i The first sampling period j The values ​​of water quality parameters, i The value range is [1, N ], j The value range is [1, M ]; μ j and σ j Respectively N No. j Mean and standard deviation of the values ​​of class water quality parameters; Represents x i,j The corresponding standardized value is Constitute the i The water quality data after standardization.

4. The water quality parameter prediction modeling method according to claim 1, wherein: In the step 3, b training sample matrix T b With the b +1 training sample matrix T b+1 Interval △ C Sampling period, △ C The value range is △ C ≥1.

5. The water quality parameter prediction modeling method according to claim 1, wherein: In the step 4, b The training sample matrix after position encoding 、 No. b The label matrix after position encoding The expression is as follows: ； Among them: matrix PE T Located in the middle ma Row, No. na Elements of a column PE T ( ma , na ) is as follows: matrix PE L Located in the middle mb Row, No. nb Elements of a column PE L ( mb , nb ) is as follows: in, ma The value range is [1, C ], na The value range is [1, M ], mb The value range is [1, λC ], nb The value range is [1, M ].

6. The water quality parameter prediction modeling method according to claim 1, wherein: The multi-head attention model includes Number encoders, Number decoders, and a linear mapping layer connected to the output of the Number decoder, where the output of the linear mapping layer is the output of the multi-head attention model, and Number is a preset constant value; The output of the ua-th encoder is the input of the ua+1-th encoder, where ua=1,2,…,Number-1; The output of the ubth decoder is the input of the ub+1th decoder, where ub=1,2,…,Number-1; The output of the number-th encoder is used as two inputs of the multi-head attention mechanism module of each decoder; In step 5, the steps of training the multi-head attention model specifically include: Step 5-1: b The training sample matrix after position encoding As the input matrix of the first encoder of the multi-head attention model, and the first b The first input matrix is ​​used as the input matrix of the first decoder of the multi-head attention model, and the first decoder of the multi-head attention model is obtained. b output matrix; in, b =1,2,……,B, the number of rows of each first input matrix is λC +1, the number of columns is M , the first row of each first input matrix M The value of each element is Set, b The La+1th row of the first input matrix and the b The label matrix after position encoding The same as the Lath row, La = 1, 2, 3, ..., λC , Set is the preset constant value; Step 5-2: The multi-head attention model obtained according to step 5-1 B output matrices, each label matrix L Calculate the value of the loss function; If the loss function converges, the training of the multi-head attention model is completed; If the loss function does not converge, the error back propagation algorithm is used to back propagate the gradient information of the loss function, and the Adam algorithm is used to update the values ​​of all trained parameters in the multi-head attention model, and then jump to step 5-1.

7. A method for predicting water quality parameters, characterized in that: The water quality parameter prediction method comprises: A water quality parameter prediction model according to the water quality parameter prediction model modeling method according to any one of claims 1 to 6, wherein the water quality parameter prediction model is a multi-head attention model; The water quality parameter prediction method further comprises: Step 6-1: The first matrix and the second matrix are used as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model respectively, and the output matrix of the water quality parameter prediction model is obtained according to the water quality parameter prediction model, wherein the first matrix is ​​composed of continuous C The number of rows of the collected water quality data after the standardization transformation is C , the number of columns is M The number of rows of the second matrix is λC +1, the number of columns is M , the first row of the second matrix M The values ​​of the elements are all Set, and the rest of the second matrix λC The elements of the row are all 0, and Set is a preset constant value; Step 6-2: Using the elements of the first row of the output matrix of the water quality parameter prediction model obtained in step 6-1, the elements of the second row of the second matrix are updated accordingly, and the first matrix and the updated second matrix are used as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model respectively, and the output matrix of the water quality parameter prediction model is obtained according to the water quality parameter prediction model; Step 6-3: Using the elements of the second row of the output matrix of the water quality parameter prediction model obtained in step 6-2, the elements of the third row of the second matrix are updated accordingly. The first matrix and the updated second matrix are used as the input of the first encoder and the input of the first decoder of the water quality parameter prediction model, respectively. The output matrix result of the linear mapping layer is obtained according to the water quality parameter prediction model; And so on, until the second row, the third row, ..., the λC All elements of the +1 row are updated; Perform the inverse transformation of the normalization transformation on each element in the obtained updated second matrix, and replace the values ​​of each element in the second row, the third row, ..., the first row, and the second ... λC The values ​​of each element in the +1 row correspond to the continuous C After the first sampling period, the second sampling period, ..., the λC The prediction results of each water quality parameter in each sampling period.

8. A water quality parameter prediction device, characterized in that: Including processor, respectively for acquisition M Water quality parameter data M A sensor, the processor is configured to execute the steps of the water quality parameter prediction method according to claim 7.

Images