A method for repairing and predicting dissolved oxygen concentration data

Abstract

The present invention discloses a method for repairing and predicting dissolved oxygen concentration data, comprising collecting water quality data and establishing an original data set; identifying missing values ​​and performing outlier analysis on the water quality data using a box plot to remove outliers; calculating and repairing the identified missing values ​​and removed outliers using a Lagrange interpolation method; analyzing the correlation between dissolved oxygen and other water quality parameters using an improved grey correlation analysis method; optimizing the parameters of an LSTM neural network using a sparrow search algorithm to establish an improved LSTM model; and predicting dissolved oxygen data using the improved LSTM model. The present invention utilizes the sparrow search algorithm to establish an improved LSTM model. By optimizing missing values ​​and outliers in the original data set, the improved LSTM model accurately predicts changes in dissolved oxygen concentration, providing a high-precision water quality prediction technology for estuaries with complex hydrodynamics.

Description

Technical Field

[0001] The present invention relates to the field of environmental science and technology, and in particular to a method for repairing and predicting dissolved oxygen concentration data. Background Art

[0002] With economic development, human activities such as irrational water resource utilization and excessive deforestation have led to increasingly severe water pollution and water scarcity. Urban development, driven by high water consumption for both industrial and domestic purposes, often draws water from nearby rivers. River water is affected by urban development, rainwater runoff, and even tidal currents, leading to significant fluctuations in water quality. To ensure drinking water safety, water quality monitoring stations are often installed in river sections where water quality fluctuates. These monitoring stations use data to predict future water quality changes and control water withdrawal times. Among the many water quality indicators, dissolved oxygen is a key one. When oxygen concentration in water is insufficient, sulfate and nitrate ions in the water are reduced to sulfite and nitrite ions. These substances combine with hydrogen ions to form substances that are harmful to the human body, causing damage to human functions. However, due to external conditions and instrumentation, data gaps and anomalies often occur, reducing forecast accuracy and thus compromising urban water safety.

[0003] Neural networks are currently a commonly used water quality forecasting technology, but the existing neural network models need further improvement in water quality data repair and forecasting accuracy.

[0004] To this end, the present invention provides a dissolved oxygen concentration data repair and forecasting method to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of existing neural network simulation technology and propose a dissolved oxygen concentration data repair and prediction method. On the basis of optimizing the missing values ​​and outliers in the original data set, it accurately predicts the changes in dissolved oxygen concentration and provides a high-precision water quality prediction technology for estuary areas with complex hydrodynamics.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for repairing and predicting dissolved oxygen concentration data, comprising the following steps:

[0007] Step 1: Collect water quality data and establish the original data set;

[0008] Step 2: Identify missing values ​​and use box plots to perform outlier analysis on water quality data and remove outliers;

[0009] Step 3: Use Lagrange interpolation method to calculate the missing values ​​and outliers;

[0010] Step 4: Analyze the correlation between dissolved oxygen and other water quality parameters using the improved grey correlation analysis method;

[0011] Step 5: Use the sparrow search algorithm (SSA) to optimize the five parameters of the LSTM neural network: learning rate, batch size, number of training times, number of LSTM hidden layer nodes, and number of fully connected hidden layer nodes, to establish an improved LSTM model.

[0012] Step 6: Use the improved LSTM model based on the sparrow search algorithm (SSA) to predict the dissolved oxygen data.

[0013] Preferably, the operation method of step 1 is to select a monitoring station, obtain a continuously recorded water quality data sequence, number the monitoring data, and establish an original data set D1.

[0014] Preferably, the operation method of step 2 is to identify missing values ​​in the data sequence, and find outliers of water quality parameters in the data set through data sequence box plot analysis, and eliminate outliers to establish an adjusted data set D2.

[0015] Preferably, the specific process of calculating missing values ​​and outliers using the Lagrange interpolation method in step 3 includes:

[0016] Process 3-1: Count the number of data in data set D2 as n, and establish an n-1 degree polynomial passing through n points:

[0017] y=a0+a1x+a2x 2 +L+a n-1 x n-1 (1)

[0018] Process 3-2: Set the coordinates of n points (x1, y1)(x2, y2)…(x n ,y n ) into the polynomial function, we get the following equation:

[0019]

[0020]

[0021] Process 3-3: Solve the Lagrange interpolation polynomial by solving the simultaneous equations:

[0022]

[0023] Process 3-4: Substitute the point x corresponding to the missing function value into the interpolation polynomial to obtain the approximate value L(x) of the missing value and obtain the patched data set D3.

[0024] Preferably, the process of the improved grey relational analysis method in step 4 is as follows:

[0025] Process 4-1: First, you need to determine the reference sequence and the comparison sequence:

[0026]

[0027] Process 4-2: Initialization of X i (k):

[0028]

[0029] Process 4-3: Constructing the difference matrix between the reference sequence and the comparison sequence:

[0030]

[0031] Process 4-4: Introduce and form the shape similarity correlation coefficient:

[0032]

[0033] Process 4-5: Construct the quotient matrix of reference sequence and comparison sequence:

[0034]

[0035] Process 4-6: Introduced to form the distance similarity correlation coefficient:

[0036]

[0037] Process 4-7: Calculate the comprehensive correlation:

[0038]

[0039] Preferably, the specific operation process of step 5 is:

[0040] Process 5-1: 70% of the data in the dataset D2 is used as training data to form a training set;

[0041] Process 5-2: Set the hyperparameters of the sparrow search algorithm;

[0042] Process 5-3: Population initialization, using the root mean square error as the fitness function;

[0043] Process 5-4: Input the population parameters into the LSTM neural network, calculate the individual and group fitness, and continuously update the sparrow operator;

[0044] Process 5-5: Determine whether the training is completed based on the termination conditions;

[0045] Process 5-6: If training is complete, output the optimal LSTM hyperparameters; otherwise, return to process 5-4 to continue executing the rules.

[0046] Preferably, the termination condition is that the root mean square error RMSE no longer changes or the error is less than a set numerical range, wherein the sparrow operator refers to the individual optimal value, the population optimal value, the weight factor and the fitness value.

[0047] Preferably, the root mean square error RMSE is calculated as follows, where Y i represents the true value, Y j Represents the predicted value, and n represents the amount of data:

[0048]

[0049] Beneficial effects

[0050] The present invention provides a method for repairing and predicting dissolved oxygen concentration data. Compared with the prior art, it has the following advantages:

[0051] The present invention uses the sparrow search algorithm to establish an improved LSTM model. On the basis of optimizing the missing values ​​and outliers in the original data set, it accurately predicts the changes in dissolved oxygen concentration and provides a high-precision water quality prediction technology for estuaries with complex hydrodynamics. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flow chart of a dissolved oxygen concentration data repair and forecasting method provided by the present invention;

[0053] Figure 2 The original data set D1 of the dissolved oxygen concentration data repair and prediction method provided by the present invention;

[0054] Figure 3 The Lagrangian repair data set D3 of the dissolved oxygen concentration data repair and prediction method provided by the present invention;

[0055] Figure 4 An SSA-based LSTM model improvement process for a dissolved oxygen concentration data repair and forecasting method;

[0056] Figure 5 An improved LSTM dissolved oxygen prediction value for a dissolved oxygen concentration data repair and prediction method. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below using Qiantang River as an example, in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the present invention.

[0058] like Figure 1 As shown, a method for repairing and predicting dissolved oxygen concentration data includes the following steps:

[0059] Step 1 (S1): Collect water quality data and establish an original data set;

[0060] Step 2 (S2): Identify missing values ​​and perform outlier analysis on water quality data using box plots to remove outliers;

[0061] Step 3 (S3): using Lagrange interpolation method to calculate the missing values ​​identified and the outliers removed;

[0062] Step 4 (S4): Analyzing the correlation between dissolved oxygen and other water quality parameters using an improved grey correlation analysis method;

[0063] Step 5 (S5): Using the sparrow search algorithm SSA to optimize the five parameters of the LSTM neural network, namely, the learning rate, batch size, number of training times, number of LSTM hidden layer nodes, and number of fully connected hidden layer nodes, to establish an improved LSTM model;

[0064] Step 6 (S6): Predict the dissolved oxygen data using an improved LSTM model based on the Sparrow Search Algorithm (SSA). This method uses the Sparrow Search Algorithm to establish an improved LSTM model. This model accurately predicts changes in dissolved oxygen concentration by optimizing missing values ​​and outliers in the original dataset, providing a high-precision water quality prediction technology for estuaries with complex hydrodynamics.

[0065] like Figure 2 As shown, in step 1 (S1), the Qiantang River sluice monitoring station is selected to obtain a continuously recorded dissolved oxygen data sequence (the monitoring period is 5200 hours and the data time interval is 4 hours), number the monitoring data, and establish an original data set D1.

[0066] In step 2 (S2), missing values ​​in the data series are identified, and outliers of water quality parameters in the data set are found using box plots. The outliers are then removed to establish an adjusted data set D2.

[0067] The specific process of calculating missing values ​​and outliers using the Lagrange interpolation method in step 3 (S3) includes:

[0068] Process 3-1: Count the number of data in data set D2 as n, and establish an n-1 degree polynomial passing through n points:

[0069] y=a0+a1x+a2x 2 +L+a n-1 x n-1 (1)

[0070] Process 3-2: Set the coordinates of n points (x1, y1)(x2, y2)…(x n ,y n ) into the polynomial function, we get the following equation:

[0071]

[0072] Process 3-3: Solve the Lagrange interpolation polynomial by solving the simultaneous equations:

[0073]

[0074] Process 3-4: If Figure 3 As shown, the point x corresponding to the missing function value is substituted into the interpolation polynomial to obtain the approximate value L(x) of the missing value, and the patched data set D3 is obtained.

[0075] The process of the improved grey relational analysis method in step 4 (S4) is as follows:

[0076] Process 4-1: First, you need to determine the reference sequence and the comparison sequence:

[0077]

[0078] Process 4-2: Initialization of X i (k):

[0079]

[0080] Process 4-3: Constructing the difference matrix between the reference sequence and the comparison sequence:

[0081]

[0082] Process 4-4: Introduce and form the shape similarity correlation coefficient:

[0083]

[0084] Process 4-5: Construct the quotient matrix of reference sequence and comparison sequence:

[0085]

[0086] Process 4-6: Introduced to form the distance similarity correlation coefficient:

[0087]

[0088] Process 4-7: Calculate the comprehensive correlation and determine the auxiliary factors:

[0089]

[0090] like Figure 4 As shown, the specific operation process of step 5 (S5) is:

[0091] Process 5-1: 70% of the data in dataset D2 is used as training data to form a training set;

[0092] Process 5-2: Set the hyperparameters of the sparrow search algorithm;

[0093] Process 5-3: Population initialization, using the root mean square error as the fitness function;

[0094] Process 5-4: Input the population parameters into the LSTM neural network, calculate the individual and group fitness, and continuously update the sparrow operator;

[0095] Process 5-5: Determine whether the training is completed based on the termination conditions;

[0096] Process 5-6: If training is complete, output the optimal LSTM hyperparameters; otherwise, return to process 5-4 to continue executing the rules.

[0097] The termination condition is that the root mean square error (RMSE) no longer changes or the error is less than the set value range, where the sparrow operator refers to the individual optimal value, the population optimal value, the weight factor, and the fitness value.

[0098] The calculation formula of root mean square error RMSE is as follows, where Y i represents the true value, Y j Represents the predicted value, and n represents the amount of data:

[0099]

[0100] like Figure 5 As shown in Figure 2, the improved LSTM model based on SSA is used to predict the test data. The results show that the improved LSTM can effectively simulate the changing trend of fluctuating dissolved oxygen data and can be used for dissolved oxygen forecasting.

[0101] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0102] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0103] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for repairing and predicting dissolved oxygen concentration data, characterized in that: The following steps are involved: Step 1: Collect water quality data and establish the original data set; Step 2: Identify missing values ​​and use box plots to perform outlier analysis on water quality data and remove outliers; Step 3: Use Lagrange interpolation method to calculate the missing values ​​and outliers; Step 4: Analyze the correlation between dissolved oxygen and other water quality parameters using the improved grey correlation analysis method; Step 5: Use the sparrow search algorithm (SSA) to optimize the five parameters of the LSTM neural network: learning rate, batch size, number of training times, number of LSTM hidden layer nodes, and number of fully connected hidden layer nodes, to establish an improved LSTM model. Step 6: Use the improved LSTM model based on the sparrow search algorithm (SSA) to predict the dissolved oxygen data; The operation method of step 1 is to select a monitoring station, obtain a continuously recorded water quality data sequence, number the monitoring data, and establish an original data set D1; The operation method of step 2 is to identify missing values ​​in the data sequence, and find outliers of water quality parameters in the data set through data sequence box plot analysis, and remove outliers to establish an adjusted data set D2; The specific process of calculating missing values ​​and outliers using the Lagrange interpolation method in step 3 includes: Process 3-1: Count the number of data in data set D2 as n, and establish an n-1 degree polynomial passing through n points: y=a0+a1x+a2x 2 +L+a n-1 x n-1 (1) Process 3-2: Set the coordinates of n points (x1, y1)(x2, y2)…(x n ,y n ) into the polynomial function, we get the following equation: Process 3-3: Solve the Lagrange interpolation polynomial by solving the simultaneous equations: Process 3-4: Substitute the point x corresponding to the missing function value into the interpolation polynomial to obtain the approximate value L(x) of the missing value and obtain the patched data set D3; The process of the improved grey relational analysis method in step 4 is as follows: Process 4-1: First, you need to determine the reference sequence and the comparison sequence: X0(k)={X0(1),X0(2),L X0(k)}(4) X i (k)={X i (1),X i (2),L X i (k)} Process 4-2: Initialization of X i (k): Process 4-3: Constructing the difference matrix between the reference sequence and the comparison sequence: Process 4-4: Introduce and form the shape similarity correlation coefficient: Process 4-5: Construct the quotient matrix of reference sequence and comparison sequence: Process 4-6: Introduced to form the distance similarity correlation coefficient: Process 4-7: Calculate the comprehensive correlation: The specific operation process of step 5 is as follows: Process 5-1: 70% of the data in the dataset D2 is used as training data to form a training set; Process 5-2: Set the hyperparameters of the sparrow search algorithm; Process 5-3: Population initialization, using the root mean square error as the fitness function; Process 5-4: Input the population parameters into the LSTM neural network, calculate the individual and group fitness, and continuously update the sparrow operator; Process 5-5: Determine whether the training is completed based on the termination conditions; Process 5-6: If training is complete, output the optimal LSTM hyperparameters; otherwise, return to process 5-4 to continue executing the rules.

2. A method for repairing and predicting dissolved oxygen concentration data according to claim 1, characterized in that: The termination condition is that the root mean square error RMSE no longer changes or the error is less than a set numerical range, where the sparrow operator refers to the individual optimal value, the population optimal value, the weight factor and the fitness value.

3. A method for repairing and predicting dissolved oxygen concentration data according to claim 2, characterized in that: The calculation formula of the root mean square error RMSE is as follows, where Y i represents the true value, Y j Represents the predicted value, and n represents the amount of data:

Images