River water quality prediction method and system based on big data

Abstract

The present application relates to the technical field of data processing, in particular to a river water quality prediction method and system based on big data, which solves the technical problem of large deviation of interpolation value from actual value and reduced prediction accuracy in the water quality prediction process caused by ignoring the space-time coupling of water quality data when interpolating missing values in the prior art. The method comprises: obtaining water quality data at multiple time points; the water quality data obtained at each time point is from multiple sampling positions arranged along the flow direction of the river; analyzing the spatial distribution characteristics and time sequence characteristics of the water quality data in the historical water quality data set, and interpolating the missing values in the historical water quality data set in space-time coupling to obtain a complete data set; the historical water quality data set comprises water quality data obtained at multiple historical time points; training a sequence prediction model according to the complete data set, and obtaining a water quality prediction model after training; inputting the water quality data at the current time into the water quality prediction model to output the water quality data prediction value at the future time.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method and system for predicting river water quality based on big data. Background Technology

[0002] Water quality prediction, as an important technical means for water environment management and pollution prevention and control, refers to the process of predicting the trend of river water quality changes in the future period based on real-time water body monitoring data and with the help of deep learning models. It provides key data support for decision-making such as water resource protection and emergency pollution response. The conventional implementation process usually includes three core links: data collection, data cleaning, and model training.

[0003] In the data cleaning stage of water quality prediction, existing technologies typically use interpolation to handle data gaps caused by sensor failures, data entry errors, system failures, etc. The core idea of ​​this method is to make inferences based on the distribution pattern of data in the time dimension. That is, by obtaining water quality data at adjacent times before and after the missing value, analyzing its time series characteristics to fit the missing information, thereby filling the data gaps and ensuring that the subsequent process can proceed normally.

[0004] However, changes in water quality data are not only affected by time factors, but are also closely related to spatial location. Water quality data from different sampling locations have inherent differences and influence each other. That is, water quality data has significant spatiotemporal coupling. Missing value interpolation methods that rely solely on the time dimension completely ignore this characteristic, resulting in a large deviation between the calculated interpolation value and the actual water quality data value. Inaccurate cleaned data will directly affect the effectiveness of subsequent model training, ultimately leading to a significant reduction in the accuracy of river water quality predictions based on these data, which cannot meet the requirements of practical applications for accurate prediction results. Summary of the Invention

[0005] To address the technical problem in existing technologies where the spatiotemporal coupling of water quality data is ignored during interpolation of missing values, leading to excessive deviations between interpolated and actual values ​​and reduced prediction accuracy in water quality prediction, the present invention aims to provide a river water quality prediction method and system based on big data. The specific technical solution adopted is as follows:

[0006] Firstly, a big data-based river water quality prediction method is provided, comprising: acquiring water quality data at multiple time points; these multiple time points include the current time and multiple historical time points; the water quality data acquired at each time point comes from multiple sampling locations deployed along the river's flow direction; analyzing the spatial distribution characteristics and temporal series characteristics of the water quality data in the historical water quality dataset, and performing spatiotemporal coupling imputation on missing values ​​in the historical water quality dataset to obtain a complete dataset; the historical water quality dataset includes water quality data acquired at multiple historical time points; training a sequence prediction model based on the complete dataset, and obtaining a water quality prediction model after training; inputting the water quality data at the current time point into the water quality prediction model, and outputting the predicted values ​​of water quality data for future time points.

[0007] Based on the above technical solution, the river water quality prediction method based on big data provided by this invention acquires water quality data covering current and historical moments and multiple sampling locations, providing a data foundation for analyzing the spatiotemporal characteristics of the water quality data. Then, by analyzing spatial distribution characteristics and time series characteristics, spatiotemporal coupling interpolation is performed on missing values ​​to avoid interpolation bias caused by considering only a single dimension, resulting in a complete dataset that more closely reflects the actual water quality situation. Training a sequence prediction model based on this complete dataset reduces the interference of data errors on model training. Finally, when the current water quality data is input into the trained model to output predicted values, the deviation between the interpolated and actual values ​​can be effectively reduced, improving the accuracy of river water quality prediction.

[0008] In conjunction with the first aspect above, in one possible implementation, the method for obtaining water quality data at multiple times specifically includes: collecting raw data at multiple sampling locations deployed along the river flow direction at multiple times; the raw data includes at least two of the following: turbidity, ammonia nitrogen content, water temperature, permanganate index, pH, and dissolved oxygen content; and performing uniform format conversion and normalization on the raw data to obtain water quality data at multiple times.

[0009] In conjunction with the first aspect mentioned above, in one possible implementation, the method for analyzing the spatial distribution characteristics and time series characteristics of water quality data in a historical water quality dataset, and performing spatiotemporal coupling imputation on missing values ​​in the historical water quality dataset to obtain a complete dataset, specifically includes: constructing an original water quality data sequence corresponding to the time of the missing value in the historical water quality dataset; the original water quality data sequence includes water quality data of other types besides the missing value; analyzing the spatial distribution characteristics based on water quality data from multiple sampling locations in the historical water quality dataset, and generating a first prediction sequence based on the spatial distribution characteristics; the first prediction sequence includes predicted values ​​of all types of water quality data at the sampling location corresponding to the time of the missing value; analyzing the time series characteristics based on water quality data from multiple historical times in the historical water quality dataset, and generating a second prediction sequence based on the time series characteristics; the second prediction sequence includes predicted values ​​of all types of water quality data at the sampling location corresponding to the time of the missing value; determining the imputation value of the missing value based on the original water quality data sequence, the first prediction sequence, and the second prediction sequence; and filling in the missing value based on the imputation value to obtain a complete dataset.

[0010] In conjunction with the first aspect mentioned above, in one possible implementation, the method for analyzing spatial distribution characteristics based on water quality data from multiple sampling locations in a historical water quality dataset specifically includes: extracting water quality data from different sampling locations at the same time as the missing value from the historical water quality dataset, and determining them as spatial dimension water quality data; fitting the distribution of the spatial dimension water quality data to obtain spatial distribution characteristics.

[0011] In conjunction with the first aspect mentioned above, in one possible implementation, the method for analyzing time series features based on water quality data from multiple historical moments in a historical water quality dataset specifically includes: extracting water quality data from the historical water quality dataset at different times at the same sampling location as the missing values, and determining them as water quality data in the time dimension; fitting the distribution of the water quality data in the time dimension to obtain the time series features.

[0012] In conjunction with the first aspect above, in one possible implementation, the method for determining the imputation value of missing values ​​based on the original water quality data sequence, the first prediction sequence, and the second prediction sequence specifically includes: analyzing the first distance between the first prediction sequence and the original water quality data sequence; analyzing the second distance between the second prediction sequence and the original water quality data sequence; selecting the target prediction sequence corresponding to the smaller distance between the first distance and the second distance, and using the predicted value of the type corresponding to the missing value in the target prediction sequence as the imputation value of the missing value.

[0013] In conjunction with the first aspect above, in one possible implementation, the method for filling in missing values ​​based on imputed values ​​specifically includes: determining a correction coefficient based on the data association relationship between other categories and the category corresponding to the missing value; correcting the imputed value based on the correction coefficient; and filling the position of the missing value with the corrected imputed value.

[0014] In conjunction with the first aspect mentioned above, in one possible implementation, the method of training a sequence prediction model based on a complete dataset and obtaining a water quality prediction model after training specifically includes: selecting continuous water quality data of a preset duration from the complete dataset; standardizing the continuous water quality data to convert time information into periodic features; generating sample data through a sliding window; dividing the sample data into a training set, a validation set, and a test set according to a preset ratio; training the sequence prediction model using the training set; adjusting the model parameters using the validation set; and evaluating the generalization ability of the model using the test set to obtain the water quality prediction model.

[0015] In conjunction with the first aspect mentioned above, in one possible implementation, the water quality prediction model includes multiple output heads, each corresponding to a type of water quality data.

[0016] Secondly, a big data-based river water quality prediction system is provided, comprising: a data acquisition module for acquiring water quality data at multiple times; these multiple times include the current time and multiple historical times; the water quality data acquired at each time comes from multiple sampling locations deployed along the river's flow direction; a data cleaning module for analyzing the spatial distribution and temporal series characteristics of the water quality data in the historical water quality dataset, and performing spatiotemporal coupling imputation on missing values ​​in the historical water quality dataset to obtain a complete dataset; the historical water quality dataset includes water quality data acquired at multiple historical times; a model training module for training a sequence prediction model based on the complete dataset, resulting in a water quality prediction model; and a water quality prediction module for inputting the current water quality data into the water quality prediction model and outputting predicted values ​​of water quality data for future times.

[0017] Thirdly, a big data-based river water quality prediction device is provided, comprising: a processor and a storage medium; the storage medium includes instructions, and the processor is used to execute the instructions to perform the actions described in the first aspect and any possible implementation thereof. This big data-based river water quality prediction device can be an electronic device or a chip within an electronic device.

[0018] Fourthly, a computer-readable storage medium is provided, in which instructions are stored, which, when executed on a big data-based river water quality prediction device, cause the big data-based river water quality prediction device to perform the actions described in the first aspect and any possible implementation thereof.

[0019] Fifthly, a computer program product containing instructions is provided, which, when run on a big data-based river water quality prediction device, causes the big data-based river water quality prediction device to perform the actions described in the first aspect and any possible implementation thereof.

[0020] The present invention has the following beneficial effects:

[0021] By acquiring water quality data covering current and historical moments and multiple sampling locations, a data foundation is provided for analyzing the spatiotemporal characteristics of water quality data. Then, by analyzing spatial distribution characteristics and time series characteristics, spatiotemporal coupling interpolation is performed on missing values ​​to avoid interpolation bias caused by considering only a single dimension, resulting in a complete dataset that more closely reflects actual water quality conditions. Training a sequence prediction model based on this complete dataset reduces the interference of data errors on model training. Finally, when the current water quality data is input into the trained model to output predicted values, the deviation between interpolated and actual values ​​can be effectively reduced, improving the accuracy of river water quality prediction. Attached Figure Description

[0022] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A system structure diagram of a river water quality prediction system based on big data is provided as an embodiment of the present invention;

[0024] Figure 2 A flowchart illustrating a method for predicting river water quality based on big data, provided as an embodiment of the present invention;

[0025] Figure 3 A flowchart illustrating another method for predicting river water quality based on big data, provided as an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the hardware structure of a river water quality prediction device based on big data, provided as an embodiment of the present invention. Detailed Implementation

[0027] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a river water quality prediction method and system based on big data proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0029] The following description, in conjunction with the accompanying drawings, details a specific scheme for a river water quality prediction method and system based on big data provided by this invention.

[0030] Please see Figure 1 The diagram illustrates a system structure of a big data-based river water quality prediction system according to an embodiment of the present invention. The big data-based river water quality prediction system includes: a data acquisition module 1, a data cleaning module 2, a model training module 3, and a water quality prediction module 4.

[0031] Among them, the data acquisition module 1 is the data input source of the system. Its core function is to collect and preprocess multi-source heterogeneous water quality data to provide standardized and directly usable basic data for subsequent modules. The water quality data it outputs will be transmitted to the data cleaning module 2 as the object of missing value imputation.

[0032] In some implementations, the data acquisition module 1 includes a real-time data acquisition submodule 11, a historical data acquisition submodule 12, and a data preprocessing submodule 13.

[0033] The real-time data acquisition submodule 11 deploys multiple water quality sensors at different locations along the river's flow direction. The sensors collect raw water quality data at the corresponding locations in real time, covering parameters such as turbidity, ammonia nitrogen content, water temperature, permanganate index, pH value, and dissolved oxygen content, ensuring a comprehensive reflection of the core water quality status of the water body. The acquisition frequency can be set according to actual needs (e.g., 10 minutes / time).

[0034] The historical data acquisition submodule 12 connects to data sources such as hydrological monitoring databases and watershed management platforms through a network query interface to obtain historical water quality data that corresponds one-to-one with the sensor locations in the real-time data acquisition submodule 11. The parameter types of the historical data are completely consistent with those of the real-time data (also including 6 parameters such as turbidity and ammonia nitrogen content), and the time span must meet the requirements of subsequent analysis of time series characteristics (such as covering at least the last 7 days or more of data).

[0035] The data preprocessing submodule 13 first converts the real-time data acquired by the real-time data acquisition submodule 11 and the historical data acquired by the historical data acquisition submodule 12 into a pre-defined format such as comma-separated values ​​(CSV) to avoid data incompatibility issues caused by format heterogeneity. Then, the data is merged according to the dimensions of date, time, and sampling location number to form a table containing date, time, sampling location number, and six water quality data items. Finally, the six water quality data items in the table are normalized (e.g., by mapping parameter values ​​to the [0, 1] interval through linear transformation) to eliminate the influence of differences in the dimensions of different parameters on subsequent processing, and finally outputs the water quality data to the data cleaning module 2.

[0036] Data cleaning module 2 is the core component of the system to improve data quality. It mainly processes the missing values ​​that may exist in the water quality data output by data acquisition module 1 (such as those caused by sensor failure or data entry errors). The core logic is to analyze spatiotemporal features and spatiotemporal coupling interpolation. The complete dataset obtained after processing will be transmitted to model training module 3.

[0037] In some implementations, the data cleaning module 2 includes a raw data sequence construction submodule 21, a spatiotemporal feature analysis submodule 22, a predicted sequence generation submodule 23, an interpolation value determination submodule 24, and a complete dataset generation submodule 25.

[0038] The original data sequence construction submodule 21 receives the water quality data output by the data acquisition module 1 and filters out the historical water quality data set containing historical time data. For each missing value in this set, it extracts the non-missing water quality data of the corresponding time and sampling location (e.g., if the pH value is missing at a certain time, it extracts the turbidity, ammonia nitrogen content, and other 5 parameters at that time) to construct the original water quality data sequence and clarify the contextual data environment of the missing value.

[0039] The spatiotemporal feature analysis submodule 22 can be further divided into a spatial feature analysis unit and a temporal feature analysis unit, which together provide feature support for interpolation. Specifically, the spatial feature analysis unit extracts water quality data (i.e., spatial dimension data) from the historical water quality dataset at different sampling locations at the same time as the missing value, and analyzes the distribution pattern of this type of data using a preset distribution fitting algorithm (such as Gaussian distribution fitting) to obtain spatial distribution characteristics. The temporal feature analysis unit extracts water quality data (i.e., temporal dimension data) at the same sampling location as the missing value but at different historical times, and similarly analyzes its temporal variation pattern using a distribution fitting algorithm to obtain time series characteristics.

[0040] The prediction sequence generation submodule 23 generates a prediction sequence for interpolation based on the features output by the spatiotemporal feature analysis submodule 22. Based on spatial distribution characteristics, it calculates the predicted values ​​of all types of water quality data (including predicted values ​​of missing parameters) at the corresponding time and sampling location for the missing values, forming the first prediction sequence. Similarly, based on time series characteristics, it calculates the predicted values ​​of all types of water quality data at the same time and sampling location, forming the second prediction sequence, providing two dimensions of reference for subsequent optimal interpolation.

[0041] The interpolation value determination submodule 24 calculates the similarity between the first predicted sequence, the second predicted sequence, and the original water quality data sequence (using a distance algorithm such as Euclidean distance; the smaller the distance, the higher the similarity). It selects the predicted sequence with the smaller distance as the target predicted sequence and extracts the predicted values ​​of the missing parameters from this sequence as the initial interpolation values. Then, it analyzes the correlation between water quality data (e.g., increased water temperature leads to a decrease in pH, and increased turbidity leads to an increase in pH), calculates correction coefficients, and optimizes the initial interpolation values ​​to obtain the final interpolation values.

[0042] The complete dataset generation submodule 25 fills the missing positions in the historical water quality dataset with the interpolation values ​​output by the interpolation value determination submodule 24. The process of constructing the original water quality data sequence, analyzing spatiotemporal features, generating the predicted sequence, and determining the interpolation values ​​is repeated until there are no missing values ​​in the historical water quality dataset. Finally, a complete dataset is generated and transmitted to the model training module 3.

[0043] Model training module 3 is the core of the system's prediction capability. Based on the complete dataset output by data cleaning module 2, it constructs and trains a sequence prediction model, ultimately obtaining a water quality prediction model with water quality prediction capabilities.

[0044] In some implementations, the model training module 3 includes a training sample generation submodule 31, a dataset partitioning submodule 32, a prediction model construction submodule 33, and a model training and optimization submodule 34.

[0045] The training sample generation submodule 31 selects continuous water quality data (such as 72 hours of continuous data) of a preset duration from the complete dataset. First, it performs robust standardization on the continuous data (to eliminate the interference of extreme data values ​​on the model). Then, it converts the date and time information into sinusoidal periodic features (such as quantifying the periodicity of 24 hours a day and 7 days a week into feature values). Finally, it generates model input samples through sliding window technology (such as setting the window length to 432 time steps and the sliding step size to 6 time steps to ensure that the samples can cover the continuous water quality change process).

[0046] The dataset partitioning submodule 32 divides the model input samples output by the training sample generation submodule 31 into a training set, a validation set, and a test set according to a preset ratio (e.g., 8:1:1). The training set is used for the model to learn the changing patterns of water quality data; the validation set is used to adjust model parameters (such as learning rate and number of iterations) during training; and the test set is used to evaluate the model's generalization ability (i.e., the model's prediction accuracy on unseen data). These three sets work together to ensure the effectiveness and reliability of model training.

[0047] The prediction model construction submodule 33 constructs a sequence prediction model containing multiple independent output heads. Each output head corresponds one-to-one with a water quality data point (such as turbidity, ammonia nitrogen content, etc.), ensuring that the model can accurately predict multiple water quality data simultaneously. The core structure of the model adopts an architecture that can capture time-series dependencies (such as a long short-term memory network (LSTM)) to adapt to the dynamic changes in water quality data over time. It also sets an adaptive optimizer (such as the Adam optimizer) and an iteration stopping condition (such as the loss function changing less than 5% over 20 consecutive iterations), providing a basic framework for subsequent training.

[0048] The model training and optimization submodule 34 inputs the training set into the model built by the prediction model construction submodule 33 and starts the training process. During training, the training error of the model is monitored in real time using the validation set. If the error does not decrease or overfitting occurs (e.g., the error on the training set is small but the error on the validation set is large), the model parameters are adjusted (e.g., reducing the learning rate or increasing the regularization term). After training is completed, the prediction accuracy of the model is evaluated using the test set. If the accuracy meets the preset standard (e.g., the prediction error is less than 5%), the model is identified as a water quality prediction model and transferred to the water quality prediction module 4. If the accuracy does not meet the standard, the process returns to the prediction model construction submodule 33 to optimize the model structure and restart the training process.

[0049] The water quality prediction module 4 is the application output end of the system. Based on the current water quality data provided by the data acquisition module 1 and the water quality prediction model output by the model training module 3, it can predict the water quality data at future times and output the prediction results for users (such as watershed managers and environmental protection departments).

[0050] In some implementations, the water quality prediction module 4 includes a real-time data preprocessing submodule 41, a model inference submodule 42, and a prediction result output submodule 43.

[0051] The real-time data preprocessing submodule 41 receives the current water quality data (including six parameters such as real-time turbidity and ammonia nitrogen content at each sampling location) transmitted by the data acquisition module 1. It processes the current data according to the preprocessing standards of the training sample generation submodule 31 (such as the same standardization method and periodic feature conversion rules) to ensure that the format and feature dimensions of the current data are completely consistent with the samples during model training, thus avoiding prediction errors caused by data format mismatch.

[0052] The model inference submodule 42 inputs the standardized data output from the real-time data preprocessing submodule 41 into the water quality prediction model deployed in the model training module 3, initiating the inference process. Based on the input data at the current moment, and combined with the water quality change patterns learned during training (such as the spatiotemporal parameter relationships), the model calculates and outputs predicted water quality data for future preset times (such as the next 1 hour, 6 hours, and 24 hours). The predicted values ​​cover all 6 water quality data items, and each parameter corresponds to a unique output result (generated separately by the model's independent output header).

[0053] The prediction result output submodule 43 receives the predicted water quality data for future moments from the model inference submodule 42. It then organizes the prediction results by time, sampling location, and water quality data dimensions to form a structured report (e.g., in tabular form, containing predicted values ​​for parameters such as turbidity and ammonia nitrogen content per hour and at each sampling location within the next 24 hours). Simultaneously, it supports displaying the prediction results in visual formats (e.g., line graphs, heatmaps). For example, a line graph shows the trend of pH changes at a sampling location over the next 24 hours, while a heatmap shows the distribution of turbidity across the entire basin at a given moment. Finally, the structured report and visualization results are output to the user through the system interface and application programming interface (API), providing data support for users to formulate water quality management strategies (e.g., initiating pollution warnings in advance and adjusting water conservancy project scheduling plans).

[0054] Please see Figure 2 The diagram illustrates a flowchart of a big data-based river water quality prediction method according to an embodiment of the present invention. This big data-based river water quality prediction method includes:

[0055] S1. Obtain water quality data at multiple time points.

[0056] The data includes multiple time points, including the current time and multiple historical time points. The water quality data acquired at each time point comes from multiple sampling locations deployed along the river's flow direction.

[0057] In some implementations, methods for obtaining water quality data at multiple time points can specifically include: collecting raw data at multiple sampling locations deployed along the river's flow direction at multiple time points. The raw data includes at least two of the following: turbidity (characterizing the content of suspended particles in the water), ammonia nitrogen content (characterizing the degree of nitrogen pollution in the water), water temperature (characterizing the temperature state of the water), permanganate index (characterizing the content of oxidizable organic and inorganic matter in the water), pH value (characterizing the acidity or alkalinity of the water), and dissolved oxygen content (characterizing the concentration of dissolved oxygen in the water). If limited by the sampling environment (e.g., insufficient power supply in remote areas), at least two key parameters, ammonia nitrogen content and dissolved oxygen content, must be collected, as these two are directly related to the degree of water pollution and the ecological state.

[0058] The raw data underwent a unified format conversion and normalization process to obtain water quality data for multiple time points. First, the collected real-time raw data and the queried historical raw data were converted to a unified format. Heterogeneous formats, such as binary data output from different sensors and JavaScript object notation (JSON) data stored in the database, were all converted to CSV format files. The converted files must contain nine fields: date, time, sampling location number, turbidity, ammonia nitrogen content, water temperature, permanganate index, pH, and dissolved oxygen content, ensuring that each data point accurately corresponds to a specific time, location, and parameter. Then, the six water quality data items in the CSV file, excluding date, time, and sampling location number, were normalized. For example, a min-max normalization method was used, mapping parameter values ​​to the [0, 1] interval using a formula to eliminate the interference of different parameter unit differences (such as water temperature in degrees Celsius and turbidity in NTU) on subsequent data calculations. Finally, standardized water quality data covering multiple time points and sampling locations was obtained.

[0059] S2. Analyze the spatial distribution and time series characteristics of water quality data in the historical water quality dataset, and perform spatiotemporal coupling interpolation on the missing values ​​in the historical water quality dataset to obtain a complete dataset.

[0060] The historical water quality dataset includes water quality data acquired at multiple historical moments.

[0061] In some implementations, a historical water quality dataset containing water quality data from multiple historical moments is used as the processing object. First, an original water quality data sequence containing the corresponding non-missing parameters for the missing values ​​in the dataset is constructed. Then, spatial distribution features are fitted to water quality data from different sampling locations at the same moment to generate a first predicted sequence, and time series features are fitted to water quality data from the same historical sampling location at the same time to generate a second predicted sequence. Next, the distances between the two predicted sequences and the original water quality data sequence are calculated, and the value corresponding to the sequence with the smaller distance is selected as the initial interpolation value. Subsequently, a correction coefficient is calculated by combining the correlation between water quality data and historical parameter extreme values ​​and reference values ​​to optimize the initial interpolation value and obtain the final interpolation value; finally, all missing values ​​are filled in to obtain the complete dataset.

[0062] In other implementations, the historical water quality dataset can be divided into multiple time slices (each slice contains water quality data from all sampling locations at a given time). For missing values ​​in each time slice, a Kriging interpolation algorithm (based on the spatial coordinates of the sampling location and the water quality data values ​​of adjacent locations) is used to directly calculate the parameter interpolation for the missing locations, completing the spatial dimension data of the current time slice, resulting in a spatially completed slice. Then, all spatially completed slices are sorted in chronological order. For each water quality data item at each sampling location, an average value is calculated using a three-time sliding window (e.g., the parameter value at time t on day i is averaged by the spatially completed values ​​at time t on day i-2, time t on day i-1, and time t on day i). This average value replaces the original spatially completed value, further eliminating random errors in spatial interpolation, thereby integrating all optimized time slices to form a complete dataset without missing values.

[0063] S3. Train the sequence prediction model based on the complete dataset to obtain the water quality prediction model.

[0064] In some implementations, continuous water quality data of a preset duration can be selected from a complete dataset. This continuous data is then standardized to convert temporal information into periodic features, and sample data is generated using a sliding window. The sample data is then divided into training, validation, and test sets according to a preset ratio. A sequence prediction model is trained using the training set, model parameters are adjusted using the validation set, and the model's generalization ability is evaluated using the test set, resulting in a water quality prediction model.

[0065] The water quality prediction model includes multiple output heads, each corresponding to a type of water quality data.

[0066] Specifically, continuous water quality data of varying durations are selected from the complete dataset as the foundation for model training. The preset duration must cover the complete cyclical fluctuations of the water quality data (e.g., 72 hours, corresponding to three complete days and nights, capturing the diurnal variations in dissolved oxygen and pH). The data interval must match the collection interval (e.g., 10 minutes / data point, totaling 432 data points over 72 hours), and must include all six water quality parameters (turbidity, ammonia nitrogen content, water temperature, permanganate index, pH, and dissolved oxygen content) from all sampling locations. Robust standardization is applied to the selected continuous water quality data, i.e., using a preset algorithm (e.g., removing extreme outliers and normalizing parameter values ​​to the [-1, 1] range) to eliminate the interference of extreme values ​​on model training and convert all parameters into dimensionless numbers. Simultaneously, the date d and time t information are converted into sinusoidal periodic features (e.g., converting a 24-hour day into...). or Convert 7 days a week to or This quantifies the periodicity of time into features that the model can recognize, thus avoiding insufficient capture of periodicity caused by the model relying solely on a single timestamp.

[0067] A sliding window technique is used to generate model input samples from preprocessed continuous water quality data. The preset window length is 432 time steps (corresponding to 72 hours of continuous data, ensuring that the window contains complete water quality cycle characteristics), and the sliding step size is 6 time steps (corresponding to 1 hour, balancing sample size and computational efficiency). The first 426 time steps of data in each window are used as model input features, and the last 6 time steps are used as model prediction targets (i.e., labels), generating sample data that pairs input with labels to ensure that the samples can cover water quality change patterns over different time periods.

[0068] The generated sample data is divided into a training set, a validation set, and a test set according to a preset ratio, such as 8:1:1. The training set is used to allow the model to learn the changing patterns of water quality data (e.g., the correlation between rising water temperature and decreasing pH, and the negative correlation between turbidity and dissolved oxygen). The validation set is used to adjust model parameters in real time (e.g., learning rate, number of iterations) to avoid overfitting (smaller errors in the training set but larger prediction errors on unfamiliar data). The test set is used to evaluate the model's generalization ability (i.e., the accuracy of predictions on unseen water quality data). Furthermore, the test set data must not overlap with the training and validation sets in time to ensure the reliability of the evaluation results.

[0069] A sequence prediction model with multiple independent output heads is constructed. The model's basic architecture employs a network capable of capturing long-term and short-term time dependencies (adapting to the time-series characteristics of water quality data). The input layer dimension is consistent with the input feature dimension of a single sample (e.g., 426 time steps × 6 parameters = 2556 dimensions). The hidden layer learns data features through a predetermined number of neurons (e.g., 128). The output layer is designed with 6 independent output heads, each corresponding to a class of water quality data (e.g., the first output head outputs the predicted turbidity value, and the second output head outputs the predicted ammonia nitrogen content value), ensuring that the prediction of each parameter is independent and unaffected by other parameters. Simultaneously, an adaptive optimizer (e.g., the Adam optimizer) is configured for the model, with an initial learning rate set to 0.001 (balancing training speed and accuracy). An iteration stopping condition is set; for example, when the rate of change of the loss function (e.g., mean squared error (MSE)) is less than 5% over 20 consecutive iterations, model training is stopped to avoid unnecessary iterations and increased computational costs.

[0070] The training set is input into the constructed sequence prediction model. The model calculates the predicted value through forward propagation, compares it with the label to obtain the loss value, and then adjusts the model parameters (such as the weights of hidden layer neurons) through backpropagation to gradually reduce the loss value. After each iteration (one traversal of the training set), the model's prediction error is evaluated using the validation set. If the error continues to rise (indicating overfitting), the model parameters are adjusted (such as adding a regularization term or reducing the learning rate); if the error stabilizes and the stopping condition is met, training is stopped. Finally, the average error rate between the predicted values ​​and the true values ​​on the test set is calculated. If the error rate is less than a preset threshold (such as 5%), the model is identified as a water quality prediction model; if the error rate does not meet the threshold, the model construction phase is returned to optimize the architecture (such as increasing the number of hidden layer neurons or adjusting the output head design), and the training process is restarted until the model meets the accuracy requirements.

[0071] S4. Input the current water quality data into the water quality prediction model and output the predicted water quality data for future times.

[0072] Specifically, the same robust normalization algorithm used during model training is applied to the current water quality data, mapping the six water quality data values ​​to the interval [-1, 1] to ensure consistency with the unit conversion rules of the training data. Simultaneously, the current time information is converted into a sinusoidal periodic feature (e.g., if the current time is 14:00 on a certain day, then...). or (Maintaining consistency with the time feature format during training), ultimately forming standardized current data, the dimensions of which need to match the input layer dimensions of the water quality prediction model.

[0073] Standardized current data is input into the water quality prediction model to initiate the model inference process. The model first analyzes the current water quality characteristics through a hidden layer, combining this with spatiotemporal correlations of water quality data learned during training (such as the negative correlation between water temperature and pH, and the negative correlation between turbidity and dissolved oxygen content) to calculate the water quality change trend at a preset future time. Since the model contains six independent output heads (corresponding to six water quality data points), each output head independently calculates the predicted value for its corresponding parameter during inference, avoiding interference between parameters (e.g., the prediction error for turbidity will not affect the prediction result for ammonia nitrogen content). After inference is complete, the model outputs standardized prediction values ​​for future time points, including the standardized results of the six water quality data points from all sampling locations.

[0074] The standardized predicted values ​​output by the model are destandardized. Using the inverse formula of the standardization process, the values ​​in the [-1, 1] interval are restored to the actual physical units of the water quality data. For example, the standardized values ​​of pH are restored to actual pH values ​​of 0-14, and the standardized values ​​of water temperature are restored to actual values ​​in °C, thus obtaining the actual predicted values ​​of water quality data for future times. Subsequently, the actual predicted values ​​are structured according to the dimensions of future time, sampling location, and water quality data. For example, they are presented in a table format, with each row corresponding to a specific future time and sampling location, and each column corresponding to a predicted value of a water quality data point. The confidence level of each predicted value is also labeled (calculated based on the test set error rate during model training), ensuring that the prediction results are clear and easy to understand, facilitating users to quickly locate key information (such as whether the ammonia nitrogen content at a sampling location exceeds the standard in the next 6 hours).

[0075] Furthermore, the compiled future water quality data predictions can be output in two ways: visualization and structured reports. Visualization output generates line graphs (such as a trend chart of pH changes at a sampling location over the next 24 hours) and heat maps (such as a spatial distribution heat map of turbidity across the entire watershed over the next hour) through the system interface, intuitively displaying the patterns of water quality changes. Structured report output generates a prediction report containing prediction data tables, confidence level explanations, and early warning alerts for key parameters (such as an alert and reference control recommendations when a predicted value of a parameter exceeds the national surface water quality standard).

[0076] In addition, the prediction results can be associated with the corresponding current water quality data and prediction timestamp and stored in the system database to facilitate subsequent tracking of prediction accuracy (e.g., after the actual water quality data is collected in the future, it can be compared with historical prediction results to optimize model parameters).

[0077] Based on the above technical solution, water quality data covering current and historical moments and multiple sampling locations provides a data foundation for analyzing the spatiotemporal characteristics of water quality data. Then, by analyzing spatial distribution characteristics and time series characteristics, spatiotemporal coupling interpolation is performed on missing values ​​to avoid interpolation bias caused by considering only a single dimension, resulting in a complete dataset that more closely reflects actual water quality conditions. Training a sequence prediction model based on this complete dataset reduces the interference of data errors on model training. Finally, when the current water quality data is input into the trained model to output predicted values, the deviation between interpolated and actual values ​​can be effectively reduced, improving the accuracy of river water quality prediction.

[0078] In one possible implementation, combining Figure 2 ,like Figure 3 As shown, the method in S2 described above can be specifically implemented through the following steps S21 to S25, which are explained in detail below:

[0079] S21. For missing values ​​in the historical water quality dataset, construct the original water quality data sequence corresponding to the time of the missing value.

[0080] The original water quality data sequence includes water quality data of all types except for missing values.

[0081] Specifically, the process begins by iterating through all data records in the historical water quality dataset. An index is created based on four dimensions: date, time, sampling location number, and water quality data type (e.g., index format "day i, time t - sampling location k - pH"). Each index's corresponding parameter value is then checked for missing values ​​(e.g., NULL, null values, or outliers exceeding a reasonable range, where the reasonable range is based on the physicochemical characteristics of the water quality data; for example, the reasonable range for pH is 0-14, and values ​​exceeding this range are considered missing). For records identified as missing, their missing dimension information is marked (the time of the missing value, sampling location, and parameter type must be clearly specified, e.g., marking "day 3, time 14 - sampling location 2 - pH" as missing). This creates a list of missing values, ensuring that all missing values ​​are accurately located without omissions or misjudgments.

[0082] For each missing record in the missing value list, based on its marked time and sampling location number, all non-missing parameter records at the same time and sampling location are selected from the historical water quality data set. Taking "Day 3, 14:00 - Sampling Location 2 - pH Missing" as an example, the values ​​of five non-missing parameters—turbidity, ammonia nitrogen content, water temperature, permanganate index, and dissolved oxygen content—must be extracted at that time and sampling location. During the extraction process, the consistency of time and sampling location must be strictly verified to avoid the mixing of parameters from different times (e.g., mistakenly extracting data from Day 3, 13:00) or different locations (e.g., mistakenly extracting data from sampling location 3), ensuring that the extracted non-missing parameters and missing values ​​belong to the same data scenario and can truly reflect the water quality status at the time of the missing value.

[0083] Construct the original water quality data sequence at sampling location k at time t on day i. , denoted as:

[0084]

[0085] In the formula, This represents the turbidity at sampling position k at time t on day i.

[0086] This represents the ammonia nitrogen content at sampling location k at time t on day i.

[0087] Let represent the water temperature at sampling location k at time t on day i.

[0088] Let represent the permanganate index at sampling position k at time t on day i.

[0089] This represents the acidity or alkalinity at sampling position k at time t on day i.

[0090] This represents the dissolved oxygen content at sampling location k at time t on day i.

[0091] When a value is missing, it can be recorded as "NULL", such as when the pH value is missing. .

[0092] S22. Based on the water quality data from multiple sampling locations in the historical water quality dataset, analyze the spatial distribution characteristics and generate the first prediction sequence based on the spatial distribution characteristics.

[0093] The first prediction sequence includes predicted values ​​of all types of water quality data at the sampling locations corresponding to the times when the missing values ​​are found.

[0094] In some implementations, methods for analyzing spatial distribution characteristics may specifically include: extracting water quality data from historical water quality datasets at different sampling locations at the same time as the missing values, thus defining the spatial dimension of the water quality data; and then fitting the distribution of the spatial dimension of the water quality data to obtain the spatial distribution characteristics.

[0095] Specifically, the sampling locations are numbered starting from the upstream of the river. Since upstream water quality has a more significant impact on downstream water quality, when predicting the water quality data at the k-th sampling location, water quality data from the 1st to the (k-1th)th sampling locations at the same time (time t on day i) are typically used, covering turbidity, ammonia nitrogen content, water temperature, permanganate index, pH, and dissolved oxygen content. It should be noted that predicting the water quality data at the k-th sampling location requires a certain amount of upstream data; therefore, k must be at least greater than 3.

[0096] For the above six water quality data points at each sampling location, various probability distributions (such as Gaussian and Poisson distributions) were used for fitting. The distribution with the highest goodness of fit was selected through goodness-of-fit tests (such as chi-square and R² tests). Taking the Gaussian distribution as an example, which showed the highest goodness of fit, the spatial dimension distribution model and distribution parameters (mean and variance) of each parameter were obtained, denoted as:

[0097] Spatial distribution model of turbidity , and The mean and variance of the turbidity data distribution in the spatial dimension;

[0098] Spatial distribution model of ammonia nitrogen content , and The mean and variance of the ammonia nitrogen content data distribution in the spatial dimension;

[0099] Spatial distribution model of water temperature , and The mean and variance of the water temperature data distribution in the spatial dimension;

[0100] Spatial distribution model of permanganate index , and The mean and variance of the permanganate index data distribution in the spatial dimension;

[0101] Spatial distribution model of acidity and alkalinity , and In spatial dimension The mean and variance of the value data distribution;

[0102] Spatial distribution model of dissolved oxygen content , and The mean and variance of the dissolved oxygen content data distribution in the spatial dimension are given.

[0103] Based on the spatial distribution model of the above water quality data, the predicted values ​​of 6 water quality data points at the k-th sampling location at time t on day i are calculated and integrated to form the first prediction sequence. , denoted as:

[0104]

[0105] In the formula, This represents the predicted turbidity value at time t on day i, generated based on spatial distribution characteristics.

[0106] This represents the predicted ammonia nitrogen content at time t on day i, generated based on spatial distribution characteristics.

[0107] This represents the predicted water temperature at time t on day i, generated based on spatial distribution characteristics.

[0108] This represents the predicted value of the permanganate index at time t on day i, generated based on spatial distribution characteristics.

[0109] This represents the predicted acidity / alkalinity value at time t on day i, generated based on spatial distribution characteristics.

[0110] This represents the predicted dissolved oxygen content at time t on day i, generated based on spatial distribution characteristics.

[0111] Methods for extrapolating predicted values ​​based on spatial distribution models may include:

[0112] Based on the river flow distance between the k-th location and other locations, the spatial weight is calculated using the inverse distance weighting method. The closer the distance, the greater the weight (e.g., the weight of adjacent locations may reach 0.3-0.5, while the weight of distant locations may be less than 0.05), ensuring that spatially closer water quality data have a greater impact on the prediction of the k-th location.

[0113] For each water quality data point, predicted values ​​are calculated using spatial distribution models and spatial weights. Taking turbidity as an example, spatial weights are used to weight the turbidity values ​​at other locations to obtain the mean value at the k-th location. for:

[0114]

[0115] In the formula, Let j be the turbidity value at position j on day t. Let be the spatial weight of the j-th position.

[0116] Since the mean of a Gaussian distribution is the most likely value, therefore, based on the mean... variance of spatial distribution model The mean is taken as the core predicted value, that is .

[0117] S23. Based on water quality data from multiple historical moments in the historical water quality dataset, analyze the time series characteristics and generate a second prediction sequence based on the time series characteristics.

[0118] The second prediction sequence includes predicted values ​​of all types of water quality data at the sampling locations corresponding to the times when the missing values ​​are found.

[0119] In some implementations, methods for analyzing time series features may specifically include: extracting water quality data from a historical water quality dataset at different times within the same sampling location as the missing values, thus defining the water quality data as the time dimension; and then fitting the distribution of the water quality data along the time dimension to obtain the time series features.

[0120] Specifically, if the missing time is day i at time t, then the historical time is limited to day i-1 at time t, day i-2 at time t, ... day in at time t (n≥7, ensuring that at least 7 time points are included). Records with missing parameters in the historical time need to be removed (e.g., if the ammonia nitrogen content at a certain sampling location is missing at a certain historical time, then the data at that time is removed). Only complete water quality data is retained and determined as water quality data in the time dimension.

[0121] Candidate models suitable for the characteristics of the time series are selected (such as linear distribution reflecting trends, sinusoidal distribution reflecting periodicity, and Gaussian distribution reflecting randomness). For the time dimension data of each parameter, after arranging them in chronological order, the goodness-of-fit test method consistent with S22 is used to calculate the goodness-of-fit value of the data under each candidate model. The model with the highest goodness-of-fit is selected as the optimal time series model for that parameter, and the key parameters of the model are determined. Taking the Gaussian distribution as an example where the goodness-of-fit is highest, the time dimension distribution model and distribution parameters (mean and variance) of each parameter are obtained, denoted as:

[0122] Turbidity time distribution model , and The mean and variance of the turbidity data distribution over the time dimension;

[0123] Time distribution model of ammonia nitrogen content , and The mean and variance of ammonia nitrogen content data distribution over time;

[0124] Water temperature time distribution model , and The mean and variance of the water temperature data distribution over the time dimension;

[0125] Time distribution model of permanganate index , and The mean and variance of the permanganate index data distribution over the time dimension;

[0126] Time distribution model of pH , and In the time dimension The mean and variance of the value data distribution;

[0127] Time distribution model of dissolved oxygen content , and The mean and variance of dissolved oxygen content data distribution over the time dimension.

[0128] Based on the time distribution model of the above water quality data, the predicted values ​​of 6 water quality data points at the k-th sampling location at time t on day i are calculated and integrated to form the second prediction sequence. , denoted as:

[0129]

[0130] In the formula, This represents the predicted turbidity value at time t on day i, generated based on time series characteristics.

[0131] This represents the predicted ammonia nitrogen content at time t on day i, generated based on time series characteristics.

[0132] This represents the predicted water temperature at time t on day i, generated based on time series characteristics.

[0133] This represents the predicted value of the permanganate index at time t on day i, generated based on time series characteristics.

[0134] This represents the predicted pH value at time t on day i, generated based on time series characteristics.

[0135] This represents the predicted dissolved oxygen content at time t on day i, generated based on time series characteristics.

[0136] One method for estimating predicted values ​​based on a time distribution model includes: In a Gaussian distribution, the mean has the highest probability density; therefore, the mean of the same historical period can be directly used as the core value for prediction at time t on day i, while also incorporating time trends for adaptation. Taking turbidity as an example, if there is a slow upward trend in turbidity during the same historical period, the product of the daily average change and the time interval can be superimposed on the mean to obtain the predicted turbidity value at time t on day i. for:

[0137]

[0138] In the formula, This is the daily average change, determined based on the actual application scenario, such as an average daily increase of 0.1 NTU in turbidity over the past 7 days. The time interval for the time distribution model is 1 day.

[0139] S24. Determine the imputation values ​​for missing values ​​based on the original water quality data sequence, the first prediction sequence, and the second prediction sequence.

[0140] In some implementations, the method for determining the imputation value of the missing value may specifically include: analyzing a first distance between the first predicted sequence and the original water quality data sequence, and a second distance between the second predicted sequence and the original water quality data sequence.

[0141] Specifically, calculate the predicted sequence and the first The sampling location at the ... sky The Euclidean distance between the predicted values ​​of water quality data (excluding missing values) at time points and the true values ​​in the original water quality data sequence is used. The sum of the Euclidean distances of the water quality data (excluding missing values) in the predicted sequence is taken as the inter-sequence distance, and the predicted value of the predicted sequence with the smallest distance is used as the missing value for imputation. The original water quality data sequence at any given time. Taking the missing pH value as an example, the first distance... , denoted as:

[0142]

[0143] Second distance , denoted as:

[0144]

[0145] In the formula, This represents the Euclidean distance between data A and data B.

[0146] Then, the target prediction sequence corresponding to the smaller distance between the first and second distances is selected, and the predicted value of the category corresponding to the missing value in the target prediction sequence is used as the imputation value for the missing value. If the smaller distance is the first distance, it means that the predicted value in the spatial dimension is closer to the actual upstream and downstream water quality changes at the time of the missing value; if the smaller distance is the second distance, it means that the predicted value in the temporal dimension is closer to the actual water quality changes at the time of the missing value. In extreme cases where the first distance equals the second distance, the predicted value in the first prediction sequence can be selected by default, or the selection can be based on the characteristics of the missing parameter (e.g., parameters with strong spatial correlation, such as water temperature and turbidity, should prioritize the spatial dimension; parameters with strong temporal continuity, such as pH and dissolved oxygen, should prioritize the temporal dimension), ensuring that the selection of the initial imputation value follows clear rules.

[0147] S25. Fill in the missing values ​​according to the imputed values ​​to obtain the complete dataset.

[0148] In some implementations, methods for imputing missing values ​​may include: determining a correction coefficient based on the data association between other categories and the category corresponding to the missing value; then correcting the imputed value according to the correction coefficient; and finally filling the missing value with the corrected imputed value.

[0149] Specifically, the values ​​of each parameter in the water quality data are not isolated but mutually influential. The different parameter values ​​at different times may also cause deviations in the prediction results.

[0150] Taking pH as an example, the pH value of water varies under different water temperatures. Higher temperatures lead to greater self-dissociation of water, resulting in an increase in free hydrogen ions and consequently a decrease in pH. Similarly, the pH value of water varies depending on its dissolved oxygen content. The further the dissolved oxygen content deviates from saturation, the stronger the tendency for acid production, leading to… Decrease; Turbidity There is also an impact; the higher the turbidity of the water, the more suspended particles (clay, humus, etc.) there are in the water. These particles will adsorb hydrogen ions, thus leading to... rise.

[0151] Based on the above analysis, a correction coefficient was designed. This is used to correct the interpolated values ​​calculated by the above spatiotemporal distance comparison method, and is denoted as:

[0152]

[0153] In the formula, Indicates the current water temperature. and These represent the highest and lowest historical water temperatures, respectively.

[0154] This represents the saturation value of dissolved oxygen in the water body. This represents the current dissolved oxygen content in the water body.

[0155] This represents the current turbidity of the water body.

[0156] , , The influence factor coefficients are empirical coefficients obtained by fitting complete historical data. They are used to quantify the influence weight of each parameter on pH, and are all positive numbers, quantifying the degree of influence. They are typically within the range of (0, 1). Specifically, they are obtained by collecting complete data samples of known historical pH values. , , Substituting into the formula, with the goal of minimizing the error between the predicted pH and the actual pH, the optimal influence factor coefficient is obtained by fitting.

[0157] The first item ( This is used to quantify the degree to which water temperature deviates from historical extremes and to negatively correct the pH prediction value. The greater the deviation of the water temperature from the minimum value, the stronger the negative effect of the correction coefficient, and the lower the pH prediction value.

[0158] The second item ( This is used to quantify the degree to which dissolved oxygen deviates from saturation and to negatively correct the pH prediction value. The greater the deviation of dissolved oxygen from saturation, the stronger the negative effect of the correction coefficient, and the lower the pH prediction value.

[0159] Third item It is used to quantify the positive adsorption effect of turbidity on pH and to positively correct the pH prediction value; the higher the turbidity, the higher the pH.

[0160] Furthermore, based on the correction factor interpolation values Make corrections and use the corrected interpolation values. :

[0161]

[0162] Furthermore, value range constraint logic needs to be added after correction to ensure that the final value is within a reasonable range. If the calculated corrected interpolated value is greater than the maximum parameter value, the maximum value will be taken. Then the definition is taken If the calculated corrected interpolation value is less than the minimum parameter value, the minimum value will be taken. Then the definition is taken .

[0163] Specifically, if the correlation parameter in the above formula is also missing at the current time, the initial interpolated value of the correlation parameter determined in step S24 can be directly substituted into the formula for calculation. Furthermore, an iterative correction strategy can be adopted: first, calculate the first round of correction coefficients based on the initial interpolated values ​​and update all missing values; then, use the updated values ​​as input and substitute them into the formula again to calculate the correction coefficients; repeat the above steps until the deviation between the corrected interpolated values ​​calculated in two consecutive steps is less than a preset threshold (e.g., 0.01), and output the final interpolated value.

[0164] Based on the above technical solution, the original water quality data sequence with missing values ​​is first constructed to clarify the contextual baseline. Then, the distribution characteristics of the water quality data are analyzed from the spatial and temporal dimensions to generate a dual prediction sequence. The optimal initial interpolation value is selected by distance comparison. Finally, the interpolation value is optimized by introducing a correction coefficient based on the correlation between water quality data. This achieves accurate interpolation of missing values ​​in the historical water quality data set through spatiotemporal coupling. It effectively solves the problem of deviation between interpolation and actual values ​​caused by ignoring the spatiotemporal coupling of water quality data. The generated complete dataset is free of missing values ​​and has high fidelity, providing a high-quality data foundation for the subsequent training of river water quality prediction models based on big data. It ensures the accuracy and reliability of water quality prediction from the data source.

[0165] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0166] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0167] In this embodiment of the invention, the functional units of the big data-based river water quality prediction device can be divided according to the above method example. For example, each function can be divided into its own functional units, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0168] This invention also provides a schematic diagram of the hardware structure of a river water quality prediction device based on big data, see below. Figure 4 The big data-based river water quality prediction device 400 includes a processor 401, and optionally, a memory 402 connected to the processor 401.

[0169] In the first possible implementation, see Figure 4The river water quality prediction device 400 based on big data also includes a transceiver 403. The processor 401, memory 402, and transceiver 403 are connected via a bus. The transceiver 403 is used to communicate with other devices or communication networks. Optionally, the transceiver 403 may include a transmitter and a receiver. The device in the transceiver 403 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of the present invention. The device in the transceiver 403 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of the present invention.

[0170] Based on the first possible implementation method Figure 4 The schematic diagram shown can be used to illustrate the structure of the big data-based river water quality prediction device involved in the above embodiments.

[0171] in, Figure 4 This can also be illustrated by the system chip in a big data-based river water quality prediction device. In this case, the actions performed by the aforementioned big data-based river water quality prediction device can be implemented by this system chip. The specific actions performed can be found above and will not be repeated here.

[0172] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in this embodiment can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0173] The processor in this invention may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., which are various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a standalone semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may be integrated with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a System-on-a-Chip (SoC), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), or logic circuits that implement dedicated logic operations.

[0174] The memory in the embodiments of the present invention may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable read-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0175] This invention also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0176] This invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0177] This invention also provides a chip, which includes a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run computer programs or instructions to implement the above-described method. The interface circuit is used to communicate with other modules outside the chip.

[0178] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0179] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings and the disclosure, will understand and implement other variations of the disclosed embodiments in carrying out the claimed invention. In this invention, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several of the functions listed in this invention.

[0180] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely illustrative of the invention and are to be considered as covering any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications of the invention fall within the scope of the invention and its equivalents, the invention is also intended to include such modifications and modifications.

Claims

1. A method for predicting river water quality based on big data, characterized in that, include: Water quality data is acquired at multiple time points, including the current time and multiple historical time points. The water quality data acquired at each time point comes from multiple sampling locations deployed along the river's flow direction. Analyze the spatial distribution and temporal series characteristics of water quality data in the historical water quality dataset, and perform spatiotemporal coupling imputation on missing values ​​in the historical water quality dataset to obtain a complete dataset, including: For the missing values ​​in the historical water quality dataset, an original water quality data sequence corresponding to the time of the missing value is constructed; the original water quality data sequence includes other types of water quality data besides the missing value. Based on water quality data from multiple sampling locations in the historical water quality dataset, the spatial distribution characteristics are analyzed, and a first prediction sequence is generated based on the spatial distribution characteristics; the first prediction sequence includes predicted values ​​of all types of water quality data at the sampling locations corresponding to the time of the missing value. Based on water quality data from multiple historical moments in the historical water quality dataset, the time series characteristics are analyzed, and a second prediction sequence is generated based on the time series characteristics; the second prediction sequence includes predicted values ​​of all types of water quality data at the sampling locations corresponding to the moments of the missing values. Based on the original water quality data sequence, the first prediction sequence, and the second prediction sequence, determine the imputation value for the missing value; The correction coefficient is determined based on the data association relationship between other categories and the categories corresponding to the missing values; The interpolated values ​​are corrected according to the correction coefficient, and the corrected interpolated values ​​are used to fill in the positions of the missing values ​​to obtain the complete dataset; the historical water quality data set includes water quality data acquired at multiple historical moments; A sequence prediction model is trained based on the complete dataset, and a water quality prediction model is obtained after training. Input the current water quality data into the water quality prediction model, and output the predicted water quality data for future times.

2. The river water quality prediction method according to claim 1, characterized in that, The acquisition of water quality data at multiple time points includes: Raw data is collected at multiple sampling locations along the river's flow direction at multiple times; the raw data includes at least two of the following: turbidity, ammonia nitrogen content, water temperature, permanganate index, pH, and dissolved oxygen content. The original data is converted and normalized to a unified format to obtain water quality data at multiple time points.

3. The river water quality prediction method according to claim 1, characterized in that, The step of analyzing the spatial distribution characteristics based on water quality data from multiple sampling locations in the historical water quality dataset includes: Extract water quality data from different sampling locations at the same time as the missing value from the historical water quality data set, and determine them as spatial dimension water quality data; The spatial distribution characteristics are obtained by fitting the distribution of water quality data in the spatial dimension.

4. The river water quality prediction method according to claim 1, characterized in that, The step of analyzing the time series characteristics based on water quality data from multiple historical moments in the historical water quality dataset includes: Extract water quality data from the historical water quality data set that are at the same sampling location but at different times as the missing value, and determine them as water quality data in the time dimension. The distribution of water quality data in the time dimension is fitted to obtain the time series features.

5. The river water quality prediction method according to claim 1, characterized in that, The step of determining the imputation value of the missing value based on the original water quality data sequence, the first prediction sequence, and the second prediction sequence includes: Analyze the first distance between the first predicted sequence and the original water quality data sequence; Analyze the second distance between the second predicted sequence and the original water quality data sequence; Select the target prediction sequence corresponding to the smaller distance between the first distance and the second distance, and use the predicted value of the category corresponding to the missing value in the target prediction sequence as the imputation value of the missing value.

6. The river water quality prediction method according to claim 2, characterized in that, The step of training a sequence prediction model based on the complete dataset to obtain a water quality prediction model after training includes: Select continuous water quality data of a preset duration from the complete dataset, standardize the continuous water quality data, convert time information into periodic features, and generate sample data through a sliding window; The sample data is divided into a training set, a validation set, and a test set according to a preset ratio; The sequence prediction model is trained using the training set, the model parameters are adjusted using the validation set, and the generalization ability of the model is evaluated using the test set to obtain the water quality prediction model.

7. The river water quality prediction method according to claim 6, characterized in that, The water quality prediction model includes multiple output heads, each corresponding to a type of water quality data.

8. A river water quality prediction system based on big data, characterized in that, include: The data acquisition module is used to acquire water quality data at multiple times; the multiple times include the current time and multiple historical times; the water quality data acquired at each time comes from multiple sampling locations deployed along the river flow direction; The data cleaning module is used to analyze the spatial distribution and temporal series characteristics of water quality data in the historical water quality dataset, and to perform spatiotemporal coupling imputation on missing values ​​in the historical water quality dataset to obtain a complete dataset; the historical water quality dataset includes water quality data acquired at multiple historical moments. The data cleaning module is specifically used to construct the original water quality data sequence corresponding to the missing values ​​in the historical water quality data set; the original water quality data sequence includes other types of water quality data besides the missing values; Based on water quality data from multiple sampling locations in the historical water quality dataset, the spatial distribution characteristics are analyzed, and a first prediction sequence is generated based on the spatial distribution characteristics; the first prediction sequence includes predicted values ​​of all types of water quality data at the sampling locations corresponding to the time of the missing value. Based on water quality data from multiple historical moments in the historical water quality dataset, the time series characteristics are analyzed, and a second prediction sequence is generated based on the time series characteristics; the second prediction sequence includes predicted values ​​of all types of water quality data at the sampling locations corresponding to the moments of the missing values. Based on the original water quality data sequence, the first prediction sequence, and the second prediction sequence, determine the imputation value for the missing value; The correction coefficient is determined based on the data association relationship between other categories and the categories corresponding to the missing values; The imputed values ​​are corrected according to the correction coefficient, and the corrected imputed values ​​are filled into the positions of the missing values ​​to obtain the complete dataset; The model training module is used to train a sequence prediction model based on the complete dataset, and the water quality prediction model is obtained after training. The water quality prediction module is used to input the current water quality data into the water quality prediction model and output the predicted water quality data for future times.

Images