A method for construct a prediction and assessment model of time series surface water quality big data

Abstract

The invention discloses a method for constructing a prediction and evaluation model of time series type surface water quality big data, which firstly clears the numerical value obviously contrary to common sense, then finds out the time point nearest to the Markov distance according to all the data at the time point where the vacancy value exists, and uses the data at the time point to fill the vacancy value. Then the outliers in the water quality data are detected by using the improved KMeans + + clustering algorithm and Z-fraction detection algorithm, and the outliers are filled by support vector regression. Then stochastic forest algorithm is used to extract the important characteristics of water quality indicators, and the indicators with high importance are selected to evaluate the overall state of water quality. Then the LSTM model is used to predict the time series of the whole state of water quality. Finally, the MapReduce program of Hadoop is used to realize the parallel execution of the program, which improves the execution efficiency of each algorithm, completes the final prediction and evaluation model construction, and improves the efficiency, integrity and accuracy ofwater quality big data analysis.

Description

technical field [0001] The invention relates to the technical field of water quality big data prediction and evaluation model construction, and is a method for building a time series surface water quality big data prediction and evaluation model, in particular to an improved KMeans++ clustering and Z score calculation based on Mahalanobis distance Outlier detection method based on particle swarm optimization algorithm, vacancy value filling method of support vector regression based on particle swarm optimization algorithm, importance analysis method based on random forest algorithm, time series prediction method based on particle swarm optimization algorithm improved LSTM model, and MapReduce-based Water quality big data prediction and evaluation model construction method based on parallel computing method. Background technique [0002] There are a wide range of physical, chemical, and biological factors affecting water quality, and biological treatment still exhibits time-v...

AI Technical Summary

Problems solved by technology

[0003] In view of the deficiencies of the above-mentioned prior art, the purpose of the present invention is to provide a method for constructing a prediction and evaluation model oriented to time-series surface water quality big data, aiming at solving Problems in water quality data analysis, and realize a unified and automated "water quality data cleaning-water quality data prediction-water quality assessment" process, and establish the final water quality big data analysis system

Images