Time-series data cleaning method for pipe net modeling

Abstract

The invention discloses a time-series data cleaning method for pipe net modeling. The time-series data cleaning method for the pipe net modeling comprises the steps of searching and elimination of duplicate values, data dispersion degree analysis, judgment of outliers, denoising noisy points of curve smoothing, interpolation completion of missing data. The time-series data cleaning method for the pipe net modeling introduces variation coefficient to achieve standardization processing of pressure information and flow data of different dimensions, judges the dispersion degree of arrays and screen the dispersion degree of arrays at the same time. The time-series data cleaning method for the pipe net modeling is characterized in that outlier data at first is searched and processed by a utilizing three times standard deviation method and then is fitted by a least square method, which greatly reduces the effects of outliers on fitting results. At the same time, data smoothing of the noisy points is processed by fitting functions, which can further reduce the presence of outlier data. The least square method can satisfy the data processing which does not conform to the normal distribution. Compared with linear interpolation, cubic spline interpolation utilized in the end can make the data inserted more smooth. The time-series data cleaning method for the pipe net modeling has the advantages of preprocessing the data before the data is imported into a model for calculation, achieving the effect of data cleaning, and providing a guarantee for the calculation of the model.

Description

technical field [0001] The invention belongs to the technical field of data processing, and in particular relates to a time series data cleaning method for pipe network modeling. Background technique [0002] The process of pipe network modeling involves a large amount of monitoring data processing. For example, the time-series data involved mainly include monitoring data of water plant outlet pressure and outlet flow, residential water consumption and water consumption pattern data, and data used for model verification. The pressure and flow data of the monitoring points of the pipe network, etc. However, some of these data are correct, while some due to some unspecific factors such as mechanical instrument errors, there will inevitably be problems such as outliers at certain time points, missing data, and repeated data. If it is not screened, it will inevitably have a certain impact on the model calculation results and may even directly lead to the occurrence of model cal...

AI Technical Summary

Problems solved by technology

For the above-mentioned outliers, missing data, and duplication of data in the modeling and monitoring data, there is currently no complete set of standardized processing procedures

Images