Historical data compression method based on numerical ordering and linear fitting

Abstract

The invention discloses a historical data compression method based on numerical ordering and linear fitting, and the method comprises the following steps: 1) ordering the obtained historical data according to values; 2) linear fitting of numerical range; 3) storing according to the range; and 4) data processing and finishing the judgment. When processing data compression, the method of ordering data according to the values is adopted, for steady-state or linear data, the linear trend is more obvious after the ordering, and the aim of separating noise points can be achieved; and for non-linearor non-steady-state data, most data show the characteristic of normal distribution after the ordering, i.e. gathering most values in a smaller value interval. Therefore, the data are regularized by ordering and are suitable for compression, and then the data are fitted in a certain accuracy range by adopting the linear fitting method, thus achieving the aim of data compression and basically solving the influence of the non-steady-state data and the noise points on the compression efficiency; and the method has higher actual application value.

Description

technical field [0001] The invention relates to a historical data compression method based on numerical ordering linear fitting, which is especially suitable for processing large-capacity real-time data collected from industrial sites. Background technique [0002] The data generated in the industrial field environment often has the characteristics of huge quantity, high collection frequency and long duration. The processing of these large batches of sampling point data, especially how to store the real-time data of the sampling points in real time and how to quickly and efficiently access and analyze the stored historical data, has become the focus and difficulty in the realization of real-time database technology. [0003] For these large amounts of stored data, if you simply store all the original data directly, on the one hand, a large amount of data needs to be stored in a short period of time, which requires very high performance requirements for hardware storage media...

AI Technical Summary

Problems solved by technology

[0007] The problem to be solved by the present invention is: for the data generated in the industrial field environment, it is necessary to store the data in real time and to access and analyze the stored historical data quickly and efficiently, and to store a large amount of historical data in the limited physical storage space. It is necessary to consider that data can be stored and accessed quickly. Existing data compression methods cannot meet these requirements, and are susceptible to noise, wild points, and noise interference, and the compression effect is not ideal.

Images