Methods for lossy compression and decompression in real-time database through dynamic prediction

Abstract

The invention discloses a method for lossy compression in a real-time database through dynamic prediction. According to the steps of the method, system initialization is carried out, two real-time data are read in, the first datum is written into the database and is saved as 'the final read-in datum', and according to the former two data and the error permitted range, the top and bottom limitation of a slope factor is calculated, and each modification value is initialized as zero; new data are read in, and the top and bottom limitation of the slope factor is calculated according to the new data, 'the final read-in datum' and the error permitted range; the top and bottom limitation of the slope factor of the new data and the top and bottom limitation of the current slope factor are compared to determine whether the new data are saved; and the top and bottom limitation of the current slope factor and a slope factor modification value are adjusted, and the second step is repeated. The invention discloses a decompression method for lossy compression in the real-time database through dynamic prediction. The decompression method for lossy compression in the real-time database through dynamic prediction includes the following steps: query time is set as t, the latest three data p1(t1,v1), p2(t1,v2) and p3(t3,v3) before the t are read in, and then pt (t,vt) is calculated in the following calculating method. Compared with the existing calculation, the methods for lossy compression and decompression in real-time database through dynamic prediction have the advantages of being large in compression ratio, small in error, and the like.

Description

technical field [0001] The invention relates to a method in the field of real-time databases, specifically, a method for lossy compression of data through dynamic correction, prediction and inspection of real-time data. Background technique [0002] The real-time database is mainly used in the field of industrial monitoring, such as the plant-level monitoring system of thermal power plants. Real-time data in this field has the following characteristics: [0003] 1. There are many measuring points. A new 300WM thermal power plant has more than 10,000 monitoring system measuring points. More measuring points means more concurrent tasks in the system, which requires high operating efficiency of the system. [0004] 2. Huge amount of data. The data sampling frequency of the industrial monitoring system is mostly at the second level, that is, each measuring point obtains one data per second. Therefore, the data processing algorithm must complete the data processing within a ...

AI Technical Summary

Problems solved by technology

[0004] 2. Huge amount of data

Therefore, the data processing algorithm must complete the data processing within a limited time so as not to cause a bottleneck

On the other hand, the huge amount of data requires a lot of hard disk space for storage

[0005] 3. Slow data changes and many redundancy

[0007] 5. Unavoidable noise

[0012] After literature search, it was found that Peter A.James proposed the SLIM lossy compression algorithm in the article "Data Compression For Process Historians" in 1995. The compression rate and accuracy of this algorithm have certain advantages compared with the revolving door algorithm, but it is not Introduced and used in the real-time database system

And the algorithm still has room for improvement

Images