A storage method for real-time database

Abstract

The invention discloses a storage method of a real-time database. The storage method comprises the following steps: filtering historical data by a lossy compression algorithm; when the historical data is subjected to an exception examination of lossy compression, writing the historical data into a current memory table and an operation log buffer zone; when the current memory table is full, compressing and writing the historical data in the current memory table into a minute-level historical data file, and applying for a new memory table as the current memory table; combining a plurality of minute-level historical data files into a day-level historical data file by a combination thread at preset time of each day; combining a plurality of day-level historical data files into a month-level historical data file at the preset time of each month; combining a plurality of month-level historical data files into a year-level historical data file in the preset time of each year. The storage method supports multi-core optimization and solid state disk optimization, and data writing-in full-sequence operation is guaranteed since a file block structure design is adopted. The storage method has higher historical data reading and writing speed, and requirements on historical data processing by an integrated supervision and control system can be met.

Description

technical field [0001] The invention relates to a real-time database technology, in particular to a storage method of a real-time database. Background technique [0002] Currently, relational databases are generally used in many fields to store historical data. However, in many application scenarios with high-frequency sampling and massive data (for example, comprehensive monitoring of rail transit), relational databases have encountered bottlenecks in read and write performance and storage capacity, and it is difficult to meet the needs of comprehensive monitoring systems for historical data processing. . Taking the field of integrated rail transit monitoring as an example, operations such as report statistics, accident inversion, and training simulation all need to process massive amounts of historical operation data. Therefore, integrated monitoring systems have higher and higher requirements for historical data processing, and relational databases It is becoming more a...

AI Technical Summary

Problems solved by technology

However, this method cannot take advantage of the file system characteristics of the current operating system, and the operation is too low-level to deal with problems such as sector failures, so it can only be applied to embedded systems

[0015] In addition, modern databases in the prior art generally use locks or multi-version technology to achieve concurrent reading and writing. However, the degree of read and write concurrency using locks or multi-version technology is not very high.

In the prior art, the application number is CN103336828A, and the Chinese patent application titled "real-time database reading and writing method" mentions a locking method, which can effectively improve the performance of reading and writing, but it cannot support multi-writing concurrently. operate

[0016] It can be seen that there are more or less some of the above-mentioned problems in the storage methods of the historical data of the real-time database in the prior art, so it is difficult to meet the needs of the storage of the historical data of the real-time database

Images