Data storage method and device

Abstract

The invention discloses a data storage method. Due to the fact that only maximum data and minimum data in a designated partition section of a table partition are stored in a partition index and the maximum data, minimum data, the total quantity of stored data, only the total quantity of empty data, the sum of stored data and the mean value of the stored data in column-stored files are stored in ablocking index during data storage, both the storage space occupied by the partition index and the storage space occupied by the blocking index are very small, the partition index and the blocking index can be also basically stored in a memory even during massive data storage, and frequent memory replacement caused by huge indexing files is avoided. Therefore, by adopting the data storage method,the partition index and the blocking index can be completely stored in the memory during massive data storage, and the data query speed can be improved. In addition, the invention also discloses a data storage device having the above advantages.

Description

technical field [0001] The invention relates to the field of data storage, in particular to a data storage method and device. Background technique [0002] With the rapid development of the Internet, the amount of data has become larger and larger, especially some non-core data such as buried point logs and monitoring logs have become very large. If you continue to use traditional data storage methods to store these massive data, Not only will it cause a lot of waste of storage resources, but it will also make the query speed of data very slow. [0003] For example, the currently widely used Mysql database with low compression ratio uses a B+ tree index that occupies a large memory space. Due to the low compression ratio of the Mysql database, if the size of the stored data is a, the storage space occupied by the stored data is approximately a, then the storage space occupied by the B+ tree index corresponding to the stored data The size is also approximately a. This is e...

AI Technical Summary

Problems solved by technology

Therefore, although the Mysql database is used to store a small amount of data, although the B+ tree index corresponding to the small amount of data is relatively large, the complete B+ tree corresponding to the small amount of data can still be stored because the amount of data itself is small. The indexes are all stored in the memory, which will not cause memory replacement, so that the data query speed is very fast. However, when the Mysql database is used to store massive data, due to the large B+ tree index corresponding to the massive data, the complete The B+ tree keys are all stored in memory, resulting in memory replacement, which leads to a very slow data query speed

Images