Distributed time sequence database storage engine method and device and medium

Abstract

The invention relates to the field of data storage, in particular to a distributed time sequence database storage engine method.The method comprises the steps that a linear data model structure is adopted for data partitioning, data points are partitioned according to time, each partition can independently provide services for the outside, and each partition comprises all the data points within the time range; each partition uses an independent directory, the directory comprises two files, namely metadata and time series data, and meta-information recorded by the metadata is accessed in an mmap mapping byte array; and for time series data, the data storage space is reduced in a second-order incremental coding mode. Compared with the prior art, the throughput of the distributed time sequence database is improved, heap memory occupation is reduced, the storage space is reduced through a second-order coding mode, the problem of out-of-order time point writing is solved, and it is supported that memory data are prevented from being lost through log pre-writing.

Description

technical field [0001] The invention relates to the field of data storage, and specifically provides a storage engine method for a distributed time series database. devices and media. Background technique [0002] Time series data is a collection of indicator values ​​with timestamps and multiple labels, and it is usually used to record data that changes over time. The workload characteristics of time series data are: [0003] For writes, most operations are written, sometimes 95% to 99%, or even higher. Writes are almost always sequentially appended, rarely updated, and most of the writes are recent, and rarely write indicators generated at a certain time in the past. Generally, they are deleted in batches, deleting data in the historical time range of a certain indicator, There are very few random deletions. [0004] For reading, the data is much larger than the memory. Since the proportion of reading operations is very low, the system is limited by IO. Reading usually...

AI Technical Summary

Problems solved by technology

Due to the use of a general-purpose storage engine, the index and label columns are stored multiple times during data storage, resulting in waste of space

Images