A time series retrieval index system and method for super large-scale data

Abstract

The invention discloses an index system and an index method for retrieving time sequences of ultra-large-scale data. The index system comprises a vertical hierarchical index module and a corresponding time axis index module. The vertical hierarchical index module comprises a plurality of hierarchies of indexes, each hierarchy of indexes comprises a hash function and a plurality of data sets, and data in the data sets are mapped into a corresponding next hierarchy of data set by the aid of the hash function of the corresponding next hierarchy of indexes; the time axis index module is used for creating event lists and time lists. The index system and the index method have the advantages that original big data sets can be ultimately distributed into a plurality of small data sets by means of hierarchy -by- hierarchy hash mapping by the aid of the hierarchical index module, operation such as query processing, data loading and storage optimization can be independently executed on each small data set, accordingly, the vertical hierarchical index module can be combined with the time axis index module, risks of full-table scanning operation in time sequence retrieval operation procedures can be prevented, and the time sequence retrieval speeds can be greatly increased; the index system and the index method for retrieving the time sequences of the ultra-large-scale data can be widely applied to the field of data processing.

Description

technical field

[0001] The invention relates to the field of data processing, in particular to a time-series indexing system and method for ultra-large-scale data. Background technique

[0002] The field of time-series data management of time-series database also involves the time-series retrieval operation on the data in the database. Internally, the time-series database also implements various time-series data management functions efficiently by establishing a data index for the time-series data. In general, these data indexes in the time series database are mainly divided into two categories, one is the index based on the B+ tree structure, and the other is the index based on the R tree structure. For example, there are several specific index structures such as Time Index, Snapshot Index, CheckpointIndex, Archivable Time Index, Overlapping B+ tree, etc.

[0003] Timeline Index is an index structure proposed by Martin Kaufman et al. in 2013. It mainly serves the time-ser...

Images