Distributed time sequence database storage engine method and device and medium

A storage engine and time series data technology, applied in the field of distributed time series database storage engines, can solve the problem of space waste, reduce storage space, improve throughput, and reduce heap memory usage

Pending Publication Date: 2022-04-01
上海沄熹科技有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the use of a general-purpose storage engine, the index and label columns are stored multiple times during data storage, resulting in waste of space

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed time sequence database storage engine method and device and medium
  • Distributed time sequence database storage engine method and device and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention will be further described in detail below in conjunction with specific embodiments. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

[0030] A preferred embodiment is given below:

[0031] Such as Figure 1-2 As shown, a distributed time-series database storage engine method in this embodiment adopts a linear data model structure for data partitioning, and partitions data points according to time. Each partition can provide external services independently, and each A partition contains all data points within the time range.

[0032] Each partition uses a separate d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of data storage, in particular to a distributed time sequence database storage engine method.The method comprises the steps that a linear data model structure is adopted for data partitioning, data points are partitioned according to time, each partition can independently provide services for the outside, and each partition comprises all the data points within the time range; each partition uses an independent directory, the directory comprises two files, namely metadata and time series data, and meta-information recorded by the metadata is accessed in an mmap mapping byte array; and for time series data, the data storage space is reduced in a second-order incremental coding mode. Compared with the prior art, the throughput of the distributed time sequence database is improved, heap memory occupation is reduced, the storage space is reduced through a second-order coding mode, the problem of out-of-order time point writing is solved, and it is supported that memory data are prevented from being lost through log pre-writing.

Description

technical field [0001] The invention relates to the field of data storage, and specifically provides a storage engine method for a distributed time series database. devices and media. Background technique [0002] Time series data is a collection of indicator values ​​with timestamps and multiple labels, and it is usually used to record data that changes over time. The workload characteristics of time series data are: [0003] For writes, most operations are written, sometimes 95% to 99%, or even higher. Writes are almost always sequentially appended, rarely updated, and most of the writes are recent, and rarely write indicators generated at a certain time in the past. Generally, they are deleted in batches, deleting data in the historical time range of a certain indicator, There are very few random deletions. [0004] For reading, the data is much larger than the memory. Since the proportion of reading operations is very low, the system is limited by IO. Reading usually...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/21G06F16/22G06F16/27
Inventor 张炜刚贾德星
Owner 上海沄熹科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap